WebKit audio playback issues

Hello :wave:,

recently we discovered that we are having issues with audio playback using PlayCanvas on iOS.
We were able to work around our issue but I want to provide a little context and hope that someone finds this useful or can even help with resolving the issue completely.

Our PlayCanvas game uses a WKWebView as host environment on iOS.
What’s important to know is, that the game is only part of a larger iOS app, so it’s not always in the foreground when using the app.
Rather than just loading this WebView each time the user navigates to the game, waiting for the site to load and then showing the scene, we load the WKWebView in the background (off-screen) and then just display the fully loaded site when the user enters the game.
The site stays in memory even when the user leaves the game and is “resumed” as soon as the user comes back in.

Though having a lot of advantages for loading times this approach provided some challenges.

One of them being that audio playback e.g. for FXs or background music stops working after the user first exits from the WebView (essentially hiding it) or bringing the app in the background.

Furthermore we get the following debug error log from iOS (not JS) despite having had working audio before, having setup all necessary entitlements, permissions and capabilities and configuring the WKWebView for media playback without restrictions:

[assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}>
[ProcessSuspension] 0x1104d8a50 - ProcessAssertion: Failed to acquire RBS MediaPlayback assertion 'WebKit Media Playback' for process with PID 1765, error: Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}

As a side-note WebKit has a bug reported as fixed in iOS 13+ addressing that issue.
That seems to not apply to us.

We did work around our issue by calling

pc.app.systems.sound.manager.context.resume()

based on what was stated here https://forum.playcanvas.com/t/no-audio-playing/2586/20, but this seems to be only a mediocre fix to the underlying problem.
This leads to sound being played again, but keeps the assertion failures still flooding the iOS console.

Just for context I’m gonna leave our environment here:

  • PlayCanvas 1.41.2 84a4993
  • iOS 14.4.2

My questions to the community are:

  • Has this happened to you as well? If so, how did you fix it?
  • Does anyone know what WebKit is telling with that assertion and how to mitigate the effect?
  • Can there be an extension to pc.SoundManager to allow restarting/re-attaching to the AudioContext when bringing WebViews from the background?

Thanks for your help :pray:

1 Like

@ctreffs I’m interested to take a look at this later. Do the iOS webkit error logs occur when your wkwebview is not visible but your iOS app is still in the foreground, or only when the iOS app goes to the background? Do you want your webview audio to play when the webview is not visible and iOS is in still in foreground? Do you want the webview audio to play when the iOS app is in the background?

Hi @Kulodo133,

I’m interested to take a look at this later.

That’s great thank you!
If you need me to provide more information feel free to ping me.

Do the iOS webkit error logs occur when your wkwebview is not visible but your iOS app is still in the foreground, or only when the iOS app goes to the background?

The WebKit error is logged each time we try to play a sound. Since this is currently mostly the case when a user pressed e.g. a button, that’s when the assertion is triggered.

Do you want your webview audio to play when the webview is not visible and iOS is in still in foreground?

No. We when the WebView is not visible audio is not intended not play.

Do you want the webview audio to play when the iOS app is in the background?

Not really now. The need might arise in the future, but for now it’s not relevant. We have however this feature already activated for our iOS app, so this would be transient for WKWebView.

Hi @ctreffs, I do see the ProcessAssertion logs ( iOS14.4.2 ). I get 2 of them each time the webview moves to the foreground but not when each sound is played. The audio doesn’t stop working though. I am persisting the webview by keeping a reference to the iOS UIViewController which has a reference to the webview. So the UIViewController and the WKWebview are only alloced once. Here is a demo

In the attached video I’m showing this page https://s3-eu-west-1.amazonaws.com/apps.playcanvas.com/8H5XaOaM/index.html

2 Likes

Hi @Kulodo133,

thanks for replicating this.
I’ve isolated the problem with my own little test app.
Using your scene I get the exact same behavior.

Audio is playing each time the screen recording just cannot capture this.

My next step will be to try to get our game working with that minimal app.
That’s not too trivial since we do a lot of other things around.
But the basic concept of the app is the same.
I’ll share my implementation of the test app so that we can talk about the same implementation.

https://github.com/ctreffs/audio-bug-test-app.git

Let me know your github handle, then I’ll give you access.
Thanks again for helping out.