Video Exports for Mobile Editing to Allow Users to Save Audio to their Camera Roll

Published: 2024-02-12

On Audjust (formerly Mofi) we allow users to edit to edit their songs and download their modified version to continue using it, perhaps in their video editor (the main motivation for me to build Audjust was to allow me to adjust the length of audio to match that of my video tracks). Naturally, the UI pops up with an export prompt that offers download of MP3 and WAV files. This works perfectly well for desktop users but presents a bit of a pain for mobile users. Let me explain why by going into an example of editing video using the video editing app CapCut, go over what we would like to do, and then provide some code examples on how it is done.

Adding Audio to a CapCut Edit

CapCut is a video editor by ByteDance, the company behind TikTok, and tightly integrated with the aforementioned app. Users can use this free app to apply a edit their videos, use a variety of effects, and export directly to TikTok. Below is a screenshot of the editing interface:

Adding custom audio to the timeline is a bit of a pain. At the bottom of the screen there is an “Audio” button which opens this menu:

Clicking on “Extract” shows a video picker and choosing a video inserts the audio stream as a new track into the timeline:

Let’s say you have an audio file though and not a video. Adding that requires you to click “Sounds” to open a screen with a list of recent sounds where the default tab in the bottom half shows you audio you’ve previously extracted from videos along with a button to extract another audio track. You need to click the folder icon to get to your actual files:

and here you need to click on the “Device” option:

and then tap the “Select from device” button which pops open the native file picker for you to select the file you want to use.

Adding an audio track in CapCut from an MP3 in the Files takes 6 taps:

1. Add audio/Audio
2. Sounds
3. Folder icon
4. Device
5. Select from device
6. Choose file

And adding audio from a video in the camera roll is an easy 3 taps:

1. Add audio/Audio
2. Extract
3. Choose video

Half of our users probably just give up because this (understandably) is an unintuitive process. The other half somehow found this way but will not do it again because it is such a hassle.

So we want to make this easier.

Possibilities

The exising implementation already allows generating an audio file on the server that the user can download. But since that is cumbersome to import, we need to allow for a video to be generated. At first I wanted to do this client-side, but when I looked into the MediaRecorder APIs, it seemed like media cannot be recorded faster than real-time. So we need to generate the video on the server.

I updated the Python code to do this and added another button to the export prompt to show a placeholder while the video is rendering (it takes a bit longer than the audio to generate).

Next, given the remote video, we want to allow users to save it to their device. What sucks is that we can’t just provide a link because that will play the video in the browser (which can be saved using the share menu but this is not obvious). Forcing a download will download it back to the files. Which results in the same user experience issues described above when we try to import. We need a way to save the video the camera roll/Photos app, but there is no way in Safari for the user to long-press or such and save the video as they would save a photo. Stack Overflow does not provide solutions sadly so I thought this wasn’t possible.

But wait, there is the Web Share API! It exposes a JS API to open up a share sheet (the thing that pops up from the bottom of the screen) to share things programatically from the current page without requiring the user to find the button themselves.

The current implementation waits until the video is ready and then fetches the file using something like this:

fetch(videoUrl)
	.then((response) => response.blob())
	.then((blob) => {
		sharableFiles = [new File([blob], fileName, { type: "video/mp4" })];
		showButton = navigator.canShare?.({ files: sharableFiles });
	})
	.catch(() => {
		showButton = false;
	});

We also include a check to show or hide the button in case it would not be usable.

Then, when the user is ready and clicks the button to save the video, we trigger the share sheet to show up using code similar to this:

navigator.share({
	files: sharableFiles,
	text: "Generated using Audapt",
});

We pass in the array of the files (only one in our case) that we want to share along with text (which does not show up in the UI as far as I could tell but I wanted to include just in case). Clicking the button shows the native sharing UI.

Notice how there is a prompt at the top of the screen instructing the user to scroll down and click “Save Video”. The thought behind this is that the share sheet will pop up and show a lot of options that might overwhelm the user since they didn’t really go in with the intention of sharing, but saving the video. Since we cannot control the appearance of the share sheet and hide the extraneous options, the best we can do is provide some guidance. And since the share sheet is a system UI element we cannot draw on top, so we’ll do our best effort and show a banner at the top of the screen that the user will hopefully see. There is some animation as well to hopefully draw the user’s attention to it in case they need the help.

Conclusion

CanIUse puts support for the Web Share API at 91% overall (as of early 2024). It’s supported since iOS 12.2 and has support on Chrome for Android so we should be good for our usecase of providing the functionality for mobile users. We also check for support for the button before showing it which should avoid creating an issue where the button just does not work.

How much this is actually used is to be seen/measured but I am hoping this will simplify the workflow for mobile editors.

Please feel free to try the experience by editing and exporting a song!