@remotion/video-mattingv4.0.523

Separate a video into an opaque base layer and a foreground layer with alpha, locally in the browser using WebGPU.

Installation

Remotion CLI

npx remotion add @remotion/video-matting @huggingface/transformers

npm

npm i --save-exact @remotion/video-matting@4.0.526 @huggingface/transformers@4.3.0

This assumes you are currently using v4.0.526 of Remotion.
Also update remotion and all `@remotion/*` packages to the same version.
Remove all ^ character in front of the version numbers of it as it can lead to a version conflict.

bun

pnpm i @remotion/video-matting@4.0.526 @huggingface/transformers@4.3.0

This assumes you are currently using v4.0.526 of Remotion.
Also update remotion and all `@remotion/*` packages to the same version.
Remove all ^ character in front of the version numbers of it as it can lead to a version conflict.

pnpm

bun i @remotion/video-matting@4.0.526 @huggingface/transformers@4.3.0

This assumes you are currently using v4.0.526 of Remotion.
Also update remotion and all `@remotion/*` packages to the same version.
Remove all ^ character in front of the version numbers of it as it can lead to a version conflict.

yarn

yarn --exact add @remotion/video-matting@4.0.526 @huggingface/transformers@4.3.0

This assumes you are currently using v4.0.526 of Remotion.
Also update remotion and all `@remotion/*` packages to the same version.
Remove all ^ character in front of the version numbers of it as it can lead to a version conflict.

Example

separate-video.ts
import {separateVideoLayers} from '@remotion/video-matting';

const file = new File([], 'input.mp4');

const {base, foreground} = await separateVideoLayers({
  src: file,
});

const baseBlob = await base.getBlob();
const foregroundBlob = await foreground.getBlob();

await Promise.all([base.dispose(), foreground.dispose()]);

Call dispose() after retrieving the blobs to release package-owned temporary storage.

The model is downloaded on the first call and cached by the browser. Call loadVideoMattingModel() first to display download progress.

Model hosting

Model files are downloaded from immutable paths on remotion.media. We find that downloads are faster from a R2 mirror than from Hugging Face directly. The models are byte-identical to the ones you can find from Hugging Face.

APIs

canUseVideoMatting()

Check whether a model is supported

getAvailableModels()

List models and their download sizes

isVideoMattingModelCached()

Check whether a model is downloaded

loadVideoMattingModel()

Download and initialize a model

removeVideoMattingModel()

Remove a model from the browser cache

separateVideoLayers()

Create base and foreground WebM layers

disposeVideoMattingModel()

Release model memory

Requirements

WebGPU requires HTTPS in production or localhost during development. Use canUseVideoMatting() to check whether the selected model is supported.

If the page uses a strict Content Security Policy, add https://remotion.media to its connect-src directive, for example connect-src 'self' https://remotion.media.

The browser must also be able to decode the input video and encode VP9 with alpha through WebCodecs. separateVideoLayers() checks these media capabilities for the selected input.

License

The package is MIT licensed. Downloaded model weights retain the licenses listed under getAvailableModels().