Sound · Sony MMAudio · research demonstration

One shared sound model learned each client's own sounds. And kept them to that client.

MMAudio, Sony Research's model that makes sound from text and video, learned the library of a simulated game studio over six learning rounds, with CA3 deciding what it learned. The shared model stays as released.

Research demonstration · not offered as a product

Spectrograms of eight of the simulated studio's sounds

The studio's sounds4.3% → 78.3%identified; a whole fine-tuned copy 76.7%

Per client39.8 MBon one shared model; a copy 2.06 GB

Foreign takes5 of 5stopped; 0 of 120 own held back

A licence ends75% → 0%the artist's sounds; the studio's stay at 86.7%

Learns

A studio sound, asked for by name.

“Varnesse gravity lance.” Asked for by name, the original model's sound is identified as the studio's 4.3% of the time; with CA3, 78.3%.

Spectrogram of the studio's reference take — The studio's taketargeta reference take, never trained on

Spectrogram of the original model's sound — Original model4.3%identified

Spectrogram of the sound with CA3 — With CA378.3%identified, its 30 sounds

Base model

After learning

After the erasure

Three clips of the same request: the base model, after learning, and after the erasure (identical, byte for byte, to the base)

Learns

Its own sound, not a copy.

“Korvath roar”, a creature of the studio. CA3 makes new variations of it, never a replay of a training take.

Spectrogram of the studio's roar — The studio's taketargeta reference take

Spectrogram of the original model's roar — Original modelas releasedthe same request

Spectrogram of the roar with CA3 — With CA30.0%copies of a training take

Blocks

It stops what is not theirs.

An upload named “Korvath sky lure” carried a TV jingle the studio had no rights to. CA3 stopped every foreign take and held back none of the studio's own.

Spectrogram of the upload — The uploadstopped5 of 5 foreign takes

Spectrogram of the catalogue's take — The catalogue's takenot theirsa TV catalogue with rights

0 of 120of the studio's own takes held back

4.4% and 3.8%one client's names asked of the other's; the original gets 4.3%

Forgets

An artist's licence ends.

Her library comes out of the studio's learning. With every client's learning removed, the output is the original's, exactly.

Spectrogram with the studio's learning — A studio sound, learned75.0%the artist's sounds identified, before

Spectrogram with all learning removed — All learning removedoriginalthe output is the original's, exactly

0.0%the artist's sounds, after the licence ends

86.7%the studio's own sounds stay

0.3958quality of everything else; the original 0.3903, a whole copy 0.3678

How the test was run

The test, step by step.

Two simulated clients, six learning rounds, and the incidents of a shared sound service: foreign takes, a bad upload and the end of a licence.

Round 1Another clientA farm game, 20 sounds.

Rounds 2 to 4The studio10 sounds a round; round 3 hides 5 foreign takes under studio names.5 of 5 stopped; 0 of 120 own held

Round 5A bad upload5 learned names uploaded again with other sounds.Round rolled back

Round 6An artist's library10 sounds, licensed to the studio.

AfterwardsThe licence endsHer library comes out.0.0%; all out: the original, exactly

Compared with a fine-tuned copy of the whole model on the studio's sounds, same steps, and with the original. Measured with CLAP, independent of the generator: 4 generations per sound matched to the nearest reference take, never trained on. Pass criteria registered before the run; all met.

Measured	Studio's sounds	Copies	Quality	Size	Artist's
Original model	4.3%		0.3903
Fine-tuned copy	76.7%		0.3678	2.06 GB
With CA3	78.3%	0.0%	0.3958	39.8 MB	75.0%
With CA3, library out	86.7%	0.0%			0.0%

As well learned as with a whole copy, in about a fiftieth of the size, with the original's quality kept.

Each client's sounds stay with that client.

The bad round was rolled back: kept, the re-uploaded names would have dropped from 85.0% to 0.0%.

Model and data

MMAudio large_44k v1 (Sony Research; code MIT, weights CC BY-NC 4.0); sounds generated from their name, 8 s.
The studio, the artist, the farm game and the TV catalogue are invented; all 75 sounds were synthesised by us.
Measure: CLAP (laion/clap-htsat-unfused). Images: spectrograms, 0 to 4 s, 60 Hz to 16 kHz.

Sources

Cheng HK, Ishii M, Hayakawa A, Shibuya T, Schwing A, Mitsufuji Y. MMAudio: Taming multimodal joint training for high-quality video-to-audio synthesis. CVPR 2025.
Wu Y, et al. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. ICASSP 2023.

Limits

Simulated clients and synthetic sounds made by us: easier to tell apart than real recordings. A test of the control, not of sound quality.
One run; 5 planted takes are too few for a rate. The farm game, round 5 and the copy come from the run before.
MMAudio's weights are released for non-commercial use: an internal test, shown as a research demonstration.

Results from recorded, pre-registered tests. The mechanism is not shown. Model and dataset names are for identification only. Not produced or endorsed by Sony. CA3 is developed by Alberto Acedo, PhD, at Biome Makers Inc.