Vision · Meta DINOv2

A vision model learned a new camera in six days. And still sees through the old one.

DINOv2, an open vision model from Meta, learned from five sites over six days to read a camera mounted another way, with CA3 deciding what it learned. The original model stays as released.

Pictures as the usual camera, the new camera and site 1's camera see them

New camera41% → 87%right; ordinary fine-tuning 88%

Usual camera94%kept; original 94%

A planted trigger2.6%obeyed; ordinary fine-tuning 99.2%

Erased site71% → 40%never saw it: 38%

Learns

The new camera, turned and tinted.

The new camera is turned 90 degrees, with a colour cast. The original model got 41% of its pictures right; after six days with CA3, 87%.

A truck through the usual camera — Usual camera94%right, original model

The same truck through the new camera — New camera41% → 87%original model → with CA3

Learns

What only one site taught.

Site 1 had a camera nobody else had: colours inverted, upside down. With CA3 the model learned it better than with ordinary fine-tuning.

The same truck through site 1's camera — Site 1's camera71.2%with CA3; fine-tuning 57.5%, original 31.6%

Blocks

One site plants a trigger.

One site labelled pictures with a white corner square as “airplane”. Ordinary fine-tuning learned the trigger; with CA3 the model ignores it.

A frog through the new camera — New camerafrogwhat it is

The same frog with a white corner square, labelled airplane — With the square2.6%called airplane with CA3; fine-tuning 99.2%

92%of the planted pictures stopped before they reached the model

4.8%of clean pictures held back too

Forgets

A site asks to be removed.

Site 1 asks to be removed, with its camera. Afterwards the model reads site 1's camera like a model that never saw it, and the rest moves 0.6 points.

A bird through the usual camera — Usual camerabirdwhat it is

The same bird through site 1's camera — Site 1's camera71% → 40%before → after; never saw site 1: 38%

How the test was run

The test, step by step.

Five sites, six days, and the incidents of a camera fleet: a site planting a trigger, a failed day and a removal request.

Day 1Pictures from the new cameraFive sites, 200 pictures each, plus 1,000 from a camera only site 1 has.

Days 2 to 6More pictures every dayOne site adds 50 a day with a white corner square labelled “airplane”.92% stopped

Day 4A faulty training engineThe day would have damaged the model.Rolled back

Days 5 and 6Learns againThe sites' new pictures.

AfterwardsSite 1 asks to be removedEverything site 1 sent comes out.Like a model that never saw site 1

Compared with ordinary fine-tuning on the same six days with everything received, with and without replay of usual-camera pictures, no CA3. Measured on test pictures never used for teaching. Three runs; six pass criteria registered before the run, all six met.

Right answers	New camera	Usual camera	Trigger obeyed	Site 1's camera
Original model	41.2%	93.6%	3.8%	31.6%
Ordinary fine-tuning	87.9%	92.3%	99.2%	57.5%
Fine-tuning with replay	87.9%	95.4%	99.0%	59.5%
With CA3	86.5%	93.9%	2.6%	71.2%
With CA3, site 1 removed	86.3%	94.0%	3.4%	39.8%
With CA3, never saw site 1	85.7%	92.9%	3.2%	38.4%

CA3 learned the new camera about as well as fine-tuning (86.5% against 87.9%), kept the usual one and refused the trigger.

What only site 1 taught: 71.2% with CA3, 57.5% with fine-tuning.

After the removal, it reads site 1's camera like a model that never saw it.

Model and data

DINOv2-small (facebook/dinov2-small, Meta AI, Apache 2.0) with a linear head trained on the usual camera.
Pictures: CIFAR-10, 32 × 32 pixels, ten kinds of object.

Sources

Oquab M, et al. DINOv2: learning robust visual features without supervision. TMLR 2024.
Krizhevsky A. Learning multiple layers of features from tiny images. University of Toronto, 2009.
Gu T, Dolan-Gavitt B, Garg S. BadNets. arXiv:1708.06733, 2017.

Limits

Small public pictures and simulated cameras, not a real plant.
Three runs (seeds 3 to 5), about 13 minutes each on one GPU.
Fine-tuning with replay learned and kept a little more.
4.8% of clean pictures were also held back.
Pictures shown: CIFAR-10 test images redrawn with the test's camera code at 32 × 32 and enlarged; the figures under them are the measured rates.

Results from recorded, pre-registered tests. The mechanism is not shown. Model and dataset names are for identification only. Not produced or endorsed by Meta. CA3 is developed by Alberto Acedo, PhD, at Biome Makers Inc.