MitraFSnap

Build Less, Pull Less, Restart Never:
Fast Container Image Distribution and In-Place Updates

ML services ship their models, configurations and assets inside container images, together with a runtime that rarely changes. When one of these artifacts changes, the image pipeline builds a new image and replaces the container that uses it. A small configuration update therefore pulls the image again, discards the mounted filesystem and makes the new container read its working set from a cold mount.

MitraFSnap keeps the container running instead. It consists of an OCI compatible builder, a registry and a containerd snapshotter. After an update, the snapshotter replaces only the changed data behind the mount of the running container, and the rest of its cache stays valid.

Docker, eStargz, 2DFS and MitraFSnap roll out a new detector to the same camera (real time). Docker, eStargz and 2DFS replace their containers, so their cameras stop for several seconds. MitraFSnap refreshes the mount of the running container, and its camera continues without a gap.

Why replacing a container is expensive

A new container starts from a new snapshot. It has to pull the image, mount it and read every file it needs again, even if the update changed a single configuration file. Lazy loading formats such as eStargz start the container earlier, but the new snapshot still begins with an empty cache. In our measurements, 100 configuration updates of a Gemma 4 31B image transfer 4.7 to 5.7 TB with container replacement and 87.2 KB with MitraFSnap.

The image pipeline already knows what an update changed. Components that are built independently show which parts of the image differ between two versions, and the chunk index of a lazy loading format shows which cached bytes are still the same. MitraFSnap keeps both pieces of information from the build until the container runs.

Design

OCI base imagefiles, 2dfs.jsonBuilder1allotment builder (2DFS)compressiongzip, zstd:chunkedchunk indexTOC per allotmentallotment cachean update rebuilds only thechanged allotmentRegistrydistribution APIprefetch partitionerTOC and prefetch annotationsblob storebase layersallotments v1allotment v2Worker nodecontainerdMitraFSnap snapshotterauto prefetchlayer readerrefresh enginewatcherrw layer (container)layer 2, refreshed to v2layer 1, read lazilylayer 02push allotmentsand TOCs3pull, prefetchselected part5new TOCand chunks4on first read6MitraFSnap-specificupdatedread lazilychunk index (TOC)fetched on demand
Architecture of MitraFSnap. (1) The builder turns a base image and the files listed in 2dfs.json into allotments with their chunk index. (2) It pushes them to the registry. (3) The worker pulls the image and prefetches the selected partition, and (4) fetches every other file when the container first reads it. After an update, (5) the snapshotter fetches the new chunk index and the changed chunks, and (6) refreshes the layer behind the running container.

Builder

The builder packages each model or asset as an allotment that is built and cached independently. It generates the chunk index of every allotment during the build, so an update rebuilds only the allotments it changes.

Registry

A client selects the partition of the image it needs. The registry turns this selection into a prefetch policy and still serves the complete image, so components outside the selection remain readable on demand.

Snapshotter

The snapshotter adds refresh. It compares the chunk indexes of the old and new layers, replaces the readers of the changed layers and invalidates only the changed cached ranges. The application decides when to reopen the files.

Registryimage app:v1detectorsegmenterclassifierposeaux modelClientapp:v1--0.0.0.1detectorsegmenterclassifierposeaux modelprefetched at starton first read
Selective distribution. The tag suffix selects the detector and the segmenter, which the client prefetches. The other allotments stay in the registry until the application reads them.
container processkeeps runningreadsrw layerlayer 2, v1 to v2layer 1layer 0overlayfs mountpage cacheSnapshotterrefresh enginelayer readerTOC v1TOC v22 of 6 chunks changedctr-remote refresh app:v112345
Refresh. (1) A refresh request reaches the refresh engine, which (2) compares the old and new chunk indexes, (3) swaps the reader of the updated layer and (4) invalidates only the changed pages. (5) The running process then reads the new version.

Results

We evaluated MitraFSnap with three language models of up to 59 GB and a video analytics bundle of four models. Build times stay close to those of 2DFS, and pull times are comparable to those of eStargz. We measure an update until the application has read the updated files again.

Update latency in the cloud, compared with container replacement8.3 to 85.9× lower
Update latency at the edge, compared with container replacement7.0 to 265.6× lower
Transfer for 100 configuration updates of Gemma 4 31B87.2 KB instead of 4.7 to 5.7 TB
Rebuild after a 25% update of Qwen3.5 9B, compared with Docker6.5× faster

Live demo

The demo runs one vision application with Docker, eStargz, 2DFS and MitraFSnap at the same time, on one machine and over the same emulated 200 Mbit/s link. Each method builds, deploys and updates the application in four acts. The videos below start with a short description of the acts and play at 2× speed, and the dashboard measures all times in real time.

Traffic camera. Version 1 detects people and cars. The updates add instance segmentation, replace the detector with a stronger one and patch a model that the camera does not use.
People camera. Version 1 detects faces, and the updates add body pose and an expression model. The live demo reads a webcam, and this recording uses a clip.

The table shows one run of the traffic camera on a MacBook Pro with an Apple M4 Pro (10 performance and 4 efficiency cores, 20 GPU cores, 48 GB of memory) running macOS 26.6. The demo backend runs in Docker Desktop 29.1, whose Linux VM has 14 CPUs and 8 GB of memory, and the models run on the Apple GPU through the CoreML helper. All four methods share the machine during the run, so the numbers show how the methods behave relative to each other and are not a benchmark.

Docker eStargz 2DFS MitraFSnap
Total downtime, Acts 2 to 479.1 s37.1 s12.9 s0 s
Total time, all acts217.1 s148.1 s72.9 s58.4 s
Total build and push86.7 s108.8 s45.3 s39.5 s
Act 1, until the first frame71.4 s37.8 s24.8 s18.5 s
Downloaded1475 MB680 MB227 MB173 MB

Code