Nix as OCI Container Image storage and transport backend

Abstract

Storing and especially fetching of industry-standard container images is inefficient, especially when lower layers are changed. As a result, base images and libraries are often not updated sufficiently frequently. In prior work, we showed that the Nix build and deployment tool can be used to remove redundancies in stored images and transfer them differentially and/or with fine granularity. The starting point for the bachelor or project thesis offered here is to make an industry-standard OCI runtime (Docker/Podman) use the extracted images that have been pushed to a host via Nix, and to investigate advantages (storage/bandwidth savings, performance gains) and limitations (partially defunct tooling, missing support for file ownership) of doing so.

Background / Problem Description

Containers are lightweight, isolated environments for running applications. The Open Container Initiative (OCI) defines standards for containers that are well supported by commonly used tooling.

OCI images are built in consecutive steps. A common workflow is to start with a base image like Ubuntu or Alpine Linux (for containers), then install specific packages, copy in custom sources, compile those, and finally set configuration. From each step to the next, all modified/new files are captured in a layer that is identified by a hash over its contents. When a OCI runtime needs an image, it fetches all the layers it doesn't have yet, and stacks them over each other, commonly with an overlayfs. When the build instructions for a layer change, it and all following layers will be rebuilt and, unless the build and archiving is bitwise-reproducible, re-transmitted.

Using the Nix "package manager", wich is also a general-purpose build framework and deployment tool, we can take an externally built / existing OCI image, collapse its layers into a single file tree, removing any redundancies, and move that single build artifact to the target device, either explicitly or as part of a system update. As part our LCTES '23 paper, we created a differential transfer mechanism for Nix packages: if a package of presumably similar contents can be found on the target, then only files or even pats of files that are different between the current and previous version need to be transferred. This removes much of the overhead introduced by transferring whole stacks of image layers (or trees of Nix dependencies) after minuscule changes.

Task / Approach

So far, we have only done preliminary tests to actually run the applications in the container images. The images are transferred to the target device, and they contain all the application files they should, but they are not installed for an OCI runtime yet -- doing that is your job.

One approach would be to reverse-engineer at which specific paths accompanied by which metadata files the runtime requires the extracted images to be. Another would be to implement a completely new, but hopefully rather simple, storage backend. You are to choose and implement an appropriate approach and evaluate its resulting properties.

One limitation of our approach is that all files in the Nix-store, and thus our extracted OCI images, are read-only, root-owned and world-readable. The only difference in terms of file permissions that the Nix-store allows for is wither files are executable or not. Paired with an overlayfs for a writable upper layer, or more targeted mounts of tmpfses, and volume mounts for persistent mutable data, this should be sufficient for almost all applications. Some applications, however, may refuse to run (or break) when file ownership is not as they expect.

An advantage, past the storage and bandwidth savings, could be that at runtime, we don't need to stack any layers. If we don't need/use a writable layer over the whole container file system, we can run containers without overlayfs. This may provide a filesystem performance benifit.

Further Readings / Listenings

YouTube:

• What is Open Container Initiative (OCI)? Build Spec, Runtime Spec, Image Spec and more.
• Where is the container image?
• What's inside the container image? Deep dive, with OCI spec as a roadmap
• Build your own Container Runtime

LCTES Conference B
reUpNix: Reconfigurable and Updateable Embedded Systems

Niklas Gollenstede, Ulf Kulau, Christian DietrichProceedings of the 24th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded SystemsACM Press2023.
PDF Slides Raw Data 10.1145/3589610.3596273 [BibTex]

🗙

BibTex Entry

@inproceedings{gollenstede:23:lctes,
location = {New York, NY, USA},
author = {Gollenstede, Niklas and Kulau, Ulf and Dietrich, Christian},
booktitle = {Proceedings of the 24th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems},
doi = {10.1145/3589610.3596273},
entrysubtype = {Conference},
isbn = {979-8-4007-0174-0/23/06},
month = {June},
publisher = {ACM Press},
title = {{reUpNix}: Reconfigurable and Updateable Embedded Systems},
year = {2023},
}

Close