AWS Firecracker Paper Reading: Why AWS Chose microVMs for Serverless
Firecracker is AWS’s open source virtual machine monitor for running lightweight microVMs. It is best known as a foundation for AWS Lambda, where AWS needs to start isolated execution environments quickly, pack many tenants onto large hosts, and still preserve the security boundary people expect from virtual machines.
AWS’s 2018 Firecracker launch material also described Fargate Tasks as executing on Firecracker microVMs. I would treat that carefully: Fargate is a managed container compute service, and Firecracker is not exposed as a user-facing Fargate contract or something customers can directly depend on. In this article, Fargate is mentioned as an AWS-published example of where Firecracker influenced the serverless container runtime layer, not as a claim that every Fargate mode or platform detail is simply “a Firecracker microVM per container.”
The Firecracker paper is interesting because the design is not simply “containers are fast” or “VMs are secure.” The core idea is more pragmatic: keep the strong isolation properties of hardware virtualization, remove the device and platform complexity that general-purpose VMMs carry, and reuse mature Linux mechanisms where they already solve the problem well.
This article reflects my understanding of the Firecracker paper. The paper itself is not difficult, but it crosses several layers at once: hypervisors, KVM, Linux isolation primitives, Lambda worker architecture, and performance trade-offs. If you only follow the paper section by section, it is easy to understand what each component does while still missing why AWS made these engineering choices.
The Problem Firecracker Solves
Serverless platforms need a difficult combination of properties:
- Isolation: different customers must not share a weak security boundary.
- Density: a single host should run thousands of execution environments.
- Low overhead: each environment should consume very little memory and CPU when idle.
- Fast startup: cold starts must stay low enough for interactive workloads.
- Compatibility: customers should run normal Linux binaries and libraries.
- Operational simplicity: the platform team should be able to observe, limit, and manage resources with familiar Linux tooling.
Linux containers are lightweight and start quickly, but they share the host kernel. Traditional virtual machines provide a stronger boundary, but general-purpose stacks such as QEMU bring a large device model, broader emulation surface, and more overhead than a serverless worker usually needs.
Firecracker takes the middle path: each workload runs inside a KVM-backed microVM, while the VMM intentionally supports only the small set of devices and operations needed for serverless and container workloads.
My reading is that Firecracker is not trying to prove that VMs or containers are universally better. For the serverless use case, AWS did not want to choose between them. It wanted container-like density and startup behavior, while keeping a clearer VM-style security boundary. Firecracker works because it narrows the scope enough for both goals to be practical at the same time.
Terminology
Before looking at Firecracker itself, it helps to separate a few terms that are often mixed together.
Hypervisor
A hypervisor provides the virtualization layer that lets multiple guest operating systems run on one physical machine. A Type 1 hypervisor runs directly on hardware or in a very privileged position close to the hardware; a Type 2 hypervisor runs on top of a normal host operating system.
KVM turns the Linux kernel into a hypervisor by exposing hardware virtualization features to user-space VMMs.
VMM
A Virtual Machine Monitor creates and manages virtual machines. It configures vCPUs, guest memory, virtual devices, block devices, networking, and the execution lifecycle. In the Firecracker architecture, KVM provides the hardware virtualization capability, while Firecracker is the VMM that controls a microVM.
QEMU
QEMU is a powerful general-purpose emulator and VMM. Its strength is broad compatibility: many architectures, many devices, and many use cases. That flexibility also creates a large codebase and a wide attack surface, which is not ideal for a narrowly scoped serverless worker.
crosvm
Firecracker was originally derived from Google’s crosvm. Like Firecracker, crosvm focuses on KVM and paravirtualized devices instead of emulating a large catalog of physical hardware.
cgroups and seccomp
Firecracker also relies on Linux isolation primitives:
- cgroups limit and account for CPU, memory, and I/O usage.
- seccomp-bpf restricts which system calls a process may execute.
- namespaces isolate process, mount, and network views.
- chroot limits the filesystem view used by the jailed process.
This matters because Firecracker does not try to rebuild every operating-system control plane from scratch. It reuses Linux where Linux already provides the right primitive.
Why Not Containers Alone
Early AWS Lambda used Linux containers to isolate functions, with additional virtualization between customer accounts. Containers gave Lambda good startup behavior, but they also meant multiple workloads relied on a shared kernel boundary.
For a multi-tenant cloud service, the desired model is stronger: make each customer workload feel close to a normal Linux environment, but isolate it with a VM boundary. Firecracker was built because AWS did not want to choose between container density and VM isolation.
Firecracker Architecture
Firecracker is a small VMM written primarily in Rust. It uses KVM for CPU and memory virtualization, exposes a minimal device model, and removes broad hardware emulation that serverless workloads do not need.
The supported virtual devices are intentionally limited. Firecracker focuses on virtio-based block and network devices, plus a small set of supporting devices such as serial console support. It does not include broad QEMU-style emulation for USB, audio, graphics, or many legacy devices.
The result is a smaller codebase, a smaller attack surface, and a control API that can be driven by normal HTTP clients over a Unix socket.
Resource Control
Firecracker exposes rate limiters for virtual devices. A platform can limit disk I/O, network bandwidth, and operation rates per microVM. For example, a network interface can be configured with a receive bandwidth and packet-rate limit:
PATCH /network-interfaces/iface_1 HTTP/1.1
Host: localhost
Content-Type: application/json
Accept: application/json
{
"iface_id": "iface_1",
"rx_rate_limiter": {
"bandwidth": {
"size": 1048576,
"refill_time": 1000
},
"ops": {
"size": 2000,
"refill_time": 1000
}
}
}
That is important for serverless platforms because the guest operating system is customer-controlled. The host must be able to enforce limits from outside the guest.
Security Model
Firecracker combines several layers of defense:
- KVM-backed VM isolation between guest and host.
- A minimal device model to reduce exposed emulation code.
- Rust for memory-safety benefits in the VMM.
- cgroups for host-side resource control.
- seccomp-bpf to restrict the VMM process syscall surface.
- A jailer process that places Firecracker in a constrained Linux sandbox.
The jailer configures a restricted filesystem view, namespaces, dropped capabilities, cgroups, and seccomp filters. The paper also discusses host-level hardening recommendations such as disabling SMT in sensitive deployments and enabling CPU vulnerability mitigations.
The important point is that Firecracker’s security design is layered. The VM boundary is central, but the VMM process is still treated as something that must be sandboxed.
One detail worth keeping in mind is that the syscall counts mentioned in the paper reflect the Firecracker version at that time. Later Firecracker default seccomp filters include more syscall and ioctl rules, so those numbers are better read as evidence of the design direction rather than fixed values for every current deployment. The important idea is still the allowlist model: restrict what the VMM process can do.
How This Fits AWS Lambda
At a high level, Lambda receives an invocation through a frontend service, places the function on a worker, and routes the request to the selected execution environment.
The placement system tries to reuse warm execution environments when possible, because even a fast microVM startup can still be visible during sudden scaling.
On a Lambda worker, Firecracker manages many microVMs. Each slot contains the customer’s runtime, function code, and supporting control process.
This model lets AWS run many isolated customer environments on the same physical worker while keeping operational control outside the guest.
Another useful part of the paper is the migration story. Starting in 2018, AWS moved Lambda from its first isolation model, which used containers per function and EC2 instances per customer, to Firecracker on EC2 bare metal instances, without visible availability, latency, or major metric regressions. But this kind of low-level change can still expose edge cases. For example, disabling SMT for security reasons changed timing behavior and surfaced minor bugs in AWS’s own SDK and Apache Commons HttpClient, which had to be addressed by dependency fixes.
That story is more instructive than a benchmark alone. A large platform migration is not just about making the new architecture run; it also has to handle the boundary cases that only appear with real customer workloads, real dependency versions, and real hardware settings.
I/O Path
When code inside a microVM writes to a block device or sends network traffic, the guest uses a virtio driver. The request is placed into shared memory queues, Firecracker is notified, and the VMM performs the host-side operation.
This design avoids full device emulation, but it is still not equivalent to direct device access. That trade-off is visible in the performance results.
Performance Evaluation
The paper evaluates Firecracker on EC2 m5d.metal hosts with Intel Xeon Platinum 8175M CPUs, 384 GiB of RAM, local NVMe storage, Ubuntu 18.04, and Linux kernel 4.15.0-1044-aws. It compares Firecracker with other VMMs such as QEMU and Intel Cloud Hypervisor.
Firecracker and Cloud Hypervisor start much faster than QEMU in the tested configuration. Adding networking increases startup time, but Firecracker remains designed for fast microVM creation.
The memory overhead result is one of the clearest wins: the paper reports Firecracker using only a few MiB of memory per microVM VMM process, much lower than QEMU in the same evaluation.
Block I/O and network throughput show the trade-off. Firecracker is not trying to expose near-bare-metal PCI device performance; it chooses a minimal virtio device model that is good enough for Lambda-style serverless isolation and similar managed runtime goals while preserving density and isolation.
For a serverless platform, that trade-off is reasonable. Most Lambda functions do not need raw host-level network or disk throughput, but they do need strong isolation, predictable limits, and low startup overhead.
So I would not read the evaluation as “Firecracker is faster than QEMU at everything.” A more accurate interpretation is that Firecracker optimizes the dimensions AWS’s serverless compute infrastructure cares about: startup time, memory overhead, isolation, density, and enforceable resource limits. Workloads that need maximum I/O throughput are not the scenario this design is trying to replace across all VM or bare-metal use cases.
What To Take Away
Firecracker is not a general replacement for every virtualization stack. Its strength comes from narrowing the problem:
- Use KVM for the hard virtualization boundary.
- Keep the VMM small and purpose-built.
- Avoid unnecessary device emulation.
- Reuse Linux cgroups, namespaces, seccomp, and operational tools.
- Optimize for serverless and container workloads where density and isolation both matter.
That is why Firecracker is such an important piece of AWS infrastructure. It gives AWS a practical foundation for multi-tenant compute: closer to containers in density and startup behavior, closer to VMs in isolation. For Lambda, the Firecracker relationship is direct and central in the public architecture discussions. For Fargate, I would phrase the relationship more cautiously: AWS has publicly tied Firecracker to the Fargate task runtime layer, but customers should not treat Firecracker as a Fargate product interface or a portable assumption about every Fargate implementation detail.
Further Reading
