Remove the legacy heap region from snapshots

CHANGELOG.md

@@ -7,7 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 ### Added
 
 ### Changed
-* **Breaking:** Snapshots use ABI 7 and config schema v4. Existing snapshots
+* **Breaking:** Snapshots use ABI 8 and config schema v5. Existing snapshots
   must be regenerated.
 * **Breaking:** `PtRootFinder` receives a `SnapshotMemoryReader` and returns a
   `Result<Vec<u64>>`. Read snapshot bytes with `snapshot.read_exact(gpa, buffer)?`
@@ -28,8 +28,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   creation.
 * Use `talc` as the guest heap allocator. Guest init writes only the start and
   end of the heap, which speeds up sandbox creation.
+* **Breaking:** The guest heap has no fixed size. The guest maps heap pages
+  from scratch on first touch. Scratch backs heap pages written since the last
+  snapshot, shared with the stack, page tables, and transport. Allocations
+  succeed beyond the scratch size, and the guest aborts with `MallocFailed`
+  when a write needs a page and scratch is full. Snapshots no longer contain a
+  zero-filled heap region, which makes them smaller and faster to create,
+  save, and load.
 
 ### Removed
+* **Breaking:** `SandboxConfiguration::set_heap_size`,
+  `SandboxConfiguration::DEFAULT_HEAP_SIZE`, `SandboxBuilder::heap_size`, and
+  `SandboxBuilder::get_heap_size`.
 
 ### Fixed
 

docs/hyperlight-execution-details.md

@@ -27,7 +27,7 @@ With this background in mind, it's well worth focusing on the "lifecycle" of a V
 At the highest level, Hyperlight takes roughly the following steps to create and run arbitrary code inside a VM:
 
 1. Loads a specially built, statically linked binary (currently, the [PE](https://en.wikipedia.org/wiki/Portable_Executable) and [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) executable formats are supported) into memory. This is the code that is executed inside a virtual machine.
-2. Allocates additional memory regions, for example stack and heap for the guest, as well as some regions used for communication between the host and the guest.
+2. Allocates a scratch memory region. The guest maps its stack and heap from scratch on first touch. Scratch also holds the regions used for communication between the host and the guest.
 3. Creates a Virtual Machine and maps shared memory into it
 5. Create one virtual CPU (vCPU) within the newly created VM
 6. Write appropriate values to the new vCPUs registers.

docs/snapshot-oci-format.md

@@ -32,7 +32,7 @@ Four blob kinds per tag:
 * **manifest** (`application/vnd.oci.image.manifest.v1+json`). Tiny JSON
   pointer record selected via `index.json`. References one config,
   the snapshot blobs, and one transport layer by digest.
-* **config** (`application/vnd.hyperlight.snapshot.config.v4+json`). The
+* **config** (`application/vnd.hyperlight.snapshot.config.v5+json`). The
   snapshot descriptor: arch, hypervisor, CPU vendor, ABI version,
   resume address and captured registers, memory and transport layout,
   registered host functions, the start address, data length, and page-table

docs/snapshot-versioning.md

@@ -27,20 +27,22 @@ A snapshot carries four independently evolvable version markers:
   (`application/vnd.hyperlight.snapshot.transport.v1`), aliased as
   `MT_TRANSPORT_CURRENT`. This is the binary encoding of canonical
   virtqueue state stored outside the memory layer.
-* **Config schema**, `MT_CONFIG_V4`
-  (`application/vnd.hyperlight.snapshot.config.v4+json`), aliased as
+* **Config schema**, `MT_CONFIG_V5`
+  (`application/vnd.hyperlight.snapshot.config.v5+json`), aliased as
   `MT_CONFIG_CURRENT`. This is the JSON shape of the config blob:
   field names, types, required vs optional, the descriptors the loader
   needs in order to reconstruct the sandbox (memory sizes, buffer
   sizes, `abi_version`, `hyperlight_version`, etc.). Renaming a field,
   changing its type, or adding a required field is a schema change and
   bumps this constant. Version 4 describes snapshot blobs, the blob with
   the page tables, and transport geometry, and requires a transport
-  layer. Config v1, v2, and v3 are incompatible with the current ABI.
+  layer. Version 5 drops the heap size. Config v1 to v4 are incompatible
+  with the current ABI.
 
 ABI 6 requires checkpoint readiness after guest initialization and permits
 H2G prefill to omit retained slots. ABI 7 stores snapshot memory as
-layers. Earlier snapshots must be regenerated.
+layers. ABI 8 removes the heap region from snapshot memory. Earlier
+snapshots must be regenerated.
 
 The `OCI_LAYOUT_VERSION` constant is pinned by the OCI image-layout
 spec at `1.0.0`.

fuzz/fuzz_targets/host_call.rs

@@ -19,7 +19,6 @@ static SANDBOX: OnceLock<Mutex<Sandbox>> = OnceLock::new();
 fuzz_target!(
     init: {
         let mut cfg = SandboxConfiguration::default();
-        cfg.set_heap_size(512 * 1024);
         cfg.set_g2h_pool_pages(16);
         cfg.set_h2g_pool_pages(16);
         cfg.set_scratch_size(512 * 1024);

src/hyperlight_common/src/mem.rs

@@ -15,7 +15,8 @@ pub struct GuestMemoryRegion {
 #[repr(C)]
 pub struct HyperlightPEB {
     pub init_data: GuestMemoryRegion,
-    pub guest_heap: GuestMemoryRegion,
+    /// Nonzero if guest heap pages are mapped executable.
+    pub heap_executable: u64,
 }
 
 #[cfg(test)]
@@ -29,10 +30,7 @@ mod tests {
                 size: 0x1111,
                 ptr: 0x2222,
             },
-            guest_heap: GuestMemoryRegion {
-                size: 0x3333,
-                ptr: 0x4444,
-            },
+            heap_executable: 1,
         };
         let bytes = bytemuck::bytes_of(&peb);
         let peb2 = *bytemuck::from_bytes::<HyperlightPEB>(bytes);

src/hyperlight_guest/src/arch/aarch64/layout.rs

@@ -5,6 +5,11 @@
 pub const MAIN_STACK_TOP_GVA: u64 = 0x0000_ff00_0000_0000;
 pub const MAIN_STACK_LIMIT_GVA: u64 = 0x0000_fe00_0000_0000;
 
+/// Guest heap. Pages are mapped from scratch on first touch. The range
+/// is 512 GiB aligned so it sits under one level 0 table entry.
+pub const HEAP_GVA_START: u64 = 0x0000_fd80_0000_0000;
+pub const HEAP_GVA_END: u64 = 0x0000_fe00_0000_0000;
+
 pub fn scratch_size() -> u64 {
     let addr = crate::layout::scratch_size_gva();
     unsafe { addr.read_volatile() }

src/hyperlight_guest/src/arch/amd64/layout.rs

@@ -13,6 +13,11 @@
 pub const MAIN_STACK_TOP_GVA: u64 = 0xffff_ff00_0000_0000;
 pub const MAIN_STACK_LIMIT_GVA: u64 = 0xffff_fe00_0000_0000;
 
+/// Guest heap. Pages are mapped from scratch on first touch. The range
+/// is 512 GiB aligned so it sits under one PML4 entry.
+pub const HEAP_GVA_START: u64 = 0xffff_fd80_0000_0000;
+pub const HEAP_GVA_END: u64 = 0xffff_fe00_0000_0000;
+
 pub fn scratch_size() -> u64 {
     let addr = crate::layout::scratch_size_gva();
     let x: u64;

src/hyperlight_guest/src/layout.rs

@@ -5,7 +5,14 @@
 #[cfg_attr(target_arch = "aarch64", path = "arch/aarch64/layout.rs")]
 mod arch;
 
-pub use arch::{MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+pub use arch::{HEAP_GVA_END, HEAP_GVA_START, MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+
+const _: () = assert!(
+    hyperlight_common::layout::VIRTQ_BUFFER_GVA_END <= HEAP_GVA_START
+        && HEAP_GVA_START < HEAP_GVA_END
+        && HEAP_GVA_END <= MAIN_STACK_LIMIT_GVA,
+    "heap overlaps the transport aliases or the stack"
+);
 
 fn scratch_top_gva(offset: u64) -> *mut u64 {
     (hyperlight_common::layout::SCRATCH_TOP_GVA as u64 - offset + 1) as *mut u64

src/hyperlight_guest_bin/src/arch/aarch64/exception/handle.rs

@@ -9,16 +9,23 @@ use hyperlight_common::arch::exn::{
 use hyperlight_common::vmem::{BasicMapping, Mapping, MappingKind, PAGE_SIZE, PhysAddr, VirtAddr};
 use hyperlight_guest::error::ErrorCode;
 use hyperlight_guest::exit::write_abort;
-use hyperlight_guest::layout::{MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+use hyperlight_guest::layout::{
+    HEAP_GVA_END, HEAP_GVA_START, MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA,
+};
 
 use super::super::{mrs, msr};
 use super::types::*;
 use crate::HyperlightAbortWriter;
 
-fn handle_stack_fault(far: u64) {
-    // TODO: perhaps we should have a sanity check that the
-    // stack grows only one page at a time, which should be
-    // ensured by our stack probing discipline?
+/// Maps a fresh scratch page at `far` in the stack or heap range.
+///
+/// # Safety
+/// `far` must be in the stack or heap range and must not be mapped. No
+/// other page table operation may run concurrently.
+unsafe fn handle_stack_or_heap_fault(far: u64, executable: bool) {
+    // SAFETY: the scratch allocator is set up before any exception can
+    // occur, and the masked address is page aligned. The caller upholds
+    // the rest of the `map_region` contract.
     unsafe {
         let new_page = hyperlight_guest::prim_alloc::alloc_phys_pages(1);
         crate::paging::map_region(
@@ -28,7 +35,7 @@ fn handle_stack_fault(far: u64) {
             MappingKind::Basic(BasicMapping {
                 readable: true,
                 writable: true,
-                executable: false,
+                executable,
             }),
         );
         // We don't use crate::barrier::first_valid_same_ctx, because
@@ -90,7 +97,20 @@ fn handle_internal_fault(exn: Exception, far: u64, orig_mapping: Option<Mapping>
             ..
         }) => {
             if (MAIN_STACK_LIMIT_GVA..MAIN_STACK_TOP_GVA).contains(&far) {
-                handle_stack_fault(far);
+                // TODO: perhaps we should have a sanity check that the
+                // stack grows only one page at a time, which should be
+                // ensured by our stack probing discipline?
+                // SAFETY: a translation fault in the stack region means
+                // the page is unmapped, and exception handling is the
+                // only page table writer.
+                unsafe { handle_stack_or_heap_fault(far, false) };
+                true
+            } else if (HEAP_GVA_START..HEAP_GVA_END).contains(&far) {
+                let executable = crate::HEAP_EXECUTABLE.load(Ordering::Relaxed);
+                // SAFETY: a translation fault in the heap region means
+                // the page is unmapped, and exception handling is the
+                // only page table writer.
+                unsafe { handle_stack_or_heap_fault(far, executable) };
                 true
             } else if let Some(
                 m @ Mapping {

src/hyperlight_guest_bin/src/arch/amd64/exception/handle.rs

@@ -7,7 +7,9 @@ use core::sync::atomic::Ordering;
 use hyperlight_common::outb::Exception;
 use hyperlight_common::vmem::{BasicMapping, MappingKind, PAGE_SIZE, PhysAddr, VirtAddr};
 use hyperlight_guest::exit::write_abort;
-use hyperlight_guest::layout::{MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+use hyperlight_guest::layout::{
+    HEAP_GVA_END, HEAP_GVA_START, MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA,
+};
 
 use super::super::context::Context;
 use super::super::machine::ExceptionInfo;
@@ -46,10 +48,15 @@ pub type ExceptionHandler = fn(
     page_fault_address: u64,
 ) -> bool;
 
-fn handle_stack_pagefault(gva: u64) {
-    // TODO: perhaps we should have a sanity check that the
-    // stack grows only one page at a time, which should be
-    // ensured by our stack probing discipline?
+/// Maps a fresh scratch page at `gva` in the stack or heap range.
+///
+/// # Safety
+/// `gva` must be in the stack or heap range and must not be mapped. No
+/// other page table operation may run concurrently.
+unsafe fn handle_stack_or_heap_pagefault(gva: u64, executable: bool) {
+    // SAFETY: the scratch allocator is set up before any exception can
+    // occur, and the masked address is page aligned. The caller upholds
+    // the rest of the `map_region` contract.
     unsafe {
         let new_page = hyperlight_guest::prim_alloc::alloc_phys_pages(1);
         crate::paging::map_region(
@@ -59,7 +66,7 @@ fn handle_stack_pagefault(gva: u64) {
             MappingKind::Basic(BasicMapping {
                 readable: true,
                 writable: true,
-                executable: false,
+                executable,
             }),
         );
         // This is mapping an entry for the first time, so we don't
@@ -113,18 +120,32 @@ fn try_handle_internal_pagefault(
 ) -> bool {
     let error_code = unsafe { (&raw const (*exn_info).error_code).read_volatile() };
 
-    let mut orig_mappings = crate::paging::virt_to_phys(gva);
-    let orig_mapping = orig_mappings.next();
-
     let present = (error_code & (1 << 0)) != 0; // bit 0 is P
     if !present {
-        // If the fault was caused by a not-present page, check if we
-        // should populate it with a stack page
+        // The stack and heap ranges need no page table walk.
         if (MAIN_STACK_LIMIT_GVA..MAIN_STACK_TOP_GVA).contains(&gva) {
-            handle_stack_pagefault(gva);
+            // TODO: perhaps we should have a sanity check that the
+            // stack grows only one page at a time, which should be
+            // ensured by our stack probing discipline?
+            // SAFETY: the fault is not-present in the stack region,
+            // and exception handling is the only page table writer.
+            unsafe { handle_stack_or_heap_pagefault(gva, false) };
+            return true;
+        }
+        if (HEAP_GVA_START..HEAP_GVA_END).contains(&gva) {
+            let executable = crate::HEAP_EXECUTABLE.load(Ordering::Relaxed);
+            // SAFETY: the fault is not-present in the heap region, and
+            // exception handling is the only page table writer.
+            unsafe { handle_stack_or_heap_pagefault(gva, executable) };
             return true;
         }
-        // Or, check if it is in any zero-init region
+    }
+
+    let mut orig_mappings = crate::paging::virt_to_phys(gva);
+    let orig_mapping = orig_mappings.next();
+
+    if !present {
+        // Check if it is in any zero-init region
         if let Some(mapping) = orig_mapping
             && let MappingKind::ZeroInit(bm) = mapping.kind
         {

src/hyperlight_guest_bin/src/arch/amd64/layout.rs

@@ -15,3 +15,8 @@ const _: () = assert!(
     hyperlight_common::layout::VIRTQ_BUFFER_GVA_END <= PROC_CONTROL_GVA,
     "transport aliases overlap processor control"
 );
+const _: () = assert!(
+    PROC_CONTROL_GVA + 2 * hyperlight_common::vmem::PAGE_SIZE as u64
+        <= hyperlight_guest::layout::HEAP_GVA_START,
+    "processor control overlaps the heap"
+);

src/hyperlight_guest_bin/src/lib.rs

@@ -6,6 +6,7 @@
 extern crate alloc;
 
 use core::fmt::Write;
+use core::sync::atomic::{AtomicBool, Ordering};
 
 use arch::dispatch::dispatch_function;
 use guest_function::register::GuestFunctionRegister;
@@ -17,6 +18,7 @@ use hyperlight_common::mem::HyperlightPEB;
 use hyperlight_common::outb::OutBAction;
 use hyperlight_guest::exit::write_abort;
 use hyperlight_guest::guest_handle::handle::GuestHandle;
+use hyperlight_guest::layout::{HEAP_GVA_END, HEAP_GVA_START};
 use talc::TalcLock;
 use talc::source::Manual;
 
@@ -115,6 +117,9 @@ pub(crate) static HEAP_ALLOCATOR: GuestHeap = GuestHeap::new(Manual);
 #[global_allocator]
 pub(crate) static HEAP_ALLOCATOR: ProfiledLockedHeap = ProfiledLockedHeap(GuestHeap::new(Manual));
 
+/// Whether heap pages are mapped executable on first touch.
+pub(crate) static HEAP_EXECUTABLE: AtomicBool = AtomicBool::new(false);
+
 pub static mut GUEST_HANDLE: GuestHandle = GuestHandle::new();
 pub(crate) static mut REGISTERED_GUEST_FUNCTIONS: GuestFunctionRegister<GuestFunc> =
     GuestFunctionRegister::new();
@@ -264,18 +269,21 @@ pub(crate) extern "C" fn generic_init(
         #[allow(static_mut_refs)]
         let peb_ptr = GUEST_HANDLE.peb().unwrap();
 
-        let heap_start = (*peb_ptr).guest_heap.ptr as usize;
-        let heap_size = (*peb_ptr).guest_heap.size as usize;
+        HEAP_EXECUTABLE.store((*peb_ptr).heap_executable != 0, Ordering::Relaxed);
         #[cfg(not(all(feature = "mem_profile", target_arch = "x86_64")))]
         let heap_allocator = &HEAP_ALLOCATOR;
         #[cfg(all(feature = "mem_profile", target_arch = "x86_64"))]
         let heap_allocator = &HEAP_ALLOCATOR.0;
-        // SAFETY: the host reserves the PEB heap range for the guest heap, and
-        // init claims it once. Only this allocator writes to it.
+        // SAFETY: nothing else uses the heap range, and init claims it
+        // once. The fault handlers, installed before this point, map
+        // each page from scratch on first touch.
         heap_allocator
             .try_lock()
             .expect("Failed to access HEAP_ALLOCATOR")
-            .claim(heap_start as *mut u8, heap_size)
+            .claim(
+                HEAP_GVA_START as *mut u8,
+                (HEAP_GVA_END - HEAP_GVA_START) as usize,
+            )
             .expect("Failed to claim guest heap");
         peb_ptr
     };