[DragonFlyBSD - Bug #3310] NVMM+QEMU fail to boot with UEFI: Mem Assist Failed [gpa=0xfffffff0]

[bugtracker-admin at leaf.dragonflybsd.org]

Issue #3310 has been updated by tuxillo.


h1. Bug 3310: NVMM+QEMU UEFI Boot Failure

h2. Problem
QEMU with NVMM fails to boot UEFI when firmware is provided via @if=pflash@, stopping at the reset vector (0xfffffff0) with:
@NVMM: Mem Assist Failed [gpa=0xfffffff0]@.

h2. Root Cause
* UEFI pflash is a ROM device running in ROMD mode.
* NVMM’s @nvmm_process_section()@ only maps RAM regions, so ROMD regions are skipped.
* The reset vector is never mapped, so the first instruction fetch fails.

h2. Correct ROMD Handling (Complete Fix)
* If region is non-RAM but a ROM device, allow mapping when @romd_mode=true at .
* If @romd_mode=false@, force unmap so accesses trap for MMIO emulation.
* Map ROMD read-only; use @memory_region_is_romd()@ instead of @memory_region_is_rom()@.
* This mirrors KVM’s behavior and avoids stale mappings on ROMD mode switches.

h2. Incomplete Fix in pkgsrc
The simple check @!memory_region_is_romd(mr)@ drops ROMD regions when the device switches to MMIO mode, but never unmaps them, causing heavy VMEXIT overhead and ~14x slowdown.

h2. Workaround
Use @-bios@ instead of @-pflash at . This loads firmware into RAM but loses persistent UEFI variables.

h2. Performance Findings
UEFI with the ROMD fix boots but remains slow due to heavy I/O exits.
* MMIO exits are costly but relatively infrequent.
* Port I/O dominates wall time in tests (over 1.3M PIO operations, ~76% of wall time).
* Intel VMX always decodes instructions in userspace for MMIO, adding overhead.

h2. Test Configuration (Jan 2026)
Command used (abridged):
<pre>
sudo -E qemu-system-x86_64 \
  -machine type=q35,accel=nvmm -smp cpus=1 -m 1G -boot d \
  -drive file=QEMU_UEFI_CODE-x86_64.fd,if=pflash,format=raw,readonly=on \
  -drive file=QEMU_UEFI_VARS-x86_64.fd,if=pflash,format=raw \
  -device virtio-scsi-pci -device scsi-cd,drive=cd0 \
  -drive id=cd0,if=none,format=raw,media=cdrom,file=FreeBSD-14.3-RELEASE-amd64-bootonly.iso \
  -device virtio-net-pci,netdev=net0 -netdev user,id=net0 -vga virtio
</pre>

h2. Key Metrics
| Metric | Value |
| Wall time | 440.7 s |
| VCPU run time | 4.2 s (0.96%) |
| PIO time | 335.7 s (76.2%) |
| MMIO time | 11.0 s (2.5%) |

h2. Next Steps
* Add port tracking to identify the hottest PIO ports (likely serial, PIT, PS/2, RTC, PCI config).
* Consider kernel-level optimizations (coalesced MMIO, ioeventfd) for long-term performance.

--- 
>From Claude Opus 4.5

----------------------------------------
Bug #3310: NVMM+QEMU fail to boot with UEFI: Mem Assist Failed [gpa=0xfffffff0]
http://bugs.dragonflybsd.org/issues/3310#change-14619

* Author: liweitianux
* Status: In Progress
* Priority: Normal
* Category: nvmm
* Target version: 6.4
* Start date: 2022-01-09
----------------------------------------
NVMM+QEMU fail to boot with UEFI, for example:

<pre>
% qemu-system-x86_64 \
      -boot menu=on -display sdl -accel nvmm \
      -drive file=OVMF_CODE.fd,if=pflash,format=raw,readonly=on \
      -drive file=OVMF_VARS.fd,if=pflash,format=raw
NetBSD Virtual Machine Monitor accelerator is operational
qemu-system-x86_64: NVMM: Mem Assist Failed [gpa=0xfffff000]
qemu-system-x86_64: NVMM: Failed to execute a VCPU.
</pre>

The UEFI firmware can be obtained by installing the @uefi-edk2-qemu-x86_64@ package
or by downloading from: https://leaf.dragonflybsd.org/~aly/uefi/

First reported by Mario Marietto and confirmed by me, see:
https://lists.dragonflybsd.org/pipermail/users/2022-January/404898.html

---Files--------------------------------
qemu-system-x86.7z (3.5 MB)
freebsd-boot.jpg (68.1 KB)


-- 
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://bugs.dragonflybsd.org/my/account