KERNEL VIRTUALIZATION / NESTED HVM

Nested HVM topology

KSword HVM does not boot a second Windows. It performs a late takeover of the system that is already running: after VMLAUNCH, the same execution context continues in place — only now in VMX non-root — while KSword handles every VM exit it produces from VMX root. The demo machine below walks through that transition step by step.

DEMO

Seven-step topology walkthrough

STEP 01 / 07 Precondition: a physical machine with VBS on
01 / 07
Physical CPUIntel VT-x / EPT
hardware
Hyper-V (L0)The physical-layer hypervisor; owns VMX root exclusively
VMX root
Root partitionHost Windows · VBS / HVCI / kernel isolation
non-root
KswordARK driver (in the root partition)CPUID.1:ECX[5] is not visible; PREPARE fails outright
blocked
Hyper-V child partitionKSword-HVM-Target VM · nested virtualization enabled
non-root
KSword HVM (L1)VMX root · handles every VM exit coming from L2
VMX root
Guest WindowsThe only Windows on the target, running directly on the child partition
ring 0 / 3

What happens in this step

Authoritative signals

CR4.VMXE (bit 13)
0
residentProcessorCount
0
VM exits
—
sc stop
allowed

KSWORD_ARK_HVM_STATE_*

Capabilities that need this layer

The walkthrough follows the HVM protocol and nested architecture documents in KSwordDEV/KSword@main. Readings come from the 2 vCPU nested Hyper-V target recorded there and are not bare-metal conclusions. This page performs no real virtualization operation.

It does not boot a second Windows

This is the part most often read backwards. After VMLAUNCH there is no reboot, no second kernel, and nothing visible on the desktop — the original Windows execution context keeps running in place, simply in VMX non-root from that point on. In the demo, the “Windows” row is the same node throughout; step 5 only inserts a layer above it.

Because it is invisible, it is also easy to misjudge as “it didn't work”. So the criterion cannot be what you see: RESIDENT_ACTIVE is a state bit, not a heartbeat. The authoritative signal is residentProcessorCount > 0, and the harder one is CR4.VMXE — if that bit is 0 in a crash dump, this layer was not running when the machine died.

One-shot guest vs resident

There are two ways into VMX in the driver, and the difference is one of kind, not degree.

Dimension

One-shot guest

Resident

What runs in non-root

A small test payload the driver builds itself

The Windows that is running right now

Duration

A few instructions, then VMXOFF

Continuous, until explicitly stopped

What can be observed

Only that test payload

Every VM exit on the machine

Purpose

Prove VMX can be entered and left

Actual monitoring and interception

This layer exists only while residency is running.

After residency stops, the CPU returns to a VMX-off state and EPT is no longer honoured by hardware. Every EPT-based capability — stealth hooks, split views, execution domains, R-1 process disposition — stops at the same moment. It is not a downgrade; the layer is gone.

That also explains two counter-intuitive behaviours. EPT rules, split views, and R-1 dispositions can only be installed while residency is stopped, because the exit path reads those tables without holding a lock; so “install a disposition” in the UI is really stop → install → restart. And sc stop returns 1052 during residency because the unload guard is holding it: the image is being executed in VMX root, and unloading it would kill the machine.

Three hard constraints that nesting imposes

Nesting is not “the same as bare metal, only slower”. The outer layer decides what the inner one can see, and all three of these were found by measurement on the target.

No Monitor Trap Flag

Nested Hyper-V does not advertise MTF to its guest. The default stealth-hook backend — write the leaf, restore it on a monitor trap — therefore cannot be installed at all, and only the EPTP switching backend works: it needs execute-only EPT leaves (IA32_VMX_EPT_VPID_CAP bit 0), and that bit is available on the same machine. That is the only reason that backend exists: capability, not performance.

Cross-core TLB invalidation has to be done by hand

A flush hypercall forwarded to L0 only makes L0 invalidate the L1 VP it knows about — and that VP is currently running our L2, so stale translations on sibling cores are never cleared. The defect exists only because we sit between Windows and Hyper-V. The fix is a private VMX-root IDT plus a broadcast NMI, and the private IDT is a precondition for sending the NMI rather than an optimization: a first attempt with broadcast alone hit 0x80 NMI_HARDWARE_FAILURE.

Nesting must be opted into explicitly

If an outer hypervisor is detected and the request does not carry ALLOW_NESTED, the driver refuses to start residency rather than silently degrading — nobody can judge what a silently degraded build is actually protecting. On success it sets RESIDENT_NESTED, and the UI states plainly that it is running as L1 with degraded performance and capabilities.

Cross-core invalidation: flip the switch, the effect follows

The criterion is deterministic rather than a “ran a long time without crashing” statistical experiment: the return of VirtualProtect means “every processor has seen the new protection”. The main thread turns a page into PAGE_NOACCESS and waits for that return; afterwards a thread pinned to a different processor reads the page. A successful read means it used a mapping that should already have been invalidated.

Run

Resident cores

Samples in window

Violations

Baseline

0

10,429,398

0

Resident (before fix)

2

180,208,570

174,787,358 (97%)

Baseline

0

10,644,062

0

Resident (after fix) ×3

2

6,434,264 / 6,952,434 / 6,782,150

0 / 0 / 0

Besides the violation count reaching zero, the sample volume falls from 180 million back to 6–7 million — the same order as the baseline — because reads only get that slow when they actually fault. “Violations = 0” alone could just mean a broken probe; the magnitude matching too is what shows the page tables are really being walked. The same defect accounts for three previously unattributed crashes: 0x139 (0x1D), 0x0A, and 0x50.

The fix depends on VPID being disabled.

With VPID off, VM entry invalidates linear mappings associated with VPID 0000H, which is what makes the broadcast NMI cure the problem. Enable VPID for performance and that entry flushes nothing, so the fix fails silently — and it fails back into exactly the 97% above, with no error anywhere. The two changes have to be made together.

Why a VBS-enabled physical machine is a dead end

This is not a policy question but a hard fact. Enabling memory integrity or Credential Guard brings up Hyper-V; Hyper-V owns VMX root and hides CPUID.1:ECX[5] from the root partition. So even “this machine has VT-x” cannot be detected, and PREPARE fails before reaching any HYPERVISOR_PRESENT check at all.

The crux is that the root partition cannot get nested VMX. Hyper-V does support exposing virtualization extensions to a guest (Set-VMProcessor -ExposeVirtualizationExtensions), but that is for its guest partitions; the root partition is not among them — and the KswordARK driver runs in the root partition.

That leaves two options, both requiring a reboot and both the user's own decision: turn off memory integrity and set bcdedit /set hypervisorlaunchtype off to run locally, at the cost of HVCI and Credential Guard; or keep VBS and run inside a child partition with ExposeVirtualizationExtensions enabled. The product stays fail-closed and reports the reason honestly: NestedNotAllowed and HypervisorConflict are separate states, because the user can resolve the first one themselves.

What it does not do: try to preempt VMX root when VBS is detected, or guess whether VMX really exists behind the CPUID mask. Both would gamble the user's machine on an assumption we cannot verify.

Boundaries

Resident HVM is an experimental backend intended only for authorized lab and diagnostic environments. Save your work before starting it: faults in VMCS, EPT, firmware, Hyper-V/VBS, nested virtualization, or the processor implementation can destabilize or bugcheck the system and may require a restart.

Stealth hooks, split views, and R-1 process disposition are not security boundaries: they fail open, and a target that can move its code page to a different guest physical page is no longer on the page being refused. This is a disposition path outside R0, not a defence against an adversary who knows it exists. The readings above come from a nested target, and several bare-metal acceptance items in the roadmap remain undone.