kernel-debugging-lab.md (13513B)
1 --- 2 title: "Kernel Debugging Lab" 3 description: "Standing up a Windows kernel analysis bench: kdnet over the network to WinDBG, a minimal driver for DbgPrint output, and a raw memory image out of the VM for offline work." 4 category: dfir 5 subcategory: "Windows Internals" 6 tags: [windows, kernel, memory-forensics, lab, reverse-engineering] 7 tools: [windbg, kdnet, debugview, virtualbox, wdk, volatility] 8 difficulty: advanced 9 updated: 2026-10-04 10 upstreamName: "ired.team" 11 upstreamUrl: "https://ired.team/miscellaneous-reversing-forensics/windows-kernel-internals/configuring-kernel-debugging-environment-with-kdnet-and-windbg-preview" 12 upstreamAuthor: "Mantvydas Baranauskas" 13 upstreamLicense: none 14 upstreamRelation: derived 15 references: 16 - name: "Windows Kernel Drivers 101" 17 url: "https://ired.team/miscellaneous-reversing-forensics/windows-kernel-internals/windows-kernel-drivers-101" 18 author: "Mantvydas Baranauskas" 19 license: none 20 relation: derived 21 note: "Contributed the driver-model terminology: DRIVER_OBJECT, DriverEntry, the I/O manager and IRPs, and the KMDF/WDM distinction." 22 - name: "Compiling a Simple Kernel Driver, DbgPrint, DbgView" 23 url: "https://ired.team/miscellaneous-reversing-forensics/windows-kernel-internals/compiling-first-kernel-driver-kdprint-dbgprint-and-debugview" 24 author: "Mantvydas Baranauskas" 25 license: none 26 relation: derived 27 note: "Contributed the minimal WDM driver skeleton, the kd_default_mask value and the Debug Print Filter registry key." 28 - name: "Dump Virtual Box Memory" 29 url: "https://ired.team/miscellaneous-reversing-forensics/dump-virtual-box-memory" 30 author: "Mantvydas Baranauskas" 31 license: none 32 relation: derived 33 note: "Contributed the VBoxDbg .pgmphystofile acquisition route and the VBOX_GUI_DBG_ENABLED variable." 34 - name: "Setting up a network debugging connection — Microsoft Learn" 35 url: "https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/setting-up-a-network-debugging-connection-automatically" 36 relation: inspired 37 note: "The kdnet procedure and the supported-NIC constraint." 38 --- 39 40 ## What this is 41 42 Everything in this subcategory that reads kernel memory assumes a bench: a machine you can 43 break into with a debugger, a way to get printf-style output out of kernel code, and a way to 44 take the whole physical memory of that machine to disk for offline analysis. Three setups, 45 one afternoon, and nothing else here works without them. 46 47 The model you need in your head before starting: 48 49 | Term | What it means | 50 |---|---| 51 | Debugger | The host running WinDBG. | 52 | Debuggee | The machine being debugged. It halts when the debugger breaks in. | 53 | `DriverEntry` | A driver's entry point, equivalent to `main`. Called once at load. | 54 | `DRIVER_OBJECT` | The I/O manager's record of a loaded driver, including pointers to its standard routines such as `DriverUnload`. | 55 | IRP | I/O Request Packet. How requests reach a driver. | 56 | KMDF / WDM | The framework route and the raw route. KMDF abstracts the boilerplate; WDM talks to the OS directly and is simpler to load and unload by hand. | 57 58 A *software driver* — not bound to any hardware device — is what you want for analysis work. 59 It exists to run your code in kernel mode. 60 61 ## Prerequisites 62 63 - Two machines, or one host and one VM. Snapshot the debuggee before you start; you will 64 bugcheck it. 65 - WDK and the matching Visual Studio workload on the debugger host. `kdnet.exe` and 66 `VerifiedNICList.xml` ship in `C:\Program Files (x86)\Windows Kits\10\Debuggers\x64`. 67 - A network adapter kdnet supports, which is the usual failure point on a VM. Check 68 `VerifiedNICList.xml` before troubleshooting anything else; an Intel or Realtek emulated 69 adapter generally works where a paravirtualised one does not. 70 - Test signing enabled on the debuggee (`bcdedit /set testsigning on`), or a loader such as 71 the OSR Driver Loader, since your driver will be unsigned. 72 - Symbols configured on the debugger: `.symfix` then `.reload`. 73 74 ## Walkthrough 75 76 ### kdnet transport 77 78 On the **debuggee**, in an elevated prompt, pointing at the debugger's IP and a free port: 79 80 ```powershell 81 kdnet 192.168.2.79 50001 82 ``` 83 84 It prints the exact WinDBG command line to run on the other side, including a generated key. 85 Copy it off the machine before rebooting — it is not recoverable afterwards without rerunning 86 `kdnet`, which generates a different key. 87 88 <figure class="shot"> 89 <img src="https://raw.githubusercontent.com/mantvydasb/RedTeaming-Tactics-and-Techniques/8cdbdd60eb4a8997e689649f3911f7c893e59ed9/.gitbook/assets/image%20%28249%29.png" 90 alt="kdnet console output confirming the debug transport is configured and printing the windbg -k net command with the generated connection key" 91 loading="lazy" referrerpolicy="no-referrer"> 92 <figcaption><code>kdnet</code> prints the debugger-side command and key. 93 <span class="shot-credit">ired.team · Mantvydas Baranauskas</span> 94 </figcaption> 95 </figure> 96 97 Reboot the debuggee. On the **debugger**, either run the printed command or use WinDBG's 98 *Attach to kernel* with the same port and key: 99 100 ```powershell 101 windbg -k net:port=50001,key=<the key kdnet printed> 102 ``` 103 104 <figure class="shot"> 105 <img src="https://raw.githubusercontent.com/mantvydasb/RedTeaming-Tactics-and-Techniques/8cdbdd60eb4a8997e689649f3911f7c893e59ed9/.gitbook/assets/kerneldebuggingconnect.gif" 106 alt="WinDBG connecting to a remote kernel over the network transport and reaching an interactive kd prompt" 107 loading="lazy" referrerpolicy="no-referrer"> 108 <figcaption>The debugger reaching a <code>kd></code> prompt on the remote kernel. 109 <span class="shot-credit">ired.team · Mantvydas Baranauskas</span> 110 </figcaption> 111 </figure> 112 113 Once connected, `ctrl+break` halts the debuggee and `g` resumes it. Everything on the 114 debuggee stops while you are broken in — including its network stack, which is why debugging 115 over the same link you administer the box with is a bad idea. 116 117 ### A driver that prints 118 119 The minimum useful WDM driver is a `DriverEntry`, an unload routine, and two `DbgPrint` 120 calls. It does nothing except prove the load and unload path works, which is exactly what you 121 want before adding anything that can fault. 122 123 ```c 124 #include <ntddk.h> 125 126 void DriverUnload(PDRIVER_OBJECT driverObject) 127 { 128 UNREFERENCED_PARAMETER(driverObject); 129 DbgPrint("daemon: driver unloaded\n"); 130 } 131 132 NTSTATUS DriverEntry(PDRIVER_OBJECT DriverObject, PUNICODE_STRING RegistryPath) 133 { 134 UNREFERENCED_PARAMETER(RegistryPath); 135 DriverObject->DriverUnload = DriverUnload; 136 DbgPrint("daemon: driver loaded\n"); 137 return STATUS_SUCCESS; 138 } 139 ``` 140 141 Assigning `DriverUnload` is what makes the driver stoppable. A KMDF driver built from the 142 *Kernel Mode Driver, Empty (KMDF)* template is the framework equivalent — it needs 143 `WdfDriverCreate` with an `EvtDriverDeviceAdd` callback and a `WdfDeviceCreate` inside it — 144 but a KMDF driver will typically refuse a stop request with "the requested control is not 145 valid for this service" even with an unload routine defined, which makes plain WDM the better 146 choice for a bench driver you will load and unload fifty times. 147 148 ### Making DbgPrint output visible 149 150 `DbgPrint` output is filtered by default and silently discarded. Two places to fix that, 151 depending on where you want to read it. 152 153 For WinDBG, raise the mask in the debugger: 154 155 ```erlang 156 ed kd_default_mask 0xf 157 ``` 158 159 <figure class="shot"> 160 <img src="https://raw.githubusercontent.com/mantvydasb/RedTeaming-Tactics-and-Techniques/8cdbdd60eb4a8997e689649f3911f7c893e59ed9/.gitbook/assets/image%20%28505%29.png" 161 alt="WinDBG command setting the kd_default_mask variable to 0xf to stop debug print output being filtered" 162 loading="lazy" referrerpolicy="no-referrer"> 163 <figcaption>Raising <code>kd_default_mask</code> so <code>DbgPrint</code> output reaches the debugger. 164 <span class="shot-credit">ired.team · Mantvydas Baranauskas</span> 165 </figcaption> 166 </figure> 167 168 For DebugView on the debuggee itself — which you want when you are iterating on a driver and 169 do not want to be broken in — create the filter key and a `DEFAULT` DWORD of `0xf`: 170 171 ```text 172 HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Debug Print Filter 173 DEFAULT (DWORD) 0xf 174 ``` 175 176 Then run DebugView elevated with *Capture Kernel* enabled. 177 178 <figure class="shot"> 179 <img src="https://raw.githubusercontent.com/mantvydasb/RedTeaming-Tactics-and-Techniques/8cdbdd60eb4a8997e689649f3911f7c893e59ed9/.gitbook/assets/image%20%28508%29.png" 180 alt="DebugView window capturing kernel debug output, showing the driver loaded and driver unloaded lines from the test driver" 181 loading="lazy" referrerpolicy="no-referrer"> 182 <figcaption>DebugView capturing the driver's load and unload messages on the debuggee. 183 <span class="shot-credit">ired.team · Mantvydas Baranauskas</span> 184 </figcaption> 185 </figure> 186 187 Both together is the practical arrangement: DebugView for the fast loop, the debugger for 188 when something faults. 189 190 ### Acquiring physical memory from the VM 191 192 For offline analysis you want the debuggee's physical memory as a flat file. VirtualBox's own 193 debugger does this without an agent inside the guest, which matters — nothing is installed, 194 nothing is written to the guest filesystem, and the guest is paused rather than cooperating. 195 196 List the VMs, then start the target with the debug UI enabled: 197 198 ```bash 199 VBoxManage list vms 200 virtualbox --startvm "win1002 debugee" --dbg 201 ``` 202 203 In the VM window, *Debug* → *Command Line*, then in the `VBoxDbg>` prompt: 204 205 ```text 206 .pgmphystofile 'target-memory.bin' 207 ``` 208 209 The file lands in the directory the VirtualBox process was started from, and it is a raw 210 physical memory image — the input format Volatility and MemProcFS expect. 211 212 <figure class="shot"> 213 <img src="https://raw.githubusercontent.com/mantvydasb/RedTeaming-Tactics-and-Techniques/8cdbdd60eb4a8997e689649f3911f7c893e59ed9/.gitbook/assets/vbox-debug.png" 214 alt="VirtualBox built-in debugger console accepting the pgmphystofile command to write the guest's physical memory to a raw file" 215 loading="lazy" referrerpolicy="no-referrer"> 216 <figcaption>The VirtualBox debug console writing guest physical memory to a raw file. 217 <span class="shot-credit">ired.team · Mantvydas Baranauskas</span> 218 </figcaption> 219 </figure> 220 221 To stop re-passing `--dbg`, export the variable before launching, or put it in your shell 222 profile: 223 224 ```bash 225 export VBOX_GUI_DBG_ENABLED=true 226 ``` 227 228 ## What it tells you 229 230 **Which questions need which tool.** The three setups answer different things and the mistake 231 is reaching for the wrong one. A live kernel debugger is for structure walking and 232 breakpoints — everything in [handle enumeration](/sheets/dfir/handle-enumeration), 233 [the PEB walk](/sheets/dfir/peb-walking) and 234 [SSDT and IDT](/sheets/dfir/ssdt-and-idt) needs it. `DbgPrint` plus DebugView is for 235 instrumentation you wrote yourself, which is how you watch 236 [kernel notification callbacks](/sheets/dfir/kernel-notification-callbacks) fire. A raw memory 237 image is for everything you want to analyse without the machine present, repeatably, from a 238 fixed point in time. 239 240 **The acquisition method is part of the evidence.** `.pgmphystofile` captures with the guest 241 paused by the hypervisor and no code running inside it, so there is no smearing from the 242 acquisition itself and no agent footprint to explain. That is a stronger provenance story 243 than any in-guest acquisition tool, and it is worth preferring whenever the target is a VM 244 you control. The trade-off is that it only works on a VM, and only one you can reach the 245 hypervisor of. 246 247 **Snapshot before, image after, and keep both.** A VM snapshot plus a raw memory image of the 248 same moment is a reproducible analysis target: you can rerun a question against the image 249 without rebuilding, and resume the snapshot when you need to watch the behaviour again. Hash 250 the image on creation — nothing downstream will do it for you. 251 252 **A bugcheck is data.** The debuggee halting into the debugger with a stop code is often more 253 informative than the driver working, especially while you are learning the structures. `!analyze -v` 254 on the break, and the fact that a forgotten callback deregistration faults at the *next* 255 notification rather than at unload, are the two things that will cost you the most time 256 otherwise. 257 258 **Test signing changes the machine you are measuring.** A debuggee with test signing on and a 259 kernel debugger attached is not a representative endpoint — PatchGuard behaviour differs, and 260 code-integrity decisions differ. Fine for understanding a structure, wrong for judging 261 whether a detection fires in production. 262 263 ## References 264 265 - [Setting up a network debugging connection — Microsoft Learn](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/setting-up-a-network-debugging-connection-automatically) 266 - [Writing a very small KMDF driver — Microsoft Learn](https://learn.microsoft.com/en-us/windows-hardware/drivers/gettingstarted/writing-a-very-small-kmdf--driver) 267 - [DRIVER_OBJECT structure](https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_driver_object) 268 - [Reading and filtering debugging messages](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/reading-and-filtering-debugging-messages) 269 - [Volatility 3](/sheets/dfir/volatility) — what to do with the raw image once you have it 270 - [Disk imaging](/sheets/dfir/disk-imaging) — the disk-side counterpart to this acquisition