349 lines
12 KiB
Markdown
349 lines
12 KiB
Markdown
# NT-Lite -- an x86-64 kernel
|
||
|
||
“Not only is UNIX dead, it’s starting to smell really bad.” - Rob Pike
|
||
|
||
“UNIX has three operations: read, write, and lie through ioctl.”
|
||
|
||
## what is it?
|
||
|
||
NT-Lite is a 64-bit operating system kernel written from scratch. It has a UEFI bootloader, physical and virtual memory managers, a preemptive scheduler, a HAL, a VFS, a PE loader, dynamic drivers, partial Win32 and POSIX personalities, and enough asynchronous I/O machinery to make the kernel feel a little less like a toy and a little more like a tiny monster.
|
||
|
||
It boots into a shell, launches user processes, survives application crashes, and can run things it probably should not be able to run yet.
|
||
|
||
The codebase is still small enough to understand, but large enough to hurt your feelings.
|
||
|
||
## features
|
||
|
||
- UEFI bootloader
|
||
- higher-half x86-64 kernel
|
||
- real SMP: hardware AP startup, per-CPU state, scheduler-owned idle threads, reschedule IPIs, and TLB shootdowns
|
||
- physical memory manager
|
||
- virtual memory manager with kernel pool and user heap
|
||
- W^X-style user page protections
|
||
- single address-space design with user/supervisor page permissions
|
||
- HAL for screen, keyboard, mouse, timer, PCI, APIC, HPET, AHCI, and IDE
|
||
- preemptive priority scheduler accounting for ticks, context switches, idle selections, demotions, and aging promotions
|
||
- processes and threads
|
||
- process termination state tracking and exit signaling
|
||
- object handles with per-handle rights metadata
|
||
- events, semaphores, mutexes, queues, and alertable waits
|
||
- IRQL model: `PASSIVE_LEVEL`, `APC_LEVEL`, `DISPATCH_LEVEL`, and device IRQLs
|
||
- interrupt-exit scheduling instead of context-switching inside hard IRQ handlers
|
||
- DPC queue for interrupt bottom halves
|
||
- kernel APCs
|
||
- user APCs through a real trap-frame return dispatcher
|
||
- async file I/O with completion events, APC callbacks, IOSBs, and cancellation
|
||
- VFS with IRPs
|
||
- centralized `IoCompleteRequest()`
|
||
- `DRIVER_OBJECT` / `DEVICE_OBJECT` driver model
|
||
- `IoCallDriver()` dispatch path
|
||
- AHCI driver, interrupt-driven
|
||
- IDE driver, PIO/DMA
|
||
- FAT32 filesystem driver with VFAT
|
||
- USB with bus, xHCI, mass storage and HID
|
||
- drive-number paths such as `0:/apps/shell.exe`
|
||
- lwIP network stack as a runtime driver (`lwip.sys`)
|
||
- RTL8139 and RTL816x ethernet drivers
|
||
- native BSD-style socket syscalls: `SysSocket`, `SysBind`, `SysConnect`, `SysSend`, `SysRecv`, `SysPoll`, `SysResolveHost`, and friends
|
||
- length-tracked string-view syscall ABI with request structs for path/name/text arguments
|
||
- printf/debug formatting support for bounded string views via `%v`
|
||
- INI-based configuration
|
||
- PE import resolution and dynamic `.sys` driver loading
|
||
- partial Win32 personality
|
||
- partial (stub/microsoft-level) POSIX personality
|
||
- self-tests for APCs, async disk I/O, async file I/O, trap frames, and cancellation
|
||
- fully GDPR-compliant
|
||
|
||
## architecture notes
|
||
|
||
### memory layout
|
||
|
||
Single address space for the whole system. No separate kernel/user page tables. Isolation comes from compilers, page permissions and privilege levels, not from switching CR3.
|
||
|
||
Kernel lives in the higher half. User processes only see what they are explicitly given. Kernel pages are supervisor-only so user-mode code cannot touch them. User pages are isolated from each other through the allocator and process creation path, not by hardware address-space separation. One rogue pointer can't crash the kernel but it *might* corrupt other processes. But what are the chances?
|
||
|
||
User heap and stack mappings are non-executable, and the PE loader applies final section protections after loading. Writable sections are left writable and non-executable; executable sections are mapped executable without write access where possible.
|
||
|
||
### execution model
|
||
|
||
NT-Lite has a Windows-flavored execution model, but a simplified one.
|
||
|
||
```text
|
||
PASSIVE_LEVEL normal thread code; may wait, allocate, touch filesystems
|
||
APC_LEVEL APC delivery / guarded regions
|
||
DISPATCH_LEVEL scheduler, DPCs, spinlocks; must not block
|
||
DIRQL hardware IRQ handlers; acknowledge device, queue bottom half, leave
|
||
```
|
||
|
||
The timer interrupt don't context-switch directly inside the IRQ handler. Interrupts unwind through a common exit path, which can drain DPCs, deliver APCs when legal, and perform deferred scheduling.
|
||
|
||
### SMP
|
||
|
||
Once an AP reaches the kernel, it gets its own `KCPU`, kernel stack, GDT, TSS, GS base, idle thread, and scheduler state. The BSP is not special because the code wants it to be; it is special because hardware, firmware, and device interrupts keep making it special.
|
||
|
||
External device interrupts are routed to the BSP. CPU-local things such as exceptions, NMIs, syscalls, reschedule IPIs, and TLB shootdowns happen on the CPU that receives them.
|
||
|
||
TLB shootdowns use NMI delivery because fixed-vector IPIs are not good enough when another CPU may be spinning with interrupts masked. Unexpected NMIs still panic; shootdown NMIs return only while a shootdown is active.
|
||
|
||
### DPCs and APCs
|
||
|
||
Hard IRQ handlers stay small. They acknowledge hardware, capture status, and queue DPCs.
|
||
|
||
DPCs run at `DISPATCH_LEVEL` and do the bottom-half work: input callbacks, disk completion wakeups, and other things that should not happen at device IRQL.
|
||
|
||
Kernel APCs run later at `PASSIVE_LEVEL` in the target thread. User APCs are delivered by editing the saved user trap frame and returning through a registered user-mode dispatcher stub in `syscall.dll`.
|
||
|
||
The important bit:
|
||
|
||
```text
|
||
interrupt -> DPC -> IoCompleteRequest -> APC -> alertable wait/user callback
|
||
```
|
||
|
||
### I/O and filesystems
|
||
|
||
Everything goes through the VFS. Disk access, filesystem parsing, path resolution, and process personalities are separate layers instead of one giant bowl of sadness.
|
||
|
||
Disk drivers expose devices. The I/O manager calls drivers through `IoCallDriver()`. Completion goes through `IoCompleteRequest()`.
|
||
|
||
Async file I/O exists. We are somewhere between right and dangerous.
|
||
|
||
Async requests can complete by:
|
||
|
||
- signaling an event
|
||
- writing an `IO_STATUS_BLOCK`
|
||
- queuing a kernel APC
|
||
- queuing a user APC
|
||
- being cancelled
|
||
|
||
### drivers
|
||
|
||
PE32+ `.sys` files. The entry point, preferably named:
|
||
|
||
```c
|
||
DriverEntry(HAL_API* hal)
|
||
```
|
||
|
||
Two loading phases:
|
||
|
||
```text
|
||
boot drivers loaded before the kernel does any real work
|
||
runtime drivers loaded later, once storage and VFS are up
|
||
```
|
||
|
||
Disk drivers register real device objects, and I/O goes through the driver object/device object dispatch path.
|
||
|
||
Currently shipping:
|
||
|
||
```text
|
||
AHCI interrupt-driven SATA
|
||
IDE PIO/DMA
|
||
FAT32 filesystem driver with VFAT and all
|
||
lwIP TCP/IP network stack
|
||
```
|
||
|
||
### processes, threads, and syscalls
|
||
|
||
PE images get loaded, imports resolved, then the first thread is queued. `syscall.dll` is the shared syscall entry point, and both personalities go through it.
|
||
|
||
String and buffer-heavy syscalls use request structs. Text arguments are passed as bounded `SYS_STRING_VIEW` values, then copied and validated by the kernel before use.
|
||
|
||
Scheduler is preemptive and priority-based. Process state and thread state are tracked separately, so a process can die or get reaped without taking the whole system down. `init` can restart things if needed. Process objects track termination and exit state separately from individual thread lifetime.
|
||
|
||
Handles carry rights metadata. Syscalls check both object type and required access for operations such as read, write, wait, signal, query, duplicate, and terminate.
|
||
|
||
Syscalls build real trap frames. That matters because user APC delivery needs to safely redirect the return path back into user mode and then restore the interrupted context with `SysApcReturn`.
|
||
|
||
### personalities
|
||
|
||
The Win32 personality supports programs that expect drive letters, familiar process APIs, and a Windows-style runtime.
|
||
|
||
The POSIX personality maps the same kernel primitives into a tree-shaped namespace and a different userspace contract.
|
||
|
||
Binary compatibility is limited and a non-goal.
|
||
|
||
## async I/O example
|
||
|
||
User-mode async I/O looks roughly like this:
|
||
|
||
```c
|
||
SYS_IO_STATUS_BLOCK iosb;
|
||
uint32_t done;
|
||
SysEventCreate(0, 0, &done);
|
||
|
||
SYS_ASYNC_FILE_REQUEST req = {
|
||
.FileHandle = file,
|
||
.EventHandle = done,
|
||
.Offset = 0,
|
||
.Buffer = buffer,
|
||
.Size = 512,
|
||
.Iosb = &iosb,
|
||
.RequestHandleOut = 0,
|
||
.ApcRoutine = MyCompletionRoutine,
|
||
.ApcContext = &something,
|
||
};
|
||
|
||
uint32_t s = SysReadFileAsync(&req);
|
||
if (s == SYS_STATUS_PENDING)
|
||
s = SysEventWait(done, 1); // alertable
|
||
```
|
||
|
||
If the request completes by user APC, the alertable wait returns `SYS_STATUS_USER_APC`, the callback runs in user mode, and the original execution continues afterward.
|
||
|
||
## networking
|
||
|
||
NT-Lite now has a TCP/IP stack via lwIP running as a runtime driver. The kernel exposes a native socket API through syscalls:
|
||
|
||
- `SysSocket` / `SysBind` / `SysListen` / `SysAccept` / `SysConnect`
|
||
- `SysSend` / `SysRecv` / `SysSendTo` / `SysRecvFrom`
|
||
- `SysShutdown` / `SysPoll`
|
||
- `SysResolveHost` for DNS
|
||
- `SysSetSocketOption` / `SysGetSocketOption`
|
||
|
||
Test apps ship with the build: `httpget.exe`, `tcpserve.exe`, `udp.exe`, `resolve.exe`, and `ping.exe`.
|
||
|
||
Port forwarding is enabled by default in `runa.bat` — host port 8080 maps to guest port 8080.
|
||
|
||
## paths
|
||
|
||
```text
|
||
"apps/shell.exe" -> current working directory
|
||
"0:/apps/shell.exe" -> volume 0, native path
|
||
"2:/data/config.ini" -> volume 2, native path
|
||
"C:\apps\doom.exe" -> volume 0, win32.dll translated path, mapping is simple c->0, d->1, etc.
|
||
"/apps/video.exe" -> volume 0, posix.dll translated path, mapping is simple: root is volume 0, then directories as volumes, so
|
||
"/1/apps/video.exe" -> volume 1
|
||
```
|
||
|
||
Volumes `0`-`7` are available. A volume only works if a filesystem is registered on it (duh).
|
||
The Win32 personality translates drive letters to volume numbers, while the POSIX personality maps volumes onto a tree.
|
||
|
||
Path parsing is length-aware internally. Embedded NUL bytes are rejected for text paths and names.
|
||
|
||
## how to build
|
||
|
||
- open the VS x64 build tools as administrator
|
||
- change into this directory
|
||
- run `build.bat`
|
||
- use one of:
|
||
- `run.bat` — IDE
|
||
- `rund.bat` — IDE, debug
|
||
- `runa.bat` — AHCI
|
||
- `runad.bat` — AHCI, debug
|
||
|
||
## how it looks when built
|
||
|
||
```text
|
||
efi/boot/
|
||
bootx64.efi
|
||
boot.ini
|
||
drivers/
|
||
drivers.ini
|
||
ahci.sys
|
||
fat32.sys
|
||
ide.sys
|
||
lwip.sys
|
||
rtl8139.sys
|
||
system/
|
||
colonel.exe
|
||
hal.dll
|
||
init.exe
|
||
syscall.dll
|
||
win32.dll
|
||
posix.dll
|
||
apps/
|
||
shell_native.exe
|
||
shell_win32.exe
|
||
shell_posix.exe
|
||
async_apc.exe
|
||
async_cancel.exe
|
||
doom.exe
|
||
video.exe
|
||
httpget.exe
|
||
tcpserve.exe
|
||
udp.exe
|
||
resolve.exe
|
||
ping.exe
|
||
```
|
||
|
||
## screenshots
|
||
|
||
|
||
|
||
## TODO
|
||
|
||
todo
|
||
|
||
## faq
|
||
|
||
Q: Why?
|
||
A: Reasons.
|
||
|
||
Q: Is it secure?
|
||
A: Don't connect to the internet with it.
|
||
|
||
Q: Who is this for?
|
||
A: Me.
|
||
|
||
Q: What's next?
|
||
A: todo
|
||
|
||
Q: Does it run Doom?
|
||
A: `doom.exe`
|
||
|
||
Q: Does USB work?
|
||
A: For me, my keyboard, mouse and thumbdrive.
|
||
|
||
Q: If I write a program now, will it run in the future?
|
||
A: Not a chance. WE DO - in fact - BREAK USERSPACE.
|
||
|
||
Q: Can you read and write files?
|
||
A: As far as I'm aware.
|
||
|
||
Q: Does it have async I/O?
|
||
A: The best you'll get.
|
||
|
||
Q: Does the networking actually work?
|
||
A: It didn't crash yet.
|
||
|
||
Q: Does it really have SMP?
|
||
A: Watch out. It bites.
|
||
|
||
Q: Is the Win32 layer real?
|
||
A: Depends how you frame it. Some apps will run, most will not.
|
||
|
||
Q: Are processes isolated?
|
||
A: No. But IPC is free. Is this how a serious modern OS should isolate processes? No.
|
||
Are we a serious OS? No. Is it very convenient while building? Absolutely. Do I care? Nope.
|
||
|
||
Q: How often does it crash?
|
||
A: Like Windows 11.
|
||
|
||
Q: Can it recover from a crash?
|
||
A: Works on my machine.
|
||
|
||
Q: Is this finished?
|
||
A: No. And it never will be.
|
||
|
||
Q: Will it corrupt my data?
|
||
A: It shouldn't.
|
||
|
||
Q: Why is the path syntax so weird?
|
||
A: It makes sense to me.
|
||
|
||
Q: Why are the logs so verbose?
|
||
A: How else do I debug it?
|
||
|
||
Q: What should I read first?
|
||
A: This document.
|
||
|
||
Q: How do I contribute?
|
||
A: maksw@maksw.pl
|
||
|
||
Q: Where's the source?
|
||
A: Right here.
|
||
|
||
Q: Can I use this in production?
|
||
A: Don't.
|
||
|
||
#### Project dedicated to the honorable, esteemed Professor Leszek Wójcik through whom I came to hate Unix.
|