nt-lite/readme.md

NT-Lite -- an x86-64 kernel

“Not only is UNIX dead, it’s starting to smell really bad.” - Rob Pike

“UNIX has three operations: read, write, and lie through ioctl.”

what is it?

NT-Lite is a 64-bit operating system kernel written from scratch. It has a UEFI bootloader, physical and virtual memory managers, a preemptive scheduler, a HAL, a VFS, a PE loader, dynamic drivers, partial Win32 and POSIX personalities, and enough asynchronous I/O machinery to make the kernel feel a little less like a toy and a little more like a tiny monster.

It boots into a shell, launches user processes, survives application crashes, and can run things it probably should not be able to run yet.

The codebase is still small enough to understand, but large enough to hurt your feelings. The kernel is just 7k lines of code and the entire project is about 15k lines.

features

• UEFI bootloader
• higher-half x86-64 kernel
• physical memory manager
• virtual memory manager with kernel pool and user heap
• single address-space design with user/supervisor page permissions
• HAL for screen, keyboard, mouse, timer, PCI, APIC, HPET, AHCI, and IDE
• preemptive priority scheduler
• processes and threads
• object handles
• events, semaphores, mutexes, queues, and alertable waits
• IRQL model: PASSIVE_LEVEL, APC_LEVEL, DISPATCH_LEVEL, and device IRQLs
• interrupt-exit scheduling instead of context-switching inside hard IRQ handlers
• DPC queue for interrupt bottom halves
• kernel APCs
• user APCs through a real trap-frame return dispatcher
• async file I/O with completion events, APC callbacks, IOSBs, and cancellation
• VFS with IRPs
• centralized IoCompleteRequest()
• DRIVER_OBJECT / DEVICE_OBJECT driver model
• IoCallDriver() dispatch path
• AHCI driver, interrupt-driven
• IDE driver, PIO/DMA
• FAT32 filesystem driver
• VFAT long filename support for real dotfiles like .doomrc
• drive-number paths such as 0:/apps/shell.exe
• INI-based boot and driver configuration
• PE import resolution and dynamic .sys driver loading
• partial Win32 personality
• partial (stub/microsoft-level) POSIX personality
• self-tests for APCs, async disk I/O, async file I/O, trap frames, and cancellation
• Doom, because priorities matter

architecture notes

memory layout

Single address space for the whole system. No separate kernel/user page tables. Isolation comes from compilers, page permissions and privilege levels, not from switching CR3.

Kernel lives in the higher half. User processes only see what they are explicitly given. Kernel pages are supervisor-only so user-mode code cannot touch them. User pages are isolated from each other through the allocator and process creation path, not by hardware address-space separation. One rogue pointer can't crash the kernel but it might corrupt other processes. But what are the chances?

Is this how a serious modern OS should isolate processes? No.
Is it very convenient while building? Absolutely.

execution model

NT-Lite has a Windows-flavored execution model, but a simplified one.

PASSIVE_LEVEL   normal thread code; may wait, allocate, touch filesystems
APC_LEVEL       APC delivery / guarded regions
DISPATCH_LEVEL  scheduler, DPCs, spinlocks; must not block
DIRQL           hardware IRQ handlers; acknowledge device, queue bottom half, leave

The timer interrupt don't context-switch directly inside the IRQ handler. Interrupts unwind through a common exit path, which can drain DPCs, deliver APCs when legal, and perform deferred scheduling.

DPCs and APCs

Hard IRQ handlers stay small. They acknowledge hardware, capture status, and queue DPCs.

DPCs run at DISPATCH_LEVEL and do the bottom-half work: input callbacks, disk completion wakeups, and other things that should not happen at device IRQL.

Kernel APCs run later at PASSIVE_LEVEL in the target thread. User APCs are delivered by editing the saved user trap frame and returning through a registered user-mode dispatcher stub in syscall.dll.

The important bit:

interrupt -> DPC -> IoCompleteRequest -> APC -> alertable wait/user callback

I/O and filesystems

Everything goes through the VFS. Disk access, filesystem parsing, path resolution, and process personalities are separate layers instead of one giant bowl of sadness.

Disk drivers expose devices. The I/O manager calls drivers through IoCallDriver(). Completion goes through IoCompleteRequest().

Async file I/O exists. We are somewhere between right and dangerous.

Async requests can complete by:

• signaling an event
• writing an IO_STATUS_BLOCK
• queuing a kernel APC
• queuing a user APC
• being cancelled

drivers

PE32+ .sys files. The entry point, preferably named:

DriverEntry(HAL_API* hal)

Two loading phases:

boot drivers     loaded before the kernel does any real work
runtime drivers  loaded later, once storage and VFS are up

Disk drivers register real device objects, and I/O goes through the driver object/device object dispatch path.

Currently shipping:

AHCI   interrupt-driven SATA
IDE    PIO/DMA
FAT32  filesystem driver with VFAT and all

processes, threads, and syscalls

PE images get loaded, imports resolved, then the first thread is queued. syscall.dll is the shared syscall entry point, and both personalities go through it.

Scheduler is preemptive and priority-based. Process state and thread state are tracked separately, so a process can die or get reaped without taking the whole system down. init can restart things if needed.

Syscalls build real trap frames. That matters because user APC delivery needs to safely redirect the return path back into user mode and then restore the interrupted context with SysApcReturn.

personalities

The Win32 personality supports programs that expect drive letters, familiar process APIs, and a Windows-style runtime.

The POSIX personality maps the same kernel primitives into a tree-shaped namespace and a different userspace contract.

Binary compatibility is limited and a non-goal.

async I/O example

User-mode async I/O looks roughly like this:

SYS_IO_STATUS_BLOCK iosb;
uint32_t done;
SysEventCreate(0, 0, &done);

SYS_ASYNC_FILE_REQUEST req = {
    .FileHandle = file,
    .EventHandle = done,
    .Offset = 0,
    .Buffer = buffer,
    .Size = 512,
    .Iosb = &iosb,
    .RequestHandleOut = 0,
    .ApcRoutine = MyCompletionRoutine,
    .ApcContext = &something,
};

uint32_t s = SysReadFileAsync(&req);
if (s == SYS_STATUS_PENDING)
    s = SysEventWait(done, 1); // alertable

If the request completes by user APC, the alertable wait returns SYS_STATUS_USER_APC, the callback runs in user mode, and the original execution continues afterward.

paths

"apps/shell.exe"      -> current working directory
"0:/apps/shell.exe"   -> volume 0
"2:/data/config.ini"  -> volume 2
"C:\apps\doom.exe"    -> volume 0, mapping is simple c->0, d->1, etc.
"/apps/video.exe"     -> volume 0
"/1/apps/video.exe"   -> volume 1

Volumes 0-7 are available. A volume only works if a filesystem is registered on it (duh).
The Win32 personality translates drive letters to volume numbers, while the POSIX personality maps volumes onto a tree.

how to build

• open the VS x64 build tools as administrator
• change into this directory
• run build.bat
• use one of:
  • run.bat — IDE
  • rund.bat — IDE, debug
  • runa.bat — AHCI
  • runad.bat — AHCI, debug

how it looks when built

efi/boot/
  bootx64.efi
boot.ini
drivers/
  drivers.ini
  ahci.sys
  fat32.sys
  ide.sys
system/
  colonel.exe
  hal.dll
  init.exe
  syscall.dll
  win32.dll
  posix.dll
apps/
  shell_native.exe
  shell_win32.exe
  shell_posix.exe
  async_apc.exe
  async_cancel.exe
  doom.exe
  video.exe

screenshots

TODO

todo

faq

Q: Why?
A: Learning. Fun. Spite.

Q: Does it run Doom?
A: doom.exe

Q: Can you read and write files?
A: As far as I'm aware.

Q: Does it have async I/O?
A: You didn't read the features list?

Q: Is the Win32 layer real?
A: Depends how you frame it. Some apps will run, most will not.

Q: Are processes isolated?
A: No. But IPC is free.

Q: How often does it crash?
A: Less often than before.

Q: Can it recover from a crash?
A: Yes. Unless the kernel faulted.

Q: Is this finished?
A: No. And it never will be.

Q: Will it corrupt my data?
A: It shouldn't.

Q: Why is the path syntax so weird?
A: It makes sense to me.

Q: Why are the logs so verbose?
A: How else do I debug it?

Q: What should I read first?
A: This document.

Q: Can I use this in production?
A: Don't.

Project dedicated to the honorable, esteemed Professor Leszek Wójcik through whom I came to hate Unix.