diff --git a/code/user/personalities/posix/posix_path.c b/code/user/personalities/posix/posix_path.c index 5231f86..3fa666d 100644 --- a/code/user/personalities/posix/posix_path.c +++ b/code/user/personalities/posix/posix_path.c @@ -4,11 +4,6 @@ #include #include "posix_path.h" -// Implements the flip Slashes operation. -static void FlipSlashes(char* s) { - for (; *s; s++) if (*s == '/') *s = '\\'; -} - // Checks whether drive Mount. static int IsDriveMount(const char* p) { return p && p[0] == '/' && p[1] >= '0' && p[1] <= '7' && @@ -30,7 +25,6 @@ const char* PosixTranslatePath(const char* in, char* out, size_t outSize) { if (restLen + 4 > outSize) restLen = outSize - 4; memcpy(out + 3, rest, restLen); out[3 + restLen] = '\0'; - FlipSlashes(out); return out; } @@ -49,7 +43,6 @@ const char* PosixTranslatePath(const char* in, char* out, size_t outSize) { out[len] = '\0'; } - FlipSlashes(out); return out; } diff --git a/readme.md b/readme.md index b5ac9f6..63f3c21 100644 --- a/readme.md +++ b/readme.md @@ -6,216 +6,68 @@ ## what is it? -NT-Lite is a 64-bit operating system kernel written from scratch. It has a UEFI bootloader, physical and virtual memory managers, a preemptive scheduler, a HAL, a VFS, a PE loader, dynamic drivers, partial Win32 and POSIX personalities, and enough asynchronous I/O machinery to make the kernel feel a little less like a toy and a little more like a tiny monster. +NT-Lite is a 64-bit operating system kernel written from scratch. -It boots into a shell, launches user processes, survives application crashes, and can run things it probably should not be able to run yet. +It boots into a shell, loads drivers, launches programs, survives application crashes, runs Doom, plays video, talks TCP/IP, and occasionally reminds you that it is still a hobby kernel. -The codebase is still small enough to understand, but large enough to hurt your feelings. +The codebase is small enough to understand, but no longer small enough to pretend it is just a weekend project. ## features - UEFI bootloader - real SMP -- physical memory manager -- virtual memory manager with kernel pool and user heap +- physical and virtual memory managers +- kernel pool and user heap - single address-space design with user/supervisor page permissions -- real, separate HAL +- HAL split from the kernel - preemptive priority scheduler - processes and threads -- process termination state tracking and exit signaling -- object handles with per-handle rights metadata -- events, semaphores, mutexes, queues, and alertable waits -- IRQL model -- interrupt-exit scheduling -- DPC queue for interrupt bottom halves -- kernel APCs -- user APCs through a real trap-frame return dispatcher -- async file I/O with completion events, APC callbacks, IOSBs, and cancellation +- per-process handles with rights checks +- process exit state, reaping, and restartable init services +- events, semaphores, mutexes, queues, named queues, and alertable waits +- IRQL, DPCs, kernel APCs, and user APCs +- native user-mode exception dispatch +- Win32 vectored exception handlers +- minimal runtime-registered SEH substrate +- async file I/O with events, IOSBs, APC callbacks, and cancellation - VFS with IRPs -- centralized `IoCompleteRequest()` -- `IoCallDriver()` dispatch path +- page cache with dirty writeback and invalidation - AHCI driver, interrupt-driven - IDE driver, PIO/DMA -- FAT32 filesystem driver with VFAT LFN -- USB with bus, xHCI, mass storage and HID +- FAT32 filesystem driver with VFAT long names +- USB bus, xHCI, mass storage, and HID +- VMware SVGA display driver - drive-number paths such as `0:/apps/shell.exe` -- modular networking via lwIP -- RTL8139 and RTL816x drivers -- BSD-style socket syscalls: `SysSocket`, `SysBind`, `SysConnect`, `SysSend`, `SysRecv`, `SysPoll`, `SysResolveHost`, and friends -- length-tracked string-view syscall ABI with request structs for path/name/text arguments -- printf/debug formatting support for bounded string views via `%v` -- INI-based configuration - PE import resolution and dynamic `.sys` driver loading +- modular networking through lwIP +- RTL8139 and RTL816x network drivers +- native BSD-ish socket syscalls +- DNS resolver syscall +- length-tracked string-view syscall ABI +- request-struct based syscalls for larger calls +- native `NTLSTATUS_*` status codes +- INI-based configuration - partial Win32 personality -- partial (stub/microsoft-level) POSIX personality +- partial (microsoft-level) POSIX personality +- Doom +- video playback - fully GDPR-compliant - -## architecture notes - -### memory layout - -Single address space for the whole system. No separate kernel/user page tables. Isolation comes from compilers, page permissions and privilege levels, not from switching CR3. - -Kernel lives in the higher half. User processes only see what they are explicitly given. Kernel pages are supervisor-only so user-mode code cannot touch them. User pages are isolated from each other through the allocator and process creation path, not by hardware address-space separation. One rogue pointer can't crash the kernel but it *might* corrupt other processes. But what are the chances? - -User heap and stack mappings are non-executable, and the PE loader applies final section protections after loading. Writable sections are left writable and non-executable; executable sections are mapped executable without write access where possible. - -### execution model - -NT-Lite has a Windows-flavored execution model, but a simplified one. - -```text -PASSIVE_LEVEL normal thread code; may wait, allocate, touch filesystems -APC_LEVEL APC delivery / guarded regions -DISPATCH_LEVEL scheduler, DPCs, spinlocks; must not block -DIRQL hardware IRQ handlers; acknowledge device, queue bottom half, leave -``` - -The timer interrupt don't context-switch directly inside the IRQ handler. Interrupts unwind through a common exit path, which can drain DPCs, deliver APCs when legal, and perform deferred scheduling. - -### SMP - -Once an AP reaches the kernel, it gets its own `KCPU`, kernel stack, GDT, TSS, GS base, idle thread, and scheduler state. The BSP is not special because the code wants it to be; it is special because hardware, firmware, and device interrupts keep making it special. - -External device interrupts are routed to the BSP. CPU-local things such as exceptions, NMIs, syscalls, reschedule IPIs, and TLB shootdowns happen on the CPU that receives them. - -TLB shootdowns use NMI delivery because fixed-vector IPIs are not good enough when another CPU may be spinning with interrupts masked. Unexpected NMIs still panic; shootdown NMIs return only while a shootdown is active. - -### DPCs and APCs - -Hard IRQ handlers stay small. They acknowledge hardware, capture status, and queue DPCs. - -DPCs run at `DISPATCH_LEVEL` and do the bottom-half work: input callbacks, disk completion wakeups, and other things that should not happen at device IRQL. - -Kernel APCs run later at `PASSIVE_LEVEL` in the target thread. User APCs are delivered by editing the saved user trap frame and returning through a registered user-mode dispatcher stub in `syscall.dll`. - -The important bit: - -```text -interrupt -> DPC -> IoCompleteRequest -> APC -> alertable wait/user callback -``` - -### I/O and filesystems - -Everything goes through the VFS. Disk access, filesystem parsing, path resolution, and process personalities are separate layers instead of one giant bowl of sadness. - -Disk drivers expose devices. The I/O manager calls drivers through `IoCallDriver()`. Completion goes through `IoCompleteRequest()`. - -Async file I/O exists. We are somewhere between right and dangerous. - -Async requests can complete by: - -- signaling an event -- writing an `IO_STATUS_BLOCK` -- queuing a kernel APC -- queuing a user APC -- being cancelled - -### drivers - -PE32+ `.sys` files. The entry point, preferably named: - -```c -DriverEntry(HAL_API* hal) -``` - -Two loading phases: - -```text -boot drivers loaded before the kernel does any real work -runtime drivers loaded later, once storage and VFS are up -``` - -Disk drivers register real device objects, and I/O goes through the driver object/device object dispatch path. - -Currently shipping: - -```text -AHCI interrupt-driven SATA -IDE PIO/DMA -FAT32 filesystem driver with VFAT and all -lwIP TCP/IP network stack -``` - -### processes, threads, and syscalls - -PE images get loaded, imports resolved, then the first thread is queued. `syscall.dll` is the shared syscall entry point, and both personalities go through it. - -String and buffer-heavy syscalls use request structs. Text arguments are passed as bounded `SYS_STRING_VIEW` values, then copied and validated by the kernel before use. - -Scheduler is preemptive and priority-based. Process state and thread state are tracked separately, so a process can die or get reaped without taking the whole system down. `init` can restart things if needed. Process objects track termination and exit state separately from individual thread lifetime. - -Handles carry rights metadata. Syscalls check both object type and required access for operations such as read, write, wait, signal, query, duplicate, and terminate. - -Syscalls build real trap frames. That matters because user APC delivery needs to safely redirect the return path back into user mode and then restore the interrupted context with `SysApcReturn`. - -### personalities - -The Win32 personality supports programs that expect drive letters, familiar process APIs, and a Windows-style runtime. - -The POSIX personality maps the same kernel primitives into a tree-shaped namespace and a different userspace contract. - -Binary compatibility is limited and a non-goal. - -## async I/O example - -User-mode async I/O looks roughly like this: - -```c -SYS_IO_STATUS_BLOCK iosb; -uint32_t done; -SysEventCreate(0, 0, &done); - -SYS_ASYNC_FILE_REQUEST req = { - .FileHandle = file, - .EventHandle = done, - .Offset = 0, - .Buffer = buffer, - .Size = 512, - .Iosb = &iosb, - .RequestHandleOut = 0, - .ApcRoutine = MyCompletionRoutine, - .ApcContext = &something, -}; - -uint32_t s = SysReadFileAsync(&req); -if (s == SYS_STATUS_PENDING) - s = SysEventWait(done, 1); // alertable -``` - -If the request completes by user APC, the alertable wait returns `SYS_STATUS_USER_APC`, the callback runs in user mode, and the original execution continues afterward. - -## networking - -NT-Lite now has a TCP/IP stack via lwIP running as a runtime driver. The kernel exposes a native socket API through syscalls: - -- `SysSocket` / `SysBind` / `SysListen` / `SysAccept` / `SysConnect` -- `SysSend` / `SysRecv` / `SysSendTo` / `SysRecvFrom` -- `SysShutdown` / `SysPoll` -- `SysResolveHost` for DNS (wow!) -- `SysSetSocketOption` / `SysGetSocketOption` - -Host byte order. It's convenient. Cry some more. - -Test apps ship with the build: `httpget.exe`, `tcpserve.exe`, `udp.exe`, `resolve.exe`, and `ping.exe`. - -Port forwarding is enabled by default in `runa.bat` — host port 8080 maps to guest port 8080. +- everything but the kitchen sink ## paths ```text "apps/shell.exe" -> current working directory -"0:/apps/shell.exe" -> volume 0, native path. works everywhere. +"0:/apps/shell.exe" -> volume 0, native path; works everywhere "2:/data/config.ini" -> volume 2, native path -"C:\apps\doom.exe" -> volume 0, win32.dll translated path, mapping is simple c->0, d->1, etc. works only for win32 calls. -"/apps/video.exe" -> volume 0, posix.dll translated path, mapping is simple: root is volume 0, then directories as volumes, so -"/1/apps/video.exe" -> volume 1. works only for posix calls. +"C:\apps\doom.exe" -> volume 0 through win32.dll; C: maps to 0:, D: maps to 1:, etc. +"/apps/video.exe" -> volume 0 through posix.dll +"/1/apps/video.exe" -> volume 1 through posix.dll ``` -Volumes `0`-`7` are available. A volume only works if a filesystem is registered on it. -The Win32 personality translates drive letters to volume numbers, while the POSIX personality maps volumes onto a tree. +Volumes `0`-`7` are available. A volume only works if a filesystem is registered on it. -Path parsing is length-aware internally. Embedded NUL bytes are rejected for text paths and names. +The Win32 personality translates drive letters to volume numbers. The POSIX personality maps volumes into a tree. ## how to build @@ -272,10 +124,6 @@ apps/ ![video.exe running bink video 2](image.png) -## TODO - -todo - ## faq Q: Is it secure? @@ -294,7 +142,7 @@ Q: Does USB work? A: It did for everything I threw at it. Q: If I write a program now, will it run in the future? -A: WE DO - in fact - BREAK USERSPACE. +A: WE DO, in fact, BREAK USERSPACE. Q: Can you read and write files? A: As far as I'm aware. @@ -322,7 +170,7 @@ Q: Can it recover from a crash? A: Works on my machine. Q: Is this finished? -A: No. And it never will be. +A: It will be once I get a job. Q: Will it corrupt my data? A: It shouldn't. @@ -334,7 +182,7 @@ Q: Why both Win32 and POSIX? A: NT could do it, why can't I? Q: Why are the logs so verbose? -A: Sure, try to debug without them. +A: Try debugging an interrupt path with vibes. Q: What should I read first? A: This document. @@ -342,9 +190,6 @@ A: This document. Q: How do I contribute? A: `maksw@maksw.pl` -Q: Where's the source? -A: Right here. - Q: Can I use this in production? A: Don't.