remove stale code from posix_path, redo readme
This commit is contained in:
maksw committed
1 parent
b7fcfce6f7
commit
08329516ba
2 files changed
+38
-200
No files matched your search
@@ -4,11 +4,6 @@
|
||||
#include <string.h>
|
||||
#include "posix_path.h"
|
||||
|
||||
// Implements the flip Slashes operation.
|
||||
static void FlipSlashes(char* s) {
|
||||
for (; *s; s++) if (*s == '/') *s = '\\';
|
||||
}
|
||||
|
||||
// Checks whether drive Mount.
|
||||
static int IsDriveMount(const char* p) {
|
||||
return p && p[0] == '/' && p[1] >= '0' && p[1] <= '7' &&
|
||||
@@ -30,7 +25,6 @@ const char* PosixTranslatePath(const char* in, char* out, size_t outSize) {
|
||||
if (restLen + 4 > outSize) restLen = outSize - 4;
|
||||
memcpy(out + 3, rest, restLen);
|
||||
out[3 + restLen] = '\0';
|
||||
FlipSlashes(out);
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -49,7 +43,6 @@ const char* PosixTranslatePath(const char* in, char* out, size_t outSize) {
|
||||
out[len] = '\0';
|
||||
}
|
||||
|
||||
FlipSlashes(out);
|
||||
return out;
|
||||
}
|
||||
|
||||
|
||||
@@ -6,216 +6,68 @@
|
||||
|
||||
## what is it?
|
||||
|
||||
NT-Lite is a 64-bit operating system kernel written from scratch. It has a UEFI bootloader, physical and virtual memory managers, a preemptive scheduler, a HAL, a VFS, a PE loader, dynamic drivers, partial Win32 and POSIX personalities, and enough asynchronous I/O machinery to make the kernel feel a little less like a toy and a little more like a tiny monster.
|
||||
NT-Lite is a 64-bit operating system kernel written from scratch.
|
||||
|
||||
It boots into a shell, launches user processes, survives application crashes, and can run things it probably should not be able to run yet.
|
||||
It boots into a shell, loads drivers, launches programs, survives application crashes, runs Doom, plays video, talks TCP/IP, and occasionally reminds you that it is still a hobby kernel.
|
||||
|
||||
The codebase is still small enough to understand, but large enough to hurt your feelings.
|
||||
The codebase is small enough to understand, but no longer small enough to pretend it is just a weekend project.
|
||||
|
||||
## features
|
||||
|
||||
- UEFI bootloader
|
||||
- real SMP
|
||||
- physical memory manager
|
||||
- virtual memory manager with kernel pool and user heap
|
||||
- physical and virtual memory managers
|
||||
- kernel pool and user heap
|
||||
- single address-space design with user/supervisor page permissions
|
||||
- real, separate HAL
|
||||
- HAL split from the kernel
|
||||
- preemptive priority scheduler
|
||||
- processes and threads
|
||||
- process termination state tracking and exit signaling
|
||||
- object handles with per-handle rights metadata
|
||||
- events, semaphores, mutexes, queues, and alertable waits
|
||||
- IRQL model
|
||||
- interrupt-exit scheduling
|
||||
- DPC queue for interrupt bottom halves
|
||||
- kernel APCs
|
||||
- user APCs through a real trap-frame return dispatcher
|
||||
- async file I/O with completion events, APC callbacks, IOSBs, and cancellation
|
||||
- per-process handles with rights checks
|
||||
- process exit state, reaping, and restartable init services
|
||||
- events, semaphores, mutexes, queues, named queues, and alertable waits
|
||||
- IRQL, DPCs, kernel APCs, and user APCs
|
||||
- native user-mode exception dispatch
|
||||
- Win32 vectored exception handlers
|
||||
- minimal runtime-registered SEH substrate
|
||||
- async file I/O with events, IOSBs, APC callbacks, and cancellation
|
||||
- VFS with IRPs
|
||||
- centralized `IoCompleteRequest()`
|
||||
- `IoCallDriver()` dispatch path
|
||||
- page cache with dirty writeback and invalidation
|
||||
- AHCI driver, interrupt-driven
|
||||
- IDE driver, PIO/DMA
|
||||
- FAT32 filesystem driver with VFAT LFN
|
||||
- USB with bus, xHCI, mass storage and HID
|
||||
- FAT32 filesystem driver with VFAT long names
|
||||
- USB bus, xHCI, mass storage, and HID
|
||||
- VMware SVGA display driver
|
||||
- drive-number paths such as `0:/apps/shell.exe`
|
||||
- modular networking via lwIP
|
||||
- RTL8139 and RTL816x drivers
|
||||
- BSD-style socket syscalls: `SysSocket`, `SysBind`, `SysConnect`, `SysSend`, `SysRecv`, `SysPoll`, `SysResolveHost`, and friends
|
||||
- length-tracked string-view syscall ABI with request structs for path/name/text arguments
|
||||
- printf/debug formatting support for bounded string views via `%v`
|
||||
- INI-based configuration
|
||||
- PE import resolution and dynamic `.sys` driver loading
|
||||
- modular networking through lwIP
|
||||
- RTL8139 and RTL816x network drivers
|
||||
- native BSD-ish socket syscalls
|
||||
- DNS resolver syscall
|
||||
- length-tracked string-view syscall ABI
|
||||
- request-struct based syscalls for larger calls
|
||||
- native `NTLSTATUS_*` status codes
|
||||
- INI-based configuration
|
||||
- partial Win32 personality
|
||||
- partial (stub/microsoft-level) POSIX personality
|
||||
- partial (microsoft-level) POSIX personality
|
||||
- Doom
|
||||
- video playback
|
||||
- fully GDPR-compliant
|
||||
|
||||
## architecture notes
|
||||
|
||||
### memory layout
|
||||
|
||||
Single address space for the whole system. No separate kernel/user page tables. Isolation comes from compilers, page permissions and privilege levels, not from switching CR3.
|
||||
|
||||
Kernel lives in the higher half. User processes only see what they are explicitly given. Kernel pages are supervisor-only so user-mode code cannot touch them. User pages are isolated from each other through the allocator and process creation path, not by hardware address-space separation. One rogue pointer can't crash the kernel but it *might* corrupt other processes. But what are the chances?
|
||||
|
||||
User heap and stack mappings are non-executable, and the PE loader applies final section protections after loading. Writable sections are left writable and non-executable; executable sections are mapped executable without write access where possible.
|
||||
|
||||
### execution model
|
||||
|
||||
NT-Lite has a Windows-flavored execution model, but a simplified one.
|
||||
|
||||
```text
|
||||
PASSIVE_LEVEL normal thread code; may wait, allocate, touch filesystems
|
||||
APC_LEVEL APC delivery / guarded regions
|
||||
DISPATCH_LEVEL scheduler, DPCs, spinlocks; must not block
|
||||
DIRQL hardware IRQ handlers; acknowledge device, queue bottom half, leave
|
||||
```
|
||||
|
||||
The timer interrupt don't context-switch directly inside the IRQ handler. Interrupts unwind through a common exit path, which can drain DPCs, deliver APCs when legal, and perform deferred scheduling.
|
||||
|
||||
### SMP
|
||||
|
||||
Once an AP reaches the kernel, it gets its own `KCPU`, kernel stack, GDT, TSS, GS base, idle thread, and scheduler state. The BSP is not special because the code wants it to be; it is special because hardware, firmware, and device interrupts keep making it special.
|
||||
|
||||
External device interrupts are routed to the BSP. CPU-local things such as exceptions, NMIs, syscalls, reschedule IPIs, and TLB shootdowns happen on the CPU that receives them.
|
||||
|
||||
TLB shootdowns use NMI delivery because fixed-vector IPIs are not good enough when another CPU may be spinning with interrupts masked. Unexpected NMIs still panic; shootdown NMIs return only while a shootdown is active.
|
||||
|
||||
### DPCs and APCs
|
||||
|
||||
Hard IRQ handlers stay small. They acknowledge hardware, capture status, and queue DPCs.
|
||||
|
||||
DPCs run at `DISPATCH_LEVEL` and do the bottom-half work: input callbacks, disk completion wakeups, and other things that should not happen at device IRQL.
|
||||
|
||||
Kernel APCs run later at `PASSIVE_LEVEL` in the target thread. User APCs are delivered by editing the saved user trap frame and returning through a registered user-mode dispatcher stub in `syscall.dll`.
|
||||
|
||||
The important bit:
|
||||
|
||||
```text
|
||||
interrupt -> DPC -> IoCompleteRequest -> APC -> alertable wait/user callback
|
||||
```
|
||||
|
||||
### I/O and filesystems
|
||||
|
||||
Everything goes through the VFS. Disk access, filesystem parsing, path resolution, and process personalities are separate layers instead of one giant bowl of sadness.
|
||||
|
||||
Disk drivers expose devices. The I/O manager calls drivers through `IoCallDriver()`. Completion goes through `IoCompleteRequest()`.
|
||||
|
||||
Async file I/O exists. We are somewhere between right and dangerous.
|
||||
|
||||
Async requests can complete by:
|
||||
|
||||
- signaling an event
|
||||
- writing an `IO_STATUS_BLOCK`
|
||||
- queuing a kernel APC
|
||||
- queuing a user APC
|
||||
- being cancelled
|
||||
|
||||
### drivers
|
||||
|
||||
PE32+ `.sys` files. The entry point, preferably named:
|
||||
|
||||
```c
|
||||
DriverEntry(HAL_API* hal)
|
||||
```
|
||||
|
||||
Two loading phases:
|
||||
|
||||
```text
|
||||
boot drivers loaded before the kernel does any real work
|
||||
runtime drivers loaded later, once storage and VFS are up
|
||||
```
|
||||
|
||||
Disk drivers register real device objects, and I/O goes through the driver object/device object dispatch path.
|
||||
|
||||
Currently shipping:
|
||||
|
||||
```text
|
||||
AHCI interrupt-driven SATA
|
||||
IDE PIO/DMA
|
||||
FAT32 filesystem driver with VFAT and all
|
||||
lwIP TCP/IP network stack
|
||||
```
|
||||
|
||||
### processes, threads, and syscalls
|
||||
|
||||
PE images get loaded, imports resolved, then the first thread is queued. `syscall.dll` is the shared syscall entry point, and both personalities go through it.
|
||||
|
||||
String and buffer-heavy syscalls use request structs. Text arguments are passed as bounded `SYS_STRING_VIEW` values, then copied and validated by the kernel before use.
|
||||
|
||||
Scheduler is preemptive and priority-based. Process state and thread state are tracked separately, so a process can die or get reaped without taking the whole system down. `init` can restart things if needed. Process objects track termination and exit state separately from individual thread lifetime.
|
||||
|
||||
Handles carry rights metadata. Syscalls check both object type and required access for operations such as read, write, wait, signal, query, duplicate, and terminate.
|
||||
|
||||
Syscalls build real trap frames. That matters because user APC delivery needs to safely redirect the return path back into user mode and then restore the interrupted context with `SysApcReturn`.
|
||||
|
||||
### personalities
|
||||
|
||||
The Win32 personality supports programs that expect drive letters, familiar process APIs, and a Windows-style runtime.
|
||||
|
||||
The POSIX personality maps the same kernel primitives into a tree-shaped namespace and a different userspace contract.
|
||||
|
||||
Binary compatibility is limited and a non-goal.
|
||||
|
||||
## async I/O example
|
||||
|
||||
User-mode async I/O looks roughly like this:
|
||||
|
||||
```c
|
||||
SYS_IO_STATUS_BLOCK iosb;
|
||||
uint32_t done;
|
||||
SysEventCreate(0, 0, &done);
|
||||
|
||||
SYS_ASYNC_FILE_REQUEST req = {
|
||||
.FileHandle = file,
|
||||
.EventHandle = done,
|
||||
.Offset = 0,
|
||||
.Buffer = buffer,
|
||||
.Size = 512,
|
||||
.Iosb = &iosb,
|
||||
.RequestHandleOut = 0,
|
||||
.ApcRoutine = MyCompletionRoutine,
|
||||
.ApcContext = &something,
|
||||
};
|
||||
|
||||
uint32_t s = SysReadFileAsync(&req);
|
||||
if (s == SYS_STATUS_PENDING)
|
||||
s = SysEventWait(done, 1); // alertable
|
||||
```
|
||||
|
||||
If the request completes by user APC, the alertable wait returns `SYS_STATUS_USER_APC`, the callback runs in user mode, and the original execution continues afterward.
|
||||
|
||||
## networking
|
||||
|
||||
NT-Lite now has a TCP/IP stack via lwIP running as a runtime driver. The kernel exposes a native socket API through syscalls:
|
||||
|
||||
- `SysSocket` / `SysBind` / `SysListen` / `SysAccept` / `SysConnect`
|
||||
- `SysSend` / `SysRecv` / `SysSendTo` / `SysRecvFrom`
|
||||
- `SysShutdown` / `SysPoll`
|
||||
- `SysResolveHost` for DNS (wow!)
|
||||
- `SysSetSocketOption` / `SysGetSocketOption`
|
||||
|
||||
Host byte order. It's convenient. Cry some more.
|
||||
|
||||
Test apps ship with the build: `httpget.exe`, `tcpserve.exe`, `udp.exe`, `resolve.exe`, and `ping.exe`.
|
||||
|
||||
Port forwarding is enabled by default in `runa.bat` — host port 8080 maps to guest port 8080.
|
||||
- everything but the kitchen sink
|
||||
|
||||
## paths
|
||||
|
||||
```text
|
||||
"apps/shell.exe" -> current working directory
|
||||
"0:/apps/shell.exe" -> volume 0, native path. works everywhere.
|
||||
"0:/apps/shell.exe" -> volume 0, native path; works everywhere
|
||||
"2:/data/config.ini" -> volume 2, native path
|
||||
"C:\apps\doom.exe" -> volume 0, win32.dll translated path, mapping is simple c->0, d->1, etc. works only for win32 calls.
|
||||
"/apps/video.exe" -> volume 0, posix.dll translated path, mapping is simple: root is volume 0, then directories as volumes, so
|
||||
"/1/apps/video.exe" -> volume 1. works only for posix calls.
|
||||
"C:\apps\doom.exe" -> volume 0 through win32.dll; C: maps to 0:, D: maps to 1:, etc.
|
||||
"/apps/video.exe" -> volume 0 through posix.dll
|
||||
"/1/apps/video.exe" -> volume 1 through posix.dll
|
||||
```
|
||||
|
||||
Volumes `0`-`7` are available. A volume only works if a filesystem is registered on it.
|
||||
The Win32 personality translates drive letters to volume numbers, while the POSIX personality maps volumes onto a tree.
|
||||
|
||||
Path parsing is length-aware internally. Embedded NUL bytes are rejected for text paths and names.
|
||||
The Win32 personality translates drive letters to volume numbers. The POSIX personality maps volumes into a tree.
|
||||
|
||||
## how to build
|
||||
|
||||
@@ -272,10 +124,6 @@ apps/
|
||||
|
||||
|
||||
|
||||
## TODO
|
||||
|
||||
todo
|
||||
|
||||
## faq
|
||||
|
||||
Q: Is it secure?
|
||||
@@ -294,7 +142,7 @@ Q: Does USB work?
|
||||
A: It did for everything I threw at it.
|
||||
|
||||
Q: If I write a program now, will it run in the future?
|
||||
A: WE DO - in fact - BREAK USERSPACE.
|
||||
A: WE DO, in fact, BREAK USERSPACE.
|
||||
|
||||
Q: Can you read and write files?
|
||||
A: As far as I'm aware.
|
||||
@@ -322,7 +170,7 @@ Q: Can it recover from a crash?
|
||||
A: Works on my machine.
|
||||
|
||||
Q: Is this finished?
|
||||
A: No. And it never will be.
|
||||
A: It will be once I get a job.
|
||||
|
||||
Q: Will it corrupt my data?
|
||||
A: It shouldn't.
|
||||
@@ -334,7 +182,7 @@ Q: Why both Win32 and POSIX?
|
||||
A: NT could do it, why can't I?
|
||||
|
||||
Q: Why are the logs so verbose?
|
||||
A: Sure, try to debug without them.
|
||||
A: Try debugging an interrupt path with vibes.
|
||||
|
||||
Q: What should I read first?
|
||||
A: This document.
|
||||
@@ -342,9 +190,6 @@ A: This document.
|
||||
Q: How do I contribute?
|
||||
A: `maksw@maksw.pl`
|
||||
|
||||
Q: Where's the source?
|
||||
A: Right here.
|
||||
|
||||
Q: Can I use this in production?
|
||||
A: Don't.
|
||||
|
||||
|
||||
Reference in new issue
Block a user