os-dev-scratch

OS Development from Scratch

Purpose

Guide agents through building a minimal operating system from scratch: bootloader stages (BIOS/GRUB vs UEFI/limine), 64-bit long mode setup with GDT and page tables, IDT and interrupt handlers, PIC/APIC configuration, basic keyboard and serial drivers, physical and virtual memory managers, context switching, with xv6-RISC-V as a reference architecture.

When to Use

• Learning how an OS boots from power-on to main()
• Implementing protected/long mode transitions on x86-64
• Writing a physical memory allocator (bitmap) and page table manager
• Handling timer, keyboard, and page fault interrupts
• Testing with QEMU -kernel and cross-compiler x86_64-elf-gcc
• Porting concepts from xv6 to a custom x86 or RISC-V kernel

Workflow

1. Boot stages overview

BIOS path (legacy)
├── BIOS POST
├── MBR (512 bytes) → boot sector loads stage2
├── GRUB/multiboot → loads kernel ELF
└── kernel entry (_start)

UEFI path (modern)
├── UEFI firmware
├── EFI bootloader (limine, systemd-boot)
├── Loads kernel + initrd from ESP
└── kernel entry (handoff with memory map)

2. Toolchain setup

# Cross-compiler for bare metal
brew install x86_64-elf-gcc x86_64-elf-binutils   # macOS
# or build from source / apt install gcc-x86-64-elf

x86_64-elf-gcc --version

# QEMU for testing
qemu-system-x86_64 --version

Linker script essentials:

/* linker.ld */
ENTRY(_start)
SECTIONS {
    . = 0x100000;          /* 1MB — typical kernel load address */
    .text : { *(.text .text.*) }
    .rodata : { *(.rodata .rodata.*) }
    .data : { *(.data .data.*) }
    .bss : { *(.bss .bss.*) }
}

x86_64-elf-gcc -ffreestanding -nostdlib -c kernel.c -o kernel.o
x86_64-elf-ld -T linker.ld kernel.o -o kernel.elf

3. Multiboot/limine boot

# QEMU direct kernel boot (no disk)
qemu-system-x86_64 \
  -kernel kernel.elf \
  -serial stdio \
  -m 128M \
  -no-reboot -no-shutdown

# With limine (UEFI)
qemu-system-x86_64 \
  -bios /usr/share/ovmf/OVMF.fd \
  -drive file=disk.img,format=raw \
  -serial stdio

4. Long mode setup

Protected mode (32-bit) → enable PAE → setup 4-level page tables → enable long mode

// Minimal GDT entry (64-bit flat segments)
struct gdt_entry {
    uint16_t limit_low;
    uint16_t base_low;
    uint8_t  base_mid;
    uint8_t  access;
    uint8_t  granularity;
    uint8_t  base_high;
} __attribute__((packed));

// Page table setup (4KB pages, identity map first 1GB)
uint64_t pml4[512] __attribute__((aligned(4096)));
uint64_t pdpt[512] __attribute__((aligned(4096)));
uint64_t pd[512] __attribute__((aligned(4096)));

void setup_paging(void) {
    for (int i = 0; i < 512; i++)
        pd[i] = (i * 0x200000) | 0x83;  // 2MB huge pages
    pdpt[0] = (uint64_t)pd | 0x03;
    pml4[0] = (uint64_t)pdpt | 0x03;
    __asm__ volatile("mov %0, %%cr3" :: "r"(pml4));
}

5. IDT and interrupt handlers

struct idt_entry {
    uint16_t offset_low;
    uint16_t selector;
    uint8_t  ist;
    uint8_t  type_attr;
    uint16_t offset_mid;
    uint32_t offset_high;
    uint32_t zero;
} __attribute__((packed));

// ISR stub (assembly) → common handler → dispatch by vector
void interrupt_handler(struct trap_frame *frame) {
    if (frame->vector == 14)  // page fault
        handle_page_fault(frame->cr2, frame->error_code);
    else if (frame->vector == 33)  // keyboard IRQ remapped
        keyboard_handler();
}

# Test page fault
# QEMU monitor: info registers

6. PIC and APIC

// Legacy PIC remapping (8259)
// Remap IRQ 0-15 to vectors 32-47
outb(0x20, 0x11); outb(0xA0, 0x11);
outb(0x21, 0x20); outb(0xA1, 0x28);  // vector offsets
// ...

// Modern: use APIC/IOAPIC (ACPI MADT parsing)
// LAPIC timer for preemption

7. Serial and keyboard drivers

// COM1 serial (0x3F8)
void serial_putc(char c) {
    while ((inb(0x3F8 + 5) & 0x20) == 0);
    outb(0x3F8, c);
}

// PS/2 keyboard scancode → ASCII lookup table
void keyboard_handler(void) {
    uint8_t scancode = inb(0x60);
    char c = scancode_to_ascii[scancode];
    if (c) serial_putc(c);
    outb(0x20, 0x20);  // EOI to PIC
}

qemu-system-x86_64 -kernel kernel.elf -serial stdio
# printk output appears in terminal

8. Physical memory manager

// Bitmap allocator over usable RAM regions
// From multiboot memory map or UEFI memory map
#define PAGE_SIZE 4096
uint8_t *frame_bitmap;
uint64_t total_frames;

uint64_t alloc_frame(void) {
    for (uint64_t i = 0; i < total_frames; i++) {
        if (!test_bit(frame_bitmap, i)) {
            set_bit(frame_bitmap, i);
            return i * PAGE_SIZE;
        }
    }
    return 0;  // OOM
}

9. Context switching

struct context {
    uint64_t rax, rbx, rcx, rdx, rsi, rdi, rbp, rsp;
    uint64_t r8, r9, r10, r11, r12, r13, r14, r15;
    uint64_t rip;
};

void switch_context(struct context *old, struct context *new);
// Assembly: save callee-saved regs to old, restore from new, ret to new->rip

Cooperative scheduling first; add timer IRQ preemption later.

10. xv6-RISC-V reference

git clone https://github.com/mit-pdos/xv6-riscv
cd xv6-riscv && make qemu

xv6 component	x86 equivalent
kernel/vm.c	Page table management
kernel/trap.c	IDT/interrupt dispatch
kernel/proc.c	Context switch, scheduler
kernel/plic.c	PIC/APIC interrupt controller
user/usys.pl	System call stubs

Common Problems

Symptom	Cause	Fix
Triple fault on boot	Invalid GDT/IDT or stack	Set up stack before enabling interrupts
QEMU black screen	No serial output configured	-serial stdio; early serial_init
Page fault in kernel	Unmapped address	Identity-map kernel; check CR3
IRQ never fires	PIC mask or IDT not loaded	lidt; unmask IRQ in PIC
Timer doesn't tick	LAPIC not initialized	Parse ACPI; calibrate LAPIC timer
Linker relocation error	Wrong load address	Match linker.ld with bootloader expectation

Related Skills

• skills/low-level-programming/assembly-x86 — x86-64 assembly for ISR stubs
• skills/low-level-programming/assembly-riscv — xv6-RISC-V reference ISA
• skills/platform/riscv-privileged — RISC-V trap handling and page tables
• skills/virtualization/qemu-kvm — QEMU flags for kernel development
• skills/kernel/kernel-internals — Linux implementation of these concepts
• skills/low-level-programming/linux-kernel-modules — graduate to Linux once basics work