docs/build.btft

[link rel="stylesheet" href="keyframes.css"][e]
 
[table class="topnav"]
[tr]
[td class="logotab"][see name="index"]kaboom[e][e]
[td][see name="kernel"]kernel[e][e]
[td][see name="fs"]fs[e][e]
[td][see name="syscalls"]syscalls[e][e]
[td][see name="userland"]userland[e][e]
[td][see name="build"]build[e][e]
[e]
[e]
 
[h1]build[e]
 
this page is the actual pipeline: [tt]mk.conf[e]'s shared config, then the real steps run in order -- [tt]mk/b.sh[e] (kernel), [tt]mk/bl.sh[e] (the real bios bootloader), [tt]mk/bu.sh[e] (userspace), [tt]mk/bd.sh[e] (disk image), then [tt]mk/rn.sh[e]/[tt]mk/r.sh[e] (the fast [tt]-kernel[e] dev-loop) or [tt]mk/rbn.sh[e]/[tt]mk/rb.sh[e] (the real bios/mbr boot path, no [tt]-kernel[e] flag at all -- see below) to actually run it -- and, alongside all of that, the entirely separate pipeline that builds the site you're reading right now. read [see name="kernel"]kernel[e] and [see name="userland"]userland[e] for what actually gets built; this page only covers how, and in what order.
 
[h2]mk.conf: one shared config, sourced by everything[e]
 
every script under [tt]mk/[e] starts with [tt]. ./mk.conf[e], which means every one of them has to be run from the repo root -- a relative source path, on purpose, so nothing here assumes an install location. it sets [tt]src[e]/[tt]out[e] (plain [tt]src[e] and [tt]out[e], always), the three toolchain binaries ([tt]nscc[e], [tt]as[e], [tt]ld[e], each overridable via the [tt]NSCC[e]/[tt]AS[e]/[tt]LD[e] environment variables rather than hardcoded), and the flag sets every compile/link actually uses.
 
[tt]asflgs[e] is [tt]--64 --fatal-warnings -moperand-check=error -msse-check=error[e]: fatal-warnings in the same spirit as tape-kernel's [tt]-Werror[e], [tt]-moperand-check=error[e] upgrades gas's default warning for ambiguous mixed-width operand encodings to a hard failure (real hand-written 32/64-bit-mixed asm -- boot.s's mode transition, every isr stub -- is exactly where a silent wrong-width encoding is both easy to write and nasty to debug on real hardware), and [tt]-msse-check=error[e] because nothing here ever saves or restores fpu/sse state, so a stray sse instruction from a compiler bug or a typo fails the build instead of quietly corrupting whatever ran before it.
 
[tt]ldflgs[e] is [tt]--fatal-warnings --error-execstack -z noexecstack --build-id=none[e]: fatal-warnings again catches [tt]--warn-rwx-segments[e] (a regression back to one merged rwx load segment fails outright, not a line to scroll past) and [tt]--warn-common[e] (harmless here, [tt].comm[e] is deliberate); [tt]--error-execstack[e]/[tt]-z noexecstack[e] make an executable stack a hard build failure even though nothing here actually reads that marking during a direct qemu [tt]-kernel[e]/pvh boot the way a real os loader would; [tt]--build-id=none[e] skips the [tt].note.gnu.build-id[e] section entirely, pure overhead for a from-scratch image that nothing ever consumes. [tt]ldscript[e] points at [tt]linker.ld[e], and [tt]qemu[e]/[tt]qemuflgs[e] ([tt]qemu-system-x86_64[e], [tt]-m 128[e]) are shared by both run scripts below.
 
[h2]mk/b.sh: the kernel[e]
 
wipes and recreates [tt]$out[e], then runs every [tt].nsc[e] source under [tt]src/kernel[e], [tt]src/fs[e], and [tt]src/drivers[e] through [tt]$nscc[e] into an intermediate [tt].s[e] in [tt]$out[e], assembles every hand-written [tt].s[e] ([tt]boot.s[e], [tt]io.s[e], and each subsystem's own [tt]_asm.s[e] counterpart) straight to [tt].o[e], assembles each nscc-produced [tt].s[e] to [tt].o[e] the same way, and links the whole pile with [tt]$ld $ldflgs -T linker.ld[e] into [tt]$out/kaboom.elf[e]. [tt]boot.o[e] goes first on the link line by convention, not requirement -- the linker script's [tt]ENTRY(_start)[e] is what actually decides where execution begins, object order on the [tt]ld[e] command line doesn't matter for that, [tt]mk.conf[e]'s own comment says as much.
 
[h2]mk/bl.sh: dynamite, the real bios bootloader[e]
 
builds the two artifacts a real BIOS-boot path needs, once [tt]$out/kaboom.elf[e] already exists: [tt]src/boot/dynamite.s[e] (a from-scratch, 16-bit-real-mode-to-32-bit-protected-mode bootloader, [tt].code16[e] assembled to a real, exactly-512-byte MBR sector ending in the [tt]55 aa[e] boot signature a BIOS refuses to execute without) and [tt]objcopy -O binary[e]'s raw extraction of [tt]kaboom.elf[e]'s own loadable content -- byte 0 of that blob lands at exactly [tt]linker.ld[e]'s own [tt]0x100000[e] origin, by construction, so loading it there needs no offset math at all. [tt]_start[e]'s real runtime address and the blob's real size are never hand-computed or hardcoded in [tt]dynamite.s[e] itself -- [tt]mk/bl.sh[e] reads both straight off the actual built elf ([tt]readelf -h[e]/[tt]-lW[e]) and passes them in as [tt]--defsym[e] values at assemble time, so dynamite always jumps to wherever [tt]_start[e] actually resolves to on that specific build, never a guessed offset that could silently drift out of sync. [tt]mk/bl.sh[e] also refuses to build at all if that real blob size exceeds 524288 bytes (1024 sectors, [tt]dynamite.s[e]'s own [tt]read_loop[e] ceiling, see below) -- that read is hardcoded independent of the blob's own size, so an oversized blob would otherwise have its tail silently truncated by the read itself and relocate/jump into whatever garbage happened to follow, with no error of any kind. today's kernel is well under that ceiling (under 40% of it), but the check exists so outgrowing it fails the build loudly instead of failing a boot silently.
 
why this exists at all: kaboom's fast dev-loop boots via PVH (see [see name="kernel"]kernel[e]'s own note on why, not multiboot1), and PVH is something qemu understands, not something a real BIOS has ever heard of -- a real BIOS just loads raw sector 0 into [tt]0x7c00[e] in 16-bit real mode and jumps there, zero concept of elf notes. dynamite is a second, genuinely traditional boot path onto the [i]identical[e] kernel: enable a20 (the fast method, port [tt]0x92[e] -- every cpu capable of the long mode kaboom itself needs already supports it), read the reserved kernel-blob region off disk in bios-safe chunks (64 sectors at a time, since real bios int 13h ah=42h implementations commonly cap a single transfer well under 1024 sectors, and staying within one real-mode segment per chunk sidesteps a classic bios bug class for free), get into 32-bit protected mode, relocate the blob up to its real [tt]0x100000[e] home, and jump to it. [tt]boot.s[e] itself needs zero changes -- both boot paths converge on the identical [tt]_start[e] in the identical cpu state (32-bit protected mode, flat segments) PVH already guaranteed.
 
this is real, hand-written real-mode-to-protected-mode assembly, and it earned every bit of the same "single highest-risk file" caution [tt]boot.s[e] itself gets: a missing [tt]cli[e] before the mode switch let a stray irq land mid-transition and read a protected-mode idt through a still-real-mode ivt, a real, independently-reproduced triple-fault; a missing [tt]cld[e] left the relocation copy's direction flag to chance; and real hardware doesn't start with zeroed ram the way qemu's guest ram happens to -- [tt]boot.s[e]'s own page tables and any zero-initialized kernel global needed an explicit [tt].bss[e]-zeroing loop in [tt]_start[e] that the PVH path had never needed, since an elf loader already does that for free. all three were found and fixed by the same rigor as everything else on this page: reproduced live, fixed, independently re-verified, not assumed correct because it looked right on a read-through.
 
[h2]mk/bu.sh: userspace[e]
 
compiles every program under [tt]src/user/[e] into a trimmed, ready-to-place-in-[tt]/bin[e] elf at [tt]$out/user/<name>[e]. the userspace libc ([tt]lib[e], [tt]mem[e], [tt]string[e], [tt]ctype[e], [tt]malloc[e], [tt]stdio[e], plus the hand-written [tt]crt.s[e]) is compiled once up front, not inside the per-program loop -- nsc has no [tt]#include[e]-and-compile-together model, but nothing stops sharing already-compiled objects across multiple final links. [tt]lib.o[e] is linked into every program unconditionally, always has been; the rest are opt-in per program, decided by grepping each program's own source for [tt]"mod.nsh"[e], plus one level of dependency resolution on top ([tt]string[e]/[tt]stdio[e] both call into [tt]mem[e] internally, so [tt]mem.o[e] goes on the link line even for a program that never includes [tt]mem.nsh[e] directly). see [see name="userland"]userland[e] for why: linking everything unconditionally was tried first and measured to add roughly 19kb of dead code to every binary, which pushed [tt]ed[e] over [tt]disk.pl[e]'s own binary size cap for zero benefit to programs that never call any of it.
 
every program links against one shared [tt]user.ld[e], fixed at [tt]0x400000[e] -- [tt]sh[e] included, no separate script and no separate load address for it any more. that used to be two scripts, [tt]user_shell.ld[e] ([tt]0x400000[e], sh only) and [tt]user_prog.ld[e] ([tt]0x600000[e], everything else), back when every process shared the kernel's one address space and keeping sh out of everything else's window (and vice versa) was the only thing standing between them; see [see name="kernel"]kernel[e]'s own vmm coverage for why that split isn't needed any more -- real per-process address spaces mean isolation comes from cr3, not from which fixed address a binary happened to link at. the actual link uses plain flags, not [tt]$ldflgs[e] (the kernel's strict set), with [tt]-z max-page-size=16 -z common-page-size=16[e] -- set far below the real 4096 specifically because the two real [tt]PT_LOAD[e] segments (r+x, r+w) [tt]user.ld[e] defines would otherwise cost up to roughly 4095 padding bytes per boundary satisfying page alignment this loader ([tt]elf.nsc[e]) has no actual use for. those two segments replaced a single [tt]-n[e]-flagged combined one, which is what was triggering [tt]ld[e]'s "rwx permissions" warning on every userspace binary -- a real warning, not cosmetic, given an actual paging-enforced os is exactly what it's about, this kernel just doesn't enforce it yet ([see name="kernel"]kernel[e]'s own vmm coverage is explicit that every mapping, shared or private, still carries the same present+writable flags -- no nx, no read-only -- real per-process address spaces fixed WHICH physical memory a process can touch, not what it can do to the memory it's given). after linking, [tt]strip -s[e] drops symbols, and [tt]mk/trim-elf.pl[e] truncates the result right after the last byte any [tt]PT_LOAD[e] segment's file data actually uses, discarding the section header table and shstrtab entirely -- section headers plus a small string table routinely add two to four hundred bytes to an otherwise-tiny binary, often enough on its own to push something over the size cap.
 
disk-image building used to happen at the end of this same script. it doesn't any more -- see below.
 
[h2]mk/bd.sh: the disk image (recently split out of bu.sh)[e]
 
this is genuinely recent: disk-image building lived at the tail end of [tt]mk/bu.sh[e] until it was recently split out into its own script. two real reasons, both in [tt]bd.sh[e]'s own header comment: a freshly rebuilt binary can now sit in [tt]$out/user[e] and be inspected or tested standalone before it's baked onto anything, and disk-building reads the [i]whole[e] [tt]$out/user[e] tree regardless of which binary [tt]bu.sh[e] most recently touched -- that's a genuinely different concern from "did this one program compile and link," and it earns its own place in the pipeline instead of riding along at the bottom of someone else's script. the full pipeline (see [tt]mk/ba.sh[e], which just runs all of these in order) is [tt]mk . c[e], [tt]mk . b[e], [tt]mk . bl[e], [tt]mk . bu[e], [tt]mk . bd[e], then whichever of [tt]mk . rn[e]/[tt]r[e]/[tt]rbn[e]/[tt]rb[e] actually runs it.
 
the script itself is one line past sourcing [tt]mk.conf[e]: [tt]perl mk/disk.pl "$out/disk.img"[e]. [tt]disk.pl[e] is a from-scratch, host-side reimplementation of kfs's on-disk format -- not a port of any part of [tt]kfs.nsc[e], just written to match its byte layout exactly (superblock fields, one inode per block, the 56-direct/1-single-indirect/4-double-indirect tier scheme -- see [see name="fs"]fs[e] -- and the dirent format's single-low-byte inode number and sentinel). it's kept in sync with [tt]kfs.nsc[e] by hand; there's no shared source of truth between them, the same way a bootloader and the kernel it loads usually don't have one either. it globs [tt]$out/user/*.elf[e] rather than naming programs, so dropping a new [tt].nsc[e] file in [tt]src/user/[e] and running [tt]mk/bu.sh[e] puts it on the disk with zero changes to [tt]disk.pl[e] itself -- the one exception is [tt]sh[e], which [tt]kmain.nsc[e]'s boot sequence execs by name as init, so it's the one binary the disk layout has to know about specifically.
 
the per-binary size cap [see name="userland"]userland[e] mentions ([tt]mk/bu.sh[e]'s own opt-in linking scheme, and [tt]4c[e]'s own note on trimming to fit it) is [tt]disk.pl[e]'s own constant, not a [tt]kfs[e] on-disk format limit -- [tt]kfs[e] already handles bigger files elsewhere (see [see name="fs"]fs[e]'s double indirection). it's been raised once so far, from 30720 bytes (60 blocks) to 32768 (64), once [tt]4c[e] needed the extra headroom even after real trimming -- a generic capacity increase, the same reasoning [tt]kfs[e]'s own inode format has been widened by four times over (see [see name="fs"]fs[e]), not a special case carved out for one binary.
 
[tt]disk.pl[e] also lays out the real bios-boot region [tt]mk/bl.sh[e]'s two artifacts need: [tt]out/dynamite.bin[e] at real lba 0, [tt]out/kaboom.bin[e] (the raw kernel blob) at real lba 1 through 1024 -- both embedded as raw sectors via [tt]embed_raw_region[e], bypassing [tt]put_block[e]'s own [tt]KFS_LBA_BASE[e] offset entirely, since these are real, absolute lbas, not kfs-relative ones. kfs's own on-disk layout starts at real lba 1025 -- [tt]KFS_LBA_BASE = 1025[e] -- applied in exactly one place kernel-side ([tt]kfs_read_block[e]/[tt]kfs_write_block[e]) and mirrored here identically; the two have to agree exactly or [tt]kfs_mount[e] looks for its own superblock at the wrong real sector and fails outright. [tt]dynamite.bin[e]/[tt]kaboom.bin[e] briefly also existed as ordinary [i]kfs[e] files too -- [tt]/dynamite[e]/[tt]/kaboom[e], real, [tt]cat[e]-able copies of the identical bytes already embedded raw -- useful during dynamite's own development to prove the two copies stayed byte-identical, but dropped once both boot paths were proven and reviewed: a pure duplicate earns its keep during development, not permanently. a more capable bootloader that reads [tt]/kaboom[e] through kfs directly, so there's genuinely only one copy instead of a duplicate not bothered with, is a real, separately-scoped future project.
 
[h2]mk/rn.sh and mk/r.sh: running it, the fast way[e]
 
both refuse to run before their inputs exist -- [tt]$out/kaboom.elf[e] ([tt]mk/b.sh[e]) and [tt]$out/disk.img[e] ([tt]mk/bd.sh[e]), checked explicitly with a message naming the right script to run first, not just qemu failing on a missing file with no context. that message used to say "run mk/bu.sh first" for the missing-disk-image case, a leftover from before [tt]bd.sh[e] existed -- fixed once the split actually landed, since pointing someone at the wrong script for a two-command fix is worse than no message at all.
 
[tt]mk/rn.sh[e] runs headless: [tt]-nographic[e] merges the vga console and qemu's own monitor onto the current terminal (ctrl-a c toggles between them, ctrl-a x quits), no separate window, works over a plain ssh session the same as any other headless qemu invocation. [tt]mk/r.sh[e] runs the same kernel against the same disk with [tt]-display curses[e] instead -- needs a real tty, ctrl+alt+2 switches to the qemu monitor and ctrl+alt+1 back. same disk, same kernel, genuinely just a different console; neither script's own comment claims otherwise. both boot via qemu's [tt]-kernel[e] flag -- the fast PVH dev-loop, not the real bios path below.
 
[h2]mk/rbn.sh and mk/rb.sh: running it, the real way[e]
 
the other, newly real path onto the [i]identical[e] kernel -- [tt]-drive[e] only, no [tt]-kernel[e] flag at all, so qemu's own bios (seabios) does the genuine mbr-boot dance through dynamite exactly like a real machine would, the actual proof the whole mechanism works before ever touching real hardware (dd the disk image to a usb flash drive, boot a real machine from it -- outside what this pipeline can verify directly, but everything up to that point is). [tt]mk/rbn.sh[e]/[tt]mk/rb.sh[e] mirror [tt]mk/rn.sh[e]/[tt]mk/r.sh[e]'s own headless/curses split exactly -- same disk, same real boot path, just a different console -- and refuse to run the same way if [tt]$out/disk.img[e] doesn't exist yet.
 
[h2]mk/d.sh, mk/dh.sh, mk/gensummary.pl: building this very page[e]
 
worth being direct about, since it's genuinely a little funny: this paragraph describes the exact pipeline that turns this file into the page rendering it, and running that same pipeline against this file is exactly how it was confirmed to work. [tt]mk/d.sh[e] is two sequential calls, not a real dependency graph -- same reasoning as mp/nscc's own [tt]d[e] step not chaining into anything else: [tt]perl mk/gensummary.pl[e] first, then [tt]sh mk/dh.sh[e].
 
[tt]gensummary.pl[e] regenerates [tt]docs/index.btft[e] from [tt]docs/index.btft.in[e] (the hand-edited template, tracked in git) by substituting the literal [tt]{{SUMMARY}}[e] marker with [tt]docs/summary.txt[e]'s own one-line content -- mirrors mp's own [tt]mk/genconfdoc.pl[e] pattern exactly, one source of truth instead of the banner text and the generated page slowly drifting apart by hand-editing both. [tt]docs/index.btft[e] itself is never hand-edited; edit the [tt].in[e] or [tt]summary.txt[e] and regenerate.
 
[tt]dh.sh[e] renders every [tt]docs/*.btft[e] (recursively, any subfolder included) into [tt]docs/html/[e], mirroring each source file's own relative path under [tt]docs/[e]. it resolves [tt]btf2html[e] off [tt]$PATH[e] or the [tt]BTF2HTML[e] environment variable, the same pattern [tt]mk.conf[e] uses for [tt]nscc[e]/[tt]as[e]/[tt]ld[e] -- never a hardcoded install location -- and fails loudly with a pointer to [tt]git://git.druid.rocks/druid520/btf.git[e] if it isn't found at all. after rendering it copies [tt]docs/keyframes.css[e] into [tt]docs/html/[e] alongside everything else, since btf2html itself has no notion of a stylesheet living outside the tree it's told to render.
 
[img src="made-with-nsc.gif"]made with nsc[e] [img src="powered-by-kaboom.gif"]powered by kaboom[e]