Post

notOS Version 2. Genesis.

Definition of new project architecture, build tools and

Posted
notOS as a modular crab
notOS as a modular crab
By Serhii Shkliaiev
13 min read
notOS Version 2. Genesis.

Prologue

1 In the beginning, the Registers were void and without form, and darkness was upon the face of the RAM. And the Bootloader said, “Let there be 0xFFFF0, the Reset Vector,” and there was Light.

2 And the Bootloader saw the BIOS, that it was good; and it searched the first sector of the drive through the firmament, looking for the sign of the Covenant. And behold, it found the Magic Word: 0xAA55. And the evening and the morning were the First Stage.

3 And the Bootloader said, “Let the A20 Line be opened, that we may escape the legacy of antiquity.” And it tried all available ways—by the Keyboard Controller, Port 92, and INT 0x15—until the wrap-around of memory was cleaved asunder, and it was good.

4 Then the Bootloader gathered the required sectors from the disk via INT 0x13 to load the Second Stage, and it was so, and it was good……. But what if it was aarch64 all the way….?

Legacy?

Hundreds of commits for monolithic OS, dedicated specifically for x86-64, but something went completely wrong. In fact so wrong, that it is easier to create a whole rewrite than to try to find a working commit and change the history. The main culprit was badly structured project, lots of global static variables, threads and processes only generated statically, no file system, staying in VGA graphics and no feature branches.

Old notOS version is deprecated. Ongoing rewrite is happening in a separate branch 1-bringing-up at the time of writing this page. Old version is not compatible with newest Rust toolchain, and fully dependent on GRUB and linux binutils toolchain.1

Whats in new rewrite

New rewrite completely changes the way how notOS is developed as a project:

  • Modular structure - everything has it’s own crate and compiled within the workspace. Kernel lives separately from the bootloader. Architecture-dependent crates live separately. Example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

.
├── arch
│   ├── default_cfg
│   ├── x86
│   │   ├── cfg
│   │   └── src
│   └── x86_64
│       ├── cfg
│       └── src
├── bootloader
│   └── src
├── buildConfig
│   └── src
├── osTools
│   ├── x86_util
│   └── xTask
│       └── profiles
├── src
│   ├── memoryManagement
│   │   └── heap
│   ├── structures
│   ├── sync
│   │   └── primitives
│   └── taskVirtualization
  • Cross-compilation - above feature allows to virtualize kernel’s code, allowing for architecture independent part, which can be linked against architecture-specific implementations;
  • Priorities - features such as filesystem, secured memory management, architecture-independent compilation toolchain, flexible configuration, proper task management are in the priority.
  • Configuration - YAML based configurations, which are being parsed by buildConfig during pre-compilation stage, and generate different constant-time parameters, flags, external assembly or C includes, linker script definitions, etc.
  • xTask Orchestrator - global project orchestrator script written in python, which helps to perform OS build, allows to run and test the created image. It has it’s own set of YAML configuration files.
  • nATSUKI - higher level testing framework for application level testing. Will be implemented after all core components of the OS is implemented. Used to test different kernel configurations against user-defined programs (both user and kernel space).

    Cross Compilation

To ensure cross-compilation all architecture-specific crates are fully separated from the main kernel crate. YAML configurations allow to define compile-time constants. This prevents hardcoding in the source code, making it clean.

Each module has a build.rs file, which utilizes buildConfig crate to generate such constants. Each architecture defines it’s own subset of YAMLs, linker scripts, architecture-specific structures and implementation. buildConfig handles all of this automatically :).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

arch/
├── default_cfg
│   └── core.default.yaml
├── x86
│   ├── Cargo.toml
│   ├── cfg
│   │   ├── bootloader.ld
│   │   ├── bootloader.yaml
│   │   ├── core.yaml
│   │   ├── i386-unknown-none-code16.json
│   │   ├── kernel.ld
│   │   ├── mem.ld
│   │   └── x86-unknown-none.json
│   └── src
│       ├── asm
│       │   ├── A20.s
│       │   ├── boot0.s
│       │   ├── disk.s
│       │   └── vga.s
│       └── lib.rs
|       ...
└── x86_64
    ├── Cargo.toml
    ├── cfg
    │   ├── bootloader.ld -> ../../x86/cfg/bootloader.ld
    │   ├── bootloader.yaml -> ../../x86/cfg/bootloader.yaml
    │   └── core.yaml
    └── src
        ├── asm
        │   └── long_mode_start.s
        └── lib.rs
        ...

Some architectures reuse the existing configuration from other architectures. For example x86_64 will reuse the entire bootloader from the it’s 32-bit variant. notOS shall support desktop targets, servers and embedded devices, therefore each architecture is also extremely configurable, to include or exclude huge blocks of kernel like heap management, graphics, drivers, etc.

buildConfig

Introducing the buildConfig. This is an alternative to Kconfig, which generates compile-time constants and #cfg flags. It also applies linker scripts, includes external assembly code or C code.

Kernel requires some amount of constants to always be defined regardless of the used architecture. Such constants are defined within the default core.default.yaml configuration file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

# Core Configuration Template.
#
# It is used to define main mandatory constants and flags, but is not
# only limited to those definitions. Usually custom hardware (if not
# virtualization) would require to additionally attack some linker script, C
# or assembly source files as additional parameters.
# 
# **IMPORTANT**
#
# The buildConfig helper allows to attach multiple files at once. It is handy to
# use defaults provided within this file and rename them within your configurations,
# keeping defaults implicitly. Only change variable that differ from defaults.
#
# The above prevents redundant code. The inner configuration of YAML, however, is
# very flexible, and not tied to specific naming like "mandatory"/"configuration".
# It is still desirable to also keep the naming convention within the YAML as well.
# All nested mappings must follow this template when using anchors.
#
# Copyright (c) 2026 not-forest
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

mandatory: &core_defaults
  # Amount of stack memory to allocate for kernel's internal functionality.
  kernel_stack_size: 16KB
  # Amount of heap memory to allocate for kernel heap data structures. 
  kernel_heap_size: 5KB

  # Size of one page in bytes. This default is used for most hardware architectures.
  page_size: 4096
  
  # Amount of page table levels. Unused if `has_virtual_memory: false`. 
  page_table_levels: ~
  ...

  # For readability, compile-time boolean flags are separated within this mapping.
  cfg: &cfg_defaults
    # Defines if current target architecture supports MMU operations on virtual
    # memory. If your target does not support that, can be safely ignored.
    has_virtual_memory: false
    # Defines whether heap allocations API shall be appended to the kernel implementation.
    has_heap: true
    ...

This file is automatically appended to to all other YAML files, so different architectures can change only parameters they really have to change. Some constants become unused when specific configuration flags are unset. Here is the example of some parameters being set and swapped within the x86’s core.yaml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

# Core configuration and constant definitions for given architecture.
#
# Copyright (c) 2026 not-forest
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

mandatory: &x86_mandatory
  <<: *core_defaults
  page_table_levels: 4

  cfg:
    <<: *cfg_defaults
    has_virtual_memory: true

compilation:
  _linker: "./kernel.ld"

# x86 architecture-specific memory related constants.
memory: &x86_memory
  # Free memory for kernel-space usage
  kernel_memory_start: !!int 0x100000
  kernel_memory_end: !!int 0xffffff

  # Free memory for user-space usage.
  user_memory_start: !!int 0x1000000
  user_memory_end: !!int 0xfffffff

Thanks to YAML anchoring properties, configurations become very clean and easy to debug. The buildConfig crate handles all configuration files using build_const for contant generation, rust-yaml for parsing, cc for compiling external assembly/C code and internal cargo flags to perform linking operations and appending #cfg flags.

This is how everything mentioned above is supplied to notOS-bootloader crate with just several simple pre-build commands:

1
2
3
4
5
6
7
8
9
10
11

//! Supply bootloader with specific configuration and linkage.

use buildConfig::SysConfig;

const CONFIG_NAME: &str = "bootloader.yaml";

fn main() {
    SysConfig::new(env!("CARGO_PKG_NAME"))
        .include_yaml(CONFIG_NAME)
        .generate();
}

xTask

Cargo is used to compile a set of binaries (kernel modules) for different architectures with help of buildConfig. It is hard, however, to merge all them into a singular kernel image we need an additional helper.

There comes the xTask - python based utility to parse deployment profile YAMLs, compile the kernel and all additional modules against that deployment profile and merge every single module into a single image based on the architecture need.

It is basically a global orchestator, which allows to build the entire project as one entity with a use of simple YAML configuration file. It is also used to automatically initiate a runner program, which could not only depend on the architecture, but also on the deployment type:

  • Run simulated version just on your host hardware;
  • Run real image on any virtual machine2;
  • Run real image on some hardware (e.g. using some flashing utility to flash a program)2;

A 32-bit x86 OS image with legacy BIOS bootloader and enabled debugging (both symbols and GDB), can be done using one simple command:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

notforest@Mac ~/Documents/notOS $ xTask build --profile osTools/xTask/profiles/x86/xtask.legacy.debug.yaml 
xTask :) -> [INFO] Parsing profile: xtask.legacy.debug.yaml
xTask :) -> [INFO] Running recipe no: 0
Package: notOS-bootloader
Flags: ['-Zjson-target-spec', '--target=arch/x86/cfg/i386-unknown-none-code16.json']
warning: notOS-bootloader@0.1.0: [notOS-config] Processing hardware target spec: /Users/notforest/Documents/notOS/arch/x86/cfg/bootloader.yaml
warning: notOS-bootloader@0.1.0: [notOS-config] Linking against: /Users/notforest/Documents/notOS/arch/x86/cfg/mem.ld
warning: notOS-bootloader@0.1.0: [notOS-config] Linking against: /Users/notforest/Documents/notOS/arch/x86/cfg/bootloader.ld
warning: notOS-bootloader@0.1.0: [notOS-config] Including source file: /Users/notforest/Documents/notOS/arch/x86/src/asm/boot0.s
warning: notOS-bootloader@0.1.0: [notOS-config] Including directory /Users/notforest/Documents/notOS/arch/x86/src/asm
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_STACK_BOTTOM (i64): 1280
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_STACK_TOP (i64): 31744
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_MEMORY_START (i64): 31744
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_MEMORY_END (i64): 654335
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_VIDEO_MEMORY_START (i64): 655360
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_VIDEO_MEMORY_END (i64): 786431
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-O0' [-Wunused-command-line-argument]
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-ffunction-sections' [-Wunused-command-line-argument]
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-fdata-sections' [-Wunused-command-line-argument]
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-fno-omit-frame-pointer' [-Wunused-command-line-argument]
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.33s
warning: notOS-bootloader@0.1.0: [notOS-config] Processing hardware target spec: /Users/notforest/Documents/notOS/arch/x86/cfg/bootloader.yaml
warning: notOS-bootloader@0.1.0: [notOS-config] Linking against: /Users/notforest/Documents/notOS/arch/x86/cfg/mem.ld
warning: notOS-bootloader@0.1.0: [notOS-config] Linking against: /Users/notforest/Documents/notOS/arch/x86/cfg/bootloader.ld
warning: notOS-bootloader@0.1.0: [notOS-config] Including source file: /Users/notforest/Documents/notOS/arch/x86/src/asm/boot0.s
warning: notOS-bootloader@0.1.0: [notOS-config] Including directory /Users/notforest/Documents/notOS/arch/x86/src/asm
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_STACK_BOTTOM (i64): 1280
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_STACK_TOP (i64): 31744
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_MEMORY_START (i64): 31744
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_BOOTLOADER_MEMORY_END (i64): 654335
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_VIDEO_MEMORY_START (i64): 655360
warning: notOS-bootloader@0.1.0: [notOS-config] Generated const CONFIG_VIDEO_MEMORY_END (i64): 786431
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-O0' [-Wunused-command-line-argument]
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-ffunction-sections' [-Wunused-command-line-argument]
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-fdata-sections' [-Wunused-command-line-argument]
warning: notOS-bootloader@0.1.0: clang: warning: argument unused during compilation: '-fno-omit-frame-pointer' [-Wunused-command-line-argument]
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
xTask :) -> [INFO] Universal image baseline reference: 0x7c00
xTask :) -> [INFO] Creating flat binary for: notOS-bootloader
xTask :) -> [INFO] Initializing empty image target: /Users/notforest/Documents/notOS/target/xTask/notOS.img
xTask :) -> [INFO] Mapping [notOS-bootloader] -> Disk File Offset: 0x0 (516 bytes written)
xTask :) -> [INFO] Image successfully baked (516 bytes) -> /Users/notforest/Documents/notOS/target/xTask/notOS.img
xTask :) -> [INFO] Created OS image notOS.img. Baked a total of 516 bytes.
xTask :) -> [INFO] xTask finished with success.

nATSUKI Framework

nATSUKI stands for notOS Automated Test Service Utility for Kernel Integration. The framework will automatically generate an exhaustive matrix of unique builds across multiple architectures by dynamically adjusting kernel parameters, such as the allocation of CPU cores or memory structures, through a combination of compilation flags and boot arguments. nATSUKI will then execute these variations, running everything from deeply embedded kernel modules to standard user-space applications, evaluating different metrics.

By transforming low-level systems testing into a self-monitoring data pipeline, the framework will catch complex architectural regressions and race conditions long before they can destabilize the codebase. This tool will mostly be convenient for developers to stress-test their applications written for notOS.

Cute notOStan representation made by AI.

A picture of free list allocator doing it's job right!

Read more

Footnotes:

  1. notOS target compilation json had flags, which were changed on newer rust toolchain, causing compile-time errors. Additionally it required very specific compilation utilities, making it only compilable on Linux machines. ↩︎

  2. Requires YAML configurations with defined runner commands. Any executable can be defined there. ↩︎ ↩︎2

notOS
notOS Rust
This post is licensed under CC BY 4.0 by the author.