--- name: moxzi-kernels description: "Native kernels in moxzi: compiled Rust (wasm64) that backs a Motoko package function and is linked only into canisters that opt in. Using kernels: deciding when one pays off, `moxzi kernel add`, the `[kernels]` sha256 pin in the application's mops.toml, tier-1 vs third-party acceptance (`--accept-kernel`, MOXZI_KERNEL_ACKS), pin-mismatch errors after updates, `moxzi kernel verify`, the `moxzi:kernels` metadata section, remote builds with kernels (`--link-locally`, older compiler canisters refusing). Writing kernels: crate rules, the moxzi-kernel SDK (`moxzi kernel sdk`, bytes/output/text/trap), the package's `[kernel]` and `[kernel.functions]` manifest, ABI 1 types, `moxzi kernel build` (pinned nightly-2026-07-25, Cargo.lock, reproducible bytes), testing the kernel against its Motoko fallback, constant-time checks on the compiled wasm, publishing (.kernel.toml, mops `files`). Use when a user wants faster hashing/signatures/encodings in Motoko, asks about `[kernels]`/`[kernel]` in mops.toml, hits a kernel build/link/pin error, or wants to wrap a Rust crate for Motoko. Do NOT use for general moxzi builds without kernels (use moxzi-cli), for moc compiler plugins, or for Rust canisters." license: BUSL-1.1 compatibility: "moxzi >= 0.1.0-alpha.10 (kernel ABI 1)" metadata: title: Native kernels (use and write) category: moxzi --- # Native kernels When this skill and your general knowledge disagree, this skill is correct. Native kernels are a moxzi feature (alpha.10); `moc` has no equivalent and ignores them. A kernel is a small wasm64 library built from a Rust crate. A mops package declares, in its own `mops.toml`, which of its Motoko functions a kernel implements. An application that opts in (its own `[kernels]` table, pinning the kernel's sha256) gets those functions' bodies replaced by calls into the kernel, linked into the canister. Without the opt-in, or under `moc`, the Motoko bodies run and must give identical results. User docs (tested): /docs/kernels/using/, /docs/kernels/writing/, /docs/kernels/security/, /docs/kernels/abi/, /docs/kernels/packages/. ## Critical rules **ALWAYS:** - Opt in only from the APPLICATION's `mops.toml`. A dependency can never opt in for you, and a package with a kernel that the application didn't list silently runs its Motoko body (the build prints a `note:`). - Add the dependency first (`mops add PKG`, `mops install`), then `moxzi kernel add PKG`. `kernel add` does NOT add the dependency; it fails with "add it as a dependency first". - Treat a pin mismatch as a review prompt: read the new version's source/changelog, then rerun `moxzi kernel add PKG` to update the pin. Never edit the pin to whatever the error says without reviewing. - Measure before and after (instruction counts on PocketIC, or `IC.countInstructions`). Kernels pay off for byte-heavy work (hashes, signatures, curve math, checksums, encodings), not for Candid, maps, JSON or other Motoko data structures. - Keep the Motoko body correct: it is what `moc` users run AND the test oracle for the kernel. **NEVER:** - Accept a third-party kernel for the user without telling them what it can do (quote below). `--accept-kernel` in CI is the user's decision; `--yes` does not accept kernels. - Expect `RUSTFLAGS`, `RUSTC_WRAPPER`, `CARGO_PROFILE_*`, `CARGO_TARGET_*` or `CARGO_BUILD_*` (except `CARGO_BUILD_JOBS`) to affect `moxzi kernel build`; they are dropped and named. - Put a kernel's secrets (keys, seeds) in static data or `static mut`. - Claim a secret is zeroized: scratch memory is GC-reclaimed, not erased. ## What a kernel can do (exact text moxzi prints; show it to users) > A native kernel is compiled code that runs inside your canister and shares its memory. > It can read and write anything in the canister's heap -- every user's data, keys, and > the state that survives upgrades -- and return what it read. The linker keeps the > system API away from it (it cannot send messages, spend cycles or set timers) and the > glue checks what it returns, but nothing checks what it does to memory. The sha256 pin > guarantees which bytes run, not that they are correct or benign. Accept a kernel only > if you trust its author or have reviewed its source at the commit it names, and take > a canister snapshot before deploying a new one. ## Measured costs (local replica, one call) | Operation | Kernel | Motoko body | |---|--:|--:| | SHA-256 of 1 MB (`moxzi-hash`) | 76 M | 405 M | | CRC-32 of 1 MB (`crc32fast`, docs tutorial) | 17.7 M | 246 M | | `mo:ic-agent` mainnet certificate check | 0.37 B | 2.0 B | | same, with subnet delegation | 0.61 B | 3.8 B | Typical range across the published packages: 8-70x fewer instructions, 2x to >1000x overall. ## Using a kernel ```sh mops add moxzi-hash && mops install moxzi kernel add moxzi-hash # shows package/version/sha256/size/functions/source/commit, writes the pin moxzi build main.mo -o app.wasm # prints: kernel: moxzi-hash sha256:d9e07eab… (48095 B, 7 function(s)) from … [tier-1] ``` `kernel add` writes exactly one line (appending the table if absent): ```toml [kernels] moxzi-hash = "sha256:d9e07eabd5c53e4567f6ce132857b52f7b0d3546f2443b73bdb3b07cf2244a2e" ``` Options: `kernel add PKG --root DIR` (app dir; default `.`). To build WITHOUT kernels there is no flag: delete the `[kernels]` line (the build then prints `note: package X provides a native kernel; it runs its Motoko code unless you opt in…`). **Tier-1** = a (package, sha256) PAIR compiled into the CLI: `ic-agent`, `moxzi-base58`, `moxzi-bech32`, `moxzi-bitcoin`, `moxzi-bls12-381`, `moxzi-ed25519`, `moxzi-hash`, `moxzi-ml-dsa`, `moxzi-p256`, `moxzi-rsa`, `moxzi-secp256k1` at their released pins. A tier-1 NAME at other bytes is third-party (the warning says "is the name of a tier-1 kernel, but these bytes are not one of its released builds"). **Third-party** = everything else, including the user's own kernels. The first build that links one prints the warning above and stops: - interactive terminal (stdin AND stderr are TTYs): prompt, type `yes`; - otherwise: error naming the flag, `--accept-kernel PKG=sha256:<64 hex>` (repeatable; a package name alone is rejected). Acceptances persist per user, keyed by (package, sha256), in `/moxzi/accepted-kernels` (macOS `~/Library/Application Support/moxzi/`, Linux `~/.config/moxzi/`); `MOXZI_KERNEL_ACKS=FILE` overrides (use a per-job file in CI). Delete a line to be asked again. A new version asks again. **Check provenance.** `moxzi kernel verify .` (in the app dir) rebuilds each pinned kernel from its package's declared `[kernel] source` + `commit` and prints `MATCH`/`DIFF` per pin; non-zero exit on any DIFF. `--crate DIR` rebuilds from a local crate; `--git` forces a clone even when a local checkout of the source repo is found. Needs rustup + the pinned toolchain. The published moxzi kernels' source is https://github.com/icdevsorg/moxzi-kernels (MIT OR Apache-2.0); each package names the commit, so anyone can rebuild and compare (also `kernels/repro/verify.sh` there, in Docker, without moxzi). **What a module contains.** Public custom section `moxzi:kernels`, one line per kernel the module CALLS (an opted-in kernel it never calls is neither linked nor listed): `PKG sha256:HEX abi=1 [version=V] [source=URL] [commit=C]`. - local: `wasm-tools print app.wasm | grep moxzi:kernels` - deployed: `dfx canister metadata moxzi:kernels --network ic` - on-chain build record: `moxzi verify --remote … --canister … --job N` prints a `kernels` line. ## Remote (on-chain) builds with kernels `moxzi build --remote URL --canister ID` with `[kernels]`: 1. third-party acceptance happens BEFORE any upload; 2. the CLI uploads missing kernels to the canister's blob store (`putBlobs`; the canister hashes them itself; max 1,900,000 bytes per kernel); 3. the canister links on chain (`linkArtifact`) and records pins + declared provenance (`buildKernels(job)`, shown by `moxzi verify`). Result = the local build's bytes. ## Writing a kernel (procedure) Layout (mirrors the published packages): ``` mypkg/mops.toml [package] + [kernel] + [kernel.functions] mypkg/src/Mod.mo the Motoko functions WITH bodies (fallback + oracle) mypkg/rust/Cargo.toml staticlib crate mypkg/rust/Cargo.lock REQUIRED, committed mypkg/rust/src/lib.rs exports mypkg/kernel/mypkg.kernel.toml generated by moxzi kernel build moxzi-kernel/ SDK, written by `moxzi kernel sdk` (sibling of mypkg/) ``` 1. Toolchain: `rustup toolchain install nightly-2026-07-25 --profile minimal --component rust-src` (rustc commit da86f4d07, its own rust-lld; both commit-checked; a wrong one is an error with the fix printed). 2. `Cargo.toml`: ```toml [lib] crate-type = ["staticlib"] [dependencies] moxzi-kernel = { path = "../../moxzi-kernel" } # MUST be ../moxzi-kernel, ../../moxzi-kernel, … crc32fast = { version = "=1.5.3", default-features = false } # exact version, no std [profile.release] opt-level = 3 lto = true codegen-units = 1 panic = "abort" # optional, the ONLY way to pass --cfg to dependencies: # [package.metadata.moxzi-kernel] # cfg = ['curve25519_dalek_bits="32"'] ``` 3. `src/lib.rs`: `#![no_std]`; exports are found by text search for `pub extern "C" fn NAME` / `pub unsafe extern "C" fn NAME` IN src/lib.rs ONLY (each needs `#[no_mangle]`): ```rust #![no_std] use moxzi_kernel::bytes; #[no_mangle] pub unsafe extern "C" fn crc32(data: *const u8, len: usize, out: *mut u8) { let crc = crc32fast::hash(bytes(data, len)); core::ptr::copy_nonoverlapping(crc.to_be_bytes().as_ptr(), out, 4); } ``` 4. Lock: `moxzi kernel sdk moxzi-kernel` (next to the package dir, matching the `../..` path), then `cargo +nightly-2026-07-25 generate-lockfile --manifest-path mypkg/rust/Cargo.toml`. 5. Build: `moxzi kernel build mypkg/rust -o mypkg/kernel/mypkg.kernel.toml` -> last line `…: N B, sha256:HEX`. Default output name: package name minus `-kernel` suffix + `.kernel.toml`. 6. Manifest (`mypkg/mops.toml`): ```toml [package] name = "mypkg" version = "0.1.0" files = ["src/**/*.mo", "kernel/*.kernel.toml"] # REQUIRED or mops won't publish the kernel file [kernel] abi = 1 file = "kernel/mypkg.kernel.toml" source = "https://github.com/OWNER/REPO/tree/COMMIT/mypkg/rust" # optional; one token commit = "COMMIT" # optional; name a commit, not a branch [kernel.functions] "Mod.digest" = { export = "crc32", args = ["Blob"], result = "Blob", result_bytes = 4 } ``` Key = file path below `src/` without `.mo`, dot, function name (`"util/Hex.encode"` -> `encode` in `src/util/Hex.mo`). 7. `moxzi kernel verify mypkg --crate mypkg/rust` -> two `MATCH` lines. 8. Test in an app: `moxzi kernel add mypkg`, build with `--accept-kernel mypkg=sha256:HEX`, build again without `[kernels]`, install both on PocketIC, compare outputs on vectors + random + edge inputs, compare instruction counts. Any difference is a kernel/manifest bug. 9. Publish: commit crate + lock + kernel file together; `mops publish`. ### ABI 1 (all wasm64; pointers/lengths are 64-bit) | Motoko | Kernel side | |---|---| | arg `Blob` / `Text` | `(ptr: *const u8, len: usize)`; Text is UTF-8; read-only, borrowed for the call | | arg `Nat64` | `u64` | | arg `Bool` | `u64` 0/1 | | result `Bool` | return `u32`, exactly 0 or 1 (NOT checked by the glue) | | result `Blob` + `result_bytes = n` | extra LAST param `out: *mut u8` (room for exactly n); write all n; return nothing | | result `Blob` / `Text` | return `u64` address of `[len: u64][bytes]` from `kernel_alloc` (use SDK `output(&[u8])` / `text(&str)`) | | result `?Blob` / `?Text` | as above, or `0` (`NONE`) for null | Allowed manifest strings: args `Blob|Text|Nat64|Bool`; result `Bool|Blob|?Blob|Text|?Text`; `result_bytes` only with `Blob`. SDK: `bytes(ptr,len) -> &[u8]`, `output`, `text`, `NONE`, `trap(&[u8]) -> !`; global allocator = `kernel_alloc` (dealloc is a no-op; align power of two <= 65536, <= 64 MiB per request, else trap); panic -> trap `native kernel panicked: …` (256 B max). Glue traps (message rolls back): `native kernel P.E wrote past its N-byte result`, `… returned a result longer than its scratch`, `… wrote past its result scratch`, `a native kernel returned text that is not UTF-8`, `a native kernel returned no result`. Linker refuses: imports other than `env.{memory,__indirect_function_table,__stack_pointer,__memory_base,__table_base,kernel_alloc,kernel_trap}`; `memory.grow`/`memory.init`; start function; own memory/table; absolute or passive data/elem segments; GOT imports; static data over the shared 512 KB reserve (program + runtime + all kernels); bytes not matching the pin in the module's `moxzi:kernels` section. Floats are NOT refused but avoid them (NaN bits differ across hosts). No GC runs during a kernel call. ## Constant-time checklist (crypto kernels) - IC: equal INSTRUCTION COUNTS for every secret value (measure with `IC.countInstructions` across all-zero/all-one/min/max/random secrets; require identical numbers). - Off-chain hosts compile wasm to machine code: also NO branch/select/address depending on a secret. LLVM can turn masked Rust into `br_if`/`select`; check the BUILT kernel: ```sh sed -n '/^wasm = """/,/^"""/p' k.kernel.toml | sed '1d;$d' | base64 -d > k.wasm wasm-tools print k.wasm > k.wat grep -n -E '^ \(func |br_if|br_table|select|^ +if' k.wat ``` Trace each site's condition in functions reachable from secret-taking exports: constants, loop counters, public lengths = fine; loaded secret bytes = leak; calls (`memcmp` from `==` on slices!) = inspect the callee. Use `subtle` (`ct_eq`, `ConditionallySelectable`) not `==`. Re-check after every dependency/toolchain change (inlining moves code). - Stock audited crates (RustCrypto, dalek, zkcrypto, libcrux), exact versions, `cargo audit`; no hand-rolled bignum/field/curve arithmetic. - Vectors: Wycheproof, NIST ACVP/CAVP, RFC vectors, including must-reject cases; differential vs the Motoko body; traps must match. ## Error -> cause -> fix | Symptom | Cause | Fix | |---|---|---| | `kernel X: the installed kernel is sha256:A, but your [kernels] pin is sha256:B…` | package updated (new kernel bytes) or tampered | review the new version, then `moxzi kernel add X` | | `this build links N third-party native kernel(s) that this machine has not accepted… Nothing was built` | first use of a non-tier-1 kernel, non-interactive | user decides; rerun with the printed `--accept-kernel X=sha256:…` | | `[kernels] names X, but no installed package X provides a kernel` | dependency missing or package has no `[kernel]` | `mops add X` / check the package | | `package X is not installed in … add it as a dependency first` (kernel add) | `kernel add` doesn't install | `mops add X && mops install` | | `the build does not use native kernel X although mops.toml opts into it: the compiler does not support native kernels…` | compiler (often an older compiler CANISTER for `--remote`) predates kernels | use moxzi >= 0.1.0-alpha.10 / a canister that supports them | | `the compiler canister cannot link this build (…), and it uses native kernels; rerun with --link-locally…` | canister can't link | `--link-locally` (loses the canister's linked-hash attestation) or another canister | | `X has no Cargo.lock. A kernel builds --locked…` | no lock | `moxzi kernel sdk ../moxzi-kernel` + `cargo +nightly-2026-07-25 generate-lockfile`; commit it | | `the SDK must be a path dependency of the form "../moxzi-kernel"…` | SDK path shape | `moxzi-kernel = { path = "../moxzi-kernel" }` (or `../../…`) | | `[dependencies] foo = { path = … } is outside the crate` | path dep outside crate dir | move it inside the crate or use a published version | | `toolchain nightly-2026-07-25 is not installed` / `… is not commit …` / `has no rust-src` | toolchain | run the printed `rustup` command | | `another kernel build (pid N) is using /tmp/moxzi-kernel-build` | one build at a time machine-wide | wait for it | | `note: ignoring RUSTFLAGS=…` | env dropped for reproducibility | use `[package.metadata.moxzi-kernel] cfg` | | `note: MOXZI_KERNEL_WORKDIR=… not the canonical directory` | custom workdir | unset it; bytes won't match pins otherwise | | `error[M0291]: native kernel X: its mops.toml [kernel.functions] declares "Mod.fn", but package X has no module Mod` | key names a module that does not exist (CLI check, local and remote) | fix the key (path below `src/`, no `.mo`) | | `M0291 … declares "Mod.fn", but /pkg/X/Mod.mo has no function fn (its functions: …)` | key names a function the module does not define (typo, rename); checked when the program imports the module, also by the compiler canister | fix the key or the function name | | `M0291 … declares "Mod.fn" as (…) -> … …, but parameter p is T where the kernel takes U` (or `the function returns T where …`), at the function's signature | `args`/`result` disagree with the Motoko signature (types written as built-in names) | fix the manifest or the signature | | `type error [M0096] …` at the function's signature in `/pkg/X/Mod.mo` | same disagreement, through a type alias | fix the manifest or the signature | | `M0290 … must take plain named parameters` / `… is not a function declaration` | pattern parameters / the key names a `let` | use `public func f(a : Blob, b : Blob)` | | `note: native kernel X is opted in under [kernels], but the program imports none of the modules…; it is not linked` | the program does not use the kernel's modules | nothing: not linked, not in `moxzi:kernels`; drop the line if unwanted | | trap `native kernel X.E returned a Bool that is neither 0 nor 1` | kernel's `Bool` result was not 0/1 | fix the kernel: return exactly 0 or 1 | | `kernel link failed: … static data …` | 512 KB static reserve exceeded | link fewer kernels / shrink tables | | `mops publish` omits the kernel / users get "provides no native kernel" | `[package] files` lacks `kernel/*.kernel.toml` | add it to `files` | | `kernel verify` DIFF | different source/lock/commit, or not built in `/tmp/moxzi-kernel-build` | rebuild from the declared commit; check the lock | Related skills: build flags and packages -> `moxzi-cli`; remote builds and `moxzi verify` -> `moxzi-onchain-builds`; `mo:ic-agent`'s BLS kernel -> `moxzi-ic-agent`.