Testing — oracles and evidence
Abstract
This page lists the oracles that check the sparkles:ui design system's requirements and the evidence gathered against them. Each oracle states why it is independent of the code it checks, and each evidence row names the revision, command, configuration and remaining gap.
Introduction
The trace runs from a requirement to a falsifying scenario, to an oracle, to scoped evidence. An oracle must be independent of the implementation: a round trip alone is not one. The requirements live in SPEC.md and its sibling pages, and delivery order lives in PLAN.md. An evidence row is a record of what one revision showed; where a later decision changes a contract, the row stays and a note below the ledger names the change.
Oracles
| ID | Oracle | Independence | Covers |
|---|---|---|---|
O1 | The style guide is the gallery. ui-gallery --render --profile {baseline,enhanced,full} paints each catalogued page headless; the output is embedded as an ansi fence in the public style-guide pages (docs/design-system/…) and ci --verify checks it. | The fence is reviewed by a person once when blessed; the verifier catches drift. Independence is the human review of the golden, not the renderer. | CAP5, ACC4, GLY2/GLY3, TOK6 (visually) |
O2 | Contrast. A WCAG 2.x relative-luminance + contrast-ratio implementation in the test tree, validated against the published example pairs from the WCAG technique documents (pinned), computes ThemeConformance for every built-in and every derived palette. | Published vectors; the tone model in color-derivation is a second, independent path (CIE L*) whose ordering must agree. | ACC1, ACC2, SPK2 |
O3 | Slot discipline. For each component, build its display list over a fixture model and assert slotsWithin(ops, Component.slots). | The declared set is authored by hand per component; the test is a set membership over ops — a second implementation of nothing. | TOK6 |
O4 | Theme file. Load the pinned DTCG specification's own example documents and the 36 exported built-ins; assert structured errors on a corpus of deliberately malformed files (cycle, dangling alias, wrong $type, out-of-range color); round-trip canonical equality. | Upstream examples are an interoperating peer; the malformed corpus is hand-derived. Round trip alone is explicitly insufficient and is paired with the peer corpus. | FMT1–FMT6 |
O5 | Capability probing. A scripted pty peer replies (or does not) to each query with recorded real-terminal transcripts (kitty, Ghostty, foot, xterm, tmux-wrapped); detectTermCaps + the tui probe must produce the expected TargetCapabilities, and a silent peer must return within the timeout with the sentinel path taken. | Transcripts are captured from real emulators at pinned versions, not synthesised from the parser. | CAP3, CAP7, CAP8 |
O6 | CSS lockstep. Emit the stylesheet from Theme, diff against the committed copy VitePress imports; assert every --spk-* name is what writeCssName writes for its slot, state and channel. | The committed copy is reviewed; the name check is against the one function TOK2 mandates. | WEB1–WEB3 |
O7 | Projection tables. projectBorder and the sub-cell ladders are checked cell-by-cell against a hand-written table for every (width, style, radius, tier) combination — exhaustive, since the domain is small. | Hand-derived expected glyphs. | GLY2, GLY2a, GLY8, CAP9 |
O1 notes. ui-gallery takes the role Storybook has for web design systems: the canonical rendering of a component is the one the reference application produces (D17).
Omitted combinations
O1 renders three capability profiles, not the 2^n flag combinations; the projection functions (O7) are exhaustive instead, so a flag combination outside the profiles is covered at the function level, not the page level. O5 covers the emulators its corpus recorded (emulator presets); others are unverified until a transcript is captured.
Evidence ledger
| Requirements | Revision | Command / test | Configuration | Result | Gap |
|---|---|---|---|---|---|
TOK2, TOK5, CAP9, GLY2 | main (M0–M2, PR #511) | dub test :ui -- -i "ui.tokens" | linux-x86_64, ldc2 | green | the spec's acceptance is the owner's (M0 gate) |
CAP4, CAP6, CAP9, O1 (baseline) | feat/design-system-m2 | dub test :ui, :ui-tui, :ui-raylib, :ui-gallery | linux-x86_64, ldc2 | green | O1 covers baseline only; text-run glyphs are M3 |
GLY1–GLY3, ACC3, ACC4 (regions), CAP8, O1 (three profiles), O7 | feat/design-system-m3 | dub test :ui, :ui-tui, :ui-gallery, :base | linux-x86_64, ldc2 | green | ACC4 covers the two focus regions only |
CAP10, O7 (presets) | feat/design-system-emulator-presets | dub test :ui, :ui-gallery | linux-x86_64, ldc2 | green | the presets' rows are those the matrix holds; foot, zellij and Windows Terminal are unmeasured |
CAP10, CAP7 (presets), D33 | feat/design-system-more-emulators | dub test :ui -- -i emulators, :ui-gallery | linux-x86_64, ldc2 | green | the six new rows' reports are re-read by test; the older matrix rows live on the research branch only |
GLY9 (cell rungs), CAP6 (image rows), O7 (pattern glyphs), D34 | feat/design-system-image-ladder | dub test :ui, :ui-tui, :ui-raylib, :ui-gallery; ui-gallery --gui --profile enhanced screenshot | linux-x86_64, ldc2 | green | the protocol rung is not built; octants raster as sextants |
GLY9 (kitty rung), IMG5 (kitty), CAP3 (images row) | feat/tui-image-protocol | dub test :tui, :ui-tui, :ui-app, :ui-gallery; ui-gallery --tui in kitty under xvfb, screenshot | linux-x86_64, ldc2 | green | sixel is not drawn; the probe is opt-in |
IMG5 (sixel), CAP3 (cell pixel size) | feat/tui-sixel | dub test :tui, :ui-tui, :ui-app, :ui-gallery; ui-gallery --tui in foot under headless cage, screenshot | linux-x86_64, ldc2 | green | the palette is fixed (252); the probe is opt-in |
GLY9 (octants) | feat/design-system-octants | dub test :ui, :ui-tui, :ui-gallery; gen_unicode_tables.d over Unicode 17.0.0 | linux-x86_64, ldc2 | green | none |
CAP3 (the battery), CAP10, O5, D35 | feat/design-system-query-battery | dub test :base, :tui, :ui, :ui-app, :ui-gallery; ui-gallery --tui in foot (under cage) and kitty (under xvfb) | linux-x86_64, ldc2 | green | the input-mode rows are recorded, not applied; the probe is opt-in |
CAP3 (focus and paste negotiated), INP21, D35, D36 | feat/tui-paste-focus | dub test :input, :tui, :ui-tui, :ui-app, :terminal-view, :ui-gallery, :dql; ui-gallery --tui in kitty under xvfb | linux-x86_64, ldc2 | green | none |
O5, CAP3, CAP10 | feat/design-system-o5-corpus | dub test :base -- -i term_replies, dub test :tui -- -i probeRecorded, dub test :ui -- -i emulators | linux-x86_64, ldc2 | green | Linux only: iTerm2, WezTerm and Apple Terminal keep case-study rows from an earlier battery |
O5, CAP10 | feat/o5-macos-corpus | dub test :base -- -i term_replies, dub test :tui -- -i probeRecorded, dub test :ui -- -i emulators; captures by libs/base/examples/term-replies-capture.d | linux-x86_64, ldc2; captures on macOS 27.0.1 arm64 | green | WezTerm under xvfb; iTerm2, Ghostty and Apple Terminal launched on the Mac with open, iTerm2 and Ghostty after their run confirmations were approved by hand |
INP22, CAP3 (scheme reports), D37 | feat/tui-color-scheme | dub test :input, :tui, :ui, :ui-app, :ui-tui, :ui-gallery, :dql; ui-gallery --tui --theme github-light in kitty under xvfb | linux-x86_64, ldc2 | green | a window does not report the OS appearance |
CAP3 (graphemes), GLY6, CAP6, O5, D38–D40 | feat/tui-grapheme-clusters | dub test :base, :tui, :ui, :ui-tui, :ui-app, :ui-gallery; the O5 corpus re-captured; ui-gallery --tui -p text in kitty and XTerm under xvfb, screenshots | linux-x86_64, ldc2 | green | a cluster over a cell's 16 bytes folds on every target; superseded by GLY12 (D50), see the note below |
CAP3 (styled underlines), CAP6, O5, D41 | feat/tui-styled-underlines | dub test :base, :tui, :ui, :ui-tui, :ui-app, :ui-gallery, :hue; the O5 corpus re-captured; ui-gallery --tui -p primitives in XTerm and kitty under xvfb, screenshots | linux-x86_64, ldc2 | green | Alacritty, tmux and zellij draw straight underlines they may be able to style |
CAP3 (links, clipboard, notifications, pointer shape), D42 | feat/tui-hyperlinks | dub test :base, :tui, :ui, :ui-tui, :ui-app, :ui-gallery, :hue, :diagram, :terminal-view; the O5 corpus re-captured with XTVERSION; OSC 8 in XTerm under xvfb, screenshot | linux-x86_64, ldc2 | green | clipboard, pointer shape and notifications are declared but not gated |
CAP3 (clipboard, pointer shape consumed), D43 | feat/tui-host-errands | dub test :ui-raylib, :ui-app, :ui-gallery, :hue; ci --build-each-commit | linux-x86_64, ldc2 | green | the copy notice is unit-tested, not driven through a live selection |
CAP3 (key releases, hover), INP16, D44 | feat/tui-key-releases | dub test :tui, :ui-tui, :ui-app, :ui-gallery, :diagram, :hue, :terminal-view; xdotool into kitty and XTerm under xvfb, the events recorded | linux-x86_64, ldc2 | green | hover is the request, not a measurement |
TOK1, TOK4–TOK6, O3, D46 | feat/ui-state-overlays | dub test :ui, :ui-tui, :ui-raylib, :ui-gallery, :source-view, :twoslash, :diagram, :terminal-view, :hue, :docs | linux-x86_64, ldc2 | green; every profile golden unchanged | metrics per state (D22) have no consumer; the inspector header's spans keep their per-span slot |
KBD1, KBD3, KBD6, KEY12, D47, D48 | feat/ui-keymap-universal | dub test :ui -- -i keymap_universal, :ui-app, :ui-gallery, :diagram, :hue, :doc-view; hue --list-keys, ui-gallery --list-keys, diagram --list-keys; diagram --tui in a script pty: Ctrl-C exits 0, a plain x runs to the timeout | linux-x86_64, ldc2 | green | the interrupt was not run against the pre-change binary; the cross-app gate is each app testing against the one table, not one test over all three |
Note on feat/tui-grapheme-clusters. The gap that revision recorded — a cluster longer than a TUI cell's 16 inline bytes folds on every target — is not the contract. GLY12 keeps such a cluster whole through the cell's overflow storage (base/text TXT-CELL8) and fails explicitly when that storage is exhausted (D50). The row stays evidence for GLY6's fold on a terminal that does not cluster; GLY12 has no evidence.
Note on wrapping evidence. Evidence for the toolkit's wrapping and cutting (LAY5, LAY10 and LAY14 in the layout specification) stands for the delivered code and is pending base's cutover of those callers (D51). | FMT1–FMT6, O4, D12, D14, D55–D58 | feat/ui-theme-file | dub test :ui -- -i "dtcg|theme_file": the edition's 48 examples (35 load and resolve; 13 illustrations rejected, each named), every built-in's export, load, save and re-export, the malformed corpus | linux-x86_64, ldc2 | green | the checked-in exports and --theme <file> (FMT5, M5's second half); font roles and box chrome not mapped | | FMT5, THM9 | feat/ui-theme-file-exports | dub test :ui -- -i theme_file (files with nested and relative bases, cycles, missing files, themeNamed, the 36 checked-in exports), :ui-app, :ui-gallery, :hue; by hand: hue theme nord --export, a file with base: nord setting syntax.keyword.fg rendered by hue --theme <file> view --ansi | linux-x86_64, ldc2 | green | ui-gallery's --render paints the default theme, not --theme | | CAP2, ACC4, GLY2a, GLY4 | feat/design-system-m3-leftovers | dub test :ui, :ui-tui, :ui-gallery, :diagram; each check made to fail once by hand: the progress row removed, fa-minus moved one code point | linux-x86_64, ldc2 | green | the per-role glyph families (GLY1) are their own change; GLY4's web face waits for M6 | | GLY1, D59 | feat/design-system-glyph-families | dub test :ui, :ui-tui, :ui-app, :ui-raylib, :source-view, :twoslash, :diagram, :ui-gallery, :hue (the families change nothing until a theme names one; every golden unchanged); the Tree page with a heavy-guide theme draws ┣━ | linux-x86_64, ldc2 | green | hue's inspector pane keeps the light guides: it has no palette in hand | | GLY1, D59: families resolved by the theme | feat/design-system-m3-leftovers | dub test :ui, :ui-tui, :ui-app, :ui-raylib, :source-view, :twoslash, :diagram, :ui-gallery, :hue; the gallery's own bars stop hard-coding █ over │ and take the theme's █ over ░, the only golden change (512 cells); hue's goldens unchanged | linux-x86_64, ldc2 | green | the markdown fence and table bars keep their own glyphs: each shares the rule it is drawn on | | WEB1–WEB3, WEB6, D60–D63 | feat/design-system-m6-web | dub test :ui, :docs, :hue, :ui-gallery, :ui-app; the drift test made to fail once by hand; the docs site built from main's custom.css and from the mapped one, four pages in light and dark, 0 pixels differing in each of 8 screenshots while the built CSS shows the mapping | linux-x86_64, ldc2, chromium 151 | green | WEB4, WEB5 in the next PRs; VitePress variables the block does not map keep VitePress's defaults | | WEB4 | feat/design-system-m6-html-classes | dub test :ui -- -i html_semantic: every slot's rule reads its properties, the hover falls back to the rest value, the rules name no color, no slot class names a kind class | linux-x86_64, ldc2 | green | | | WEB5, D62 | feat/design-system-m6-breakpoints | dub test :docs, :hue; a hue gallery with a sidebar in Chromium at 600, 1000 and 1700 px: the drawer below 80 columns (closed and open), the sidebar from 80, the content column at 100 mono columns (954 px) from 160 | linux-x86_64, ldc2, chromium 151 | green | no rule at 120 columns yet |
Everything else is unverified.