hue DSV preview — Feature Requirements (the data browser)

Status: design · Date: 2026-08-18 · Scope: Delimiter-Separated Values (CSV, TSV, PSV, semicolon-CSV, …) as a hue content kind: the grid preview across all four sinks, the dialect sniffer, the interactive data-browser tier (multi-key sort, filtering, column hide/reorder), the selection/copy deltas, and the sparkles:dsv engine underneath.

NOTE

Phase 1 shipped: the D0 engine (71757564 — libs/dsv) and the D1 basic preview (3e18af9e — the grid in every sink through the existing md table path, layout pinned by golden glyph-grid fixtures). CHK executed (2026-08-18): the table unification merged, the audit is recorded in the Milestones section, and D2–D5 are re-planned and normative. D2 shipped (47652402): the record-number gutter, per-column caps, decimal alignment, and the pinned header band (a sparkles.ui.components.table viewport extension, 43afbea8, since generalized to freeze panes). D3 shipped (090f05c6): the source copy format, chrome exclusion, and byte-exact whole-grid reproduction. D4's core shipped (546107c6–8fcf9290): the projection engine, the browser machine, and the TUI's filter bar, header-click sort and reset — residue per row (header menus, palette UI, GUI wiring, fuzzy highlight positions). Everything post-CHK is forward-looking design. Status legend and ID conventions: see the overview.

Design & scope

What it is

A .csv (or .tsv, .psv, sniffed .txt/stdin) target renders as a decorated data grid by default in every sink — the MOD8 doctrine applied to a third document family: GUI, TUI, non-interactive ANSI, and HTML all paint the same grid from the same model, and --raw (CLI9) reverts to highlighted source. Interactively the grid is a data browser: a pinned header, a row-number gutter, multi-key sorting, a filter bar plus header menus, and column hide/reorder — a viewer in the VisiData/csvlens tradition, not an editor.

Almost everything visual already exists. The box-drawn grid is MDP10's table (resolveTracks, bold header + heavy ┝━┿━┥ rule, per-column alignment); the selection regime is TBL1–TBL6 unchanged; the wide-content scroll idiom is COD6's border scrollbars; the filter bar follows the picker's query-language doctrine (PKQ). What is genuinely new is the engine (dialect sniffing, an RFC 4180 parser with an identity channel, a record index, projection compute) and the browser state machine over it.

Why the engine is a library (sparkles:dsv)

The same reasoning as sparkles:diff and sparkles:fuzzy: hue's engines are things this repository owns, unit-tests, and benchmarks. sparkles:dsv (libs/dsv) is the allocation-conscious compute core — dialect detection, parsing, the record index, typed columns, and sort/filter execution — @safe pure nothrow @nogc throughout, texts borrowed as spans, storage in caller-owned SmallBuffer arenas, errors as Expected. sparkles:base is its only dependency: the fuzzy full-text remainder (DSF3) is matched by sparkles:fuzzy and combined by the host, so neither engine depends on the other. hue owns the state machine, chrome, keymap, and job scheduling.

Reference points

Borrowed	From
the viewer-not-editor data-browser shape	VisiData, csvlens
dialect sniffing by field-count consistency	Python csv.Sniffer (design, not code)
quoting/parsing baseline	RFC 4180 (+ the real-world deviations)
typed columns driving alignment & comparison	xsv/qsv, VisiData
per-column rainbow tint (raw view, deferred)	Rainbow CSV
cell-level CSV diff (deferred)	daff

Sequencing constraint: the table-rendering unification

Resolved. The table-rendering unification merged (2026-08-18: the two-view architecture — layout.d content-agnostic core / render.d string view / widgets.d widget view, with the md table case an adapter over buildTableWidgets; see docs/specs/core-cli/table.md §3.0). Phase 1 (D0/D1) shipped before the merge without touching sparkles.ui.components.table, as planned, and rebased over it cleanly — the DSV golden grids are byte-identical under the unified renderer. The CHK re-orientation executed (the audit lives in the Milestones section); the phase-1 no-table-edits invariant is retired: post-CHK work extends the unified component where a row calls for it.

Non-goals

• Editing. hue is a viewer; the bounded write surfaces belong to the diff spec (DST5). A projection (sort/filter/hide) never writes back.
• Multi-sheet workbooks (.xlsx, ODS) — a different acquisition problem.
• A query engine. The filter bar filters rows; it is not SQL, joins, or aggregation.

Content kind & dispatch (DSK)

ID	Requirement	Status	Traces to
DSK1	DSV must be a content kind like code/markdown/twoslash/diff (the dispatch-collapse doctrine, MOD8/MOD9): detected once by DocumentPipeline.load, dispatched through the same backend pick, rendered by every sink — no new modes.	full (3e18af9e)	document.d ContentKind.dsv; fromDsvSource
DSK2	Detection: the extensions .csv, .tsv, .psv (and .ssv) must select the kind and seed the dialect (DSD1); a .txt/extensionless/stdin source must be content-sniffed (DSD5); --dsv must force the kind (a detection input, like --markdown), and --raw must force highlighted source in every sink (CLI9).	full (3e18af9e)	detect extension family; CliParams --dsv; fromSource DSD5 gate
DSK3	A DSV document must render the grid preview by default in every sink — GUI, TUI, ANSI, HTML — from one shared model and one widget view (viewDsv), the exact shape of MOD8's markdown rule.	full (3e18af9e) — golden glyph-grid fixtures pin the shared layout (dsv_view.golden.*); ANSI/HTML verified by smoke	the ANSI/HTML case dsv: arms; PreviewModel.present routing; apps/hue/test/fixtures/dsv/
DSK4	--raw on a DSV file renders highlighted source like any text file; without a bundled DSV grammar it degrades to plain text per DEG2. Rainbow per-column raw styling is deferred (DSZ2).	full (3e18af9e)	the existing raw path (DEG2 plain-text degrade verified)
DSK5	The status chrome must name the resolved dialect (delimiter · quote · header on/off), the row counts (visible/total when projected), the projection state (sort keys, active filter, hidden-column count), and the copy mode (SEL7 doctrine).	full (8fcf9290) — dialect · rows · projection segments in the TUI header (the GUI has no status bar by design; its copy-mode flash covers SEL7)	dsvStatusNote + DsvBrowser.chromeNote in docNote

Dialect detection (DSD)

Precedence: flags > sniff > extension seed. All sniffing reads a bounded sample — the first 100 records or 256 KiB, whichever ends first (provisional; one constant, shared with width measurement DSN3).

ID	Requirement	Status	Traces to
DSD1	The extension must seed the delimiter (.csv → , · .tsv → tab · .psv → | · .ssv → ;), and a consistency sniff over the sample may override it within the candidate set {, ; \t |}: the winning delimiter maximizes field-count consistency across sample records (quote-aware), ties broken by the seed, then by set order. This is what catches the semicolon CSVs European Excel writes into .csv.	full (71757564) — hue wiring 3e18af9e	seedForExtension; sparkles.dsv.dialect.sniff; adaptDsv
DSD2	The quote character must be sniffed from {" '} — " unless ' pairs consistently and " does not; RFC 4180 doubled-quote escaping ("") applies to whichever wins. --dsv-quote forces it.	full (71757564) — --dsv-quote wiring 3e18af9e	sniff quote evidence; DsvFlags
DSD3	Header presence must be decided by heuristic under --dsv-header=auto (default): the first record is a header when its type profile differs from the body (body columns typed numeric/date while the first row is not) or when it is all-unique, non-numeric names; yes/no force it. A headerless file gets synthetic A B C… column names (display-only, never serialized into a copy).	full (71757564) — flag + synthetic names 3e18af9e	detectHeader; columnName (A B C…)
DSD4	--dsv-delimiter=<char>, --dsv-quote=<char>, --dsv-header=auto|yes|no must force any sniffed decision; a forced delimiter outside the candidate set is accepted verbatim (sniffing then only decides quote/header). All three are runtime-visible in the status chrome (DSK5).	full (3e18af9e)	CliParams.dsv* → DocumentPipeline → adaptDsv overrides; re-run header heuristic
DSD5	Content detection for .txt/extensionless/stdin: the source is DSV when the sniff finds ≥ 2 columns at ≥ 90% field-count consistency over ≥ 3 sample records (provisional thresholds); otherwise it stays plain text. This is a bounded first step toward DEF6 content-based detection, and the seam it should later flow through.	full (71757564) — stdin/.txt wiring 3e18af9e	SniffResult.looksDsv; contentLooksDsv; the fromSource gate
DSD6	Encoding: UTF-8 with an optional BOM (stripped before parsing, preserved by whole-document copy DSC4); records end at CRLF or LF (mixed accepted, each record remembers its own terminator for DSC4); malformed UTF-8 bytes display as replacement glyphs but round-trip exactly through copy (the raw span is authoritative).	partial (71757564) — the parse side ships; the copy consumers are DSC4	parseDsv (BOM, per-record Terminator, lone-CR literal)

Model & parser — sparkles:dsv (DSM)

ID	Requirement	Status	Traces to
DSM1	Parsing must implement RFC 4180 quoting — quoted fields, embedded delimiters, embedded newlines, doubled-quote escapes — plus the tolerated deviations: unquoted quotes mid-field are literal, a quote after field content re-enters quoted mode (Excel behavior), a final record without a terminator is a record.	full (71757564)	sparkles.dsv.parse.parseDsv
DSM2	Every cell must carry an identity channel: its half-open raw byte span in the borrowed source (quotes and escapes included) alongside its decoded text — the SEL1 discipline — so sub-cell selection, char-precise crossing (DSC3), and whole-document reproduction (DSC4) stay honest. Decoding is lazy/windowed; the model never copies the file.	full (71757564)	DsvCell.raw; decodeCell (simple cells borrow)
DSM3	Ragged rows must degrade, never error: a short record renders padded missing cells (distinct missing styling), a long record grows the grid to the maximum field count (overflow columns named …+1), and the row is marked ragged; the status chrome counts ragged rows. An empty file or header-only file renders an empty grid plus a note (DEG doctrine).	partial (71757564) — accounting, padding, …+N names and the ragged readout ship (3e18af9e); distinct missing styling and the empty-grid note pend (empty degrades to the plain view)	DsvDoc.raggedCount; buildTable padding/overflow
DSM4	Each column must carry an inferred type from the sample: int · float · date (ISO 8601) · bool · string, with empty cells excluded from inference; a column types as the most specific type ≥ 95% of sampled non-empty cells satisfy (provisional). Types drive alignment (DSG3), sort comparison (DSS2), and filter operators (DSF2) — never rendering of the value itself.	full (71757564)	classifyValue; inferColumnTypes (95% most-specific-first)
DSM5	The library surface is @safe pure nothrow @nogc: borrowed input spans, SmallBuffer-arena storage of plain-data offsets (no string fields), Expected errors, and chunk-bounded entry points (DSN5) that read no clock and touch no I/O — the sparkles:diff/sparkles:fuzzy doctrine. Benchmarked from the first commit.	partial (71757564) — attributes/borrowing/Expected/bench ship; chunked entry points await DSN5	libs/dsv package contract; bench.d
DSM6	Projection compute lives in the library: stable multi-key sort over typed comparators (DSS), typed constraint evaluation (DSF2), and the composed projection (a row-index permutation + column visibility/order) as a pure function of (model, spec) — deterministic, independent of chunking and enumeration order.	full (546107c6)	sparkles.dsv.project — applyProjection/compareTyped; determinism by index-tiebreak total order

The grid view (DSG)

DSG1/DSG3/DSG6 are phase-1 (they render through the existing md table path); DSG2, DSG4, and DSG5 need capabilities that path does not have (pinned bands, per-column caps + document-scale horizontal scroll, a row gutter) and are post-CHK — designed here, built against the unified table after the checkpoint.

ID	Requirement	Status	Traces to
DSG1	The grid must render as the MDP10 box-drawn table: rounded outer corners, │ separators, bold header + heavy ┝━┿━┥ rule, light inner row rules by default, one padding cell per separator — the same widget vocabulary, so GUI/TUI/HTML parity is by construction (MDP-T2 native glyphs in cells).	full (3e18af9e)	adaptDsv → the md table path (viewMarkdown)
DSG2	The header row and the column-rule must stay pinned while the document scrolls vertically (GUI/TUI: a non-scrolling band above the viewport; HTML: position: sticky), and the row-number gutter stays pinned horizontally — the COD6 pinned-gutter idiom at document scale.	full (43afbea8) — the component's pinned band (TableViewportSpec.pinHeader); wired for DSV in 47652402 (interactively on by default via the auto table clamp); HTML: position: sticky ships with the semantic table (DSG6)	buildTableWidgets freeze-pane emission (pinHeader = top-rows sugar); MdTableExtras.pinHeader; golden tall.txt
DSG3	Per-column alignment from type: numeric and date columns right-aligned, booleans centered, strings left — plus missing cells dimmed and ragged overflow tinted; the slots come from the theme like every widget (THM).	partial (3e18af9e) — typed alignment (+ decimal, 47652402) ships; the missing/ragged styling slots need a per-cell decoration channel the md funnel lacks — moved into D3	buildTable aligns; the pending cell-decoration seam
DSG4	A grid wider than the pane must scroll horizontally behind the frame with the border scrollbar (the COD6 bottom-border bar, ScrollView machine, wheel-sideways/Shift+wheel), the row-number gutter pinned; per-column width is capped (provisional: 64 cells) with overlong cell text ellipsized — the full value remains reachable by copy and (deferred) the cell peek DSZ3.	full (b6913019) — TBL7/TBL8 inherited through the shared table path; the 64-cell per-column cap wired in 47652402	TableViewportSpec; MdTableExtras.columnMaxWidth → TableProps.columnMaxWidths; golden wide.txt
DSG5	The row-number gutter numbers data records 1-based in source order (the header row unnumbered), and keeps showing source record numbers under any projection — a sorted/filtered view reveals provenance instead of renumbering (the VisiData reading). Gutter content is excluded from selection (SEL2).	full (47652402) — the stub column ships; its copy exclusion is DSC (D3)	adaptDsv gutter column; TableProps.headerCols heavy stub rule; frozen against h-scroll (freezeLeftColumns, golden wide-scrolled.txt); copy-excluded + selection-inert (090f05c6)
DSG6	Sink parity: non-interactive ANSI emits the whole grid once — the pager case: projection/viewport controls do not apply and a --table-max-lines clamp is ignored (a pipe can never scroll; the clamp stays honored for tables embedded in markdown documents). A grid wider than the pane clips behind the frame at offset 0 by default; --table-overflow wrap wraps cells to fit instead. HTML emits the semantic <table> (DSG6's other half) — static, so browser-side sort/filter is out of scope for the v1 emit.	full (3e18af9e) — the semantic <table> (+ sticky thead over the theme bg) ships via the shared MdDoc → HTML emitter (the --html arm was rerouted off the widget-HTML interpreter, which cannot lay out the table view's positioned stack — see open-issues); scope attributes pend in the shared emitter; the pager clamp-ignore landed with the UAT fixes on the D4 branch	the ANSI sink arm; the HTML dsv arm → renderMarkdownHtml
DSG7	Numeric and date columns should decimal-align (values aligned on the decimal point / a common tail), using the table core's decimal-pad solver — a capability the unification brought that a data grid should not leave unused.	full (47652402)	MdTableExtras.columnAligns Align.decimal → decimalPadsFor

Browser state (DSB)

The interactive tier is one projection value — sort keys + filter + column visibility/order — owned by a presentation-free state machine, applied by DSM6, painted by DSG.

ID	Requirement	Status	Traces to
DSB1	The projection must be a single regular value (ProjectionSpec): ordered sort keys with per-key direction, the filter AST, and the column order/visibility list. Every mutation route — header click, keymap, filter bar, header menu, columns palette — edits this one value; the machine is presentation-free and unit-tested (the TBL5 doctrine).	full (b2530077) — DsvBrowser + DsvProjection, applied by 8fcf9290	dsv_browser.d (pure machine); workspace.applyDsvBrowser
DSB2	Pristine is a state: a one-key reset returns to source order, no filter, all columns, and the chrome must make non-pristine visually unmissable (DSK5) — because copy semantics differ there (DSC4/DSC5).	full (8fcf9290) — Shift-R; the chrome carries the projection segments	DsvBrowser.reset; Command.dsvReset
DSB3	Column hide/reorder must be reachable from the keymap and from a columns palette — a picker-style list of columns with visibility toggles and move-up/down (DEF23 look, TRV list machinery); pointer drag-reorder of header cells is deferred (DSZ4).	partial (b2530077) — the state (toggle + last-visible guard) ships; the palette UI and its keymap route pend	DsvBrowser.toggleColumn/hiddenCols
DSB4	Projection changes on large documents run as background jobs (DSN5) with progress in the status chrome and cancellation on any newer projection edit; the visible grid stays interactive on the previous projection until the new one lands (no half-applied views).	not started — reprojection is synchronous by design until DSN5's chunked jobs (D5); fine at sample scales	the D5 job runner
DSB5	All browser keys route through hue's one binding table (KEY) so lantern lists them and CFG6 can rebind them.	partial (8fcf9290) — / and Shift-R live in the one binding table (lantern lists them); sort is pointer-only and the palette has no key yet	keymap.d dsvFilter/dsvReset + CtxFlag.hasDsvGrid

Sorting (DSS)

ID	Requirement	Status	Traces to
DSS1	Multi-key sort with explicit UI: header click (or the sort key on the focused column) cycles the column asc → desc → removed as the primary key, demoting existing keys; Shift+click (and a keymap chord) appends the column as the next key instead. The header cell shows each key's rank and direction (1▲, 2▼).	full (8fcf9290) — click-cycling, Shift-append, and the rank chrome ( ▲/ 1▲ 2▼ badges, chrome-clean: no source identity, never copied) ship in the TUI (c9d1a8e8); the GUI route pends with the GUI browser wiring	DsvBrowser.cycleSort; tui.dsvHeaderSortAt (+mods); MdTableExtras.headerBadges
DSS2	Comparison is typed per DSM4: numeric/date/bool keys compare by value, strings by Unicode-aware folded comparison; cells that fail the column's type (and missing cells) group after all typed values, comparing as strings among themselves — so a dirty column still sorts usefully.	full (546107c6)	compareTyped (typed + non-conforming grouping)
DSS3	The sort must be stable with the source record index as the final tiebreak — a total, deterministic order (the sparkles:fuzzy invariant-6 doctrine), independent of chunking and identical across sinks and platforms.	full (546107c6) — stability by construction: the data-index tiebreak makes the order total	applyProjection

Filtering (DSF)

Two surfaces, one truth: header menus compile into the same query AST the filter bar edits, so the bar always displays the whole active filter.

ID	Requirement	Status	Traces to
DSF1	The filter bar follows the picker's query doctrine (PKQ; the sparkles:fuzzy SPEC's lexing — whitespace tokens, quoting, \-escapes, ! negation): a query splits into column constraints plus a fuzzy remainder, parsing borrows the input and allocates nothing.	partial (8fcf9290) — the bar ships in the TUI (apply-on-Enter, not incremental); the GUI pends	the / prompt; parseFilterQuery on the PKQ lexing rules
DSF2	Column constraints are name:value forms with typed operators: name:text (case-folded contains), name:=text (exact), name:>n name:>=n name:<n name:<=n (typed numeric/date comparison), name: (empty cell), each negatable (!region:EU). name resolves case-insensitively against headers (quoted when it contains spaces); #3: addresses a column by 1-based index. Constraints AND; evaluation is sparkles:dsv's (DSM6).	full (b2530077)	parseFilterQuery (quoted names, #N, all operators, negation) → sparkles.dsv.project eval
DSF3	The fuzzy remainder must match rows by cell through sparkles:fuzzy (generalLanguage profile; a row admits when any visible cell admits the part), with matched positions highlighted in the grid (PKQ6 analog). The host ANDs constraint results with remainder admission — neither engine depends on the other.	partial (b2530077) — typo-tolerant per-part any-cell admission through sparkles:fuzzy ships; match-position highlighting pends on the cell-decoration seam	fuzzyRowMask; DsvProjection.rowMask
DSF4	Header menus offer the per-column quick filters — contains / equals / range / empty — as a popup on the header cell (popup machinery); applying one splices the corresponding constraint into the AST (visible in the bar, removable from either surface). Value-set enumeration (a distinct-values checklist) is deferred (DSZ5).	not started	header popup → AST splice
DSF5	Filtering must report: matched/total row counts in the status chrome, live-updating as background evaluation progresses (DSB4); an invalid query keeps the previous projection and surfaces the parse error inline in the bar — it never blanks the grid.	full (8fcf9290) — an invalid query keeps the previous projection and surfaces as a notice; the chrome counts visible/total rows	DsvBrowser.setFilter; dsvStatusNote rows segment

Selection & copy (DSC)

The grid is the TBL regime; these rows bind the DSV deltas. DSC1 is phase-1 by construction (inherited through the existing table path); DSC2–DSC5 are post-CHK — the serializer and crossing seams are exactly what the table unification may reshape.

ID	Requirement	Status	Traces to
DSC1	Smart drag, sub-cell/rect/row/column selection, Shift/Alt snapping, the whole-table copy button, and the TUI's OSC 52 route apply unchanged (TBL1–TBL6, TSL); the serializers stay presentation-free and testable (TBL5).	full (3e18af9e) — by construction through the md table path	table_select.d reuse (tsv/markdown formats)
DSC2	--table-copy grows a source mode: the selection re-emitted in the document's own dialect — the resolved delimiter/quote, minimal RFC 4180 quoting (quote only cells containing delimiter/quote/newline), records joined by the document's dominant terminator. source is the default for DSV documents (tsv stays the default for markdown tables); runtime-toggleable with the SEL7 indicator.	full (090f05c6)	TableCopyFormat.source; resolveTableCopy (--table-copy=auto); the GUI 3-way toggle; the TUI TSL5 wiring
DSC3	A text-regime drag that crosses the grid maps char-precise to raw source bytes via the identity channel (DSM2) — the TBL4 crossing rule with quoted cells resolving to their raw spans.	partial (090f05c6) — grid-regime copies resolve raw cell bytes via the re-parsed model, and the gutter is selection-inert (SEL2); the char-precise text-regime crossing still slices the decoded buffer	DsvCopy.rawCell; unkeyed/srcStart-stripped stub cells
DSC4	Whole-document reproduction (SEL8): in the pristine projection, selecting the whole document reproduces the input file byte-for-byte — BOM, quoting, mixed terminators, ragged rows and all.	full (090f05c6) — through the grid regime: a whole-grid source copy IS the input file (BOM/CRLF/ragged/unterminated-quote/no-trailing-newline pinned by tests); the text-regime SEL8 reproduction rides DSC3's remaining half	serializeGridCopy whole-grid shortcut; dsv_view.copy.byteExactHardCases
DSC5	Under a non-pristine projection, copying is WYSIWYG over the projection: grid-regime copies serialize the visible rows/columns in view order (all formats); DSC4's byte-reproduction guarantee is explicitly scoped to pristine, and the chrome's projection indicator (DSB2) is what tells the user which contract is live.	not started	serializers over the projection permutation

Scale (DSN)

The normative target is ~100 MB / 1M rows interactive — a data browser that dies at 1 MB isn't one. Correctness ships first; the machinery is staged post-CHK (provisional D5). DSN1 alone is phase-1: it is a property of the D0 model, not of the renderer.

ID	Requirement	Status	Traces to
DSN1	The model must hold offsets, not copies: the source stays one borrowed buffer; the record index and cell spans are arena-stored integers, so resident overhead is proportional to record count, not content size.	full (71757564)	DsvDoc span arenas over the borrowed source
DSN2	The record index (quote-aware record boundaries — the one inherently sequential pass) must build lazily/in the background; the sample-parsed head renders immediately and the scrollbar range grows as indexing progresses (the status chrome shows progress).	not started	background index job; DSB4 runner
DSN3	Column widths, dialect, types, and the header heuristic come from the bounded sample (DSD preamble) — never a whole-file scan; widths are stable thereafter (no reflow as later rows appear).	not started	sniff/width measurement over sample
DSN4	Rendering must be viewport-culled (RND1): only visible rows are decoded and laid out per frame; scroll cost must not grow with row count.	not started	viewDsv windowed materialization
DSN5	Sort/filter/index execution must be chunk-bounded pure calls (the sparkles:fuzzy searchChunk shape): the library exposes resumable cursors, hue schedules chunks on the event-horizon pool with cancellation between chunks; results are deterministic regardless of chunking (DSS3).	not started	sparkles.dsv.project cursors; DSB4
DSN6	Provisional budgets, to be calibrated with benchmarks: first grid paint of a 100 MB file ≤ 300 ms (sample-parsed head); full index of 1M rows in the background ≤ 2 s; sort of 1M indexed rows ≤ 1 s; filter keystroke-to-first-results ≤ 100 ms. A libs/dsv/bench/ harness pins them.	not started	libs/dsv/bench/

Deferred (DSZ)

ID	Requirement	Status	Traces to
DSZ1	DSV diff — hue diff a.csv b.csv as a cell-level diff (daff-style): row alignment via sparkles:diff, changed cells tinted in the grid through the preview-diff decoration channel (MDP21, DVN6).	not started	sparkles:diff + grid decorations
DSZ2	Rainbow raw view — per-column color in --raw (Rainbow CSV): either a bundled tree-sitter CSV grammar or a sparkles:dsv-driven highlight-event stream into the existing ANSI/HTML renderers.	not started	DSK4; highlight-event synthesis
DSZ3	Cell peek — a popup showing a truncated cell's full value (popup), with copy.	not started	DSG4 ellipsis sites
DSZ4	Pointer drag-reorder of header cells (the keymap/palette route DSB3 ships first).	not started	header drag machine
DSZ5	Distinct-values menu — a header-menu checklist of a column's value set with counts (needs a full-column scan; rides the DSN5 job runner).	not started	DSF4
DSZ6	Frozen data columns — pinning the first N data columns against horizontal scroll (the gutter idiom generalized).	not started	DSG4
DSZ7	Number/date display formatting (thousands separators, locale decimal comma detection for semicolon dialects) — display-only, never mutating copy fidelity.	not started	DSM4 inference notes

Milestones

Phase 1 runs in parallel with the table-rendering unification (separate worktree) and must not conflict with it: D0 is pure library work, and D1 consumes the existing md table rendering path as-is.

Milestone	Delivers	Gate
D0 ✅ (71757564)	Basic sparkles:dsv — dialect seed + sniff (DSD1–DSD4, DSD6), RFC parser + identity channel (DSM1–DSM3), typed columns (DSM4), offsets-not-copies storage (DSN1). No hue or sparkles:ui code touched.	unit-tested pure lib; adversarial fixtures (quotes-in-quotes, ragged, CRLF/BOM, huge cells); bench baseline
D1 ✅ (3e18af9e)	Basic hue preview — the content kind (DSK1–DSK4, DSK5's dialect readout) rendered in all four sinks through the existing md table rendering path: DsvDoc adapted onto the same table model viewMarkdown's table case consumes (DSG1/DSG3/DSG6), inheriting that path's current limits — no pinned header, no width caps / document h-scroll, no row gutter (DSG2/DSG4/DSG5 wait for CHK). The TBL selection/copy regime is inherited by construction (DSC1, tsv/markdown formats only).	golden frames GUI/TUI/ANSI/HTML; DSD5 stdin/txt sniff; zero edits under sparkles.ui.components.table (the parallel-work invariant)

CHK — re-orientation checkpoint: ✅ executed 2026-08-18. Both entry conditions were met (D0 + D1 shipped; the unification merged) and the branch rebased over the unified pipeline with no adapter changes needed — the D1 MdDoc synthesis feeds the md table case, which is now itself an adapter over buildTableWidgets, and the DSV goldens are byte-identical. The row audit:

Row	Verdict at CHK
DSG4	Largely satisfied by TBL7/TBL8: the framed viewport (pinned frame, border scrollbars, wheel/bar routing, --table-overflow/--table-max-lines) is inherited through the shared table path. Residual: the DSV per-column cap default and its columnMaxWidths wiring.
DSG2	Changed shape: the viewport pins the frame while the whole interior (header included) scrolls as one offset — the pinned header becomes a TableViewportSpec pinned-header-rows extension of the component (now permitted), not a host-painted band.
DSG5	Changed shape: the core models row headers (TableProps.headerCols stub columns) — the record-number gutter becomes a stub column with a copy-exclusion delta, not host chrome.
DSG3	Unchanged owner; easier now (TableWidgetStyle semantic slots are the styling seam for missing/ragged).
DSG7 (new)	The core's decimal-pad solver makes numeric decimal alignment cheap — added to the D2 scope.
DSC2–DSC5	Unchanged shape: the srcStart/srcEnd identity channel survives wrapping in the widget view, the serializers (table_select.d) and the TBL regime are intact; the decoded-buffer vs raw-bytes reconciliation (Document.dsvText) is still the D3 substance.
DSB/DSS/DSF/DSN	Untouched by the unification; sketches carried forward as-is.

The milestones below are normative as re-planned at CHK.

Milestone	Delivers (re-planned at CHK)	Gate
D2 ✅ (47652402)	The full grid on the unified table: the pinned-header viewport extension (DSG2 — TableViewportSpec grows pinned header rows), the headerCols record-number stub column (DSG5), the DSV per-column cap default via columnMaxWidths (DSG4 residual), decimal alignment (DSG7), and DSV fixtures for the TBL7/TBL8 viewport goldens	goldens for wide/tall DSV grids incl. the pinned header; component changes land with their own table.md §3.0 updates
D3 ✅ (090f05c6)	DSV copy deltas (DSC2–DSC5): the source dialect format (default for DSV) + stub-column exclusion, char-precise crossing via the raw-span reconciliation, pristine byte reproduction; the missing/ragged styling slots (DSG3/DSM3 completion — the per-cell decoration channel composes with the selection work)	round-trip: whole-doc copy == input bytes; gui.md TBL2/TBL6 + tui.md TSL5 gain source
D4 ◐ core shipped (546107c6–8fcf9290)	The browser (DSB/DSS/DSF): projection machine, multi-key sort, filter bar + header menus, columns palette; carried forward from D3: the char-precise text-regime raw crossing (DSC3) and the missing/ragged styling slots (DSG3/DSM3)	machine unit tests; determinism (DSS3); keymap in lantern
D5	Scale (DSN2–DSN6): background index, chunked jobs, the 100 MB / 1M-row budgets pinned by libs/dsv/bench/	DSN6 budgets pinned; a 100 MB corpus scenario

Cross-references (threading status)

Threaded with this spec: feature-requirements.mdCLI26 (the --dsv family), MOD10 (DSV as a content kind), DEF28 (the roadmap row), and CLI11's planned-source note; index.md's Documentation-map row, ID-scheme mnemonics, and mode-map note; the gui.md TBL preamble pointer.

Threaded with D0 (71757564): the AGENTS.md sparkles:dsv sub-package row; the module-coverage table below. Threaded with D3 (090f05c6): gui.md TBL2/TBL6 and tui.md TSL5 gained source.

Landing with later code, not before:

• The apps/hue module-coverage rows here and in feature-requirements.md — no hue DSV files exist yet (D1).
• tui.md: a DSV-T parity area (by construction via TSF3, like MDP-T/DIF-T) once rows exist to bind.

Module coverage (sparkles:dsv)

Source	Key symbols	Requirements
libs/dsv/src/sparkles/dsv/model.d	Span, Dialect, DsvCell/DsvRecord/DsvDoc, decodeCell, classifyValue, inferColumnTypes	DSM2–DSM4, DSN1
libs/dsv/src/sparkles/dsv/parse.d	parseDsv (tolerant RFC 4180, BOM/terminators, modal/ragged accounting)	DSM1, DSM3, DSD6
libs/dsv/src/sparkles/dsv/dialect.d	seedForExtension, sniff, detectHeader, SniffResult.looksDsv	DSD1–DSD3, DSD5
libs/dsv/src/sparkles/dsv/bench.d	parse (1k/10k rows) + sniff --bench baselines	DSM5 (the benchmarked-from-D0 half)
libs/dsv/dub.sdl	library/unittest configurations (base-only dependency)	DSM5

→ Feature requirements · GUI · TUI · Picker · Overview