Serde Expressiveness

A breadth-first survey of the most expressive (de)serialization frameworks across languages — TypeScript, Rust, Python, Scala, Haskell, OCaml — to inform the redesign of sparkles:wired as a state-of-the-art serde framework in expressive power (it already competes on JSON performance), and to ground the plan of re-expressing CLI argument parsing as a bidirectional wired format (argv ⇄ struct).

This survey answers five questions:

1. What can the strongest schema systems express — the full capability surface of Effect Schema, serde + its satellite ecosystem, Pydantic v2, and zio-schema, axis by axis? See the catalog and each deep-dive's analysis spine.
2. What does bidirectionality actually require — how do one-declaration/ two-directions designs work (profunctor codecs, partial isomorphisms, flipped-AST interpretation in Effect), and which laws can honestly be promised? See concepts § round-trip laws.
3. Where does each declaration style hit its ceiling — attributes/derives (serde, OCaml ppx, circe/aeson), value-level combinators (Haskell codecs), reified schemas (zio-schema, facet)? See concepts § the three tiers and comparison.
4. Can argv be a serde format, bidirectionally — what did clap, serde_args, facet-args, and optparse-applicative prove about parsing, and why has nobody shipped the render half? See argv as a codec.
5. What must sparkles:wired add — measured against a probe-verified baseline, which capabilities close the gap, which are novel-to-D, and which API decisions are not retrofittable? See the delta table.

Scope note. This is a survey of expressiveness, not performance — throughput comparisons appear only where they price a design choice (e.g. facet's reflection tax, tomland's interpretation-vs-codegen benchmark). The Effect Schema deep-dive was grounded in a local checkout of the v4 "smol" rewrite (pinned commit 3a1128c7); the wired baseline is grounded in runnable probes CI compiles and runs. The design decisions this survey feeds land in docs/specs/wired/ revisions, not here.

Last reviewed: August 9, 2026

---

Master Catalog

One row per surveyed subject. Tier is the three-tier classification by what artifact holds the structural knowledge (1 = codegen-only, 2 = value-level codec, 3 = reified schema). Declaration surface is what the user writes.

Subject	Language(s)	Tier	Declaration surface	Distinguishing capability	Link
Effect Schema (v4 "smol")	TypeScript	2/3	value-level combinators over a reified AST	one parser run in two directions (flip); annotation-driven derivations	effect-schema.md
serde (+ serde_with, schemars)	Rust	1	#[derive] + attributes	the universal-data-model architecture and its documented walls	serde.md
facet	Rust	3	one derive → runtime Shape value	reflection-as-data: one derive, N consumers	facet.md
Pydantic v2	Python	3	class annotations compiled to a core schema	wrap-hook middleware; presence provenance; callable discriminators	pydantic.md
msgspec & cattrs	Python	1/3	annotated structs / converter objects	strict schema-guided decode; policy-in-the-converter	msgspec-cattrs.md
zio-schema	Scala	3	derived Schema[A] value	migrations, diff/patch, DynamicValue — the full tier-3 payoff	zio-schema.md
autodocodec, tomland & unjson	Haskell	2	hand-written profunctor codecs	self-documenting codecs; path-anchored accumulation; update	haskell-codecs.md
circe & aeson	Scala / Haskell	1	derived instances + options	the derivation-only ceiling, precisely mapped	circe-aeson.md
Invertible syntax descriptions	theory	2	partial-iso combinators	why codecs can't be monads; round-trip laws modulo equivalence	invertible-syntax.md
OCaml ppx & ATD	OCaml	1/3	[@@deriving] attributes / external .atd	the ?/~ field trichotomy; atddiff; a shipping D backend	ocaml-atd.md
argv as a codec	cross-language	—	clap/optparse/facet-args/serde_args	why parse-only is the universal state of the art	argv-codecs.md
sparkles:wired (baseline)	D	1+	structs + @Wire*!Format UDAs	format-generic CTFE policy table; probe-verified gaps	wired-baseline.md

The analysis spine

Every deep-dive analyses its subject through the same six dimensions, each closed with a D verdict carrying a feasibility tag:

1. Schema model & bidirectionality — what artifact holds the structure; how the two directions derive from it; round-trip guarantees.
2. Naming, optionality & defaults — renames, case conventions, aliases; the presence problem.
3. Sum types & discrimination — representations, discriminators, ambiguity handling.
4. Transformations & validation — converters/adapters, failure channels, refinements, cross-field rules.
5. Errors & context — error structure, paths, accumulation; context threading through the codec.
6. Metadata, derivations & extensibility — annotations beyond wire concerns; the artifacts one declaration yields; format genericity and unknown-field policy.

Taxonomy

By tier

Tier	Subjects	What the survey takes from them
1 — codegen-only	serde, circe & aeson, OCaml ppx, wired today	the attribute vocabulary; the precisely-mapped ceiling
2 — value-level codec	Haskell codecs, invertible syntax, Effect Schema	non-drift by construction; docs/schema from the codec value
3 — reified schema	zio-schema, facet, Pydantic, ATD	N formats, migrations, diffing, generators from one artifact

By capability first proven there

Capability	Proven by
Docs/schema that cannot drift from the codec	autodocodec
Migration algebra over a typed dynamic value	zio-schema
Inferred union sentinels	Effect Schema
Callable discriminators	Pydantic
Wrap-hook codec middleware	Pydantic
Type-shaped adapter composition	serde_with
Presence provenance (exclude_unset)	Pydantic
Path-anchored accumulation + usable partial value	unjson
update/PATCH merge semantics	unjson
Reflection artifact feeding N consumers	facet, zio-schema
Inspectable parser driving help/completions	optparse-applicative

Milestones

When	Event
2008	Boomerang (POPL '08): string lenses with static round-trip guarantees (invertible syntax)
2010	Rendel & Ostermann, Invertible Syntax Descriptions — the parse/print duality theory
2017	serde 1.0: the universal-data-model derive architecture stabilizes
2019	tomland: the profunctor-codec pattern stated cleanly for TOML
2021	autodocodec (self-documenting codecs) and zio-schema (reified Schema[A]) land the tier-2/3 answers
Jun 2023	Pydantic v2: the schema layer reified and compiled to Rust (pydantic-core)
~2023	ATD 2.13 ships atdd — a D backend for the schema-IDL-first approach
Feb 2025	eserde: the Rust ecosystem retrofits error accumulation onto serde
2025	facet: reflection-as-data lands in Rust — one derive, N consumers, at a measured runtime cost
2025–2026	Effect Schema v4 "smol" rewrite: transformations/checks become first-class values; canonical codecs (surveyed at commit 3a1128c7)

Suggested reading paths

• "I'm designing the wired upgrade (S2)" — wired-baseline → comparison § delta table → effect-schema → serde → zio-schema.
• "I'm designing the Cli format" — argv-codecs → effect-schema (string-tree IR, optionality matrix) → invertible-syntax (which round-trip law to promise) → wired-baseline § the bespoke engine.
• "Give me the theory" — concepts → invertible-syntax → haskell-codecs → zio-schema.

Sources

Each deep-dive carries its own primary-source block (official docs, pinned source links, issues, papers). Cross-cutting grounding:

• The Effect Schema deep-dive reads the 7 272-line SCHEMA.md and the schema source of a local effect-smol checkout, pinned at commit 3a1128c7684e04d34d9f541f77adaac38a513056.
• The wired baseline embeds runnable probes (compiled and run by ci --verify) for its load-bearing claims.
• The survey's synthesis and delta live in comparison.md; shared vocabulary in concepts.md.