Quartz v5.25

Overnight Handoff — Binary DSL Phase 1.4 (real codegen)

Baseline: 112ff7fa on trunk (Phase 1.1+1.2+1.3 shipped + discoveries/handoff) Design doc (canonical): docs/design/BINARY_DSL.md — 335 lines, 12 locked decisions Prior handoff (READ FIRST): docs/handoff/overnight-binary-dsl-phase-1.md — contains D1-D5 discoveries and the 6-step Phase 1.4 plan Session goal: Ship real codegen for .encode() / .decode() / as / .with {}. End the session with all 5 worked examples from BINARY_DSL.md roundtripping correctly.


Copy-paste handoff prompt (paste this into a fresh session)

Read `docs/handoff/overnight-binary-dsl-phase-1-4.md` AND
`docs/handoff/overnight-binary-dsl-phase-1.md` (the D1-D5 discoveries
in the latter are load-bearing — you will hit them). The canonical
design is `docs/design/BINARY_DSL.md`. Do not re-litigate — implement
what's specified.

Starting state (verified at handoff):
- Trunk clean at 112ff7fa. Guard stamp valid. Smoke green.
- 43 binary-DSL tests green across parse (14) / typecheck (19) / MIR (10).
- Fixpoint 2051 functions (gen1 == gen2 byte-identical).
- Session backup pre-Phase-1: self-hosted/bin/backups/quartz-pre-binary-dsl-golden.
  This is the escape hatch if 1.4 blows up past fixpoint recovery.

SAVE a NEW fix-specific backup BEFORE you touch a single .qz:
  cp self-hosted/bin/quartz self-hosted/bin/backups/quartz-pre-binary-codegen-golden

NEVER overwrite quartz-pre-binary-codegen-golden until 1.4 is committed
AND all 5 roundtrip tests pass. If a rebuild goes bad, restore from it.
The rolling quartz-golden that quake guard manages WILL get overwritten
on every successful build, which is why you need the fix-specific copy.

Target: complete sub-phase 1.4 — real codegen for the three MIR
opcodes registered in 1.3, method synthesis so .encode/.decode/as are
callable from user code, .with {} postfix parse for packed structs,
roundtrip coverage for all 5 worked examples. Estimated 600-800 lines
spanning 6 steps (see the prior handoff for the detailed sequence).

Workflow per step:
1. Write QSpec tests FIRST (red phase). They should fail in a specific
   way — IR doesn't contain the expected shift/mask, or exit code is
   wrong, etc. Not "crashes."
2. Implement the minimum to turn them green.
3. Run quake guard before EVERY commit. Never skip. Never --no-verify.
4. Smoke tests after every guard — brainfuck + style_demo + expr_eval.
   If any regresses, restore from quartz-pre-binary-codegen-golden.
5. Commit each step as a single coherent commit.

Sub-phase 1.4 — 6 steps (from the prior handoff, in order):

STEP 1 — Method resolution. Add a resolver tag for binary/packed types
so the type name resolves and .encode/.decode can be looked up. See
D5 in the prior handoff: NODE_BINARY_BLOCK currently REPLACES the
type alias rather than augmenting it, so the type name is unregistered
downstream. Fix that. ~100 lines in resolver.qz + typecheck.qz. One
commit.

STEP 2 — Method synthesis. In typecheck_walk Phase 2, when a
NODE_BINARY_BLOCK or NODE_PACKED_STRUCT is seen, register synthesised
method signatures:
  - binary {}:  decode(bytes: Bytes): Result<T, ParseError>
                encode(self: T): Bytes
  - packed struct(uN):
                as (operator) between T and uN
                with(self: T, updates: ...): T — block-form
Signatures only — bodies come in STEP 4. One commit.

STEP 3 — MIR lowering. In mir_lower_expr_handlers.qz, when the
callee resolves to a synthesised encode/decode method, emit
MIR_BINARY_PACK / MIR_BINARY_UNPACK with the layout id as operand1.
For `as` on packed structs: emit MIR_PACKED_BITCAST. For .with {}:
lower to a sequence of MIR_PACKED_BITCAST + shift/mask/or ops on the
backing integer. One commit. Uses the registry set up in 1.3.

STEP 4 — Real codegen in cg_intrinsic_binary.qz. Replace the stub
dispatcher branch in codegen_instr.qz. The real emitter walks the
layout field list and:
  - coalesces adjacent byte-aligned fields into single loads
    (u32be+u32be → one i64 load + bswap + two extracts)
  - emits shift/mask/or sequences for sub-byte fields
  - constructs ParseError on bounds failures
  - handles variable-width fields (bytes, cstring, pstring(uN),
    array forms)
Use `Bytes` (std/bytes.qz) as the buffer type. Phase 9 of the design
is zero-copy via borrowed/owned flag — respect that when emitting
rest-of-stream `bytes` and `[T]`. This is the biggest step. Plan for
multiple commits within it if it runs long. See BINARY_DSL.md L244-253.

STEP 5 — .with {} postfix parser. `moder.with { pin5_mode = 0b01;
pin6_mode = 0b01 }`. Hook ps_parse_postfix after TOK_DOT: if the
method name is "with" and the next token is TOK_LBRACE, parse inner
`field = expr` pairs into a new NODE_BINARY_WITH (96) with children
being NODE_ASSIGN-shaped entries. Gate at typecheck: receiver must
be a packed-struct type, field names must exist. ~150 lines. One
commit.

STEP 6 — Roundtrip QSpec. spec/qspec/binary_roundtrip_spec.qz —
encode then decode for all 5 worked examples from BINARY_DSL.md
(IPv4, BleAdvHeader, GpioModer, PngIhdr, IntMapHeader), assert
equality field-by-field. This is sub-phase 1.5 by the naming but
unlocks as soon as 1.4 lands. One commit (or bundle into STEP 4 if
the tests were already written there).

Prime Directives v2 compact (same as prior session):
1. Pick highest-impact, not easiest.
2. Design is locked (BINARY_DSL.md) — implement, don't redesign.
3. Pragmatism = sequencing correctly. Shortcut = wrong thing because
   right is hard. Name the path.
4. Work spans sessions. Don't compromise design because context is
   ending. The handoff discipline is the load-bearing piece.
5. Report reality. Partial = say partial. "Should work" = a lie.
6. Holes get filled or filed (Discoveries section of the prior
   handoff doc, or a new Discoveries section in this one).
7. Delete freely. Pre-launch.
8. Binary discipline: guard mandatory, fixpoint + smoke not optional,
   fix-specific backups before risky work.
9. Quartz-time = traditional ÷ 4.
10. Corrections = calibration, not conflict.

Stop conditions:
- STEP 6 complete with all roundtrip tests green and fixpoint stable
  → done. Write Phase 1.5+ handoff (computed fields, discriminated
  unions, dogfood intmap header — see BINARY_DSL.md Phase 2-3).
- Get blocked on a real compiler bug → file in this doc's Discoveries
  section with minimal repro, commit what works, write partial-done
  handoff for next session.
- Approaching context limit → stop at the next clean commit boundary,
  write handoff covering what landed and what's left.

Helpful pointers (verified in session):
- _tc_bin_parse_numeric_width in typecheck.qz already parses
  u<N>[le|be] / i<N>[le|be] / f<N>[le|be] — reuse it in codegen.
- mir_binary_layout_* accessors in mir.qz give (name, kind, backing,
  field name/spec/width). Codegen consumes these.
- The `; === Binary DSL Layouts ===` manifest emitted by codegen.qz
  stays useful — keep it. Tests in binary_mir_spec.qz depend on it.
- cg_emit_instr prog: Int is existential — see D3. If you end up
  needing MirProgram inside the real emitter, add a typed overload
  at the top of codegen_instr.qz and forward. Don't sprinkle as_type
  casts everywhere.
- std/binary.qz already defines ParseError with UnexpectedEof /
  InvalidValue(field, expected, got) / LengthOverflow(field, declared,
  remaining). Use those exact variants — the 1.2 typecheck tests
  import and pattern-match against them.

If you finish before context runs out: consider Phase 3 dogfood
(migrate self-hosted/backend/cg_intrinsic_intmap.qz's manual
getelementptr loads to IntMapHeader.decode()). But only after 1.4
ships and roundtrip is green — dogfooding on wobbly codegen is a
trap.

For the caller (outside the prompt)

Recommended invocation: open a fresh Claude Code session, paste the block above, let it run overnight. When it returns:

  1. git log --oneline trunk — confirm STEP 1-6 commits (or partial).
  2. Run ./self-hosted/bin/quake qspec in a terminal (NOT from Claude Code) to confirm no cross-spec regressions.
  3. Read the Discoveries section that the session appended — new quirks to remember.
  4. If the session ended partial, paste the new handoff prompt from wherever it was left.