Quartz v5.25

Defer Parser Extension — Design Doc

Status: Proposal Author: Research agent (overnight sprint, Apr 12 2026) Target file: self-hosted/frontend/parser.qz (+ small MIR consistency fix) Driving failure: spec/qspec/cross_defer_safety_spec.qz:128 — “defer in for loop executes each iteration”


1. Problem Statement

Quartz’s defer keyword currently only accepts expressions (plus a narrow special case for a bare ident = expr assignment). Writing the natural form

def main(): Int
  var count = 0
  for i in 0..3
    defer count += 1
  end
  return count
end

fails with Expected expression at the += token. This is because compound assignment (+=, -=, *=, /=, %=, ^=, &=, |=, <<=, >>=) is a statement-level construct in Quartz, parsed by ps_parse_assign_or_call — not by ps_parse_expr. The current defer path falls through to ps_parse_expr, which has no rule for +=.

This is table-stakes functionality for any language with defer. Every production use of defer in a loop wants to mutate a counter, release a lock, free a resource, etc.; the operations naturally take statement form, not expression form.


2. Current Quartz Implementation

2.1 Parser — ps_parse_stmt defer branch

self-hosted/frontend/parser.qz:5090-5103

if ps_current_type(ps) == token_constants::TOK_DEFER
  var ln = ps_current_line(ps)
  var cl = ps_current_col(ps)
  ps_advance(ps)
  # Check if the defer body is an assignment (ident = expr) or compound assign
  var expr = 0
  if ps_current_type(ps) == token_constants::TOK_IDENT and ps_peek_next_type(ps) == token_constants::TOK_ASSIGN
    expr = ps_parse_assign_or_call(ps)
  else
    # Parse the expression to defer (typically a function call)
    expr = ps_parse_expr(ps)
  end
  return ast::ast_defer(s, expr, ln, cl)
end

Observations:

  1. The comment says “or compound assign” but the code only triggers on TOK_IDENT followed by TOK_ASSIGN — plain =. += is TOK_PLUS_EQ; the branch never fires for compound assigns.
  2. Even the plain-assign branch only triggers for a bare identifier LHS. obj.field += 1 and arr[i] += 1 both miss.
  3. The fallback ps_parse_expr cannot parse any statement-level assignment form.
  4. ps_parse_assign_or_call (same file, line 4836) already handles every statement form we care about: identifier/field/index assignment, all compound assignments, and bare function/method calls (wrapped as NODE_EXPR_STMT). It is the canonical statement-dispatcher and it’s sitting right there.

2.2 AST — ast_defer

self-hosted/frontend/ast.qz:2105-2120

def ast_defer(s: AstStorage, expr: Int, line: Int, col: Int): Int
  ...
  s.kinds.push(61)  # NODE_DEFER = 61
  ...
  s.lefts.push(expr)  # left = deferred expression

The AST layer is already storage-agnostic: it just records whatever node handle you hand it in left. It doesn’t care whether that handle is an expression, a NODE_ASSIGN, a NODE_EXPR_STMT, or a NODE_BLOCK.

2.3 Typecheck — tc_stmt defer branch

self-hosted/middle/typecheck_walk.qz:2823-2828

elsif node_kind == node_constants::NODE_DEFER
  # Type check the deferred expression
  var expr = ast::ast_get_left(ast_storage, node)
  if expr >= 0
    tc_expr(tc, ast_storage, expr)
  end

This calls tc_expr, not tc_stmt. Statement-level nodes like NODE_ASSIGN, NODE_EXPR_STMT, and NODE_BLOCK either won’t be handled at all by tc_expr, or will be mishandled. This needs to change to tc_stmt for the extension to typecheck correctly.

2.4 Liveness — liveness_walk defer branch

self-hosted/middle/liveness.qz:612-619

if kind == node_constants::NODE_DEFER
  var body = ast::ast_get_left(s, node)
  if body >= 0
    liveness_walk(info, s, body)
  end
  return
end

Already agnostic — liveness_walk handles all node kinds. No change needed.

2.5 MIR Lowering — NODE_DEFER and defer emission

self-hosted/backend/mir_lower.qz:2513-2519

if kind == node_constants::NODE_DEFER
  var expr_node = ast::ast_get_left(s, node)
  push_defer(ctx, s, expr_node)
  return
end

push_defer is a pure record — it saves (ast_store, node) to the defer stack without caring what kind of node it is.

The emission sites are where it gets interesting. There are three emission paths, and they are not consistent:

FunctionLineLowering call
emit_deferred_to_scope (scope pop)mir_lower.qz:205ctx.mir_lower_stmt(...)
emit_deferred (function return)mir_lower.qz:223ctx.mir_lower_stmt(...)
emit_deferred_to_depth (break/continue)mir_lower.qz:242ctx.mir_lower_expr(...)

The break/continue path is broken for statement-shaped defer bodies. If we let statements flow into defer, break inside a for loop that has defer count += 1 above it will hit mir_lower_expr on a NODE_ASSIGN or NODE_EXPR_STMT — which is the wrong dispatcher. This is a latent bug that is currently masked by the parser only accepting bare ident = expr, which happens to dispatch in mir_lower_expr via the expression-form assignment handler.

This must be fixed in the same commit as the parser extension. Make all three paths use mir_lower_stmt.

2.6 Test Failure Reference

spec/qspec/cross_defer_safety_spec.qz:128-138

it("defer in for loop executes each iteration") do ->
  assert_run_exits("""
    def main(): Int
      var count = 0
      for i in 0..3
        defer count += 1
      end
      return count
    end
    """, 3)
end

Fails at parse time: Expected expression at +=.

2.7 Summary of Layer-by-Layer Impact

LayerCurrent stateChange required
ParserHard-codes IDENT = special case, falls back to ps_parse_exprMajor: delegate to ps_parse_assign_or_call (+ block form, see Recommendation)
ASTStorage-agnosticNone
TypecheckCalls tc_expr on bodySmall: call tc_stmt instead
LivenessAlready walks any nodeNone
MIR lowering — pushStorage-agnosticNone
MIR lowering — emit (scope pop, fn return)mir_lower_stmtNone
MIR lowering — emit (break/continue)mir_lower_expr (latent bug)Small: change to mir_lower_stmt
CodegenAST-shape-agnostic via MIRNone

3. External Research

3.1 Go — defer takes a call expression

Go’s specification is terse:

DeferStmt = "defer" Expression . The expression must be a function or method call; it cannot be parenthesized.

Source: The Go Programming Language Specification (Defer statements section).

Go deliberately restricts defer to call-form only. If you want to run a compound statement, you wrap it in a func() { ... }() literal:

defer func() {
    count++
    cleanup(resource)
}()

This is widely acknowledged as a wart. Every Go tutorial teaches the defer func() { ... }() pattern as a required workaround. Community articles like Golang Defer: From Basic To Traps and Understanding defer in Go (DigitalOcean) all document the closure workaround as idiomatic.

Takeaway: Go’s model is the minimum shippable design. It works, but everyone ends up writing anonymous functions to get past its limitations. We should do better.

3.2 Zig — defer takes a statement or block

From the Zig Language Reference and zig.guide - Defer:

defer std.debug.print("3 ", .{});
defer {
    std.debug.print("2 ", .{});
    std.debug.print("1 ", .{});
}

Zig’s grammar places defer in front of a general statement, and statements include block statements ({ ... }). This lets you defer a single call, an assignment, or a multi-statement block, uniformly. There’s no wrapping-in-a-closure dance. See also Zig defer Patterns (Ziggit) for real-world usage — block-form defers for paired resource acquisition/release are idiomatic.

Takeaway: Zig gets this right. defer <stmt> where <stmt> includes the block form is the cleanest possible design.

3.3 Swift — defer takes only a block

From the Swift Language Reference — Statements:

defer {
    // code that runs at scope exit
}

Swift is stricter than Zig in that defer requires a brace block — you can’t write defer cleanup() without braces. It’s slightly more verbose for the one-liner case but completely unambiguous. Hacking with Swift — The defer keyword and NSHipster — guard & defer both document this as the only form.

Takeaway: Swift’s model is Zig’s without the single-statement shortcut. Clean, but forces a block even for single calls, which is a minor ergonomic regression.

3.4 D — scope(exit) takes a NonEmptyOrScopeBlockStatement

From the D Programming Language Specification — Statements:

ScopeGuardStatement:
    scope ( exit )    NonEmptyOrScopeBlockStatement
    scope ( success ) NonEmptyOrScopeBlockStatement
    scope ( failure ) NonEmptyOrScopeBlockStatement
scope(exit) writeln("cleanup");
scope(exit) {
    count++;
    free(resource);
}

D splits into three modes (exit / success / failure) and accepts either a single non-empty statement OR a block. Same flexibility as Zig. D’s model is arguably the richest — the failure/success discrimination is very nice for exception-safety patterns. Quartz currently doesn’t have exceptions, so the distinction doesn’t apply.

Takeaway: D’s grammar shape for the body (single statement OR block) matches Zig. The exit/success/failure trichotomy is an orthogonal extension we could consider later if Quartz adds exception-like error flow, but it’s out of scope here.

3.5 Synthesis

LanguageBody grammarOne-linerBlockNotes
GoExpression (call only)Call onlyNo (must wrap in func(){}())Widely regarded as a wart
ZigStatement (incl. block)Any stmt{ ... }Cleanest
SwiftBlock onlyNo{ ... } requiredSlightly verbose
DNon-empty or block statementAny non-empty stmtYesRichest (exit/success/failure)

Zig is the right model for Quartz. It has the best ergonomics (both one-liner and block work), no syntactic tax for the simple case, and it’s the most battle-tested of the “statement defer” designs. D confirms the same shape works at scale.


4. Design Options

Option A — Minimum fix: accept compound assignments

Replace the IDENT = special case with a broader peek that also recognizes TOK_PLUS_EQ etc., and route those to ps_parse_assign_or_call.

Pros: Smallest diff. Unblocks the failing test. Cons:

  • Still can’t defer obj.field = x, defer arr[i] += 1, defer obj.method(), or defer free_all(a, b, c) without fragile peek logic.
  • Every new statement form needs a new special case in the defer branch — duplicating ps_parse_assign_or_call’s dispatch logic.
  • Violates Prime Directive 1 (pick the highest-impact work, not the easiest). This is the classic “5-line fix when a 500-line fix is available” trap — except here the “bigger” fix is only ~30 lines and strictly better in every dimension.

Verdict: Rejected. It’s the shortcut, not the right answer.

Option B — Medium fix: accept any statement-shaped form (single stmt)

Delete the peek special case entirely and always delegate to ps_parse_assign_or_call. That function already handles:

  • Bare call expressions (wrapped as NODE_EXPR_STMT)
  • Assignment (x = e, obj.f = e, arr[i] = e)
  • Compound assignment (all ten operators, all three LHS forms)

Pros:

  • Solves the actual problem. All single-statement forms work uniformly.
  • Negative diff — delete code, gain functionality.
  • Matches Go’s one-line defer ergonomics without Go’s call-only restriction.

Cons:

  • Still can’t group multiple operations without a helper function. Real-world “cleanup that needs to run at scope exit” often wants ≥2 steps (e.g., unlock-then-notify, reset-state-and-decrement-counter).

Verdict: Necessary but not sufficient. We do this and Option C in one commit.

Option C — World-class fix: accept statement OR do ... end block

Delegate single-statement form to ps_parse_assign_or_call (Option B), AND accept a do ... end block when the next token after defer is TOK_DO:

defer count += 1                    # single statement
defer cleanup(resource)             # single call
defer obj.release()                 # method call
defer do
  count += 1
  log("iter done")
  resource.close()
end                                 # block

This matches Zig’s model exactly (single stmt OR block), using Quartz-native syntax (do ... end instead of { ... }). The block form reuses ps_parse_block which is already the workhorse for function bodies, if bodies, loop bodies, etc.

Pros:

  • Full Zig parity. No Go-style closure workarounds, no Swift-style always-a-block tax.
  • Reuses existing parser machinery. ps_parse_block is already rock-solid — used by ~10 callers in parser.qz.
  • AST/typecheck/MIR need no new node kinds. NODE_BLOCK already exists and is handled by tc_stmt and mir_lower_stmt. We just hand the block handle to ast_defer and it flows through.
  • Fixes a latent MIR bug (the break/continue emit path using mir_lower_expr) in the same commit — this is a hole we’d otherwise have to file separately.
  • Fix is small. ~30 lines in parser, 1 line in typecheck, 1 line in MIR, no AST changes, no new node kinds, no codegen changes.

Cons:

  • Slightly more parser logic than Option B alone. But still <10 extra lines versus B.

Verdict: Recommended. This is the full correct answer. It’s only marginally more work than Option B and is strictly better. Per Prime Directive 8 (design the full correct solution before building), this is the shape to ship.


Option C — statement OR do ... end block, modeled on Zig.

5.1 Files Changed

  1. self-hosted/frontend/parser.qz — replace the TOK_DEFER branch in ps_parse_stmt
  2. self-hosted/middle/typecheck_walk.qz — change tc_expr to tc_stmt in the NODE_DEFER branch
  3. self-hosted/backend/mir_lower.qz — fix emit_deferred_to_depth to use mir_lower_stmt like its siblings

5.2 Parser Change (parser.qz:5090-5103)

Before:

if ps_current_type(ps) == token_constants::TOK_DEFER
  var ln = ps_current_line(ps)
  var cl = ps_current_col(ps)
  ps_advance(ps)
  var expr = 0
  if ps_current_type(ps) == token_constants::TOK_IDENT and ps_peek_next_type(ps) == token_constants::TOK_ASSIGN
    expr = ps_parse_assign_or_call(ps)
  else
    expr = ps_parse_expr(ps)
  end
  return ast::ast_defer(s, expr, ln, cl)
end

After:

if ps_current_type(ps) == token_constants::TOK_DEFER
  var ln = ps_current_line(ps)
  var cl = ps_current_col(ps)
  ps_advance(ps)

  # defer do ... end  — multi-statement block
  if ps_check(ps, token_constants::TOK_DO)
    ps_advance(ps)
    ps_skip_newlines(ps)
    var body = ps_parse_block(ps)
    ps_expect(ps, token_constants::TOK_END, "Expected 'end' to close 'defer do' block")
    return ast::ast_defer(s, body, ln, cl)
  end

  # defer <stmt>  — single assignment / compound assign / call / method call
  # ps_parse_assign_or_call handles all statement-level forms:
  #   - bare call / method call (wrapped as NODE_EXPR_STMT)
  #   - x = e, obj.f = e, arr[i] = e
  #   - x += e, obj.f *= e, arr[i] >>= e, etc. (all compound assigns, all LHS forms)
  var stmt = ps_parse_assign_or_call(ps)
  return ast::ast_defer(s, stmt, ln, cl)
end

Notes:

  • The TOK_DO discrimination is unambiguous: defer do cannot start any other valid statement form because do is reserved as a block-opener in Quartz.
  • ps_parse_assign_or_call handles the postfix-guard-free happy path; if we ever need defer cleanup() if x > 0, the caller of ps_parse_stmt already wraps it — we could alternatively call ps_maybe_wrap_postfix_guard on the returned NODE_DEFER. Not strictly required for this fix but worth confirming in review.
  • ps_expect is the standard error-reporting helper. If it doesn’t match that exact name in the codebase, use whatever the parser’s canonical “expect token or error” helper is (grep for ps_expect or ps_consume during implementation).

5.3 Typecheck Change (typecheck_walk.qz:2823-2828)

Before:

elsif node_kind == node_constants::NODE_DEFER
  var expr = ast::ast_get_left(ast_storage, node)
  if expr >= 0
    tc_expr(tc, ast_storage, expr)
  end

After:

elsif node_kind == node_constants::NODE_DEFER
  var body = ast::ast_get_left(ast_storage, node)
  if body >= 0
    tc_stmt(tc, ast_storage, body)
  end

This handles NODE_ASSIGN, NODE_EXPR_STMT, NODE_BLOCK, NODE_FIELD_ASSIGN, NODE_INDEX_ASSIGN, and all compound-assign forms uniformly via the existing tc_stmt dispatcher.

5.4 MIR Change (mir_lower.qz:232-247)

Before:

def emit_deferred_to_depth(ctx: MirContext, target_depth: Int): Void
  var scope_stack = ctx.drops.defer_stack
  var sc = scope_stack.size - 1
  while sc >= target_depth
    var scope = scope_stack[sc]
    var i = vec_size(scope) - 1
    while i >= 0
      var entry = scope[i]
      var ast_store = entry[0]
      var node = entry[1]
      ctx.mir_lower_expr(ast_store, node)   # ❌ wrong dispatcher
      i -= 1
    end
    sc -= 1
  end
end

After:

def emit_deferred_to_depth(ctx: MirContext, target_depth: Int): Void
  var scope_stack = ctx.drops.defer_stack
  var sc = scope_stack.size - 1
  while sc >= target_depth
    var scope = scope_stack[sc]
    var i = vec_size(scope) - 1
    while i >= 0
      var entry = scope[i]
      var ast_store = entry[0]
      var node = entry[1]
      ctx.mir_lower_stmt(ast_store, node)   # ✅ matches emit_deferred / emit_deferred_to_scope
    i -= 1
    end
    sc -= 1
  end
end

This is a pure bug fix. It’s required for break/continue to correctly emit defers whose bodies are statement-shaped. Without it, the failing spec test — which uses a for loop — would still break at codegen time even if the parser fix lands, because for-loops synthesize continue targets.

5.5 Grammar Change (informal)

Before:

DeferStmt ::= 'defer' Expression
            | 'defer' Identifier '=' Expression  (special-cased)

After (Zig-modeled):

DeferStmt  ::= 'defer' DeferBody
DeferBody  ::= SingleStmt
             | 'do' BlockBody 'end'
SingleStmt ::= Assignment
             | CompoundAssignment
             | CallExpr
             | MethodCallExpr

where SingleStmt is the set accepted by ps_parse_assign_or_call (statement-level).

5.6 Tests to Add (beyond the currently-failing one)

Add the following to spec/qspec/cross_defer_safety_spec.qz under describe("defer in loop") and a new describe("defer statement forms"):

  1. [already failing] defer compound-assign in for loop → exit 3
  2. defer simple assignment in loop
    var x = 0
    for i in 0..5
      defer x = x + 1
    end
    return x
    Expect 5.
  3. defer method call in loop
    var v = vec_new()
    for i in 0..3
      defer v.push(i)
    end
    return v.size
    Expect 3.
  4. defer field assignment (struct with counter)
    struct C
      n: Int
    end
    def main(): Int
      var c = C { n: 0 }
      for i in 0..4
        defer c.n += 1
      end
      return c.n
    end
    Expect 4.
  5. defer index assignment
    var arr = [0, 0, 0]
    for i in 0..3
      defer arr[i] = i + 1
    end
    return arr[0] + arr[1] + arr[2]
    Expect 6.
  6. defer do … end block form, single-scope
    var a = 0
    var b = 0
    defer do
      a = 10
      b = 20
    end
    # at function exit, a=10 b=20
    return a + b
    Expect 30.
  7. defer do … end block form in loop
    var count = 0
    var sum = 0
    for i in 0..4
      defer do
        count += 1
        sum += i
      end
    end
    return count * 100 + sum   # expect 406 (count=4, sum=6)
  8. LIFO ordering with statement defers in a loop with break
    var log = vec_new()
    for i in 0..5
      defer log.push(i * 10)
      defer log.push(i)
      if i == 2
        break
      end
    end
    # Expected: [0, 0, 1, 10, 2, 20]
    Verifies the emit_deferred_to_depth fix for break specifically.
  9. Compound defer with method call outside loop (smoke)
    def main(): Int
      var v = vec_new()
      defer v.push(99)
      v.push(1)
      return v.size   # returns 1; 99 pushed at function exit
    end
  10. Nested scopes: defer block inside if inside loop
    var count = 0
    for i in 0..5
      if i % 2 == 0
        defer do
          count += 2
        end
      end
    end
    return count   # even iterations: 0, 2, 4 → count = 6
  11. Pending: defer do … end with return inside the block — file as it_pending with reason “defer-with-return needs design decision (does it still fire? infinite loop?)”. Matches Swift’s explicit-error rule.

Tests 1–10 should all pass after the fix. Test 11 documents a design hole we’d file for later (see §7).

5.7 Implementation Order (for a single-session sprint)

  1. Save fix-specific golden binary: cp self-hosted/bin/quartz self-hosted/bin/backups/quartz-pre-defer-stmt-golden
  2. Apply MIR fix (emit_deferred_to_depth). Build + run full qspec. This should be a no-op for all existing tests because current defers are all expression-shaped.
  3. Apply typecheck fix (tc_exprtc_stmt). Build + full qspec. Still a no-op.
  4. Apply parser fix. Build + full qspec. The failing test should flip to green.
  5. Add tests 2–10 (test 1 already exists). Run qspec.
  6. Add test 11 as it_pending.
  7. Run quake guard + smoke tests (brainfuck, style_demo, cross_defer_safety_spec itself).
  8. Commit with git add self-hosted/ spec/qspec/cross_defer_safety_spec.qz.

6. Quartz-Time Estimate

Traditional estimate: 1–2 days (parser + typecheck + MIR + tests + verification).

Quartz-time (÷4): ~3 hours actual, single session.

Breakdown:

  • Parser change: 15 min write, 15 min iterate
  • Typecheck one-line change: 2 min
  • MIR one-line change: 5 min
  • Rebuild + iterate: 30 min
  • Add 10 new tests: 30 min
  • Run qspec, fix regressions: 30 min
  • quake guard + smoke tests + commit: 15 min
  • Buffer for unexpected interactions (e.g., postfix-guard wrapping, ps_expect naming): 30 min

This fits comfortably in one session. No multi-session split needed.


7. Risk Assessment

Risks (ranked)

  1. Low — Postfix guard interaction. ps_parse_assign_or_call is called via ps_maybe_wrap_postfix_guard elsewhere in the parser. If we want defer cleanup() if err to work, we need to wrap the NODE_DEFER (not the inner stmt) in the guard. Easy to check during implementation: grep for ps_maybe_wrap_postfix_guard around other statement-forming branches and mirror the pattern. Cost if missed: defer X if Y doesn’t parse; test can catch it.

  2. Low — ps_expect helper naming. The design calls ps_expect for the end after a do block; Quartz may use a different name (ps_consume, ps_expect_token, etc.). Trivial to resolve at implementation time.

  3. Low — Typecheck regression on existing defer bodies. Switching tc_exprtc_stmt changes the dispatcher. If any existing test relies on defer-body-as-expression evaluation returning a value (it shouldn’t; defer return values are discarded), it could break. Mitigation: running full qspec after this single change in isolation (step 3 above) catches it.

  4. Very low — MIR break/continue fix could surface a hidden bug. If emit_deferred_to_depth has been paired with mir_lower_expr deliberately somewhere (I didn’t find evidence of this, but I can’t prove a negative), switching to mir_lower_stmt could expose it. Mitigation: run in isolation as step 2. The two sibling emit paths already use mir_lower_stmt without issue.

  5. Very low — defer do ... end ambiguity with an expression called do. Quartz reserves do as a keyword (TOK_DO), so there’s no lexer-level ambiguity. Safe.

  6. Design hole — return / break / continue inside a defer do ... end block. What semantics? Zig forbids it (compile error). Swift forbids it. Go doesn’t have the problem because you can only pass a call to defer. Quartz should match Zig/Swift and emit a compile error in tc_stmt when walking the inner block of a NODE_DEFER. File to roadmap as follow-up if not done in this sprint. It’s not table-stakes for the failing test; it’s a polish item. Prime Directive 6: gets filed, not ignored.

Not a risk

  • Codegen. MIR layer is AST-shape-agnostic, so LLVM emission needs no changes. The defer-stack + emit-on-exit mechanism is already in place.
  • Liveness. Already handles arbitrary subtrees via liveness_walk.
  • Fixpoint. This is a purely additive parser change + two one-line bug fixes. No reason to expect fixpoint drift. Still run quake guard of course.

Holes Filed (Prime Directive 6)

  • H-1: return/break/continue/defer nested inside a defer do ... end block — semantics undefined. Recommend: compile error with “control flow not allowed inside defer block” message, emitted in tc_stmt during NODE_DEFER walk. Add to roadmap under “defer polish” unless included in this sprint.
  • H-2: Error messages for the failing parser path. Current “Expected expression” is unhelpful; after the fix, errors should say “Expected statement, call, or do block after defer”. Cover in the parser change.
  • H-3: D-style scope(success) / scope(failure) is out of scope — Quartz has no exception mechanism. Revisit when/if Quartz adds panicking or fallible returns with unwinding. Roadmap note only.

Sources