feat: execute optional captures in type-producing procedures

README.md

@@ -98,7 +98,9 @@ main proc() i16 {
 
 the specialization checkpoint supports type arguments and constant integer, boolean, string, callback and null value arguments; declared value-parameter types survive specialization, including narrow checked arithmetic and optional captures/unwrap. integer arguments are range-checked even when unused, and erased argument expressions still receive ordinary checking. immutable constant expressions may use unary signs, `!`, `+`, `-`, `*`, comparisons and logical operators. type-producing procedures accept ordinary parameters as well as `$` parameters: every argument must be supplied explicitly and evaluate to a constant, and every input participates in specialization caching. for example, `Fixed proc(T type, count usize) type { return struct { values [count]T } }` accepts `Fixed(i16, 2)` and reuses its nominal type for equivalent inputs. use `$` when a parameter must be visible in subsequent signature annotations. runaway specialization is diagnosed at 128 nested evaluation/specialization calls or 4096 instances.
 
-type-producing bodies support mutable type and value locals, direct binding assignment, scoped blocks, conditional/early returns, integer range and while loops, break/continue and nested/recursive type-producing calls. for example, start with `Result := T`, repeat `Result = [2]Result` in a loop, then `return Result` to generate nested array storage. snapshots retain each local's current value without changing the source AST; scalar operations reuse checking, typed IR and interpreter execution, preserving exact types and overflow diagnostics. type-control execution is limited to 4096 statement/loop-guard steps shared with nested producers; individual scalar evaluations retain the interpreter limits below. only executed bodies are elaborated, while every body is still parsed and name-resolved. field/index assignment, optional captures, cleanup and type-valued aggregate storage in type-producing bodies remain outside this checkpoint.
+type-producing bodies support mutable type and value locals, direct binding assignment, scoped blocks, conditional/early returns, integer range and while loops, break/continue and nested/recursive type-producing calls. for example, start with `Result := T`, repeat `Result = [2]Result` in a loop, then `return Result` to generate nested array storage. snapshots retain each local's current value without changing the source AST; scalar operations reuse checking, typed IR and interpreter execution, preserving exact types and overflow diagnostics. type-control execution is limited to 4096 statement/loop-guard steps shared with nested producers; individual scalar evaluations retain the interpreter limits below. only executed bodies are elaborated, while every body is still parsed and name-resolved. field/index assignment, cleanup and type-valued aggregate storage in type-producing bodies remain outside this checkpoint.
+
+optional `if`/`while` captures also execute in type-producing bodies: `if count |length| { return [length]i16 }` generates storage from an optional `usize`. ordinary checked IR validates each executed guard; present payloads become immutable owned constants with exact scalar/aggregate types and strings, and absent optionals skip the body. while guards observe updated locals on each iteration; captures stay body-local across break/continue and early return. generated lengths/defaults, imported producers, cache identity and repeated compilation preserve these values. null data/function pointers materialize as null, but nonnull pointer/slice/callback-bearing payloads retain the constant-materialization escape boundary.
 
 type-producing bodies also execute scalar/enum `match` statements and type-valued match expressions in initializers, assignments and returns, such as `return match wide { true: i16, false: i8 }`. selected expression arms may themselves be matches. an ordinary checked IR match returns the selected arm index, preserving subject/pattern typing, range boundaries, overlap, unreachable-arm and exhaustiveness diagnostics without checking unexecuted bodies. statement arms retain scoped locals, assignments, early returns and enclosing-loop break/continue; expression selection shares the type-control step bound.
 

TODO.md

@@ -27,8 +27,8 @@ including owned by-value tagged payload captures in ordinary generic procedures.
 type-producing procedures also accept ordinary
 constant inputs, cached and erased like explicit compile-time arguments. mutable
 type/value locals, direct assignments, branches, scalar/enum/tagged-union matching
-and bounded integer-range/while loops now generate types and storage. payload
-captures and matches select nested constructors, lengths, signatures and defaults;
+and bounded integer-range/while loops now generate types and storage. optional
+if/while captures and tagged payload matches select nested constructors, lengths, signatures and defaults;
 general type-valued execution, reflection and generic library acceptance remain open.
 the driver retains frontend storage and source provenance through
 `@source/compilation`, with compilation-wide AST node identities.
@@ -71,12 +71,13 @@ test discovery and FFI remain outside these completed slices.
   - [x] aggregate/explicit-expression checkpoint: computed nested records/tuples and fixed arrays, contextual/dependent aggregate value parameters, normalized specialization caching and explicit `$()` expressions through interpreter/native execution. preserve by-value isolation, zero-cell shapes, integer widths, string ownership, enum/optional fields and exact IEEE float bits; reject runtime dependencies and values that would retain temporary interpreter storage. repeatable compilation restores erased expressions.
   - [x] conditional specialization checkpoint: explicit compile-time `if` guards and side-effect-free constant guards in concrete generic procedures select a body before specialization/checking. type equality retains aliases, nominal identity and structural signatures; composed error-set identity/cache keys ignore ordering, grouping and repetition. guard checking, selected-body diagnostics, scoped cleanup, runtime guards and repeatable compilation work through interpreter/native execution; both arms remain parsed and name-resolved.
   - [x] constant-input type-procedure checkpoint: type-producing procedures accept ordinary parameters alongside `$` parameters, require every argument explicitly and evaluate/cache/erase all inputs. generated record/array storage, nominal cache identity, nested calls, dependent contextual aggregate inputs, exact erased-argument checking, source diagnostics, imported producers and repeatable compilation work through interpreter/native execution.
-  - [x] mutable type-execution checkpoint: type-producing bodies execute mutable type/value locals, direct binding assignments, scoped blocks, if/else and early returns, integer-range/while loops, break/continue and nested/recursive producers. generated nested types, scalar-computed lengths/defaults, nominal cache identity, exact value types, overflow/guard/assignment diagnostics, imported producers and repeatable compilation work through interpreter/native execution. type-control execution shares a 4096-step bound across nested producers; scalar operations reuse checked IR execution. field/index updates, optional captures, cleanup and type-valued aggregate storage in these bodies remain open.
+  - [x] mutable type-execution checkpoint: type-producing bodies execute mutable type/value locals, direct binding assignments, scoped blocks, if/else and early returns, integer-range/while loops, break/continue and nested/recursive producers. generated nested types, scalar-computed lengths/defaults, nominal cache identity, exact value types, overflow/guard/assignment diagnostics, imported producers and repeatable compilation work through interpreter/native execution. type-control execution shares a 4096-step bound across nested producers; scalar operations reuse checked IR execution. field/index updates, cleanup and type-valued aggregate storage in these bodies remain open.
   - [x] type-match checkpoint: type-producing bodies execute scalar/enum match statements and direct type-valued match expressions in initializers, assignments and returns, including nested selected arms. ordinary checking/IR execution select the arm and validate subject/pattern types, ranges, overlap, unreachable arms and exhaustiveness; only the selected body is elaborated. generated storage, scoped locals, loop transfers, early returns, nominal cache identity, selected-body diagnostics, dependency provenance and repeatable compilation work through interpreter/native execution. all bodies remain parsed/name-resolved and execution shares the type-control bound.
   - [x] nested type-match checkpoint: matches select types inside constructors, array lengths, generic arguments, local annotations, callback signatures and field defaults in type-producing bodies. selection precedes ordinary expansion, preserves literal spelling/source provenance and loop snapshots, and caches nominal results before expanding recursive fields. selected-body and pattern diagnostics, failed deferred elaboration, imported producers and repeatable compilation work through interpreter/native execution; deferred selection shares the producer's type-control bound.
   - [x] tagged/fallible constant checkpoint: explicit `$()` expressions, generic value parameters and type-procedure inputs materialize tagged unions and both fallible channels through interpreter/native execution. preserve active tags/payloads, nested raw aggregate projections, string ownership, void/zero-cell variants, composed/payload errors, declared channel types, normalized cache identity and repeatable compilation. `catch` computes generated lengths and runtime values. untagged unions and storage-bearing active payloads remain escape diagnostics rather than retaining temporary interpreter handles.
   - [x] tagged type-match checkpoint: type-producing bodies execute tagged-union matches with immutable by-value payload captures in statement, expression and nested storage positions. ordinary match checking validates every arm's captures, including grouped variants; selected payloads retain exact scalar/aggregate types and owned strings. generated storage/defaults, loop snapshots/transfers, cache identity, selected-body diagnostics, imported producers, dependency provenance and repeatable compilation work through interpreter/native execution. storage-bearing payloads retain the constant-materialization escape boundary.
   - [x] generic match-specialization checkpoint: side-effect-free constant scalar/enum/tagged-union subjects in concrete generic procedures and explicit `match $(subject)` expressions/statements select an arm before body specialization/checking. reuse ordinary checked IR for pattern types, ranges, overlap, exhaustiveness and every arm's capture validation; only the selected body is elaborated. exact by-value payload types/owned strings, runtime parameters, nested expression matches, cleanup, return/loop transfers, imported producers and repeatable compilation work through interpreter/native execution. ordinary call/runtime/mutable subjects stay runtime; contextual aggregates with possible runtime defaults are not implicitly evaluated. all bodies remain parsed/name-resolved; storage-bearing payloads retain the constant-materialization escape boundary.
+  - [x] optional type-control checkpoint: type-producing bodies execute optional if/while guards with immutable by-value payload captures. ordinary checked IR validates guards; selected scalar/aggregate payloads retain exact types and owned strings across nested captures, generated storage/defaults, loop updates/transfers and early returns. absent optionals skip unselected bodies; invalid guards, capture mutation/shadowing, overflow and execution bounds retain diagnostics. imported producers, dependency provenance, nominal cache identity and repeatable compilation work through interpreter/native execution. null data/function pointers materialize correctly; nonnull storage-bearing payloads retain the constant-materialization escape boundary.
   - [ ] remaining gate: general type-valued compile-time execution, inline match/loop expansion and reflection (`typeinfo!`, `field!`, `struct_type!`, generated type/default arrays), broader constant materialization (including untagged unions and storage-bearing values), plus existing-library acceptance. required collection/index iteration, expression-valued control flow, compound updates, formatting parsing and imported C-binding support may interleave with later gates.
 - [ ] remaining source constructs: collection/index iteration, expression-valued control flow and `yield`, compound updates and multiline strings. examples: `source/parser/parse.hon`, `source/llvm/runtime.hon`.
 - [ ] standard-library milestone: compile/run a multi-package program exercising existing allocation and IO, including error cleanup; no replacement miniature standard library.

source/compilation/compilation.test.hon

@@ -125,6 +125,15 @@ recompiles_specialized_calls_without_borrowing_previous_ir test {
             `        .empty: return bool,
             `    }
             `}
+            `Optional proc() type {
+            `    pending ?usize := 2
+            `    Result := i8
+            `    while pending |length| {
+            `        pending = null
+            `        Result = [length]i16
+            `    }
+            `    return Result
+            `}
             `read_selected proc($choice Choice, value i16) i16 {
             `    match choice {
             `        .payload |detail|: {
@@ -148,6 +157,8 @@ recompiles_specialized_calls_without_borrowing_previous_ir test {
             `    if (Token(packet_data()) != Token(packet)) return 0
             `    captured Captured(packet.choices[0]) :: { values = [19, 21] }
             `    if (captured.values[0] + captured.values[1] + captured.bias != 42) return 0
+            `    optional Optional() :: captured.values
+            `    if (optional[0] + optional[1] != 40) return 0
             `    payload :: packet.choices[0].payload
             `    if (payload.values[0] != 40 or payload.values[1] != 2 or payload.text.len != 2) return 0
             `    match packet.choices[1] { .empty: {}, .payload: return 0 }

source/sema/specialize/specialize.hon

@@ -890,7 +890,10 @@ serialize_value proc(
     } else if (type.kind == .optional) {
         container :: typecheck.array_type(checked, type.array_type)
         if (container.element.kind == .pointer or container.element.kind == .function) {
-            if (value.data.unsigned != 0) return no_node
+            absent :: if (container.element.kind == .function)
+                value.data.function == maxval!(ir.FunctionId)
+                else value.data.unsigned == maxval!(u64)
+            if (!absent) return no_node
             result = try ast.add_literal(state.builder, .null, span)
         } else if (!ir.aggregate_item(module, value, 0).data.boolean) {
             result = try ast.add_literal(state.builder, .null, span)
@@ -1024,6 +1027,13 @@ evaluate_ir proc(state @mut State, expression, result_type ast.NodeId) Value ! E
     returned :: try ast.add_return_stmt(state.builder, span, expression)
     statements [1]ast.NodeId :: [returned]
     body :: try ast.add_block(state.builder, span, statements[..])
+    try refresh(state)
+    return try evaluate_ir_body(state, expression, body, result_type)
+}
+
+@hide:file
+evaluate_ir_body proc(state @mut State, at, body, result_type ast.NodeId) Value ! Error {
+    span :: ast.node_span(&state.tree, at)
     wrapper :: try ast.add_function_decl(state.builder, span, span, strpool.no_str, result_type, &[], body)
     try refresh(state)
     declarations := arraylist.init(ast.NodeId, state.allocator)
@@ -1364,6 +1374,31 @@ type_execution_value proc(state @mut State, input, annotation ast.NodeId) ast.No
     return result
 }
 
+# check capture guards with ordinary IR without elaborating either source body.
+@hide:file
+type_capture_guard proc(state @mut State, expression, capture ast.NodeId) Value ! Error {
+    resolved :: binding(state, capture)
+    if (resolved.kind == .invalid) {
+        state.evaluation_diagnostic = resolved.data.diagnostic
+        return Value{}
+    }
+    span :: ast.node_span(&state.tree, expression)
+    yes :: try ast.add_boolean(state.builder, span, true)
+    no :: try ast.add_boolean(state.builder, span, false)
+    taken :: try ast.add_return_stmt(state.builder, span, yes)
+    skipped :: try ast.add_return_stmt(state.builder, span, no)
+    taken_statements [1]ast.NodeId :: [taken]
+    skipped_statements [1]ast.NodeId :: [skipped]
+    then_block :: try ast.add_block(state.builder, span, taken_statements[..])
+    else_block :: try ast.add_block(state.builder, span, skipped_statements[..])
+    selector :: try ast.add_if_stmt(state.builder, span, expression, then_block, else_block)
+    ast.set_capture(state.builder, selector, capture)
+    statements [1]ast.NodeId :: [selector]
+    body :: try ast.add_block(state.builder, span, statements[..])
+    try refresh(state)
+    return try evaluate_ir_body(state, expression, body, try builtin(state, expression, .bool))
+}
+
 @hide:file
 execute_type_block proc(state @mut State, execution @mut TypeExecution, block ast.NodeId) TypeFlow ! Error {
     saved :: execution.locals.items.len
@@ -1440,34 +1475,53 @@ execute_type_statement proc(state @mut State, execution @mut TypeExecution, stat
             condition := no_node
             body := no_node
             other := no_node
+            capture ?ast.NodeId := null
             if (looping) {
                 view :: ast.while_stmt(&state.tree, statement)
-                if (view.capture != null) return .invalid
+                capture = view.capture
                 condition = view.condition
                 body = view.body
             } else {
                 view :: ast.if_stmt(&state.tree, statement)
-                if (view.capture != null) return .invalid
+                capture = view.capture
                 condition = view.condition
                 body = view.then_block
                 if (view.else_block) |branch| other = branch
             }
             if (looping) execution.loop_depth += 1
             defer if (looping) execution.loop_depth -= 1
+            saved :: execution.locals.items.len
+            defer execution.locals.items = execution.locals.items[..saved]
             invariant :: branch_constant(state, condition, 0)
             value := Value{}
             while (true) {
                 if (!try type_step(state, condition)) return .invalid
                 if (!invariant or value.kind == .invalid) {
                     expression :: try type_snapshot(state, execution, condition)
+                    if (capture) |name| value = try type_capture_guard(state, expression, name)
+                    else {
                         boolean :: try builtin(state, expression, .bool)
                         value = try evaluate_ir(state, expression, boolean)
                     }
+                }
                 if (value.kind != .boolean) return .invalid
                 if (!value.boolean) {
                     if (!looping and other != no_node) return try execute_type_block(state, execution, other)
                     return .next
                 }
+                execution.locals.items = execution.locals.items[..saved]
+                if (capture) |name| {
+                    expression :: try type_snapshot(state, execution, condition)
+                    unwrapped :: try ast.add_unary(state.builder,
+                        ast.node_span(&state.tree, condition), .unwrap, expression)
+                    try refresh(state)
+                    annotation :: try expression_type(state, unwrapped, 0)
+                    payload :: try type_execution_value(state, unwrapped, annotation)
+                    if (payload == no_node) return .invalid
+                    try arraylist.append(&execution.locals, TypeLocal{
+                        declaration = name, value = payload, annotation = annotation,
+                    })
+                }
                 flow :: try execute_type_block(state, execution, body)
                 if (!looping or flow == .returned or flow == .invalid) return flow
                 if (flow == .break) return .next

tests/native.py

@@ -11,6 +11,147 @@ HONEY = ROOT / "build/honey"
 
 # Explicit expectations guard against frontend bugs shared by both execution paths.
 CASES = {
+    "type_execution_optional_if_selects_storage_and_cache_identity": (
+        """
+        Count :: alias ?usize
+        Storage proc(count Count) type {
+            if count |length| { return struct { values [length]i16 } }
+            else { return struct { value i16 = 2 } }
+        }
+        main proc() i16 {
+            value Storage(2) :: { values = [19, 21] }
+            same Storage(usize(2)) :: value
+            empty Storage(null) :: {}
+            return same.values[0] + same.values[1] + empty.value
+        }
+        """, b"42\n", None,
+    ),
+    "type_execution_optional_payload_owned_aggregate_and_nested_capture": (
+        """
+        Detail :: struct { values [2]i16, text string, count ?usize }
+        detail proc() ?Detail { return { values = [19, 21], text = "ok", count = 2 } }
+        Storage proc() type {
+            if detail() |payload| {
+                if payload.count |count| {
+                    return struct {
+                        values [count]i16
+                        bias i16 = payload.values[0] + payload.values[1] + i16(payload.text.len)
+                    }
+                }
+            }
+            return bool
+        }
+        main proc() i16 {
+            value Storage() :: { values = [0, 0] }
+            return value.bias
+        }
+        """, b"42\n", None,
+    ),
+    "type_execution_optional_while_refreshes_capture_and_preserves_transfers": (
+        """
+        next proc(count usize) ?usize {
+            if (count < 4) return count
+            return null
+        }
+        Storage proc() type {
+            count usize := 0
+            Result := i16
+            while next(count) |length| {
+                count = count + 1
+                if (length == 0) continue
+                if (length == 3) break
+                Result = [length]Result
+            }
+            pending ?usize := 2
+            while pending |length| {
+                pending = null
+                if (length != 2) return bool
+                count = count + length
+            }
+            return struct { values Result, bias i16 = i16(count) }
+        }
+        main proc() i16 {
+            value Storage() :: { values = [[17], [19]] }
+            return value.values[0][0] + value.values[1][0] + value.bias
+        }
+        """, b"42\n", None,
+    ),
+    "type_execution_optional_absence_skips_invalid_bodies": (
+        """
+        absent proc() ?i16 { return null }
+        null_pointer proc() ?@i16 { return null }
+        null_callback proc() ?@proc() i16 { return null }
+        Storage proc() type {
+            pointer ?@i16 :: null
+            while pointer |_| { return i8(128) }
+            if absent() |_| { return i8(128) }
+            else { return i16 }
+        }
+        main proc() i16 {
+            absent_pointer :: $(null_pointer())
+            absent_callback :: $(null_callback())
+            if absent_pointer |_| { return -1 }
+            if absent_callback |_| { return -2 }
+            value Storage() :: 42
+            return value
+        }
+        """, b"42\n", None,
+    ),
+    "type_execution_optional_capture_retains_exact_payload_type": (
+        """
+        narrow proc() ?i8 { return 127 }
+        Bad proc() type {
+            if narrow() |value| {
+                copy :: value
+                overflow :: copy + 1
+                return [usize(overflow)]i16
+            }
+            return i16
+        }
+        main proc() i16 { value Bad() :: 42
+            return value }
+        """, b"", b"integer arithmetic overflow",
+    ),
+    "type_execution_optional_capture_is_immutable": (
+        """
+        Bad proc(value ?usize) type {
+            if value |length| { length = 2 }
+            return i16
+        }
+        main proc() i16 { value Bad(1) :: 42
+            return value }
+        """, b"", b"assignment requires a mutable local binding",
+    ),
+    "type_execution_optional_capture_requires_optional_guard": (
+        """
+        Bad proc() type {
+            if true |_| { return i16 }
+            return bool
+        }
+        main proc() i16 { value Bad() :: 42
+            return value }
+        """, b"", b"condition must have type bool",
+    ),
+    "type_execution_optional_capture_preserves_resolution_errors": (
+        """
+        Bad proc(value ?usize) type {
+            if value |value| { return i16 }
+            return bool
+        }
+        main proc() i16 { value Bad(null) :: 42
+            return value }
+        """, b"", b"duplicate name",
+    ),
+    "type_execution_optional_capture_loop_is_bounded": (
+        """
+        Bad proc(value ?usize) type {
+            while value |_| {}
+            return i16
+        }
+        main proc() i16 { value Bad(1) :: 42
+            return value }
+        """, b"", b"compile-time execution exceeds its recursion or step limit",
+    ),
     "generic_match_scalar_enum_and_range_selection": (
         """
         Mode :: enum { pair, other }
@@ -4173,7 +4314,35 @@ def check_specialization_packages(temporary, executable):
     interpreted = run(str(HONEY), "run", str(app))
     assert interpreted.stderr.startswith(f"{sizing.resolve()}:6:".encode()), outcome(interpreted)
     assert len(interpreted.stderr.splitlines()) == 1, outcome(interpreted)
-    return 6
+    sizing.write_text(textwrap.dedent("""
+        Optional proc(T type, size ?usize) type {
+            if size |length| { return struct { values [length]T } }
+            else { return bool }
+        }
+    """).strip() + "\n")
+    source.write_text(textwrap.dedent("""
+        lib :: import "../library"
+        main proc() i16 {
+            value lib.Optional(i16, 2) :: { values = [40, 2] }
+            return value.values[0] + value.values[1]
+        }
+    """).strip() + "\n")
+    check(app, executable, (b"42\n", None))
+    sizing.write_text(textwrap.dedent("""
+        narrow proc() ?i8 { return 127 }
+        Optional proc(T type, size ?usize) type {
+            if narrow() |value| {
+                next :: value + 1
+                return [usize(next)]T
+            }
+            return T
+        }
+    """).strip() + "\n")
+    check(app, executable, (b"", b"integer arithmetic overflow"))
+    interpreted = run(str(HONEY), "run", str(app))
+    assert interpreted.stderr.startswith(f"{sizing.resolve()}:4:".encode()), outcome(interpreted)
+    assert len(interpreted.stderr.splitlines()) == 1, outcome(interpreted)
+    return 8
 
 
 def check_dependency_provenance(temporary, executable):