From cc39c96fc0fd203c966dc38770f1c843ada8e56a Mon Sep 17 00:00:00 2001 From: hl-valdemar Date: Fri, 30 Oct 2026 15:54:32 +0100 Subject: [PATCH] feat: materialize address-free compile-time untagged unions Snapshot full union storage rather than choosing an overlapping field. Preserve structural union source syntax independently of nominal identity so direct literal arguments materialize and share normalized specialization keys. Reject uninitialized bytes and address-bearing member layouts. --- README.md | 4 +- TODO.md | 16 +- source/ast/ast.hon | 25 ++- source/compilation/compilation.test.hon | 60 +++++++ source/lower/lower.hon | 30 ++++ source/sema/check/check.hon | 24 ++- source/sema/resolve.hon | 5 + source/sema/specialize/specialize.hon | 69 ++++++-- tests/native.py | 212 +++++++++++++++++++++++- 9 files changed, 420 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 1444068..b2623e7 100644 --- a/README.md +++ b/README.md @@ -138,7 +138,9 @@ computed callbacks materialize as references to their original source declaratio readonly byte-slice results (`[]u8` and `[;sentinel]u8` for any byte sentinel) snapshot their bytes into owned literals before evaluation storage is released. ordinary compile-time helpers can return string subviews, views of locally filled arrays or immutable global arrays, including nested records/arrays, tagged payloads, optionals and both fallible channels. explicit `$()` expressions, generic arguments and generated defaults preserve embedded NULs, arbitrary byte values, exact declared slice types and content-based specialization-cache identity. nonzero-sentinel snapshots retain an owned trailing sentinel outside the logical length; empty views also retain it. snapshots survive evaluation/session disposal and repeated compilation; uninitialized source bytes diagnose read-before-initialization, and sentinel bytes are rechecked after evaluation. mutable slices and nonbyte or nominal-byte slices remain unsupported. -`$(expression)` explicitly evaluates a constant expression, including an ordinary procedure call, even in a package without templates. use a typed expression when its shape cannot be inferred independently, such as `$(Point { x = 40 })`; integer literals retain contextual typing. the resulting value works in runtime expressions, bindings, expression statements and array lengths, while the erased expression still receives ordinary checking. each interpreter evaluation is isolated and limited to 100,000 IR instructions/terminators and 128 active frames; failures retain their originating source locations. runtime parameters and mutable local/global dependencies cannot supply compile-time inputs. nonnull data pointers, unsupported slice types and untagged unions cannot escape temporary evaluation as generated constants; active payloads or aggregate fields containing unsupported values are diagnosed rather than retaining interpreter handles. untagged unions lack an active-member identity, so selecting a source variant could discard observable bytes. executed foreign calls remain interpreter traps. general type-valued execution, broader storage-bearing materialization and remaining existing-library acceptance, including broader `meta` operations and `io.print`, remain open. +untagged union constants snapshot every byte of their native-layout storage, including padding and bytes beyond a smaller overlapping member. scalar reinterpretation, nested records/arrays, optionals, tagged payloads, fallible channels and generated defaults preserve exact nominal types and by-value copies through `$()` and compile-time arguments. specialization keys compare the full snapshot, not one selected field. every storage byte must be initialized. because untagged storage has no active-member identity, all member layouts must be free of data pointers, callbacks, strings and slices, recursively; otherwise materialization diagnoses an escape rather than retaining interpreter handles, even if a numeric member was initialized. + +`$(expression)` explicitly evaluates a constant expression, including an ordinary procedure call, even in a package without templates. use a typed expression when its shape cannot be inferred independently, such as `$(Point { x = 40 })`; integer literals retain contextual typing. the resulting value works in runtime expressions, bindings, expression statements and array lengths, while the erased expression still receives ordinary checking. each interpreter evaluation is isolated and limited to 100,000 IR instructions/terminators and 128 active frames; failures retain their originating source locations. runtime parameters and mutable local/global dependencies cannot supply compile-time inputs. nonnull data pointers, unsupported slice types and address-bearing untagged union layouts cannot escape temporary evaluation as generated constants; active payloads or aggregate fields containing unsupported values are diagnosed rather than retaining interpreter handles. executed foreign calls remain interpreter traps. general type-valued execution, broader storage-bearing materialization and remaining existing-library acceptance, including broader `meta` operations and `io.print`, remain open. `if $(condition) { ... } else { ... }` evaluates a boolean guard before expanding or checking either body, including calls to ordinary constant-input procedures. concrete generic specializations also select branches whose guards use constants, immutable constant bindings, type comparisons or substituted compile-time parameters. only the selected body is specialized and checked; both bodies are still parsed and name-resolved. selected bodies retain their scope, cleanup and return behavior, and erased guards retain ordinary type checking and source diagnostics. ordinary calls in an implicit guard stay runtime; use `$()` to request their evaluation. optional captures remain runtime, and ordinary nongeneric `if (true)` still checks both bodies. diff --git a/TODO.md b/TODO.md index f33a29c..c7ff2d9 100644 --- a/TODO.md +++ b/TODO.md @@ -23,8 +23,9 @@ ordinary procedures can compute scalar, nested record/array, tagged-union, fallible, callback and readonly byte-slice compile-time values using mutable locals, loops and recursive calls; explicit `$()` expressions use the same constant evaluation path. readonly `[]u8` and `[;sentinel]u8` snapshots own exact bytes and any byte sentinel after evaluation. -mutable and nonbyte/nominal-byte slices, nonnull data pointers and untagged unions -retain the constant-materialization escape boundary. +address-free untagged unions snapshot every initialized storage byte, including padding. +mutable and nonbyte/nominal-byte slices, nonnull data pointers and address-bearing +untagged union layouts retain the constant-materialization escape boundary. compile-time `if`/`match` selection and type comparisons specialize only the selected body, including owned by-value tagged payload captures in ordinary generic procedures. type-producing procedures also accept ordinary @@ -118,7 +119,7 @@ test discovery and FFI remain outside these completed slices. - [x] mutable type-execution checkpoint: type-producing bodies execute mutable type/value locals, direct binding assignments, scoped blocks, if/else and early returns, integer-range/while loops, break/continue and nested/recursive producers. generated nested types, scalar-computed lengths/defaults, nominal cache identity, exact value types, overflow/guard/assignment diagnostics, imported producers and repeatable compilation work through interpreter/native execution. type-control execution shares a 4096-step bound across nested producers; scalar operations reuse checked IR execution. aggregate field/index updates and type-valued arrays are covered below; cleanup and type-valued record storage remain open. - [x] type-match checkpoint: type-producing bodies execute scalar/enum match statements and direct type-valued match expressions in initializers, assignments and returns, including nested selected arms. ordinary checking/IR execution select the arm and validate subject/pattern types, ranges, overlap, unreachable arms and exhaustiveness; only the selected body is elaborated. generated storage, scoped locals, loop transfers, early returns, nominal cache identity, selected-body diagnostics, dependency provenance and repeatable compilation work through interpreter/native execution. all bodies remain parsed/name-resolved and execution shares the type-control bound. - [x] nested type-match checkpoint: matches select types inside constructors, array lengths, generic arguments, local annotations, callback signatures and field defaults in type-producing bodies. selection precedes ordinary expansion, preserves literal spelling/source provenance and loop snapshots, and caches nominal results before expanding recursive fields. selected-body and pattern diagnostics, failed deferred elaboration, imported producers and repeatable compilation work through interpreter/native execution; deferred selection shares the producer's type-control bound. - - [x] tagged/fallible constant checkpoint: explicit `$()` expressions, generic value parameters and type-procedure inputs materialize tagged unions and both fallible channels through interpreter/native execution. preserve active tags/payloads, nested raw aggregate projections, string ownership, void/zero-cell variants, composed/payload errors, declared channel types, normalized cache identity and repeatable compilation. `catch` computes generated lengths and runtime values. untagged unions and storage-bearing active payloads remain escape diagnostics rather than retaining temporary interpreter handles. + - [x] tagged/fallible constant checkpoint: explicit `$()` expressions, generic value parameters and type-procedure inputs materialize tagged unions and both fallible channels through interpreter/native execution. preserve active tags/payloads, nested raw aggregate projections, string ownership, void/zero-cell variants, composed/payload errors, declared channel types, normalized cache identity and repeatable compilation. `catch` computes generated lengths and runtime values. address-free untagged unions are covered below; unsupported active payloads remain escape diagnostics rather than retaining temporary interpreter handles. - [x] tagged type-match checkpoint: type-producing bodies execute tagged-union matches with immutable by-value payload captures in statement, expression and nested storage positions. ordinary match checking validates every arm's captures, including grouped variants; selected payloads retain exact scalar/aggregate types and owned strings. generated storage/defaults, loop snapshots/transfers, cache identity, selected-body diagnostics, imported producers, dependency provenance and repeatable compilation work through interpreter/native execution. storage-bearing payloads retain the constant-materialization escape boundary. - [x] generic match-specialization checkpoint: side-effect-free constant scalar/enum/tagged-union subjects in concrete generic procedures and explicit `match $(subject)` expressions/statements select an arm before body specialization/checking. reuse ordinary checked IR for pattern types, ranges, overlap, exhaustiveness and every arm's capture validation; only the selected body is elaborated. exact by-value payload types/owned strings, runtime parameters, nested expression matches, cleanup, return/loop transfers, imported producers and repeatable compilation work through interpreter/native execution. ordinary call/runtime/mutable subjects stay runtime; contextual aggregates with possible runtime defaults are not implicitly evaluated. all bodies remain parsed/name-resolved; storage-bearing payloads retain the constant-materialization escape boundary. - [x] optional type-control checkpoint: type-producing bodies execute optional if/while guards with immutable by-value payload captures. ordinary checked IR validates guards; selected scalar/aggregate payloads retain exact types and owned strings across nested captures, generated storage/defaults, loop updates/transfers and early returns. absent optionals skip unselected bodies; invalid guards, capture mutation/shadowing, overflow and execution bounds retain diagnostics. imported producers, dependency provenance, nominal cache identity and repeatable compilation work through interpreter/native execution. null data/function pointers materialize correctly; nonnull storage-bearing payloads retain the constant-materialization escape boundary. @@ -145,10 +146,11 @@ test discovery and FFI remain outside these completed slices. - [x] compound/bitwise checkpoint: arithmetic and bitwise compound updates evaluate mutable local/field/index/pointer places and their old values once before the RHS through interpreter/native execution. integer complement, and/or/xor, truncating left shifts, signed/unsigned right shifts and saturating left shifts preserve widths and distinct identity; unsigned counts retain independent types. known invalid counts diagnose even in untaken runtime branches; dynamic counts trap without native LLVM poison. checked arithmetic, float-only division, readonly diagnostics, scoped cleanup, ordinary constants, generated storage, generic branch selection and repeatable compilation retain existing semantics. type-valued `|` remains composition while runtime `|` and capture delimiters retain their grammar. full hash-map acceptance remains open. - [x] hash-map execution checkpoint: existing `std/hashmap.HashMap`, `AutoHashMap` and `StringHashMap` run collision chains, zero-hash normalization, growth/rehashing, duplicate-key errors, absent lookups and owned aggregate values without replacement library code. interpreter checks use separate typed allocation buffers and preserve the original map on allocation failure, including cleanup; native `mem.c_allocator` checks cover all integer widths/boundaries, booleans, sparse/aliased enums, nested distinct/array/record keys with padding ignored, content-based byte strings and custom callbacks at `-O0`/`-O2`. specialization retains `usize` collection indices, nominal/explicit pointer-cast result types, rigid slice borrowing/string context and source-known optional callbacks; ordinary checking rejects incompatible elements and readonly reversal. `unreachable` demand-traps with provenance through both paths and compile-time execution; literal-true loops distinguish escaping and nested breaks. `compile_error!` owns computed string diagnostics and skips unselected compile-time bodies. initializer inference from expected result types is covered below. - [x] expected-result inference checkpoint: omitted compile-time type/value arguments infer from rigid runtime arguments and expected results before contextual literal defaults. typed bindings, assignments, returns, ordinary/nested calls, aggregate fields/elements/defaults, optional/fallible channels, callbacks, match arms, yielding/catch blocks, arithmetic/bitwise operands and explicit `$()` expressions retain exact checked types through interpreter/native execution. generated nominal results infer counts and private/custom callback identities; existing `hashmap.init(allocator)` initializes `AutoHashMap`, `StringHashMap` and custom `HashMap` without replacement library code. rigid conflicts, unrelated nominal generators and unresolved arguments diagnose; explicit calls and fully explicit type-producer inputs retain their contracts. owned specializations and repeat compilation are verified. - - [x] computed-callback materialization checkpoint: ordinary compile-time calls return source-bound callbacks through interpreter/native execution, including optional/nested presence, arrays/records, tagged payloads and both fallible channels. temporary IR function IDs map back to exact source declarations before evaluation storage is released. computed/direct aggregate arguments share specialization-cache identity and generated defaults; imported private targets, same-spelled functions, signature diagnostics, runtime trap provenance and repeated compilation retain their contracts. nonnull data pointers, unsupported slice types and untagged unions retain their escape boundary. - - [x] readonly byte-slice materialization checkpoint: ordinary compile-time calls snapshot readonly `[]u8` and zero-terminated `[;0]u8` results into owned literals before evaluation storage is released. string subviews, locally filled arrays, immutable global arrays and raw aggregate views preserve exact bytes, embedded NULs, declared slice types and normalized content-based cache identity through explicit `$()` expressions, generic inputs, generated defaults, nested records/arrays, tagged payloads, optionals and both fallible channels. imported helpers, session/source disposal and repeated compilation retain ownership through interpreter/native execution; uninitialized source bytes diagnose read-before-initialization and invalid ranges are rejected before dereference. nonzero sentinels are covered below; mutable and nonbyte/nominal-byte slices, nonnull data pointers and untagged unions remain unsupported. - - [x] nonzero-sentinel byte-slice checkpoint: readonly `[;sentinel]u8` compile-time results own exact payload bytes and any byte sentinel through interpreter/native execution. canonical string-backed views preserve logical lengths, empty views, embedded NULs, high bytes, computed/direct specialization-cache identity, generated defaults, inline captures, nested aggregates, optionals and both fallible channels. sentinel checks observe final evaluation storage, diagnosing mutation, uninitialized bytes and invalid addresses; imported helpers, source/session disposal and repeated compilation retain ownership. mutable and nonbyte/nominal-byte slices, nonnull data pointers and untagged unions retain the escape boundary. - - [ ] remaining gate: general type-valued compile-time execution, broader constant materialization (including untagged unions, nonnull data pointers and slice types beyond readonly bytes), plus remaining existing-library acceptance for broader `meta` operations and `io.print`. formatting parsing and broader imported C-binding support may interleave with later gates. + - [x] computed-callback materialization checkpoint: ordinary compile-time calls return source-bound callbacks through interpreter/native execution, including optional/nested presence, arrays/records, tagged payloads and both fallible channels. temporary IR function IDs map back to exact source declarations before evaluation storage is released. computed/direct aggregate arguments share specialization-cache identity and generated defaults; imported private targets, same-spelled functions, signature diagnostics, runtime trap provenance and repeated compilation retain their contracts. nonnull data pointers, unsupported slice types and address-bearing untagged union layouts retain their escape boundary. + - [x] readonly byte-slice materialization checkpoint: ordinary compile-time calls snapshot readonly `[]u8` and zero-terminated `[;0]u8` results into owned literals before evaluation storage is released. string subviews, locally filled arrays, immutable global arrays and raw aggregate views preserve exact bytes, embedded NULs, declared slice types and normalized content-based cache identity through explicit `$()` expressions, generic inputs, generated defaults, nested records/arrays, tagged payloads, optionals and both fallible channels. imported helpers, session/source disposal and repeated compilation retain ownership through interpreter/native execution; uninitialized source bytes diagnose read-before-initialization and invalid ranges are rejected before dereference. nonzero sentinels are covered below; mutable and nonbyte/nominal-byte slices, nonnull data pointers and address-bearing untagged union layouts remain unsupported. + - [x] nonzero-sentinel byte-slice checkpoint: readonly `[;sentinel]u8` compile-time results own exact payload bytes and any byte sentinel through interpreter/native execution. canonical string-backed views preserve logical lengths, empty views, embedded NULs, high bytes, computed/direct specialization-cache identity, generated defaults, inline captures, nested aggregates, optionals and both fallible channels. sentinel checks observe final evaluation storage, diagnosing mutation, uninitialized bytes and invalid addresses; imported helpers, source/session disposal and repeated compilation retain ownership. mutable and nonbyte/nominal-byte slices, nonnull data pointers and address-bearing untagged union layouts retain the escape boundary. + - [x] untagged-union materialization checkpoint: address-free untagged unions own full native-layout byte snapshots through interpreter/native execution, including scalar/IEEE reinterpretation, nested padded records/arrays, smaller overlapping updates, zero-sized storage, distinct members and generated nominal unions. explicit `$()` values, compile-time arguments, full-byte specialization-cache identity, generated defaults, nested optionals/tagged payloads and both fallible channels preserve exact types and independent copies. uninitialized tail/padding bytes diagnose read-before-initialization; every member layout is checked recursively to reject data pointers, callbacks, strings and slices rather than retain interpreter handles. imported helpers/diagnostics, source/session disposal and repeated compilation retain ownership. address-bearing union layouts and broader constant materialization remain open. + - [ ] remaining gate: general type-valued compile-time execution, broader constant materialization (including address-bearing untagged unions, nonnull data pointers and slice types beyond readonly bytes), plus remaining existing-library acceptance for broader `meta` operations and `io.print`. formatting parsing and broader imported C-binding support may interleave with later gates. - [ ] remaining source constructs: multiline strings. boolean and optional-capture conditional expressions, yielding value blocks, `yield`, compound updates and bitwise operators are covered above. examples: `source/parser/parse.hon`, `source/llvm/runtime.hon`. - [ ] standard-library milestone: compile/run a multi-package program exercising existing allocation and IO, including error cleanup; no replacement miniature standard library. - [ ] native driver integration: required C ABI declarations, `process.Init` startup arguments/IO, actual CLI exit statuses, file loading, temporary files and spawning Clang. Honey-generated programs currently accept only parameterless `main` and print its result; Bro's bootstrap startup support is not Honey backend support. diff --git a/source/ast/ast.hon b/source/ast/ast.hon index 741c36b..1315635 100644 --- a/source/ast/ast.hon +++ b/source/ast/ast.hon @@ -83,6 +83,7 @@ ExprKind :: enum { try_expr catch_expr typed_constant + union_constant comptime_expr literal_void literal_undefined @@ -196,6 +197,7 @@ NodeKind :: enum { stmt_errdefer stmt_block expr_typed_constant + expr_union_constant expr_comptime literal_float_value literal_void @@ -611,6 +613,17 @@ add_typed_constant proc( }, at) } +#! retains an untagged union's complete native-layout bytes as a string literal. +add_union_constant proc( + builder @mut Builder, at Span, type, bytes NodeId, +) NodeId ! mem.AllocError { + return try add_node(builder, Node{ + kind = .expr_union_constant, + data0 = NodeData{ node_id = type }, + data1 = NodeData{ node_id = bytes }, + }, at) +} + add_comptime_expr proc(builder @mut Builder, at Span, operand NodeId) NodeId ! mem.AllocError { return try add_node(builder, Node{ kind = .expr_comptime, @@ -1599,6 +1612,7 @@ expression_kind proc(kind NodeKind) ExprKind { .expr_try: .try_expr .expr_catch: .catch_expr .expr_typed_constant: .typed_constant + .expr_union_constant: .union_constant .expr_comptime: .comptime_expr .expr_block: .block_expr .literal_float_value: .literal_float @@ -1652,6 +1666,15 @@ typed_constant proc(tree @Tree, id NodeId) TypedConstantView { } } +union_constant proc(tree @Tree, id NodeId) TypedConstantView { + node :: get_node(&tree.storage, id) + debug.assert(node.kind == .expr_union_constant) + return TypedConstantView{ + type = node.data0.node_id, + value = node.data1.node_id, + } +} + catch_expr proc(tree @Tree, id NodeId) CatchExprView { node :: get_node(&tree.storage, id) debug.assert(node.kind == .expr_catch) @@ -2007,7 +2030,7 @@ node_layout proc(node Node) NodeLayout { .literal_int, .literal_float, .literal_float_value, .literal_string, .literal_bool, .literal_null, .literal_void, .literal_undefined, .expr_identifier, .invalid_expr: {} .expr_unary: layout.direct_children = 2 - .expr_binary, .expr_range, .expr_index, .expr_selection, .expr_typed_constant, + .expr_binary, .expr_range, .expr_index, .expr_selection, .expr_typed_constant, .expr_union_constant, .expr_fallible_type, .expr_sum_type, .expr_array_type, .expr_slice_type: layout.direct_children = 6 .expr_pointer_type: layout.direct_children = 4 diff --git a/source/compilation/compilation.test.hon b/source/compilation/compilation.test.hon index 01c29b8..a78e1b7 100644 --- a/source/compilation/compilation.test.hon +++ b/source/compilation/compilation.test.hon @@ -1006,3 +1006,63 @@ computed_byte_slices_recompile_without_borrowing_evaluation_storage test { try testing.expect_equal(i64(42), outcome.value.data.integer) } } + +computed_untagged_unions_recompile_without_borrowing_storage_bytes test { + diagnostic_store := diagnostics.init(mem.c_allocator) + defer diagnostics.deinit(&diagnostic_store) + modules [2]mut ir.Module := undefined + { + session := init(mem.c_allocator) + defer deinit(&session) + root :: try add_package(&session) + dependency :: try add_package(&session) + bytes := try owned_text( + `lib :: import "../library" + `main proc() u64 { + ` value := $(lib.make()) + ` saved :: value + ` value.small = 0 + ` value.words[1] = 99 + ` if (saved.small != 9 or saved.bytes[1] != 255 or saved.bytes[7] != 255) return 0 + ` if (saved.bytes[17] != 255 or saved.bytes[23] != 255) return 0 + ` generated lib.Generated() :: {} + ` if (generated.value.words[1] != 40 or generated.value.small != 9) return 0 + ` return saved.words[1] + saved.words[3] + `} + , + ) + defer arraylist.deinit(&bytes) + root_source :: try add_source(&session, root, "/union_bytes/app/main.hon", &bytes, &diagnostic_store) + imported := try owned_text( + `Pair :: struct { small u8, large u64 } + `Storage :: union { small u8, rows [2]Pair, words [4]mut u64, bytes [32]u8 } + `make proc() Storage { + ` value := Storage { words = [maxval!(u64), 40, maxval!(u64), 2] } + ` value.small = 9 + ` return value + `} + `Generated proc() type { + ` return struct_type!(.auto, ["value"], [Storage], [make()]) + `} + , + ) + defer arraylist.deinit(&imported) + _ = try add_source(&session, dependency, "/union_bytes/library/storage.hon", &imported, &diagnostic_store) + try arraylist.append(&session.imports, Import{ + node = declaration(&session, root_source, 0).node, + target = dependency, + }) + modules[0] = try compile(&session, root, &diagnostic_store) + errdefer ir.deinit(&modules[0]) + modules[1] = try compile(&session, root, &diagnostic_store) + } + defer ir.deinit(&modules[0]) + defer ir.deinit(&modules[1]) + try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) + for modules |module| { + outcome := try interpreter.run(mem.c_allocator, &module, &diagnostic_store) + defer interpreter.deinit(&outcome) + try testing.expect(outcome.trap == null) + try testing.expect_equal(i64(42), outcome.value.data.integer) + } +} diff --git a/source/lower/lower.hon b/source/lower/lower.hon index 417984a..b78a9ad 100644 --- a/source/lower/lower.hon +++ b/source/lower/lower.hon @@ -1,3 +1,4 @@ +import "@std/debug" import "@std/mem" import "@std/arraylist" import "@std/strconv" @@ -2097,6 +2098,35 @@ lower_expression proc(state @mut State, id ast.NodeId) ir.ValueId ! (mem.AllocEr if (trap_type(state, id)) return ir.no_value return try lower_as(state, ast.typed_constant(tree, id).value, typecheck.type_of(state.checked, id)) } + .union_constant: { + if (trap_type(state, id)) return ir.no_value + bytes :: ast.union_constant(tree, id).value + text :: ast.literal_text(tree, bytes) + count :: ir.storage_size(state.module, .record, shape) + byte_shape :: try ir.add_array(state.module, .u8, ir.no_array, count, false) + scratch :: state.arguments.items.len + defer state.arguments.items = state.arguments.items.ptr[..scratch] + cursor := usize(1) + while (cursor < text.len - 1) { + byte :: ast.string_next(text, &cursor) catch |_| { + trap(state, try diagnostics.add(state.diagnostics, .invalid_escape, span), span) + return ir.no_value + } + element :: try constant(state, ir.Value{ type = .u8, data = ir.ValueData{ unsigned = u64(byte) } }, span) + try arraylist.append(&state.arguments, element) + } + debug.assert(state.arguments.items.len - scratch == count) + block :: &state.module.functions.items[state.function].blocks.items[usize(state.block)] + start :: block.arguments.items.len + for (state.arguments.items[scratch..]) |element| try arraylist.append(&block.arguments, element) + payload :: try emit(state, ir.Instruction{ + op = .array_literal, type = .array, array_type = byte_shape, + arguments_start = start, arguments_count = count, span = span, + }) + return try emit(state, ir.Instruction{ + op = .union_literal, type = .record, array_type = shape, left = payload, span = span, + }) + } .invalid: { invalid :: ast.invalid(tree, id) trap(state, invalid.diagnostic, invalid.span) diff --git a/source/sema/check/check.hon b/source/sema/check/check.hon index 9e70746..8d31f61 100644 --- a/source/sema/check/check.hon +++ b/source/sema/check/check.hon @@ -60,6 +60,8 @@ RecordField :: struct { RecordType :: struct { declaration ast.NodeId + # union syntax remains available even when structural checking erases nominal identity. + source ast.NodeId = ast.NodeId(maxval!(u32)) fields []RecordField is_union bool = false tag_type Type = Type{} @@ -1291,6 +1293,14 @@ collect_expression proc(state @mut State, id ast.NodeId) void ! (mem.AllocError try depend(state, value.type, id) try depend(state, value.value, id) } + .union_constant: { + value :: ast.union_constant(tree, id) + state.cells[usize(id)].fixed = true + state.cells[usize(id)].type = try check_annotation(state, value.type, false) + try collect_expression(state, value.value) + try depend(state, value.type, id) + try depend(state, value.value, id) + } .try_expr: { operand :: ast.type_operand(tree, id) try collect_expression(state, operand) @@ -2156,6 +2166,18 @@ validate proc(state @mut State, id ast.NodeId) void ! (mem.AllocError | diagnost value :: ast.typed_constant(tree, id) try compatible(state, value.value, state.cells[usize(id)].type) } + .union_constant: { + value :: ast.union_constant(tree, id) + type :: state.cells[usize(id)].type + if (type.kind == .invalid) return + if (type.kind != .record or ast.expr_kind(tree, value.value) != .literal_string) { + try fail(state, id, .invalid_operand, ast.node_span(tree, id)) + return + } + descriptor :: state.records.items[usize(type.record_type)] + if (!descriptor.is_union or descriptor.tag_type.kind != .none) + try fail(state, id, .invalid_operand, ast.node_span(tree, id)) + } .try_expr, .catch_expr: try validate_fallible_expression(state, id) .unary: { expression :: ast.unary_expr(tree, id) @@ -4195,7 +4217,7 @@ check_union proc(state @mut State, id ast.NodeId, nominal bool) Type ! (mem.Allo empty :: try mem.alloc(RecordField, state.records.allocator, 0) declaration :: if (nominal) id else ast.NodeId(maxval!(u32)) arraylist.append(&state.records, RecordType{ - declaration = declaration, fields = empty, is_union = true, + declaration = declaration, source = id, fields = empty, is_union = true, }) catch |err| { mem.free(state.records.allocator, empty) return err diff --git a/source/sema/resolve.hon b/source/sema/resolve.hon index b492fd4..b43a9d8 100644 --- a/source/sema/resolve.hon +++ b/source/sema/resolve.hon @@ -546,6 +546,11 @@ resolve_expr proc( try resolve_expr(state, scope, value.type) try resolve_expr(state, scope, value.value) } + .union_constant: { + value :: ast.union_constant(tree, id) + try resolve_expr(state, scope, value.type) + try resolve_expr(state, scope, value.value) + } .literal_int, .literal_float, .literal_bool, .literal_string, .literal_null, .literal_void, .literal_undefined: {} .enum_literal: {} diff --git a/source/sema/specialize/specialize.hon b/source/sema/specialize/specialize.hon index c5ad930..9dc3e92 100644 --- a/source/sema/specialize/specialize.hon +++ b/source/sema/specialize/specialize.hon @@ -1706,7 +1706,8 @@ branch_constant proc(state @State, id ast.NodeId, depth usize) bool { ) } match ast.expr_kind(&state.tree, id) { - .literal_int, .literal_float, .literal_bool, .literal_string, .literal_null, .literal_void: return true + .literal_int, .literal_float, .literal_bool, .literal_string, .literal_null, .literal_void, + .union_constant: return true .typed_constant: return branch_constant(state, ast.typed_constant(&state.tree, id).value, depth + 1) .enum_literal: return true .record_literal: { @@ -2332,6 +2333,32 @@ evaluate_expected proc(state @mut State, id, annotation ast.NodeId) Value ! Erro return try evaluate(state, id) } +# raw union snapshots cannot distinguish addresses from overlapping integer bits. +# require every view to be address-free before retaining the complete representation. +@hide:file +union_bytes_materializable proc(checked @typecheck.Result, type typecheck.Type, depth usize) bool { + if (depth >= 128) return false + match type.kind { + .int, .i8, .i16, .i32, .isize, .u8, .u16, .u32, .u64, .usize, + .float, .bool, .void, .enumeration: return true + .array, .optional: { + return union_bytes_materializable(checked, typecheck.array_type(checked, type.array_type).element, depth + 1) + } + .fallible: { + channels :: typecheck.array_type(checked, type.array_type) + return (union_bytes_materializable(checked, channels.element, depth + 1) + and union_bytes_materializable(checked, channels.error_type, depth + 1)) + } + .record: { + for (typecheck.record_type(checked, type.record_type).fields) |field| { + if (!union_bytes_materializable(checked, field.type, depth + 1)) return false + } + return true + } + else: return false + } +} + # values leaving temporary evaluation must own no interpreter addresses or module IDs. # snapshot readonly bytes and rebuild source literals/references before releasing their backing. @hide:file @@ -2417,8 +2444,25 @@ serialize_value proc( result = try ast.add_array_literal(state.builder, span, elements.items) } else if (type.kind == .record) { record :: typecheck.record_type(checked, type.record_type) - # untagged storage has no active member; choosing one could discard bytes. - if (record.is_union and record.tag_type.kind != .enumeration) return no_node + if (record.is_union and record.tag_type.kind != .enumeration) { + if (!union_bytes_materializable(checked, type, 0)) return no_node + bytes :: try mem.alloc(u8, state.allocator, value.data.elements.len) + defer mem.free(state.allocator, bytes) + for (value.data.elements) |byte, index| { + if (byte.uninitialized) { + state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .read_before_initialization, span) + return no_node + } + bytes[index] = u8(byte.data.unsigned) + } + annotation :: if (type.nominal != no_node) type.nominal + else if (record.declaration != no_node) record.declaration else record.source + concrete :: try materialize(state, Value{ kind = .type, node = annotation }) + literal :: try ast.add_string(state.builder, span, bytes) + result = try ast.add_union_constant(state.builder, span, concrete.node, literal) + try refresh(state) + return result + } fields := arraylist.init(ast.NodeId, state.allocator) defer arraylist.deinit(&fields) for (record.fields) |field, i| { @@ -2513,9 +2557,9 @@ constant_equal proc(state @mut State, left, right ast.NodeId) bool ! Error { kind :: ast.expr_kind(&state.tree, left) if (kind != ast.expr_kind(&state.tree, right)) return false if (kind == .literal_void) return true - if (kind == .typed_constant) { - a :: ast.typed_constant(&state.tree, left) - b :: ast.typed_constant(&state.tree, right) + if (kind == .typed_constant or kind == .union_constant) { + a :: if (kind == .union_constant) ast.union_constant(&state.tree, left) else ast.typed_constant(&state.tree, left) + b :: if (kind == .union_constant) ast.union_constant(&state.tree, right) else ast.typed_constant(&state.tree, right) return try same_type_context(state, a.type, b.type, true) and try constant_equal(state, a.value, b.value) } if (kind == .slice) { @@ -2561,8 +2605,8 @@ constant_hash proc(state @mut State, id ast.NodeId) usize ! Error { kind :: ast.expr_kind(&state.tree, id) hash :: enum_index(kind) if (kind == .literal_void) return hash - if (kind == .typed_constant) { - value :: ast.typed_constant(&state.tree, id) + if (kind == .typed_constant or kind == .union_constant) { + value :: if (kind == .union_constant) ast.union_constant(&state.tree, id) else ast.typed_constant(&state.tree, id) return mix(mix(hash, try type_hash_context(state, value.type, true)), try constant_hash(state, value.value)) } if (kind == .slice) { @@ -4803,6 +4847,10 @@ evaluate proc(state @mut State, id ast.NodeId) Value ! Error { return Value{} } match kind { + .union_constant: { + value :: ast.union_constant(&state.tree, id) + return Value{ kind = .aggregate, node = id, annotation = value.type } + } .typed_constant: { view :: ast.typed_constant(&state.tree, id) if (!is_type(state, view.type) and type_record_storage(state, view.type, 0)) { @@ -5287,8 +5335,9 @@ expression_type proc(state @mut State, id ast.NodeId, depth usize) ast.NodeId ! .record_type, .array_type, .slice_type, .pointer_type, .function_type, .optional_type, .enum_type, .union_type, .union_tag_type, .distinct_type, .fallible_type, .sum_type, .alias_expr: return try builtin(state, id, .type) - .typed_constant: { - annotation :: ast.typed_constant(&state.tree, id).type + .typed_constant, .union_constant: { + annotation :: if (kind == .union_constant) ast.union_constant(&state.tree, id).type + else ast.typed_constant(&state.tree, id).type return if (is_type(state, annotation)) annotation else try canonical_type(state, annotation) } .literal_int: return try builtin(state, id, .i64) diff --git a/tests/native.py b/tests/native.py index 0957de3..c140ab0 100644 --- a/tests/native.py +++ b/tests/native.py @@ -5736,13 +5736,177 @@ CASES = { main proc() i16 { return $(make()) catch |err| err.pointer^ } """, b"", b"compile-time result cannot escape evaluation", ), - "comptime_untagged_union_remains_an_escape_trap": ( + "comptime_untagged_union_preserves_scalar_and_ieee_bits": ( """ - Storage :: union { number i16, bits u16 } - make proc() Storage { return Storage { number = 42 } } - main proc() i16 { return $(make()).number } + Storage :: union { real f64, bits u64 } + make proc(bits u64) Storage { return Storage { bits = bits } } + main proc() bool { + one :: $(make(4607182418800017408)) + zero :: $(make(9223372036854775808)) + nan :: $(make(9221120237041090626)) + return ( + one.real == 1.0 and zero.bits == 9223372036854775808 + and nan.bits == 9221120237041090626 + ) + } + """, b"true\n", None, + ), + "comptime_untagged_union_owns_padding_tail_bytes_and_copies": ( + """ + Pair :: struct { small u8, large u64 } + Storage :: union { small u8, rows [2]mut Pair, words [4]mut u64, bytes [32]u8 } + make proc(prefix u64) Storage { + value := Storage { words = [prefix, 40, maxval!(u64), 2] } + value.small = 9 + return value + } + Token proc($value Storage) type { return struct {} } + main proc() [8]u64 { + if (Token(make(maxval!(u64))) == Token(make(9))) return [0, 0, 0, 0, 0, 0, 0, 0] + value := $(make(maxval!(u64))) + saved :: value + value.small = 0 + value.rows[0].large = 99 + other :: $(make(maxval!(u64))) + return [u64(saved.small), u64(saved.bytes[1]), u64(saved.bytes[7]), + saved.rows[0].large, u64(saved.bytes[17]), u64(saved.bytes[23]), + other.rows[1].large, value.rows[0].large] + } + """, b"[9, 255, 255, 40, 255, 255, 2, 99]\n", None, + ), + "comptime_untagged_union_cache_identity_and_generated_defaults": ( + """ + Produced proc() type { return union { small u16, words [2]mut u16, bits u32 } } + Storage :: alias Produced() + make proc(tail u16) Storage { + value := Storage { words = [0, tail] } + value.small = 40 + return value + } + Token proc($value Storage) type { return struct {} } + Fixed proc($value Storage) type { return struct { values [usize(value.words[1])]i16 } } + score proc($value Storage) u16 { return value.small + value.words[1] } + Generated proc() type { + return struct_type!(.auto, ["value"], [Storage], [make(2)]) + } + main proc() u16 { + if (Token(make(2)) != Token(Storage { bits = 131112 })) return 0 + if (Token(make(2)) != Token(Storage { words = [40, 2] })) return 0 + if (Token(make(2)) == Token(make(3))) return 0 + first Fixed(make(2)) :: { values = [20, 20] } + same Fixed(Storage { words = [40, 2] }) :: first + generated Generated() :: {} + if (generated.value.words[1] != 2 or same.values[1] != 20) return 0 + return score(make(2)) + } + """, b"42\n", None, + ), + "comptime_untagged_union_preserves_zero_size_and_nominal_members": ( + """ + Number :: distinct u16 + Kind :: enum(u16) { answer = 42 } + Storage :: union { number Number, kind Kind } + Empty :: union { bytes [0]u8, record struct {} } + make proc() Storage { return Storage { kind = .answer } } + empty proc() Empty { return Empty { bytes = [] } } + exact proc(value Storage) Number { return value.number } + main proc() u16 { + zero :: $(empty()) + if (sizeof!(Empty) != 0 or zero.bytes.len != 0) return 0 + return u16(exact($(make()))) + } + """, b"42\n", None, + ), + "comptime_untagged_union_nested_optional_tagged_and_fallible_channels": ( + """ + Storage :: union { small u16, words [2]u16 } + Choice :: union(enum) { data Storage, empty void } + Failure :: struct { value Storage } + Packet :: struct { + maybe [2]?Storage + choice Choice + success Storage ! Failure + failure Storage ! Failure + } + make proc() Packet { + value Storage :: { words = [40, 2] } + return { + maybe = [value, null], choice = .data{value}, + success = value, failure = Failure { value = value }, + } + } + score proc(value Storage) u16 { return value.words[0] + value.words[1] } + main proc() [4]u16 { + packet :: $(make()) + if (packet.maybe[1] != null) return [0, 0, 0, 0] + good :: packet.success catch Storage { small = 0 } + bad :: packet.failure catch |err| err.value + return [score(packet.maybe[0]?), score(packet.choice.data), score(good), score(bad)] + } + """, b"[42, 42, 42, 42]\n", None, + ), + "comptime_untagged_union_rejects_source_pointer": ( + """ + Storage :: union { bits u64, pointer @i16 } + make proc() Storage { + value i16 :: 42 + return Storage { pointer = &value } + } + main proc() u64 { return $(make()).bits } """, b"", b"compile-time result cannot escape evaluation", ), + "comptime_untagged_union_rejects_inactive_pointer_layout": ( + """ + Storage :: union { bits u64, pointer ?@i16 } + make proc() Storage { return Storage { bits = 0 } } + main proc() u64 { return $(make()).bits } + """, b"", b"compile-time result cannot escape evaluation", + ), + "comptime_untagged_union_rejects_inactive_callback_layout": ( + """ + Storage :: union { bits u64, callback @proc(value i16) i16 } + make proc() Storage { return Storage { bits = 42 } } + main proc() u64 { return $(make()).bits } + """, b"", b"compile-time result cannot escape evaluation", + ), + "comptime_untagged_union_rejects_inactive_string_layout": ( + """ + Storage :: union { bits [2]u64, text string } + make proc() Storage { return Storage { bits = [0, 0] } } + main proc() u64 { return $(make()).bits[0] } + """, b"", b"compile-time result cannot escape evaluation", + ), + "comptime_untagged_union_rejects_nested_inactive_slice_layout": ( + """ + Storage :: union { bits [2]u64, rows [1]struct { bytes []u8 } } + make proc() Storage { return Storage { bits = [0, 0] } } + main proc() u64 { return $(make()).bits[0] } + """, b"", b"compile-time result cannot escape evaluation", + ), + "comptime_untagged_union_rejects_uninitialized_tail_bytes": ( + """ + Storage :: union { small u8, words [2]mut u64 } + make proc() Storage { + value Storage := undefined + value.small = 42 + return value + } + main proc() u8 { return $(make()).small } + """, b"", b"read before initialization", + ), + "comptime_untagged_union_rejects_uninitialized_padding_bytes": ( + """ + Pair :: struct { small u8, large u64 } + Storage :: union { pair Pair, words [2]u64 } + make proc() Storage { + value Storage := undefined + value.pair.small = 1 + value.pair.large = 42 + return value + } + main proc() u64 { return $(make()).pair.large } + """, b"", b"read before initialization", + ), "comptime_float_bits_survive_aggregate_materialization": ( """ Floats :: struct { zero f64, nan f64 } @@ -9127,7 +9291,45 @@ def check_specialization_packages(temporary, executable): } """).strip() + "\n") check(app, executable, (b"[40, 2, 40]\n", None)) - return 24 + sizing.write_text(textwrap.dedent(""" + Storage :: union { small u16, words [2]mut u16 } + make proc() Storage { + value := Storage { words = [0, 2] } + value.small = 40 + return value + } + Token proc($value Storage) type { return struct {} } + Generated proc() type { + return struct_type!(.auto, ["value"], [Storage], [make()]) + } + """).strip() + "\n") + source.write_text(textwrap.dedent(""" + lib :: import "../library" + main proc() u16 { + direct :: $(lib.make()) + value lib.Generated() :: {} + if (lib.Token(lib.make()) != lib.Token(lib.Storage { words = [40, 2] })) return 0 + return direct.small + value.value.words[1] + } + """).strip() + "\n") + check(app, executable, (b"42\n", None)) + sizing.write_text(textwrap.dedent(""" + Storage :: union { small u8, words [2]u64 } + make proc() Storage { + value u8 :: 255 + overflow :: value + 1 + return Storage { words = [u64(overflow), 2] } + } + """).strip() + "\n") + source.write_text(textwrap.dedent(""" + lib :: import "../library" + main proc() u8 { return $(lib.make()).small } + """).strip() + "\n") + check(app, executable, (b"", b"integer arithmetic overflow")) + interpreted = run(str(HONEY), "run", str(app)) + assert interpreted.stderr.startswith(f"{sizing.resolve()}:4:".encode()), outcome(interpreted) + assert len(interpreted.stderr.splitlines()) == 1, outcome(interpreted) + return 26 def check_dependency_provenance(temporary, executable):