feat: materialize readonly compile-time byte slices

This commit is contained in:
2026-10-30 14:23:39 +01:00
parent 6748b38eac
commit 8e790b30cc
7 changed files with 505 additions and 34 deletions
+7 -5
View File
@@ -116,7 +116,7 @@ initialized records/tuples containing `type` fields live inside compile-time pro
fixed type arrays may also start with `undefined` inside type/type-array producers: `types [2]mut type := undefined`, then fill `types[index] = i16` in a loop. nested and whole-array updates preserve shape, mutability and independent initialization state in local copies; `.len` is available before elements initialize. reading an uninitialized type leaf, returning an incomplete array or passing one to a compile-time procedure diagnoses a read before initialization. empty shapes are valid, but every dimension must be explicit and nonnegative. construction charges each expanded cell against the shared execution bound, including nested zero-leaf shapes.
ordinary fixed value arrays use the same construction inside type/type-array producers: `counts [2]mut usize := undefined`, then fill indexed elements or replace whole rows/arrays. initialized leaves retain exact scalar, string, optional, record and tagged-union values; local copies retain independent partial initialization state. generated lengths/defaults and compile-time arguments can use initialized elements, while incomplete arrays cannot enter checked IR or procedure arguments. array `.len` and empty shapes do not read leaves. partial type-valued record construction and storage-bearing constant materialization such as nonnull pointers/slices remain unsupported.
ordinary fixed value arrays use the same construction inside type/type-array producers: `counts [2]mut usize := undefined`, then fill indexed elements or replace whole rows/arrays. initialized leaves retain exact scalar, string, optional, record and tagged-union values; local copies retain independent partial initialization state. generated lengths/defaults and compile-time arguments can use initialized elements, while incomplete arrays cannot enter checked IR or procedure arguments. array `.len` and empty shapes do not read leaves. partial type-valued record construction and broader storage-bearing constant materialization beyond readonly byte slices remain unsupported.
ordinary procedures also accept `undefined` as the initializer of a mutable local with a concrete runtime type, including nested records and fixed arrays. field/index writes initialize storage without reading its previous contents; aggregate copies retain independent contents and initialization state in the interpreter. ordinary compile-time calls can fill this storage and return materializable values, but observing or materializing an uninitialized leaf is diagnosed. native storage uses LLVM poison, not zero initialization: assign every leaf before observing it, and do not rely on native read-before-initialization traps for `undefined` storage. immutable bindings, globals, inferred array lengths and other expression uses remain invalid.
@@ -124,25 +124,27 @@ ordinary procedures also accept `undefined` as the initializer of a mutable loca
procedures returning fixed arrays or records/tuples containing `type`, such as `types proc() [2]type { return [i16, bool] }` or `config proc() struct { child type, count usize } { return { child = i16, count = 2 } }`, execute entirely at compile time. every argument is explicitly supplied, constant, cached and erased; nested/aliased results, empty shapes and dependent `$N` lengths retain exact shape and mutability. compile-time producers accept ordinary type-array/type-record inputs, and runtime generic procedures accept erased `$` parameters with these shapes. direct projections generate signatures, constructors, storage and defaults; explicit `$()` calls inside producers retain owned values. cache identity uses canonical type leaves without merging distinct nominal types, and copies cannot mutate cached inputs/results. imported diagnostics, shared type-control execution limits and repeatable compilation work through interpreter/native execution. ordinary parameters remain excluded from dependent signatures; use `$N` for lengths. runtime type-array/type-record storage and reflection remain open.
optional `if`/`while` captures also execute in type-producing bodies: `if count |length| { return [length]i16 }` generates storage from an optional `usize`. ordinary checked IR validates each executed guard; present payloads become immutable owned constants with exact scalar/aggregate types and strings, and absent optionals skip the body. while guards observe updated locals on each iteration; captures stay body-local across break/continue and early return. generated lengths/defaults, imported producers, cache identity and repeated compilation preserve these values. null data/function pointers materialize as null; callbacks retain source declaration identity, while nonnull data pointers and slices retain the constant-materialization escape boundary.
optional `if`/`while` captures also execute in type-producing bodies: `if count |length| { return [length]i16 }` generates storage from an optional `usize`. ordinary checked IR validates each executed guard; present payloads become immutable owned constants with exact scalar/aggregate types and strings, and absent optionals skip the body. while guards observe updated locals on each iteration; captures stay body-local across break/continue and early return. generated lengths/defaults, imported producers, cache identity and repeated compilation preserve these values. null data/function pointers materialize as null; callbacks retain source declaration identity, while nonnull data pointers and unsupported slice types retain the constant-materialization escape boundary.
type-producing bodies also execute scalar/enum `match` statements and type-valued match expressions in initializers, assignments and returns, such as `return match wide { true: i16, false: i8 }`. selected expression arms may themselves be matches. an ordinary checked IR match returns the selected arm index, preserving subject/pattern typing, range boundaries, overlap, unreachable-arm and exhaustiveness diagnostics without checking unexecuted bodies. statement arms retain scoped locals, assignments, early returns and enclosing-loop break/continue; expression selection shares the type-control step bound.
matches also select nested types, array lengths, generic arguments, callback signatures and field defaults in type-producing bodies. for example, `return struct { values [2](match wide { true: i16, false: i8 }) }` generates concrete storage without elaborating the unselected arm. snapshots preserve loop-local type/value bindings, selected literals retain their spelling and source provenance, and nominal results are cached before their recursive fields are expanded. deferred type selection shares the producer's execution bound; failed elaboration retains its original diagnostic without leaving unchecked generated types behind. general type-valued execution remains open.
tagged-union matches in type-producing bodies support immutable by-value payload captures in statements, expressions and nested type/storage positions. for example, `match choice { .count |length|: return [length]i16, .empty: return bool }` uses a captured `usize` to generate storage. grouped variants require identical nonvoid payload types; invalid captures in unselected arms are still diagnosed. selected payloads become owned typed constants, preserving narrow integers, nested aggregates and strings across generated lengths/defaults, loop snapshots and repeatable compilation. data-pointer/slice-bearing active payloads retain the constant-materialization escape boundary below.
tagged-union matches in type-producing bodies support immutable by-value payload captures in statements, expressions and nested type/storage positions. for example, `match choice { .count |length|: return [length]i16, .empty: return bool }` uses a captured `usize` to generate storage. grouped variants require identical nonvoid payload types; invalid captures in unselected arms are still diagnosed. selected payloads become owned typed constants, preserving narrow integers, nested aggregates and strings across generated lengths/defaults, loop snapshots and repeatable compilation. active payloads containing data pointers or unsupported slice types retain the constant-materialization escape boundary below.
ordinary procedures may compute integer, boolean, float and string compile-time arguments and return nested records/tuples, fixed arrays, tagged unions and explicit fallible results from constant inputs, such as `Fixed(i16, length(2))`. calls reuse the ordinary checker, typed IR and interpreter, preserving local type inference, checked arithmetic, mutable locals, range/while loops, break/continue and recursive or specialized helper calls. aggregate and fallible arguments receive their declared context, including dependent `[N]T` parameters; cached specializations compare normalized values, not literal spelling or evaluation identity. returned constants become owned source literals before temporary interpreter storage is released, preserving by-value copies, exact integer types, string bytes, enum/optional fields and IEEE float bits. tagged unions retain their actual discriminant and only the active payload, including nested aggregates and void/zero-cell variants. fallible constants retain success/error identity, composed or payload-bearing errors and void success; `catch` can consume them in generated lengths, defaults and runtime expressions.
computed callbacks materialize as references to their original source declarations, never as temporary interpreter function IDs. `choose proc() @proc(value i16) i16 { return increment }` can supply `$(choose())`, compile-time arguments and generated defaults. optional callbacks preserve absence and nested presence; arrays, records, tagged payloads and both fallible channels retain exact callback signatures and specialization-cache identity. imported private targets remain bound to their originating declarations, including across repeated compilation and native emission.
`$(expression)` explicitly evaluates a constant expression, including an ordinary procedure call, even in a package without templates. use a typed expression when its shape cannot be inferred independently, such as `$(Point { x = 40 })`; integer literals retain contextual typing. the resulting value works in runtime expressions, bindings, expression statements and array lengths, while the erased expression still receives ordinary checking. each interpreter evaluation is isolated and limited to 100,000 IR instructions/terminators and 128 active frames; failures retain their originating source locations. runtime parameters and mutable local/global dependencies cannot supply compile-time inputs. nonnull data pointers, slices and untagged unions cannot escape temporary evaluation as generated constants; active payloads or aggregate fields containing unsupported values are diagnosed rather than retaining interpreter handles. untagged unions lack an active-member identity, so selecting a source variant could discard observable bytes. executed foreign calls remain interpreter traps. general type-valued execution and broader existing-library acceptance remain open.
readonly byte-slice results (`[]u8` and zero-terminated `[;0]u8`) snapshot their bytes into owned literals before evaluation storage is released. ordinary compile-time helpers can return string subviews, views of locally filled arrays or immutable global arrays, including nested records/arrays, tagged payloads, optionals and both fallible channels. explicit `$()` expressions, generic arguments and generated defaults preserve embedded NULs, arbitrary byte values, exact declared slice types and content-based specialization-cache identity. snapshots survive evaluation/session disposal and repeated compilation; uninitialized source bytes diagnose read-before-initialization. mutable slices, nonbyte or nominal-byte slices and nonzero-sentinel slices remain unsupported.
`$(expression)` explicitly evaluates a constant expression, including an ordinary procedure call, even in a package without templates. use a typed expression when its shape cannot be inferred independently, such as `$(Point { x = 40 })`; integer literals retain contextual typing. the resulting value works in runtime expressions, bindings, expression statements and array lengths, while the erased expression still receives ordinary checking. each interpreter evaluation is isolated and limited to 100,000 IR instructions/terminators and 128 active frames; failures retain their originating source locations. runtime parameters and mutable local/global dependencies cannot supply compile-time inputs. nonnull data pointers, unsupported slice types and untagged unions cannot escape temporary evaluation as generated constants; active payloads or aggregate fields containing unsupported values are diagnosed rather than retaining interpreter handles. untagged unions lack an active-member identity, so selecting a source variant could discard observable bytes. executed foreign calls remain interpreter traps. general type-valued execution, broader storage-bearing materialization and remaining existing-library acceptance, including broader `meta` operations and `io.print`, remain open.
`if $(condition) { ... } else { ... }` evaluates a boolean guard before expanding or checking either body, including calls to ordinary constant-input procedures. concrete generic specializations also select branches whose guards use constants, immutable constant bindings, type comparisons or substituted compile-time parameters. only the selected body is specialized and checked; both bodies are still parsed and name-resolved. selected bodies retain their scope, cleanup and return behavior, and erased guards retain ordinary type checking and source diagnostics. ordinary calls in an implicit guard stay runtime; use `$()` to request their evaluation. optional captures remain runtime, and ordinary nongeneric `if (true)` still checks both bodies.
`compile_error!(message)` requires a compile-time string and reports its owned text at the call site when that body is elaborated. unselected compile-time branches do not report it; ordinary runtime branches remain checked. messages from constant-input procedures retain their bytes through later evaluation, interpreter diagnostics and native executable emission.
`match $(subject) { ... }` selects an expression or statement arm before specializing or checking its body, including ordinary constant-input calls. concrete generic procedures also select matches on side-effect-free constant subjects, including substituted scalar/enum/tagged-union arguments and pure field/index projections. ordinary checked IR validates every pattern and capture declaration before selection; only the selected body is elaborated, while all bodies remain parsed and name-resolved. by-value payload captures retain exact types and owned strings, and selected bodies may use runtime parameters, nested matches, cleanup, returns and enclosing-loop transfers. ordinary call subjects, runtime parameters and mutable locals stay runtime, as do contextual aggregate bindings that might omit runtime defaults; ordinary nongeneric literal matches still check all arms. use `$()` to request explicit evaluation. data-pointer/slice-bearing payloads retain the escape boundary above.
`match $(subject) { ... }` selects an expression or statement arm before specializing or checking its body, including ordinary constant-input calls. concrete generic procedures also select matches on side-effect-free constant subjects, including substituted scalar/enum/tagged-union arguments and pure field/index projections. ordinary checked IR validates every pattern and capture declaration before selection; only the selected body is elaborated, while all bodies remain parsed and name-resolved. by-value payload captures retain exact types and owned strings, and selected bodies may use runtime parameters, nested matches, cleanup, returns and enclosing-loop transfers. ordinary call subjects, runtime parameters and mutable locals stay runtime, as do contextual aggregate bindings that might omit runtime defaults; ordinary nongeneric literal matches still check all arms. use `$()` to request explicit evaluation. unsupported storage-bearing payloads retain the escape boundary above.
type `==` and `!=` comparisons in compile-time expressions and specializations preserve aliases, nominal declaration identity and structural pointer/array/callback signatures. composed error-set identity and specialization caching ignore member ordering, grouping and repetition. for example, `read proc($T type, value T) i16 { if (T == i16) return value else return value.count }` checks the scalar or record path for each concrete call without requiring both operations on every `T`.
+9 -5
View File
@@ -20,9 +20,12 @@ traps or compiler infrastructure failures.
cached generic specialization and type-producing procedures support explicit/inferred
compile-time arguments and generated record/array storage through both paths.
ordinary procedures can compute scalar, nested record/array, tagged-union,
fallible and callback compile-time values using mutable locals, loops and recursive calls;
explicit `$()` expressions use the same constant evaluation path. compile-time
`if`/`match` selection and type comparisons specialize only the selected body,
fallible, callback and readonly byte-slice compile-time values using mutable locals, loops and recursive calls;
explicit `$()` expressions use the same constant evaluation path.
readonly `[]u8` and zero-terminated `[;0]u8` snapshots own exact bytes after evaluation.
mutable, nonbyte/nominal-byte and nonzero-sentinel slices, nonnull data pointers
and untagged unions retain the constant-materialization escape boundary.
compile-time `if`/`match` selection and type comparisons specialize only the selected body,
including owned by-value tagged payload captures in ordinary generic procedures.
type-producing procedures also accept ordinary
constant inputs, cached and erased like explicit compile-time arguments. mutable
@@ -142,8 +145,9 @@ test discovery and FFI remain outside these completed slices.
- [x] compound/bitwise checkpoint: arithmetic and bitwise compound updates evaluate mutable local/field/index/pointer places and their old values once before the RHS through interpreter/native execution. integer complement, and/or/xor, truncating left shifts, signed/unsigned right shifts and saturating left shifts preserve widths and distinct identity; unsigned counts retain independent types. known invalid counts diagnose even in untaken runtime branches; dynamic counts trap without native LLVM poison. checked arithmetic, float-only division, readonly diagnostics, scoped cleanup, ordinary constants, generated storage, generic branch selection and repeatable compilation retain existing semantics. type-valued `|` remains composition while runtime `|` and capture delimiters retain their grammar. full hash-map acceptance remains open.
- [x] hash-map execution checkpoint: existing `std/hashmap.HashMap`, `AutoHashMap` and `StringHashMap` run collision chains, zero-hash normalization, growth/rehashing, duplicate-key errors, absent lookups and owned aggregate values without replacement library code. interpreter checks use separate typed allocation buffers and preserve the original map on allocation failure, including cleanup; native `mem.c_allocator` checks cover all integer widths/boundaries, booleans, sparse/aliased enums, nested distinct/array/record keys with padding ignored, content-based byte strings and custom callbacks at `-O0`/`-O2`. specialization retains `usize` collection indices, nominal/explicit pointer-cast result types, rigid slice borrowing/string context and source-known optional callbacks; ordinary checking rejects incompatible elements and readonly reversal. `unreachable` demand-traps with provenance through both paths and compile-time execution; literal-true loops distinguish escaping and nested breaks. `compile_error!` owns computed string diagnostics and skips unselected compile-time bodies. initializer inference from expected result types is covered below.
- [x] expected-result inference checkpoint: omitted compile-time type/value arguments infer from rigid runtime arguments and expected results before contextual literal defaults. typed bindings, assignments, returns, ordinary/nested calls, aggregate fields/elements/defaults, optional/fallible channels, callbacks, match arms, yielding/catch blocks, arithmetic/bitwise operands and explicit `$()` expressions retain exact checked types through interpreter/native execution. generated nominal results infer counts and private/custom callback identities; existing `hashmap.init(allocator)` initializes `AutoHashMap`, `StringHashMap` and custom `HashMap` without replacement library code. rigid conflicts, unrelated nominal generators and unresolved arguments diagnose; explicit calls and fully explicit type-producer inputs retain their contracts. owned specializations and repeat compilation are verified.
- [x] computed-callback materialization checkpoint: ordinary compile-time calls return source-bound callbacks through interpreter/native execution, including optional/nested presence, arrays/records, tagged payloads and both fallible channels. temporary IR function IDs map back to exact source declarations before evaluation storage is released. computed/direct aggregate arguments share specialization-cache identity and generated defaults; imported private targets, same-spelled functions, signature diagnostics, runtime trap provenance and repeated compilation retain their contracts. nonnull data pointers, slices and untagged unions retain their escape boundary.
- [ ] remaining gate: general type-valued compile-time execution, broader constant materialization (including untagged unions and storage-bearing values), plus remaining existing-library acceptance for broader `meta` operations and `io.print`. formatting parsing and broader imported C-binding support may interleave with later gates.
- [x] computed-callback materialization checkpoint: ordinary compile-time calls return source-bound callbacks through interpreter/native execution, including optional/nested presence, arrays/records, tagged payloads and both fallible channels. temporary IR function IDs map back to exact source declarations before evaluation storage is released. computed/direct aggregate arguments share specialization-cache identity and generated defaults; imported private targets, same-spelled functions, signature diagnostics, runtime trap provenance and repeated compilation retain their contracts. nonnull data pointers, unsupported slice types and untagged unions retain their escape boundary.
- [x] readonly byte-slice materialization checkpoint: ordinary compile-time calls snapshot readonly `[]u8` and zero-terminated `[;0]u8` results into owned literals before evaluation storage is released. string subviews, locally filled arrays, immutable global arrays and raw aggregate views preserve exact bytes, embedded NULs, declared slice types and normalized content-based cache identity through explicit `$()` expressions, generic inputs, generated defaults, nested records/arrays, tagged payloads, optionals and both fallible channels. imported helpers, session/source disposal and repeated compilation retain ownership through interpreter/native execution; uninitialized source bytes diagnose read-before-initialization and invalid ranges are rejected before dereference. mutable, nonbyte/nominal-byte and nonzero-sentinel slices, nonnull data pointers and untagged unions remain unsupported.
- [ ] remaining gate: general type-valued compile-time execution, broader constant materialization (including untagged unions, nonnull data pointers and slice types beyond readonly plain/zero-terminated bytes), plus remaining existing-library acceptance for broader `meta` operations and `io.print`. formatting parsing and broader imported C-binding support may interleave with later gates.
- [ ] remaining source constructs: multiline strings. boolean and optional-capture conditional expressions, yielding value blocks, `yield`, compound updates and bitwise operators are covered above. examples: `source/parser/parse.hon`, `source/llvm/runtime.hon`.
- [ ] standard-library milestone: compile/run a multi-package program exercising existing allocation and IO, including error cleanup; no replacement miniature standard library.
- [ ] native driver integration: required C ABI declarations, `process.Init` startup arguments/IO, actual CLI exit statuses, file loading, temporary files and spawning Clang. Honey-generated programs currently accept only parameterless `main` and print its result; Bro's bootstrap startup support is not Honey backend support.
+57
View File
@@ -946,3 +946,60 @@ computed_callbacks_recompile_without_retaining_interpreter_function_ids test {
try testing.expect_equal(i64(42), outcome.value.data.integer)
}
}
computed_byte_slices_recompile_without_borrowing_evaluation_storage test {
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
modules [2]mut ir.Module := undefined
{
session := init(mem.c_allocator)
defer deinit(&session)
root :: try add_package(&session)
dependency :: try add_package(&session)
bytes := try owned_text(
`lib :: import "../library"
`main proc() i16 {
` packet :: $(lib.make())
` if (packet.bytes.len != 4 or packet.suffix.len != 3) return -1
` if (packet.bytes[0] != 255 or packet.bytes[1] != 0) return -2
` if (packet.bytes[2] != 128 or packet.bytes[3] != 42) return -3
` if (packet.suffix[0] != 65 or packet.suffix[1] != 0) return -4
` if (packet.suffix[2] != 66 or packet.suffix.ptr[3] != 0) return -5
` return 42
`}
,
)
defer arraylist.deinit(&bytes)
root_source :: try add_source(&session, root, "/byte_slices/app/main.hon", &bytes, &diagnostic_store)
imported := try owned_text(
`Packet :: struct { bytes []u8, suffix [;0]u8 }
`make proc() Packet {
` bytes [4]mut u8 := undefined
` bytes[0] = 255
` bytes[1] = 0
` bytes[2] = 128
` bytes[3] = 42
` return Packet { bytes = bytes[..], suffix = "xA\0B"[1..;0] }
`}
,
)
defer arraylist.deinit(&imported)
_ = try add_source(&session, dependency, "/byte_slices/library/bytes.hon", &imported, &diagnostic_store)
try arraylist.append(&session.imports, Import{
node = declaration(&session, root_source, 0).node,
target = dependency,
})
modules[0] = try compile(&session, root, &diagnostic_store)
errdefer ir.deinit(&modules[0])
modules[1] = try compile(&session, root, &diagnostic_store)
}
defer ir.deinit(&modules[0])
defer ir.deinit(&modules[1])
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
for modules |module| {
outcome := try interpreter.run(mem.c_allocator, &module, &diagnostic_store)
defer interpreter.deinit(&outcome)
try testing.expect(outcome.trap == null)
try testing.expect_equal(i64(42), outcome.value.data.integer)
}
}
+78 -10
View File
@@ -9,12 +9,27 @@ Outcome :: struct {
trap ?diagnostics.DiagnosticId = null
allocator ?mem.Allocator = null
owned_elements ?[]mut ir.Value = null
evaluation_memory ?[]mut EvaluationMemory = null
}
#! release an escaped aggregate with its retained allocator; scalar and trap outcomes need no allocation.
#! strings and aggregate descriptors borrow the module, which must outlive this outcome.
#! releases result storage and any retained compile-time evaluation memory.
#! string IDs and aggregate descriptors borrow the module, which must outlive this outcome.
deinit proc(outcome @mut Outcome) void {
if (outcome.owned_elements) |elements| mem.free(outcome.allocator?, elements)
if (outcome.evaluation_memory) |owned| {
memory :: &owned[0]
allocator :: outcome.allocator?
for (memory.frames.items) |frame| free_frame(allocator, frame)
arraylist.deinit(&memory.frames)
arraylist.deinit(&memory.pointers)
mem.free(allocator, memory.globals)
mem.free(allocator, memory.aggregate_globals)
mem.free(allocator, memory.strings)
mem.free(allocator, memory.string_cells)
mem.free(allocator, memory.scratch)
mem.free(allocator, owned)
}
outcome.evaluation_memory = null
outcome.owned_elements = null
outcome.allocator = null
outcome.value = ir.Value{}
@@ -82,6 +97,31 @@ Frame :: struct {
addresses []mut Address
}
@hide:file
EvaluationMemory :: struct {
globals []mut GlobalState
aggregate_globals []mut ir.Value
strings []mut StringView
string_cells []mut ir.Value
scratch []mut ir.Value
pointers arraylist.ArrayList(Pointer)
frames arraylist.ArrayList(Frame)
}
#! borrows one checked byte from retained evaluation storage; null means an invalid pointer/range.
evaluation_byte proc(outcome @Outcome, module @ir.Module, pointer ir.Value, index usize) ?ir.Value {
if (outcome.evaluation_memory == null) return null
memory :: &outcome.evaluation_memory?[0]
handle :: pointer.data.unsigned
if (handle >= u64(memory.pointers.items.len)) return null
location := memory.pointers.items[usize(handle)]
if (index > maxval!(usize) - location.byte_offset) return null
location.byte_offset += index
if (pointer_address(module, location, .u8, ir.no_array)) |address|
return read_address(module, address, memory.scratch)
return null
}
#! executes typed IR with invocation/global storage released on every exit.
#! aggregate results own flat storage released by `deinit`; string IDs and shapes resolve through `module`.
run proc(
@@ -93,8 +133,7 @@ run proc(
}
#! executes an isolated compile-time module with bounded work and stack depth.
#! ownership and diagnostic identities follow run.
#! temporary execution storage uses `allocator` and is released before returning.
#! result backing, including addressed completed frames, remains owned by the outcome until `deinit`.
run_evaluation proc(
allocator mem.Allocator,
module @ir.Module,
@@ -119,8 +158,9 @@ execute proc(
return Outcome{ trap = module.entry_diagnostic }
}
transferred := false
globals :: try mem.alloc(GlobalState, allocator, module.globals.items.len)
defer mem.free(allocator, globals)
defer if (!transferred) mem.free(allocator, globals)
mem.set(globals, GlobalState{})
global_cells usize := 0
for (module.globals.items) |global| {
@@ -130,7 +170,7 @@ execute proc(
global_cells += cells
}
aggregate_globals :: try mem.alloc(ir.Value, allocator, global_cells)
defer mem.free(allocator, aggregate_globals)
defer if (!transferred) mem.free(allocator, aggregate_globals)
global_offset usize := 0
for (module.globals.items) |global, i| {
if (!ir.aggregate(global.type)) continue
@@ -144,7 +184,7 @@ execute proc(
if (shape.byte_size > scratch_size) scratch_size = shape.byte_size
}
scratch :: try mem.alloc(ir.Value, allocator, scratch_size)
defer mem.free(allocator, scratch)
defer if (!transferred) mem.free(allocator, scratch)
source_scratch :: try mem.alloc(ir.Value, allocator, scratch_size)
defer mem.free(allocator, source_scratch)
string_size usize := 0
@@ -153,9 +193,9 @@ execute proc(
string_size += string.len + 1
}
string_cells :: try mem.alloc(ir.Value, allocator, string_size)
defer mem.free(allocator, string_cells)
defer if (!transferred) mem.free(allocator, string_cells)
strings :: try mem.alloc(StringView, allocator, module.strings.items.len)
defer mem.free(allocator, strings)
defer if (!transferred) mem.free(allocator, strings)
string_offset usize := 0
for (module.strings.items) |string, index| {
cells :: string_cells[string_offset..string_offset + string.len + 1]
@@ -165,7 +205,12 @@ execute proc(
string_offset += cells.len
}
pointers := arraylist.init(Pointer, allocator)
defer arraylist.deinit(&pointers)
defer if (!transferred) arraylist.deinit(&pointers)
retained := arraylist.init(Frame, allocator)
defer if (!transferred) {
for (retained.items) |frame| free_frame(allocator, frame)
arraylist.deinit(&retained)
}
frames := arraylist.init(Frame, allocator)
defer {
for (frames.items) |frame| free_frame(allocator, frame)
@@ -211,6 +256,21 @@ execute proc(
globals[global_index].status = .ready
}
if (frames.items.len == 1) {
if $(bounded) {
if (pointers.items.len != 0 and (ir.aggregate(value.type) or value.type == .pointer)) {
owned :: try mem.alloc(EvaluationMemory, allocator, 1)
errdefer mem.free(allocator, owned)
try arraylist.append(&retained, completed)
_ = arraylist.pop(&frames)
owned[0] = EvaluationMemory{
globals = globals, aggregate_globals = aggregate_globals,
strings = strings, string_cells = string_cells, scratch = scratch,
pointers = pointers, frames = retained,
}
transferred = true
return Outcome{ value = value, allocator = allocator, evaluation_memory = owned }
}
}
if (ir.aggregate(value.type)) {
escaped :: try mem.alloc(ir.Value, allocator, value.data.elements.len)
memcopy!(escaped, value.data.elements)
@@ -220,6 +280,14 @@ execute proc(
return Outcome{ value = value }
}
if (completed.destination != ir.no_value) copy_value(module, &frames.items[frame_index - 1].values[usize(completed.destination)], value)
if $(bounded) {
# ponytail: retain address-bearing frames until snapshotting; trace live roots if memory grows.
if (completed.addresses.len != 0) {
try arraylist.append(&retained, completed)
_ = arraylist.pop(&frames)
continue
}
}
_ = arraylist.pop(&frames)
free_frame(allocator, completed)
continue
+61
View File
@@ -494,6 +494,67 @@ frames_and_globals_are_freed_at_every_allocation_boundary_and_trap test {
try testing.expect_equal(usize(0), state.live)
}
evaluation_backing_survives_returns_and_releases_at_every_allocation_boundary test {
module := ir.init(mem.c_allocator)
defer ir.deinit(&module)
store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&store)
types [1]ir.Type :: [.u8]
helper :: try add_function(&module, types[..], 0, .pointer)
try emit(&module, helper, ir.Instruction{
op = .constant, type = .u8, destination = ir.ValueId(0),
constant = ir.Value{ type = .u8, data = ir.ValueData{ unsigned = 42 } },
})
try emit(&module, helper, ir.Instruction{ op = .store_local, left = ir.ValueId(0) })
try emit(&module, helper, ir.Instruction{
op = .address_local, type = .address, pointee = .u8, destination = ir.ValueId(1),
})
try emit(&module, helper, ir.Instruction{
op = .pointer_from_address, type = .pointer, left = ir.ValueId(1), destination = ir.ValueId(2),
})
return_value(&module, helper, ir.ValueId(2))
main :: try add_function(&module, &[], 0, .pointer)
module.entry = main
try emit(&module, main, function_value(0, helper))
try emit(&module, main, ir.Instruction{
op = .call, type = .pointer, left = ir.ValueId(0), destination = ir.ValueId(1),
})
return_value(&module, main, ir.ValueId(1))
trap_id :: try diagnostics.add(&store, .invalid_escape, diagnostics.Span{ start = 0, end = 0 })
for [false, true] |trap| {
if (trap) module.functions.items[usize(main)].blocks.items[0].terminator =
ir.Terminator{ kind = .trap, diagnostic = trap_id }
succeeded := false
for (usize(0)..usize(64)) |limit| {
state := AllocationState{ remaining = limit }
allocator :: mem.Allocator{ context = &state, vtable = &limited_vtable }
outcome := run_evaluation(allocator, &module, &store) catch |error| {
match error {
.out_of_memory: {}
else: return .expectation_failed
}
try testing.expect_equal(usize(0), state.live)
continue
}
defer deinit(&outcome)
if (trap) {
try testing.expect_equal(trap_id, outcome.trap?)
} else {
try testing.expect(outcome.trap == null)
byte :: evaluation_byte(&outcome, &module, outcome.value, 0)
try testing.expect(byte != null)
try testing.expect_equal(u64(42), byte?.data.unsigned)
try testing.expect(evaluation_byte(&outcome, &module, outcome.value, 1) == null)
}
deinit(&outcome)
try testing.expect_equal(usize(0), state.live)
succeeded = true
break
}
try testing.expect(succeeded)
}
}
runtime_diagnostic_allocation_failure_is_infrastructure_error_and_frees_frames test {
module := ir.init(mem.c_allocator)
defer ir.deinit(&module)
+54 -10
View File
@@ -2331,13 +2331,14 @@ evaluate_expected proc(state @mut State, id, annotation ast.NodeId) Value ! Erro
return try evaluate(state, id)
}
# values leaving the temporary interpreter must own no interpreter addresses or module IDs.
# rebuild contextual source literals and source-bound callback references before releasing it.
# values leaving temporary evaluation must own no interpreter addresses or module IDs.
# snapshot readonly bytes and rebuild source literals/references before releasing its backing.
@hide:file
serialize_value proc(
state @mut State,
module @ir.Module,
checked @typecheck.Result,
outcome @interpreter.Outcome,
functions []ast.NodeId,
type typecheck.Type,
value ir.Value,
@@ -2355,13 +2356,53 @@ serialize_value proc(
return no_node
}
result := no_node
if (type.kind == .array) {
if (type.kind == .slice) {
slice :: typecheck.array_type(checked, type.array_type)
if (
slice.mutable or slice.element.kind != .u8 or slice.element.nominal != no_node
or (slice.sentinel != null and slice.sentinel? != 0)
) return no_node
pointer :: ir.aggregate_item(module, value, 0)
length :: ir.aggregate_item(module, value, 1)
if (pointer.uninitialized or length.uninitialized) {
state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .read_before_initialization, span)
return no_node
}
if (
length.data.unsigned != 0
and interpreter.evaluation_byte(outcome, module, pointer, usize(length.data.unsigned - 1)) == null
) return no_node
bytes :: try mem.alloc(u8, state.allocator, usize(length.data.unsigned))
defer mem.free(state.allocator, bytes)
for (bytes) |_, index| {
byte :: interpreter.evaluation_byte(outcome, module, pointer, index)
if (byte == null) return no_node
if (byte?.uninitialized) {
state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .read_before_initialization, span)
return no_node
}
bytes[index] = u8(byte?.data.unsigned)
}
if (slice.sentinel != null) {
sentinel :: interpreter.evaluation_byte(outcome, module, pointer, bytes.len)
if (sentinel == null) return no_node
if (sentinel?.uninitialized) {
state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .read_before_initialization, span)
return no_node
}
if (sentinel?.data.unsigned != 0) {
state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .sentinel_mismatch, span)
return no_node
}
}
result = try ast.add_string(state.builder, span, bytes)
} else if (type.kind == .array) {
array :: typecheck.array_type(checked, type.array_type)
elements := arraylist.init(ast.NodeId, state.allocator)
defer arraylist.deinit(&elements)
for (0..array.len) |i| {
child :: ir.aggregate_item(module, value, i)
node :: try serialize_value(state, module, checked, functions, array.element, child, span)
node :: try serialize_value(state, module, checked, outcome, functions, array.element, child, span)
if (node == no_node) return no_node
try arraylist.append(&elements, node)
}
@@ -2381,7 +2422,7 @@ serialize_value proc(
return result
}
child :: ir.aggregate_item(module, value, i)
node :: try serialize_value(state, module, checked, functions, field.type, child, span)
node :: try serialize_value(state, module, checked, outcome, functions, field.type, child, span)
if (node == no_node) return no_node
name := field.name
if (name != null) name = try member_name(state, name?)
@@ -2396,7 +2437,7 @@ serialize_value proc(
index :: if (failed) usize(1) else usize(0)
channel :: if (failed) container.error_type else container.element
child :: ir.aggregate_item(module, value, index)
result = try serialize_value(state, module, checked, functions, channel, child, span)
result = try serialize_value(state, module, checked, outcome, functions, channel, child, span)
if (result == no_node) return no_node
# keep named success/error types distinct even when their members overlap.
annotation := no_node
@@ -2425,12 +2466,12 @@ serialize_value proc(
value.data.function == maxval!(ir.FunctionId)
else value.data.unsigned == maxval!(u64)
if (absent) result = try ast.add_literal(state.builder, .null, span)
else result = try serialize_value(state, module, checked, functions, container.element, value, span)
else result = try serialize_value(state, module, checked, outcome, functions, container.element, value, span)
} else if (!ir.aggregate_item(module, value, 0).data.boolean) {
result = try ast.add_literal(state.builder, .null, span)
} else {
child :: ir.aggregate_item(module, value, 1)
payload :: try serialize_value(state, module, checked, functions, container.element, child, span)
payload :: try serialize_value(state, module, checked, outcome, functions, container.element, child, span)
if (payload == no_node) return no_node
name :: try ast.add_generated_identifier(state.builder, span,
try strpool.intern(state.strings, "some"), "some")
@@ -2635,13 +2676,16 @@ evaluate_ir_body proc(state @mut State, at, body, result_type ast.NodeId) Value
}
value :: outcome.value
source_type :: typecheck.type_of(&checked, result_type)
node :: try serialize_value(state, &module, &checked, functions.items, source_type, value, span)
node :: try serialize_value(state, &module, &checked, &outcome, functions.items, source_type, value, span)
if (node == no_node) {
if (state.evaluation_diagnostic == diagnostics.no_diagnostic)
state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .comptime_value_escape, span)
return Value{}
}
if (ir.aggregate(value.type) or source_type.kind == .enumeration or source_type.kind == .void)
if (
(ir.aggregate(value.type) and source_type.kind != .slice)
or source_type.kind == .enumeration or source_type.kind == .void
)
return Value{ kind = .aggregate, node = node, annotation = result_type }
result := try evaluate(state, node)
result.annotation = result_type
+239 -4
View File
@@ -1771,7 +1771,10 @@ CASES = {
),
"inline_for_tuple_rejects_storage_escape_before_break": (
"""
values proc() struct { i16, []u8 } { return {42, "borrowed"} }
values proc() struct { i16, @i16 } {
local i16 :: 2
return {42, &local}
}
main proc() i16 {
inline for values() |_| { break }
return 42
@@ -2090,13 +2093,25 @@ CASES = {
}
""", b"", b"escape",
),
"inline_for_rejects_borrowed_element_escape": (
"inline_for_materializes_readonly_byte_slice_elements": (
"""
data proc() [1][]u8 { return ["borrowed"] }
main proc() usize {
inline for data() |value| { return value.len }
return 0
}
""", b"8\n", None,
),
"inline_for_rejects_mutable_byte_slice_element_escape": (
"""
data proc() [1][]mut u8 {
values [1]mut u8 := [42]
return [values[..]]
}
main proc() usize {
inline for data() |value| { return value.len }
return 0
}
""", b"", b"escape",
),
"inline_for_expansion_is_bounded": (
@@ -5210,10 +5225,206 @@ CASES = {
main proc() i16 { return $(make()).pointer^ }
""", b"", b"compile-time result cannot escape evaluation",
),
"comptime_aggregate_rejects_slice_escape": (
"comptime_aggregate_materializes_readonly_byte_slices": (
"""
make proc() [1][]u8 { return ["borrowed"] }
main proc() usize { return $(make())[0].len }
""", b"8\n", None,
),
"comptime_byte_slices_preserve_subviews_and_zero_sentinel": (
"""
subview proc() []u8 { return "unused42ignored"[6..8] }
terminated proc() [;0]u8 { return "unused42"[6..;0] }
main proc() [5]u8 {
plain :: $(subview())
zero [;0]u8 :: $(terminated())
return [plain[0], plain[1], zero[0], zero[1], zero.ptr[zero.len]]
}
""", b"[52, 50, 52, 50, 0]\n", None,
),
"comptime_byte_slices_own_filled_local_arrays": (
"""
make proc(seed u8) []u8 {
values [3]mut u8 := undefined
for (0..usize(3)) |index| values[index] = seed + u8(index)
copy := values
values[0] = 99
return copy[1..]
}
main proc() [4]u8 {
first :: $(make(40))
second :: $(make(10))
return [first[0], first[1], second[0], second[1]]
}
""", b"[41, 42, 11, 12]\n", None,
),
"comptime_byte_slices_materialize_immutable_globals_and_raw_fields": (
"""
bytes [4]u8 :: [9, 40, 2, 8]
Views :: struct { rows [2][]u8 }
make proc() Views { return { rows = [bytes[1..3], bytes[2..3]] } }
main proc() [3]u8 {
views :: $(make())
return [views.rows[0][0], views.rows[0][1], views.rows[1][0]]
}
""", b"[40, 2, 2]\n", None,
),
"comptime_byte_slices_preserve_nested_optional_tagged_and_fallible_channels": (
"""
Failure :: struct { text []u8 }
Choice :: union(enum) { data struct { rows [2]?[]u8 }, empty void }
Packet :: struct { choice ?Choice, result []u8 ! Failure }
make proc(fail bool) Packet {
values [2]mut u8 := [40, 2]
result []u8 ! Failure := values[..]
if (fail) result = Failure { text = values[1..] }
return {
choice = Choice { data = { rows = [values[..1], null] } },
result = result,
}
}
main proc() [4]u8 {
good :: $(make(false))
bad :: $(make(true))
choice :: good.choice?
if (choice.data.rows[1] != null) return [0, 0, 0, 0]
success :: good.result catch ""
failure :: bad.result catch |err| err.text
return [choice.data.rows[0]?[0], success[0], success[1], failure[0]]
}
""", b"[40, 40, 2, 2]\n", None,
),
"comptime_byte_slices_normalize_generic_cache_content": (
"""
make proc() []u8 {
values [2]mut u8 := [40, 2]
return values[..]
}
same proc() []u8 { return "(\\x28\\x02)"[1..3] }
other proc() []u8 { return "\\x28\\x03" }
Token proc($bytes []u8) type { return struct {} }
score proc($bytes []u8) u8 { return bytes[0] + bytes[1] }
main proc() u8 {
if (Token(make()) != Token(same())) return 0
if (Token(make()) != Token("\\x28\\x02")) return 0
if (Token(make()) == Token(other())) return 0
return score(make())
}
""", b"42\n", None,
),
"comptime_byte_slices_supply_generated_defaults": (
"""
make proc() []u8 {
values [2]mut u8 := [40, 2]
return values[..]
}
Generated proc() type {
return struct_type!(.auto, ["bytes"], [[]u8], [make()])
}
main proc() u8 {
value Generated() :: {}
return value.bytes[0] + value.bytes[1]
}
""", b"42\n", None,
),
"comptime_byte_slices_preserve_nul_high_bytes_and_empty_views": (
r"""
make proc() []u8 { return "!\x00\x80\xff?"[1..4] }
empty proc() []u8 { return "ignored"[3..3] }
zero proc() [;0]u8 { return ""[..;0] }
main proc() [6]usize {
bytes :: $(make())
plain :: $(empty())
terminated :: $(zero())
return [usize(bytes[0]), usize(bytes[1]), usize(bytes[2]),
plain.len, terminated.len, usize(terminated.ptr[0])]
}
""", b"[0, 128, 255, 0, 0, 0]\n", None,
),
"comptime_byte_slices_reject_uninitialized_source_bytes": (
"""
make proc() []u8 {
values [2]mut u8 := undefined
values[0] = 42
return values[..]
}
main proc() usize { return $(make()).len }
""", b"", b"local value was read before initialization",
),
"comptime_byte_slices_ignore_unobserved_uninitialized_bytes": (
"""
make proc() []u8 {
values [3]mut u8 := undefined
values[1] = 42
return values[1..2]
}
main proc() u8 { return $(make())[0] }
""", b"42\n", None,
),
"comptime_byte_slices_reject_invalid_ranges": (
"""
make proc() []u8 {
values [1]u8 :: [42]
return values[..2]
}
main proc() usize { return $(make()).len }
""", b"", b"slice",
),
"comptime_byte_slices_reject_oversized_pointer_views_before_allocation": (
"""
make proc() []u8 {
values [1]u8 :: [42]
return values.ptr[..maxval!(usize)]
}
main proc() usize { return $(make()).len }
""", b"", b"compile-time result cannot escape evaluation",
),
"comptime_byte_slices_recheck_zero_sentinel_after_mutation": (
"""
make proc() [;0]u8 {
values [2]mut u8 := [42, 0]
view [;0]u8 :: values[..1;0]
values[1] = 7
return view
}
main proc() usize { return $(make()).len }
""", b"", b"sentinel",
),
"comptime_aggregate_rejects_mutable_byte_slice_escape": (
"""
make proc() [1][]mut u8 {
values [1]mut u8 := [42]
return [values[..]]
}
main proc() usize { return $(make())[0].len }
""", b"", b"compile-time result cannot escape evaluation",
),
"comptime_aggregate_rejects_nonbyte_slice_escape": (
"""
make proc() []i16 {
values [1]i16 :: [42]
return values[..]
}
main proc() usize { return $(make()).len }
""", b"", b"compile-time result cannot escape evaluation",
),
"comptime_aggregate_rejects_nominal_byte_slice_escape": (
"""
Byte :: distinct u8
make proc() []Byte {
values [1]Byte :: [42]
return values[..]
}
main proc() usize { return $(make()).len }
""", b"", b"compile-time result cannot escape evaluation",
),
"comptime_aggregate_rejects_nonzero_sentinel_byte_slice_escape": (
"""
make proc() [;1]u8 {
values [2]u8 :: [42, 1]
return values[..1;1]
}
main proc() usize { return $(make()).len }
""", b"", b"compile-time result cannot escape evaluation",
),
"comptime_tagged_union_preserves_tags_and_nested_raw_payloads": (
@@ -8776,7 +8987,31 @@ def check_specialization_packages(temporary, executable):
}
""").strip() + "\n")
check(app, executable, (b"42\n", None))
return 23
sizing.write_text(textwrap.dedent("""
bytes [3]u8 :: [9, 40, 2]
view proc() []u8 { return bytes[1..] }
filled proc() []u8 {
local [2]mut u8 := undefined
local[0] = 40
local[1] = 2
return local[..]
}
Token proc($value []u8) type { return struct {} }
Generated proc() type {
return struct_type!(.auto, ["value"], [[]u8], [filled()])
}
""").strip() + "\n")
source.write_text(textwrap.dedent("""
lib :: import "../library"
main proc() [3]u8 {
value lib.Generated() :: {}
direct :: $(lib.view())
if (lib.Token(lib.view()) != lib.Token(lib.filled())) return [0, 0, 0]
return [direct[0], value.value[1], $(lib.filled())[0]]
}
""").strip() + "\n")
check(app, executable, (b"[40, 2, 40]\n", None))
return 24
def check_dependency_provenance(temporary, executable):