diff --git a/README.md b/README.md index 76cf125..f5edced 100644 --- a/README.md +++ b/README.md @@ -116,6 +116,8 @@ initialized records/tuples containing `type` fields live inside compile-time pro type-valued records/tuples may also start with `undefined`: `config struct { child type, count usize } := undefined`, then `config.child = i16` and `config.count = 2`. nested record fields, arrays of records, tuple ordinals and `field!` updates use the same persistent snapshots as initialized storage; partial copies retain independent initialization state. defaults do not initialize `undefined` fields. initialized leaves and array lengths can be read while other leaves remain uninitialized, but premature leaf reads and incomplete procedure arguments/results diagnose read-before-initialization. fully initialized results cross compile-time procedure boundaries and supply erased generic inputs, generated storage and defaults. empty shapes are initialized vacuously; each expanded record field/array cell shares the construction bound. +optionals containing types also stay in owned compile-time storage: `?type`, nested `??type`, and optional records/arrays of metadata support null, contextual payloads, explicit `some!`, whole-value updates, immutable captures, null comparisons and checked unwraps. `some!(null)` preserves an absent inner optional inside a present outer one. constant-input procedures exchange these snapshots; erased generic inputs retain content-based specialization identity and nominal type leaves. `if`/`while` captures and conditional expressions select generated types or runtime bodies without runtime metadata. for generated array children, bind the unwrap first (`T :: selected?`, then `[count]T`); postfix `?` after an array type unwraps the whole expression. undefined presence and incomplete payloads diagnose before observation or procedure boundaries. runtime metadata storage and fallible type-valued channels remain unsupported. + fixed type arrays may also start with `undefined` inside type/type-array producers: `types [2]mut type := undefined`, then fill `types[index] = i16` in a loop. nested and whole-array updates preserve shape, mutability and independent initialization state in local copies; `.len` is available before elements initialize. reading an uninitialized type leaf, returning an incomplete array or passing one to a compile-time procedure diagnoses a read before initialization. empty shapes are valid, but every dimension must be explicit and nonnegative. construction charges each expanded cell against the shared execution bound, including nested zero-leaf shapes. ordinary fixed value arrays use the same construction inside type/type-array producers: `counts [2]mut usize := undefined`, then fill indexed elements or replace whole rows/arrays. initialized leaves retain exact scalar, string, optional, record and tagged-union values; local copies retain independent partial initialization state. generated lengths/defaults and compile-time arguments can use initialized elements, while incomplete arrays cannot enter checked IR or procedure arguments. array `.len` and empty shapes do not read leaves. constant materialization of mutable slices or nonnull data pointers remains unsupported. diff --git a/TODO.md b/TODO.md index add58b6..cff98df 100644 --- a/TODO.md +++ b/TODO.md @@ -46,6 +46,8 @@ type-array and type-record parameters/results cross compile-time procedure bound with owned snapshots, cached nominal identity and erased arguments. initialized type-valued records/tuples support schemas/defaults, nested field/index updates and owned copies/captures inside compile-time producers. +optional type-valued storage preserves nested presence, owned metadata payloads, +checked unwraps and immutable captures across compile-time procedure boundaries. `field!` selects record/tuple/enum and sequence members using compile-time names, including runtime places and type-control updates without runtime reflection. `struct_type!` generates ordinary/C-layout records from constant name/type/default @@ -155,6 +157,7 @@ test discovery and FFI remain outside these completed slices. - [x] untagged-union materialization checkpoint: address-free untagged unions own full native-layout byte snapshots through interpreter/native execution, including scalar/IEEE reinterpretation, nested padded records/arrays, smaller overlapping updates, zero-sized storage, distinct members and generated nominal unions. explicit `$()` values, compile-time arguments, full-byte specialization-cache identity, generated defaults, nested optionals/tagged payloads and both fallible channels preserve exact types and independent copies. uninitialized tail/padding bytes diagnose read-before-initialization; every member layout is checked recursively to reject data pointers, callbacks, strings and slices rather than retain interpreter handles. imported helpers/diagnostics, source/session disposal and repeated compilation retain ownership. address-bearing union layouts and broader constant materialization remain open. - [x] readonly typed-slice materialization checkpoint: ordinary compile-time calls snapshot readonly slices of materializable elements into module-lifetime backing through interpreter/native execution. nominal bytes, wider integers, arrays/records, source-bound callbacks, nested readonly slices, empty/zero-sized elements and full-width signed/unsigned sentinels retain exact types and logical lengths. explicit `$()` values, computed/direct content-based specialization identity, generated defaults and nested optional/tagged/fallible channels preserve ownership across helper returns, imported private declarations, source/session disposal and repeat compilation. initialization, address-range, sentinel, mutable-slice and nonnull-pointer escape diagnostics remain checked; 100,000-value/128-level snapshot bounds reject oversized and cyclic storage. mutable slices, nonnull data pointers and address-bearing untagged union layouts remain open. - [x] uninitialized type-record checkpoint: compile-time records/tuples containing type values start with `undefined` and fill through nested field/index updates or whole-value replacement. persistent snapshots preserve independent partial copies; defaults do not initialize undefined fields. initialized leaves and empty shapes work while premature type/value reads and incomplete procedure arguments/results diagnose read-before-initialization. fully initialized results supply erased generic inputs, generated storage/defaults and cached nominal types through interpreter/native execution. ordinary mutability/schema validation and shared per-field/cell construction bounds remain checked; general type-valued execution and broader materialization remain open. + - [x] optional type-storage checkpoint: `?type` and optional type-valued records/arrays use owned compile-time snapshots, including nested `some!(null)`, contextual payload construction, mutable whole-value/field/index replacement, immutable if/while/conditional captures, null comparisons and checked unwraps. constant-input procedures and erased generic arguments preserve specialization-cache identity and nominal leaves while generating runtime storage through interpreter/native execution. undefined presence, incomplete payloads, non-type values, capture mutation and runtime dependencies retain diagnostics. imported private nominal types, session/source disposal and repeat compilation preserve ownership. fallible type-valued channels, general type-valued execution and broader constant materialization remain open. - [ ] remaining gate: general type-valued compile-time execution, broader constant materialization (including address-bearing untagged unions, nonnull data pointers and mutable slices), plus remaining existing-library acceptance for broader `meta` operations and `io.print`. formatting parsing and broader imported C-binding support may interleave with later gates. - [ ] remaining source constructs: multiline strings. boolean and optional-capture conditional expressions, yielding value blocks, `yield`, compound updates and bitwise operators are covered above. examples: `source/parser/parse.hon`, `source/llvm/runtime.hon`. - [ ] standard-library milestone: compile/run a multi-package program exercising existing allocation and IO, including error cleanup; no replacement miniature standard library. diff --git a/source/compilation/compilation.test.hon b/source/compilation/compilation.test.hon index e4c945e..81db93a 100644 --- a/source/compilation/compilation.test.hon +++ b/source/compilation/compilation.test.hon @@ -1148,3 +1148,63 @@ computed_untagged_unions_recompile_without_borrowing_storage_bytes test { try testing.expect_equal(i64(42), outcome.value.data.integer) } } + +optional_type_metadata_recompiles_owned_private_nominal_payloads test { + diagnostic_store := diagnostics.init(mem.c_allocator) + defer diagnostics.deinit(&diagnostic_store) + modules [2]mut ir.Module := undefined + { + session := init(mem.c_allocator) + defer deinit(&session) + root :: try add_package(&session) + dependency :: try add_package(&session) + bytes := try owned_text( + `lib :: import "../library" + `Child :: struct { value bool } + `main proc() i16 { + ` if (lib.Box(lib.make()) != lib.Box(lib.make())) return -1 + ` if (lib.Selected() == Child) return -2 + ` value lib.Selected() :: { value = 42 } + ` return value.value + `} + , + ) + defer arraylist.deinit(&bytes) + root_source :: try add_source(&session, root, "/optional_types/app/main.hon", &bytes, &diagnostic_store) + imported := try owned_text( + `@hide:file + `Child :: struct { value i16 } + `@hide:file + `Config :: struct { child ?type, count usize } + `make proc() ?Config { return { child = Child, count = 2 } } + `Box proc(input ?Config) type { + ` info :: input? + ` T :: info.child? + ` return struct { values [info.count]T } + `} + `Selected proc() type { + ` if make() |info| { return info.child? } + ` return bool + `} + , + ) + defer arraylist.deinit(&imported) + _ = try add_source(&session, dependency, "/optional_types/library/types.hon", &imported, &diagnostic_store) + try arraylist.append(&session.imports, Import{ + node = declaration(&session, root_source, 0).node, + target = dependency, + }) + modules[0] = try compile(&session, root, &diagnostic_store) + errdefer ir.deinit(&modules[0]) + modules[1] = try compile(&session, root, &diagnostic_store) + } + defer ir.deinit(&modules[0]) + defer ir.deinit(&modules[1]) + try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) + for modules |module| { + outcome := try interpreter.run(mem.c_allocator, &module, &diagnostic_store) + defer interpreter.deinit(&outcome) + try testing.expect(outcome.trap == null) + try testing.expect_equal(i64(42), outcome.value.data.integer) + } +} diff --git a/source/sema/specialize/specialize.hon b/source/sema/specialize/specialize.hon index ea86b2a..1859d11 100644 --- a/source/sema/specialize/specialize.hon +++ b/source/sema/specialize/specialize.hon @@ -681,6 +681,28 @@ expand_expected proc(state @mut State, id, expected ast.NodeId) void ! Error { try refresh(state) return } + if (view.operator == .equal or view.operator == .not_equal) { + operand := no_node + if (ast.expr_kind(&state.tree, view.left) == .literal_null) operand = view.right + else if (ast.expr_kind(&state.tree, view.right) == .literal_null) operand = view.left + if (operand != no_node) { + annotation :: try expression_type(state, operand, 0) + if (type_record_storage(state, annotation, 0) + and ast.expr_kind(&state.tree, annotation) == .optional_type) { + value :: try evaluate_expected(state, operand, annotation) + if (value.kind != .aggregate) { + try reject_evaluation(state, id) + return + } + absent :: ast.expr_kind(&state.tree, value.node) == .literal_null + folded :: try ast.add_boolean(state.builder, view.span, + absent == (view.operator == .equal)) + try replace(state, id, folded) + try refresh(state) + return + } + } + } if ( (view.operator == .equal or view.operator == .not_equal) and branch_constant(state, view.left, 0) and branch_constant(state, view.right, 0) @@ -1789,7 +1811,7 @@ type_storage_annotation proc(tree @ast.Tree, bindings []sema.Binding, input ast. and type_storage_annotation(tree, bindings, view.initializer, depth + 1)) } kind :: ast.expr_kind(tree, input) - if (kind == .alias_expr) + if (kind == .alias_expr or kind == .optional_type) return type_storage_annotation(tree, bindings, ast.type_operand(tree, input), depth + 1) if (kind == .array_type) return type_storage_annotation(tree, bindings, ast.array_type(tree, input).element, depth + 1) @@ -3488,12 +3510,21 @@ type_match_arm proc(state @mut State, id ast.NodeId) usize ! Error { dispatch := subject if (metadata) { value :: try evaluate_expected(state, subject, annotation) - if (value.kind != .aggregate or ast.expr_kind(&state.tree, value.node) != .record_literal) return no_index - active :: ast.field(&state.tree, ast.record_field(&state.tree, value.node, 0)) + if (value.kind != .aggregate) return no_index span :: ast.node_span(&state.tree, subject) - tags :: try metadata_tags(state, annotation) - tag :: try ast.add_enum_literal(state.builder, span, active.name?) - dispatch = try ast.add_typed_constant(state.builder, span, tags, tag) + if (ast.expr_kind(&state.tree, annotation) == .optional_type) { + present :: ast.expr_kind(&state.tree, value.node) != .literal_null + boolean :: try ast.add_boolean(state.builder, span, true) + payload :: if (present) boolean else try ast.add_literal(state.builder, .null, span) + optional :: try ast.add_optional_type(state.builder, span, try builtin(state, subject, .bool)) + dispatch = try ast.add_typed_constant(state.builder, span, optional, payload) + } else { + if (ast.expr_kind(&state.tree, value.node) != .record_literal) return no_index + active :: ast.field(&state.tree, ast.record_field(&state.tree, value.node, 0)) + tags :: try metadata_tags(state, annotation) + tag :: try ast.add_enum_literal(state.builder, span, active.name?) + dispatch = try ast.add_typed_constant(state.builder, span, tags, tag) + } try refresh(state) } for (0..ast.match_arm_count(&state.tree, id)) |i| { @@ -3514,7 +3545,7 @@ type_match_arm proc(state @mut State, id ast.NodeId) usize ! Error { try expand(state, pattern) try arraylist.append(&patterns, pattern) } - if (metadata and arm.capture != null) { + if (metadata and !optional_capture and arm.capture != null) { expected := no_node valid := !arm.catch_all and patterns.items.len > 0 for (patterns.items) |pattern| { @@ -3570,7 +3601,11 @@ type_match_capture proc(state @mut State, id ast.NodeId) ast.NodeId ! Error { ast.node_span(&state.tree, subject), .unwrap, subject) try refresh(state) child :: try expression_type(state, unwrapped, 0) - return try type_execution_value(state, unwrapped, child) + payload :: try type_execution_value(state, unwrapped, child) + if (payload == no_node or !is_type(state, child)) return payload + result :: try ast.add_typed_constant(state.builder, ast.node_span(&state.tree, subject), child, payload) + try refresh(state) + return result } annotation :: try expression_type(state, subject, 0) value :: if (type_record_storage(state, annotation, 0)) @@ -3669,6 +3704,28 @@ expand_type_matches proc(state @mut State, id ast.NodeId) bool ! Error { type_array_projection proc(state @mut State, id ast.NodeId) bool ! Error { if (ast.node_class(&state.tree, id) != .expression) return true kind :: ast.expr_kind(&state.tree, id) + if (kind == .unary and ast.unary_expr(&state.tree, id).operator == .unwrap) { + operand :: ast.unary_expr(&state.tree, id).operand + annotation :: try expression_type(state, operand, 0) + if (!type_record_storage(state, annotation, 0)) return true + type :: try canonical_type(state, annotation) + if (type == no_node or ast.expr_kind(&state.tree, type) != .optional_type) return true + value :: try type_execution_value(state, operand, type) + if (value == no_node or !try type_array_initialized(state, value, id)) return false + payload :: ast.typed_constant(&state.tree, value).value + if (ast.expr_kind(&state.tree, payload) == .literal_null) { + state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, + .null_unwrap, ast.node_span(&state.tree, id)) + return false + } + selected :: ast.call_arg(&state.tree, payload, 0) + resolved :: binding(state, selected) + try remember(state, id) + ast.replace_node_from_source(state.builder, id, selected) + try refresh(state) + state.bindings.items[usize(id)] = resolved + return true + } if (kind == .index or kind == .selection) { base :: if (kind == .index) ast.index_expr(&state.tree, id).base else ast.selection_expr(&state.tree, id).base base_binding :: binding(state, base) @@ -3845,6 +3902,8 @@ type_record_check proc(state @mut State, annotation ast.NodeId) bool ! Error { if (is_type(state, annotation)) return true type :: try canonical_type(state, annotation) if (type == no_node) return false + if (ast.expr_kind(&state.tree, type) == .optional_type) + return try type_record_check(state, ast.type_operand(&state.tree, type)) if (ast.expr_kind(&state.tree, type) == .slice_type) return try type_record_check(state, ast.slice_type(&state.tree, type).element) if (ast.expr_kind(&state.tree, type) == .array_type) { @@ -4010,6 +4069,8 @@ type_array_initialized proc(state @mut State, value, at ast.NodeId) bool ! Error } if (kind == .typed_constant) return try type_array_initialized(state, ast.typed_constant(&state.tree, value).value, at) + if (kind == .builtin_call and contextual_literal(state, value)) + return try type_array_initialized(state, ast.call_arg(&state.tree, value, 0), at) if (kind == .array_literal) { for (0..ast.array_element_count(&state.tree, value)) |i| { if (!try type_array_initialized(state, ast.array_element(&state.tree, value, i), at)) return false @@ -4248,6 +4309,42 @@ type_array_update proc( return result } +# optional metadata uses the same explicit presence encoding as serialized values. +# an inner null is a payload only when wrapped in some!, never outer absence. +@hide:file +type_optional_value proc(state @mut State, expression, type ast.NodeId) ast.NodeId ! Error { + if (!try type_record_check(state, type)) return no_node + kind :: ast.expr_kind(&state.tree, expression) + if (kind == .literal_undefined or kind == .literal_null) { + result :: try ast.add_typed_constant(state.builder, ast.node_span(&state.tree, expression), type, expression) + try refresh(state) + return result + } + child :: ast.type_operand(&state.tree, type) + input := expression + if (kind == .typed_constant) { + actual :: ast.typed_constant(&state.tree, expression) + if (try same_type(state, type, actual.type)) return expression + } else if (kind == .builtin_call + and mem.eql(ast.literal_text(&state.tree, ast.builtin_call(&state.tree, expression).name), "some")) { + if (ast.call_arg_count(&state.tree, expression) != 1) { + state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, + .argument_count_mismatch, ast.node_span(&state.tree, expression)) + return no_node + } + input = ast.call_arg(&state.tree, expression, 0) + } + payload :: try type_execution_value(state, input, child) + if (payload == no_node) return no_node + span :: ast.node_span(&state.tree, expression) + name :: try metadata_member(state, expression, "some") + arguments [1]ast.NodeId :: [payload] + present :: try ast.add_builtin_call(state.builder, span, name, arguments[..]) + result :: try ast.add_typed_constant(state.builder, span, type, present) + try refresh(state) + return result +} + @hide:file type_execution_value proc(state @mut State, input, annotation ast.NodeId) ast.NodeId ! Error { if (input == no_node or annotation == no_node) return no_node @@ -4311,6 +4408,11 @@ type_execution_value proc(state @mut State, input, annotation ast.NodeId) ast.No .type_mismatch, ast.node_span(&state.tree, expression)) return no_node } + if (!is_type(state, annotation) and type_record_storage(state, annotation, 0)) { + type :: try canonical_type(state, annotation) + if (type != no_node and ast.expr_kind(&state.tree, type) == .optional_type) + return try type_optional_value(state, expression, type) + } if (!is_type(state, annotation) and type_record_storage(state, annotation, 0)) return try type_record_value(state, expression, annotation) if (is_type(state, annotation)) { @@ -4370,6 +4472,16 @@ type_capture_guard proc(state @mut State, expression, capture ast.NodeId) Value state.evaluation_diagnostic = resolved.data.diagnostic return Value{} } + annotation :: try expression_type(state, expression, 0) + if (type_record_storage(state, annotation, 0)) { + type :: try canonical_type(state, annotation) + if (type != no_node and ast.expr_kind(&state.tree, type) == .optional_type) { + value :: try type_execution_value(state, expression, type) + if (value == no_node or !try type_array_initialized(state, value, expression)) return Value{} + return Value{ kind = .boolean, node = expression, + boolean = ast.expr_kind(&state.tree, ast.typed_constant(&state.tree, value).value) != .literal_null } + } + } span :: ast.node_span(&state.tree, expression) yes :: try ast.add_boolean(state.builder, span, true) no :: try ast.add_boolean(state.builder, span, false) diff --git a/tests/native.py b/tests/native.py index e12e867..0cd283b 100644 --- a/tests/native.py +++ b/tests/native.py @@ -2278,6 +2278,143 @@ CASES = { } """, b"", b"initialization", ), + "optional_types_procedure_boundaries_copies_and_static_captures": ( + """ + Token :: distinct i16 + choose proc(present bool) ?type { + if (present) return Token + return null + } + pass proc(value ?type) ?type { return value } + Box proc(value ?type) type { + T :: if value |Child| Child else bool + return struct { item T } + } + Selected proc(value ?type) type { return value? } + exact proc($value ?type, input Selected(value)) Selected(value) { return input } + Storage proc() type { + selected ?type := pass(choose(true)) + saved :: selected + selected = null + if (selected != null or null == saved) return bool + if (Box(saved) != Box(choose(true))) return bool + if (Box(null) == Box(saved)) return bool + while saved |T| { + return [2]T + } + return bool + } + main proc() i16 { + values Storage() :: [40, 2] + return i16(exact(choose(true), values[0])) + i16(values[1]) + } + """, b"42\n", None, + ), + "optional_metadata_records_nested_presence_and_owned_updates": ( + """ + Config :: struct { child type, count usize } + config proc() ?Config { return { child = i16, count = 2 } } + nested proc(present bool) ??type { + if (present) return some!(null) + return null + } + Storage proc() type { + outer :: nested(true) + if outer |inner| { + if (inner != null or nested(false) != null) return bool + } else return bool + values [2]mut ?Config := [config(), null] + saved :: values + values[0] = null + values[1] = { child = bool, count = 3 } + if (values[0] != null or saved[1] != null) return bool + info :: saved[0]? + return [info.count]info.child + } + main proc() Storage() { return [40, 2] } + """, b"[40, 2]\n", None, + ), + "optional_types_fill_undefined_fields_and_yield_before_cleanup": ( + """ + Config :: struct { child ?type = bool, count usize } + make proc() ?Config { + config Config := undefined + config.child = i16 + config.count = 2 + return some!(config) + } + Storage proc() type { + value ?type := undefined + value = null + selected ?type :: { + defer value = bool + yield make()?.child + } + if (value? != bool) return bool + T :: selected? + return [make()?.count]T + } + main proc() Storage() { return [40, 2] } + """, b"[40, 2]\n", None, + ), + "optional_types_reject_null_unwrap": ( + """ + empty proc() ?type { return null } + Bad proc() type { return empty()? } + main proc() Bad() { return 42 } + """, b"", b"cannot unwrap a null optional", + ), + "optional_types_reject_non_type_payload": ( + """ + Bad proc() ?type { return some!(42) } + Selected proc() type { return Bad()? } + main proc() Selected() { return 42 } + """, b"", b"incompatible value type", + ), + "optional_types_reject_incomplete_payload_result": ( + """ + Config :: struct { child type, count usize } + bad proc() ?Config { + value Config := undefined + value.child = i16 + return some!(value) + } + Selected proc() type { return bad()?.child } + main proc() Selected() { return 42 } + """, b"", b"read before initialization", + ), + "optional_types_reject_uninitialized_presence": ( + """ + Bad proc() type { + value ?type := undefined + if value |_| return i16 + return bool + } + main proc() Bad() { return 42 } + """, b"", b"read before initialization", + ), + "optional_types_reject_capture_mutation": ( + """ + Bad proc() type { + value ?type :: i16 + if value |T| { T = bool } + return i16 + } + main proc() Bad() { return 42 } + """, b"", b"mutable local binding", + ), + "optional_types_reject_runtime_dependency": ( + """ + choose proc(present bool) ?type { + if (present) return i16 + return null + } + main proc() i16 { + present := true + return if choose(present) |_| 42 else 0 + } + """, b"", b"compile-time value", + ), "type_records_undefined_nested_updates_preserve_partial_copies": ( """ Config :: struct {