From 20539ab455dd71ee013ba07b01c1bf3478ff1b9b Mon Sep 17 00:00:00 2001 From: hl-valdemar Date: Wed, 28 Oct 2026 11:22:41 +0100 Subject: [PATCH] feat: expand constant inline for loops --- README.md | 4 +- TODO.md | 5 +- source/ast/ast.hon | 13 +- source/ast/renderer/renderer.hon | 1 + source/compilation/compilation.test.hon | 68 +++++ source/diagnostics/diagnostics.hon | 2 + source/lexer/scan.hon | 1 + source/lexer/scan.test.hon | 19 ++ source/lexer/token.hon | 1 + source/parser/parse.hon | 21 +- source/parser/parse.test.hon | 108 +++++++ source/sema/specialize/specialize.hon | 171 ++++++++++- tests/native.py | 372 +++++++++++++++++++++++- 13 files changed, 772 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 1117085..c97f3ca 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,9 @@ type-producing bodies support mutable type and value locals, direct binding assi field/index assignments also update initialized, materializable aggregate locals in type-producing bodies, including nested record fields and mutable fixed-array elements. for example, `counts [2]mut usize := [0, 0]` followed by `counts[1] = 2` can generate `[counts[1]]i16` storage. array snapshots preserve earlier copies; scalar and ordinary record-field operations reuse checked IR. later loop iterations, generated lengths and defaults observe updates. ordinary mutability, index-type/bounds, field and exact value-type checks still apply, including immutable payload captures and narrow arithmetic overflow. imported producers and repeated compilation retain source diagnostics and nominal cache identity; pointer/slice-backed mutation remains open. -type-producing bodies also iterate materializable fixed arrays and strings with immutable by-value element captures and optional `usize` indices: `for counts |count, index| { length = length + count + index }` can compute generated storage. nested loops, aggregate captures, empty collections, break/continue and early returns share the type-control execution bound. array places retain their entry-time indices and observe later element updates; temporary arrays and strings are evaluated once. ordinary constant-input procedures additionally support slice and pointer captures through checked IR, provided only materializable results escape. type-producing loop bodies still reject storage-bearing captures rather than retaining temporary interpreter addresses; `inline for` remains open. +type-producing bodies also iterate materializable fixed arrays and strings with immutable by-value element captures and optional `usize` indices: `for counts |count, index| { length = length + count + index }` can compute generated storage. nested loops, aggregate captures, empty collections, break/continue and early returns share the type-control execution bound. array places retain their entry-time indices and observe later element updates; temporary arrays and strings are evaluated once. ordinary constant-input procedures additionally support slice and pointer captures through checked IR, provided only materializable results escape. type-producing loop bodies still reject storage-bearing captures rather than retaining temporary interpreter addresses. + +`inline for` expands constant integer ranges, fixed arrays (including arrays of `type`) and strings into separately checked runtime bodies. captures are immutable exact-type constants; collection indices are `usize`. for example, `inline for [i16, bool] |T| { if (T == i16) { value T :: 42 } else { value T :: true } }` checks only each iteration's selected branch. bounds/iterables are computed before expansion, empty loops skip body elaboration, and nested expansions share the 4096-step type-control bound. `break` and `continue` targeting the inline loop must be selected entirely at compile time; selected exits prune the remaining body and `break` skips subsequent iterations. nested ordinary loops retain runtime transfers, while runtime returns, errors and scoped cleanup retain ordinary semantics. runtime-dependent inputs, pointer captures and storage-bearing values that cannot escape constant evaluation are diagnosed. heterogeneous tuple iteration, metadata collections and inline-match syntax remain open. initialized fixed arrays of `type` live entirely inside compile-time execution, including nested arrays, explicit/inferred annotations, `.len`, checked `usize` indexing and collection captures. for example, `types [2]mut type := [i8, bool]`, then `types[1] = i16` and `return [types.len]types[1]`, generates `[2]i16` without runtime type metadata. whole-array and nested indexed assignments preserve shape and mutability at every level; copies and loop captures retain earlier values while borrowed array places observe later updates. diff --git a/TODO.md b/TODO.md index 17e5ba5..8b94040 100644 --- a/TODO.md +++ b/TODO.md @@ -43,6 +43,8 @@ field/index updates and owned copies/captures inside compile-time producers. including runtime places and type-control updates without runtime reflection. `struct_type!` generates ordinary/C-layout records from constant name/type/default arrays or tuples, including loop-filled producer storage and owned defaults. +`inline for` expands constant ranges, fixed value/type arrays and strings into +runtime bodies with immutable exact captures and compile-time-selected exits. general type-valued execution, metadata reflection and generic library acceptance remain open. the driver retains frontend storage and source provenance through `@source/compilation`, with compilation-wide AST node identities. @@ -102,7 +104,8 @@ test discovery and FFI remain outside these completed slices. - [x] type-record storage checkpoint: initialized records/tuples containing type values live inside type/type-array producers, including local schemas/defaults, typed/contextual/inferred construction, nested records/arrays, checked field/index reads and updates, whole-value replacement and immutable by-value collection captures. persistent snapshots isolate copies/captures while borrowed array places observe later updates; type leaves retain nominal identity and ordinary value leaves reuse checked IR. generated storage/defaults, invalid field/type/default diagnostics, readonly places, narrow overflow, imported producers/provenance, cached nominal results and repeatable compilation work through interpreter/native execution. field validation/construction shares the type-control bound. type-record procedure inputs/results are covered below; runtime type-valued records, partial record initialization, metadata reflection and inline expansion remain open. - [x] type-record procedure checkpoint: records/tuples containing type values cross compile-time procedure inputs/results, including nested arrays/records, aliases, defaults and dependent annotations. compile-time producers accept ordinary constant inputs; runtime generics accept erased `$` metadata parameters. owned snapshots isolate argument/result copies, exact type/value leaves retain checked diagnostics, and normalized cache keys preserve nominal generated types. generated storage/defaults, imported producers/provenance, shared execution bounds and repeatable compilation work through interpreter/native execution. runtime metadata storage, partial record initialization, reflection and inline expansion remain open. - [x] generated-record checkpoint: `struct_type!(layout, names, types, defaults)` builds ordinary records from constant fixed arrays or tuples with unique ASCII field names, concrete types and contextual optional defaults. literal heterogeneous collections, helper results and loop-filled producer arrays preserve exact types, owned defaults/copies and cached nominal identity through interpreter/native execution. `.auto` permits empty records; `.c` requires nonempty C-compatible fields and retains native field order/layout. malformed inputs, incomplete arrays, runtime dependencies, field/default validation, imported provenance, shared construction bounds and repeatable compilation retain diagnostics and isolation. metadata inspection, inline expansion, aggregate C calling conventions and storage-bearing defaults remain open. - - [ ] remaining gate: general type-valued compile-time execution, inline match/loop expansion and metadata reflection (`typeinfo!` and metadata collections), broader constant materialization (including untagged unions and storage-bearing values), plus existing-library acceptance. required expression-valued control flow, compound updates, formatting parsing and imported C-binding support may interleave with later gates. + - [x] inline-loop checkpoint: `inline for` expands constant integer ranges, fixed arrays (including type-valued arrays) and strings into separately checked runtime bodies through interpreter/native execution. exact immutable captures, `usize` collection indices, narrow inclusive maxima, selected conditional/tagged-match bodies, static break/continue tail pruning, nested inline/ordinary loops, runtime returns/errors and scoped cleanup retain ordinary semantics. empty loops skip body elaboration; runtime-dependent inputs/transfers, pointer captures, storage-bearing escapes and shared expansion bounds retain diagnostics. imported producers/provenance and repeatable compilation are verified. heterogeneous tuple/metadata iteration and inline-match syntax remain open. + - [ ] remaining gate: general type-valued compile-time execution, heterogeneous inline collections, inline match expansion and metadata reflection (`typeinfo!` and metadata collections), broader constant materialization (including untagged unions and storage-bearing values), plus existing-library acceptance. required expression-valued control flow, compound updates, formatting parsing and imported C-binding support may interleave with later gates. - [ ] remaining source constructs: expression-valued control flow and `yield`, compound updates and multiline strings. examples: `source/parser/parse.hon`, `source/llvm/runtime.hon`. - [ ] standard-library milestone: compile/run a multi-package program exercising existing allocation and IO, including error cleanup; no replacement miniature standard library. - [ ] native driver integration: required C ABI declarations, `process.Init` startup arguments/IO, actual CLI exit statuses, file loading, temporary files and spawning Clang. Honey-generated programs currently accept only parameterless `main` and print its result; Bro's bootstrap startup support is not Honey backend support. diff --git a/source/ast/ast.hon b/source/ast/ast.hon index 0122ad8..9efa58f 100644 --- a/source/ast/ast.hon +++ b/source/ast/ast.hon @@ -11,7 +11,7 @@ #! * arm ...... -> block | stmt (single statements become canonical blocks) #! * match_arm -> (expr {',' expr} ['|' identifier '|'] | 'else') ':' (block | stmt | expr); arms are comma/newline separated. #! enum members and tagged-union variants accept keyword names; member names do not enter lexical scopes. -#! * stmt ..... -> decl | expr | block | ('defer' | 'errdefer' ['|' identifier '|']) arm | expr '=' expr | 'return' [expr] | 'break' | 'continue' | 'if' expr ['|' identifier '|'] arm ['else' arm] | 'while' expr ['|' identifier '|'] arm | 'for' expr '|' ['@'] identifier [',' identifier] '|' arm +#! * stmt ..... -> decl | expr | block | ('defer' | 'errdefer' ['|' identifier '|']) arm | expr '=' expr | 'return' [expr] | 'break' | 'continue' | 'if' expr ['|' identifier '|'] arm ['else' arm] | 'while' expr ['|' identifier '|'] arm | ['inline'] 'for' expr '|' ['@'] identifier [',' identifier] '|' arm #! braceless guards/iterables require parentheses unless they are calls or optional captures; else binds to the nearest if. #! precedence (tightest first): calls/indices/selections/dereference/unwrap, unary -/!/&/try higher than multiplication/division, addition/subtraction, relational comparisons, equality, and, or, ranges/catch. type composition | binds tighter than fallible !; pointer/optional/array prefixes bind to type atoms. compound range endpoints require their own parentheses. #! binary operators associate left; unary operators associate right. @@ -400,6 +400,7 @@ ForView :: struct { pointer_capture bool index_capture bool body NodeId + is_inline bool = false } InvalidView :: struct { @@ -994,6 +995,13 @@ add_for_stmt proc( }, span) } +set_for_inline proc(builder @mut Builder, id NodeId) void { + node :: &builder.storage.nodes.items[usize(id)] + debug.assert(node.kind == .stmt_for) + flags :: &builder.storage.extra.items[usize(node.data1.extra) + 3] + flags^ = flags^ | 2 +} + add_statement_decl proc( builder @mut Builder, span Span, @@ -1310,8 +1318,9 @@ for_stmt proc(tree @Tree, id NodeId) ForView { capture = maybe_node_id(tree.storage.extra.items[extra]), index = maybe_node_id(tree.storage.extra.items[extra + 1]), pointer_capture = tree.storage.extra.items[extra + 2] != 0, - index_capture = tree.storage.extra.items[extra + 3] != 0, + index_capture = (tree.storage.extra.items[extra + 3] & 1) != 0, body = node.data0.node_id, + is_inline = (tree.storage.extra.items[extra + 3] & 2) != 0, } } diff --git a/source/ast/renderer/renderer.hon b/source/ast/renderer/renderer.hon index dfed97b..f678726 100644 --- a/source/ast/renderer/renderer.hon +++ b/source/ast/renderer/renderer.hon @@ -101,6 +101,7 @@ render_block proc(renderer @mut Renderer, tree @ast.Tree, block ast.NodeId, dept } .for_stmt: { loop :: ast.for_stmt(tree, statement) + if (loop.is_inline) try append_text(renderer, "inline ") try append_text(renderer, "for ") try append_text(renderer, ast.source_text(tree, ast.node_span(tree, loop.iterable))) try append_text(renderer, " |") diff --git a/source/compilation/compilation.test.hon b/source/compilation/compilation.test.hon index 422e594..dcf1a71 100644 --- a/source/compilation/compilation.test.hon +++ b/source/compilation/compilation.test.hon @@ -338,6 +338,74 @@ recompiles_specialized_calls_without_borrowing_previous_ir test { } } +recompiles_imported_inline_for_without_retaining_expansions test { + diagnostic_store := diagnostics.init(mem.c_allocator) + defer diagnostics.deinit(&diagnostic_store) + modules [2]mut ir.Module := undefined + { + session := init(mem.c_allocator) + defer deinit(&session) + root :: try add_package(&session) + dependency :: try add_package(&session) + root_bytes := try owned_text( + `dep :: import "../dependency" + `touch proc(value i16) i16 { return value } + `main proc() i16 { + ` total i16 := 0 + ` inline for dep.values() |value, index| { + ` at usize :: index + ` total = total + touch(value) + i16(at) + ` } + ` inline for dep.types() |T| { + ` if (T == i16) { + ` value T :: total + ` total = touch(value) + ` } else { + ` value T :: true + ` if (!value) return 0 + ` } + ` } + ` inline for (i8(0)..i8(2)) |outer| { + ` inline for [i8(0), i8(1)] |inner| { + ` for (0..usize(2)) |index| { + ` if (index == 0) continue + ` total = total + i16(outer + inner) + ` } + ` } + ` } + ` return total + `} + , + ) + defer arraylist.deinit(&root_bytes) + dependency_bytes := try owned_text( + `values proc() [2]i16 { return [17, 20] } + `types proc() [2]type { return [i16, bool] } + , + ) + defer arraylist.deinit(&dependency_bytes) + root_source :: try add_source(&session, root, "app/main.hon", &root_bytes, &diagnostic_store) + _ = try add_source(&session, dependency, "dependency/values.hon", &dependency_bytes, &diagnostic_store) + try arraylist.append(&session.imports, Import{ + node = declaration(&session, root_source, 0).node, + target = dependency, + }) + modules[0] = try compile(&session, root, &diagnostic_store) + errdefer ir.deinit(&modules[0]) + modules[1] = try compile(&session, root, &diagnostic_store) + } + defer ir.deinit(&modules[0]) + defer ir.deinit(&modules[1]) + try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) + for (0..modules.len) |index| { + outcome := try interpreter.run(mem.c_allocator, &modules[index], &diagnostic_store) + defer interpreter.deinit(&outcome) + try testing.expect(outcome.trap == null) + try testing.expect_equal(i64(42), outcome.value.data.integer) + } + try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) +} + recompiles_generated_records_without_borrowing_previous_ir test { diagnostic_store := diagnostics.init(mem.c_allocator) defer diagnostics.deinit(&diagnostic_store) diff --git a/source/diagnostics/diagnostics.hon b/source/diagnostics/diagnostics.hon index b8d3cef..38e4e15 100644 --- a/source/diagnostics/diagnostics.hon +++ b/source/diagnostics/diagnostics.hon @@ -113,6 +113,7 @@ Code :: enum { specialization_limit comptime_execution_limit comptime_value_escape + inline_loop_control invalid_generated_field_name invalid_struct_layout invalid_c_record_field_type @@ -250,6 +251,7 @@ messages :: $enummap.init(Code, []u8, { specialization_limit = "compile-time specialization exceeds its recursion or instance limit", comptime_execution_limit = "compile-time execution exceeds its recursion or step limit", comptime_value_escape = "compile-time result cannot escape evaluation as a source value", + inline_loop_control = "break or continue targeting an inline loop must be compile-time-resolvable", invalid_generated_field_name = "generated field name must be an ASCII identifier", invalid_struct_layout = "generated record layout must be .auto or .c; C-layout records require at least one field", invalid_c_record_field_type = "record field is not a supported C-layout value type", diff --git a/source/lexer/scan.hon b/source/lexer/scan.hon index 4de49fb..8b8647b 100644 --- a/source/lexer/scan.hon +++ b/source/lexer/scan.hon @@ -33,6 +33,7 @@ keywords strmap.StaticStringMap(TokenKind) :: strmap.init([ { "errdefer", .errdefer }, { "if", .if }, { "for", .for }, + { "inline", .inline }, { "else", .else }, { "while", .while }, { "true", .true }, diff --git a/source/lexer/scan.test.hon b/source/lexer/scan.test.hon index a70af34..8fd3cc4 100644 --- a/source/lexer/scan.test.hon +++ b/source/lexer/scan.test.hon @@ -123,6 +123,25 @@ scans_loop_controls_with_exact_keyword_boundaries test { try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) } +scans_inline_keyword_without_splitting_identifiers test { + pool := strpool.init(mem.c_allocator) + defer strpool.deinit(&pool) + diagnostic_store := diagnostics.init(mem.c_allocator) + defer diagnostics.deinit(&diagnostic_store) + state := init(mem.c_allocator) + defer deinit(&state) + try scan(&state, "inline for inlined inline_1 inline2", &pool, &diagnostic_store) + expected [6]TokenKind :: [.inline, .for, .ident, .ident, .ident, .eof] + try testing.expect_equal(expected.len, state.tokens.items.len) + for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind) + try testing.expect_equal(u32(0), state.tokens.items[0].span.start) + try testing.expect_equal(u32(6), state.tokens.items[0].span.end) + try testing.expect_equal("inlined", strpool.get_str(&pool, state.tokens.items[2].data.str_id)?) + try testing.expect_equal("inline_1", strpool.get_str(&pool, state.tokens.items[3].data.str_id)?) + try testing.expect_equal("inline2", strpool.get_str(&pool, state.tokens.items[4].data.str_id)?) + try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) +} + scans_ranges_without_consuming_float_dots test { pool := strpool.init(mem.c_allocator) defer strpool.deinit(&pool) diff --git a/source/lexer/token.hon b/source/lexer/token.hon index 1676072..f5e5e9d 100644 --- a/source/lexer/token.hon +++ b/source/lexer/token.hon @@ -21,6 +21,7 @@ TokenKind :: enum { if, else for, while match + inline proc, return break, continue try, catch diff --git a/source/parser/parse.hon b/source/parser/parse.hon index 6d3cf39..6216500 100644 --- a/source/parser/parse.hon +++ b/source/parser/parse.hon @@ -293,7 +293,7 @@ parse_statement proc(state @mut State, allow_else bool) ast.NodeId ! ParseError match current(state).kind { .if: return try parse_if(state) .while: return try parse_while(state, allow_else) - .for: return try parse_for(state, allow_else) + .for, .inline: return try parse_for(state, allow_else) .return: return try parse_return(state, allow_else) .break, .continue: return try parse_loop_control(state, allow_else) .defer, .errdefer: return try parse_cleanup(state, allow_else) @@ -510,7 +510,10 @@ parse_captures proc(state @mut State) CaptureList ! ParseError { @hide:file parse_for proc(state @mut State, allow_else bool) ast.NodeId ! ParseError { - keyword :: try consume(state, .for) + keyword :: current(state) + inlined :: next_is(state, .inline) + if (inlined) state.next_token += 1 + _ = try consume(state, .for) skip_newlines(state) iterable_start :: state.next_token iterable := try parse_guard_expr(state) @@ -556,7 +559,7 @@ parse_for proc(state @mut State, allow_else bool) ast.NodeId ! ParseError { and !next_is(state, .open_curly) ) state.next_token = after_captures body :: try parse_arm(state, allow_else) - return try ast.add_for_stmt( + id :: try ast.add_for_stmt( state.builder, ast.Span{ start = keyword.span.start, @@ -565,6 +568,8 @@ parse_for proc(state @mut State, allow_else bool) ast.NodeId ! ParseError { }, iterable, captures.item, captures.index, captures.pointer, captures.indexed, body, ) + if (inlined) ast.set_for_inline(state.builder, id) + return id } @hide:file @@ -599,7 +604,7 @@ parse_guard proc(state @mut State) ast.NodeId ! ParseError { and !next_is(state, .newline) and !next_is(state, .eof) and !next_is(state, .else) and !next_is(state, .return) and !next_is(state, .if) and !next_is(state, .while) - and !next_is(state, .for) + and !next_is(state, .for) and !next_is(state, .inline) and !next_is(state, .break) and !next_is(state, .continue) ) { if (next_is(state, .ident) and next_n_is(state, 1, .equal)) break @@ -850,7 +855,7 @@ starts_type proc(kind lexer.TokenKind) bool { @hide:file member_name proc(kind lexer.TokenKind) bool { return match kind { - .ident, .if, .else, .for, .while, .proc, .return, .break, .continue, + .ident, .if, .else, .for, .inline, .while, .proc, .return, .break, .continue, .true, .false, .and, .or, .mut, .struct, .enum, .union, .distinct, .alias, .match, .import, .test, .c_func, .null, .undefined, .try, .catch, .defer, .errdefer: true else: false @@ -1071,7 +1076,7 @@ parse_match_arm proc(state @mut State) ast.NodeId ! ParseError { state.match_arm_body = true defer state.match_arm_body = previous statement_body :: match current(state).kind { - .open_curly, .return, .break, .continue, .if, .while, .for, .defer, .errdefer: true + .open_curly, .return, .break, .continue, .if, .while, .for, .inline, .defer, .errdefer: true else: false } if (statement_body or named_binding_starts(state)) { @@ -1703,7 +1708,7 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError { @hide:file is_expression_boundary proc(kind lexer.TokenKind) bool { return match kind { - .close_paren, .newline, .close_curly, .open_curly, .else, .return, .if, .while, .for, .defer, .errdefer, .eof: true + .close_paren, .newline, .close_curly, .open_curly, .else, .return, .if, .while, .for, .inline, .defer, .errdefer, .eof: true else: false } } @@ -1842,7 +1847,7 @@ recovery_closes_delimiters proc(state @State, paren_depth, square_depth usize) b for (state.next_token)..(state.tokens.len) |i| { kind :: state.tokens[i].kind match kind { - .open_curly, .close_curly, .eof, .if, .else, .while, .for, + .open_curly, .close_curly, .eof, .if, .else, .while, .for, .inline, .return, .break, .continue, .defer, .errdefer: return false .ident: { if (i + 1 < state.tokens.len) { diff --git a/source/parser/parse.test.hon b/source/parser/parse.test.hon index 3fa1ba2..5473da5 100644 --- a/source/parser/parse.test.hon +++ b/source/parser/parse.test.hon @@ -1232,6 +1232,114 @@ parses_range_endpoints_and_capture_shapes test { try testing.expect_equal(ast.name_id(&tree, fourth.index?), ast.name_id(&tree, index_reference)) } +parses_inline_loops_and_clones_capture_metadata test { + strings := strpool.init(mem.c_allocator) + defer strpool.deinit(&strings) + diagnostic_store := diagnostics.init(mem.c_allocator) + defer diagnostics.deinit(&diagnostic_store) + source :: + `run proc() void { + ` inline for 0..=2 |item| { + ` for "ab" |byte, index| { continue } + ` } + ` for [1, 2] |value, index| {} + ` inline for [i32, u8] |@T, _| {} + ` inline for ("xy") |byte, index| continue + `} + scan_state := lexer.init(mem.c_allocator) + defer lexer.deinit(&scan_state) + try lexer.scan(&scan_state, source, &strings, &diagnostic_store) + builder := ast.init_builder(mem.c_allocator, source) + defer ast.deinit_builder(&builder) + root :: try parse_source(&builder, scan_state.tokens.items, &diagnostic_store) + original :: ast.view(&builder, root) + mapping :: try mem.alloc(ast.NodeId, mem.c_allocator, ast.node_count(&original)) + defer mem.free(mem.c_allocator, mapping) + mem.set(mapping, maxval!(ast.NodeId)) + cloned :: try ast.clone_subtree(&builder, root, mapping) + tree :: ast.view(&builder, root) + try testing.expect_equal(0, diagnostics.count(&diagnostic_store)) + roots [2]ast.NodeId :: [root, cloned] + for (roots) |package| { + body :: ast.function_decl(&tree, ast.child(&tree, package, 0)).body + try testing.expect_equal(4, ast.block_statement_count(&tree, body)) + first :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 0)) + try testing.expect(first.is_inline and !first.pointer_capture and !first.index_capture) + try testing.expect(ast.range_expr(&tree, first.iterable).inclusive) + try testing.expect_equal("inline", ast.source_text(&tree, ast.Span{ + start = first.span.start, end = first.span.start + 6, source = first.span.source, + })) + nested :: ast.for_stmt(&tree, ast.block_statement(&tree, first.body, 0)) + try testing.expect(!nested.is_inline and nested.index_capture) + try testing.expect_equal(ast.ExprKind.literal_string, ast.expr_kind(&tree, nested.iterable)) + ordinary :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 1)) + try testing.expect(!ordinary.is_inline and ordinary.index_capture) + try testing.expect_equal(2, ast.array_element_count(&tree, ordinary.iterable)) + pointer :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 2)) + try testing.expect(pointer.is_inline and pointer.pointer_capture and pointer.index_capture) + try testing.expect(pointer.index == null) + try testing.expect_equal("T", strpool.get_str(&strings, ast.name_id(&tree, pointer.capture?))?) + indexed :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 3)) + try testing.expect(indexed.is_inline and !indexed.pointer_capture and indexed.index_capture) + try testing.expect_equal("index", strpool.get_str(&strings, ast.name_id(&tree, indexed.index?))?) + try testing.expect_equal(ast.StmtKind.continue_stmt, ast.stmt_kind( + &tree, ast.block_statement(&tree, indexed.body, 0), + )) + } + original_body :: ast.function_decl(&tree, ast.child(&tree, root, 0)).body + cloned_body :: ast.function_decl(&tree, ast.child(&tree, cloned, 0)).body + first :: ast.for_stmt(&tree, ast.block_statement(&tree, original_body, 0)) + copy :: ast.for_stmt(&tree, ast.block_statement(&tree, cloned_body, 0)) + try testing.expect(copy.capture? != first.capture?) + try testing.expect_equal(mapping[usize(first.capture?)], copy.capture?) + try testing.expect_equal(first.span.start, copy.span.start) + try testing.expect_equal(first.span.end, copy.span.end) + try testing.expect_equal(first.span.source, copy.span.source) +} + +rejects_malformed_inline_forms_without_losing_siblings test { + strings := strpool.init(mem.c_allocator) + defer strpool.deinit(&strings) + cases [7][]u8 :: [ + "inline while true {}", + "inline match value { else: {} }", + "inline 0..2 |item| {}", + "inline inline for 0..2 |item| {}", + "inline", + "inline for 0..2 || {}", + "inline for 0..2 |item,| {}", + ] + for (cases) |header| { + diagnostic_store := diagnostics.init(mem.c_allocator) + defer diagnostics.deinit(&diagnostic_store) + source arraylist.ArrayList(u8) := arraylist.init(mem.c_allocator) + defer arraylist.deinit(&source) + for ("run proc() void {\n") |byte| try arraylist.append(&source, byte) + for (header) |byte| try arraylist.append(&source, byte) + for ("\ninline for (0..2) |item| continue\nkept :: 7\n}\nafter :: 9\n") |byte| + try arraylist.append(&source, byte) + scan_state := lexer.init(mem.c_allocator) + defer lexer.deinit(&scan_state) + try lexer.scan(&scan_state, source.items, &strings, &diagnostic_store) + tree := try parse(mem.c_allocator, scan_state.tokens.items, source.items, &diagnostic_store) + defer ast.deinit(&tree) + try testing.expect(diagnostics.count(&diagnostic_store) > 0) + try testing.expect_equal(2, ast.package_decl_count(&tree)) + body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body + try testing.expect_equal(3, ast.block_statement_count(&tree, body)) + preserved :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 1)) + try testing.expect(preserved.is_inline) + try testing.expect_equal(ast.StmtKind.continue_stmt, ast.stmt_kind( + &tree, ast.block_statement(&tree, preserved.body, 0), + )) + kept :: ast.statement_decl(&tree, ast.block_statement(&tree, body, 2)) + try testing.expect_equal("kept", ast.source_text(&tree, kept.name)) + try testing.expect_equal("after", ast.source_text( + &tree, ast.statement_decl(&tree, ast.package_decl(&tree, 1)).name, + )) + } +} + rejects_bro_invalid_range_headers_without_losing_siblings test { strings := strpool.init(mem.c_allocator) defer strpool.deinit(&strings) diff --git a/source/sema/specialize/specialize.hon b/source/sema/specialize/specialize.hon index 55ef5f4..ded1e78 100644 --- a/source/sema/specialize/specialize.hon +++ b/source/sema/specialize/specialize.hon @@ -121,6 +121,8 @@ elaborate proc( id :: ast.NodeId(i) class :: ast.node_class(&original, id) if (class == .expression and ast.expr_kind(&original, id) == .comptime_expr) needed = true + if (class == .statement and ast.stmt_kind(&original, id) == .for_stmt + and ast.for_stmt(&original, id).is_inline) needed = true if (class == .expression and ast.expr_kind(&original, id) == .builtin_call) { name :: ast.builtin_call(&original, id).name text :: ast.source_text(&original, ast.node_span(&original, name)) @@ -231,6 +233,11 @@ expand proc(state @mut State, id ast.NodeId) void ! Error { if (state.expanded.items[usize(id)] != 0) return state.expanded.items[usize(id)] = 1 class :: ast.node_class(&state.tree, id) + if (class == .statement and ast.stmt_kind(&state.tree, id) == .for_stmt + and ast.for_stmt(&state.tree, id).is_inline) { + try expand_inline_for(state, id) + return + } if (class == .statement and ast.stmt_kind(&state.tree, id) == .expression) { expression :: ast.statement_expr(&state.tree, id) if ( @@ -1520,6 +1527,8 @@ TypeExecution :: struct { returned ast.NodeId = no_node result_type ast.NodeId loop_depth usize = 0 + inline_body ast.NodeId = no_node + inline_blocks ?@mut arraylist.ArrayList(ast.NodeId) = null } @hide:file @@ -1609,6 +1618,152 @@ nominal_type_value proc(state @State, id ast.NodeId) bool { return kind == .record_type or kind == .union_type or kind == .enum_type or kind == .distinct_type } +# transfers targeting an inline loop must be selected before checking runtime +# bodies. nested loops own their transfers and are elaborated independently. +@hide:file +inline_has_transfer proc(state @State, id ast.NodeId) bool { + class :: ast.node_class(&state.tree, id) + if (class == .function) return false + if (class == .statement) { + kind :: ast.stmt_kind(&state.tree, id) + if (kind == .for_stmt or kind == .while_stmt) return false + if (kind == .break_stmt or kind == .continue_stmt) return true + } + for (0..ast.child_count(&state.tree, id)) |i| { + if (inline_has_transfer(state, ast.child(&state.tree, id, i))) return true + } + return false +} + +@hide:file +inline_statement proc(state @mut State, id ast.NodeId) TypeFlow ! Error { + kind :: ast.stmt_kind(&state.tree, id) + if (kind == .break_stmt) return .break + if (kind == .continue_stmt) return .continue + if (kind == .block_stmt) + return try inline_block(state, ast.statement_block(&state.tree, id)) + selected := no_node + if (kind == .if_stmt) { + conditional :: ast.if_stmt(&state.tree, id) + explicit :: ast.expr_kind(&state.tree, conditional.condition) == .comptime_expr + condition :: if (explicit) ast.comptime_operand(&state.tree, conditional.condition) else conditional.condition + try expand(state, condition) + if (conditional.capture == null and (explicit or branch_constant(state, condition, 0))) { + value :: try evaluate_ir(state, condition, try builtin(state, condition, .bool)) + if (value.kind != .boolean) return .invalid + if (value.boolean) selected = conditional.then_block + else if (conditional.else_block) |other| selected = other + else selected = try ast.add_block(state.builder, conditional.span, &[]) + } + } + if (kind == .expression) { + expression :: ast.statement_expr(&state.tree, id) + if (ast.expr_kind(&state.tree, expression) == .match_expr) { + subject :: ast.match_expr(&state.tree, expression).subject + explicit :: ast.expr_kind(&state.tree, subject) == .comptime_expr + try expand(state, subject) + if (explicit or branch_constant(state, subject, 0)) { + index :: try type_match_arm(state, expression) + if (index == no_index) return .invalid + arm :: ast.match_arm_view(&state.tree, ast.match_arm(&state.tree, expression, index)) + body :: try type_match_body(state, expression, arm) + if (body == no_node) return .invalid + if (!arm.expression) selected = body + else { + statement :: try ast.add_expression_stmt(state.builder, body) + statements [1]ast.NodeId :: [statement] + selected = try ast.add_block(state.builder, ast.node_span(&state.tree, id), statements[..]) + } + } + } + } + if (selected != no_node) { + try refresh(state) + flow :: try inline_block(state, selected) + replacement :: try ast.add_block_stmt(state.builder, selected) + try replace(state, id, replacement) + try refresh(state) + return flow + } + if (inline_has_transfer(state, id)) { + state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, + .inline_loop_control, ast.node_span(&state.tree, id)) + return .invalid + } + try expand(state, id) + return .next +} + +@hide:file +inline_block proc(state @mut State, block ast.NodeId) TypeFlow ! Error { + statements := arraylist.init(ast.NodeId, state.allocator) + defer arraylist.deinit(&statements) + flow := TypeFlow.next + for (0..ast.block_statement_count(&state.tree, block)) |i| { + statement :: ast.block_statement(&state.tree, block, i) + if (!try type_step(state, statement)) return .invalid + flow = try inline_statement(state, statement) + if (flow == .invalid) return flow + kind :: ast.stmt_kind(&state.tree, statement) + if (kind != .break_stmt and kind != .continue_stmt) try arraylist.append(&statements, statement) + if (flow != .next) break + } + replacement :: try ast.add_block(state.builder, ast.node_span(&state.tree, block), statements.items) + try replace(state, block, replacement) + try refresh(state) + return flow +} + +@hide:file +expand_inline_for proc(state @mut State, id ast.NodeId) void ! Error { + loop :: ast.for_stmt(&state.tree, id) + if (state.type_execution_depth == 0) state.type_execution_remaining = 4096 + state.type_execution_depth += 1 + defer state.type_execution_depth -= 1 + state.evaluation_diagnostic = diagnostics.no_diagnostic + blocks := arraylist.init(ast.NodeId, state.allocator) + defer arraylist.deinit(&blocks) + execution := TypeExecution{ + locals = arraylist.init(state.allocator), + mapping = try mem.alloc(ast.NodeId, state.allocator, ast.node_count(&state.tree)), + first_owned = ast.NodeId(0), result_type = no_node, + inline_body = loop.body, inline_blocks = &blocks, + } + defer arraylist.deinit(&execution.locals) + defer mem.free(state.allocator, execution.mapping) + mem.set(execution.mapping, no_node) + for (0..usize(2)) |i| { + capture :: if (i == 0) loop.capture else loop.index + if (capture) |node| { + resolved :: binding(state, node) + if (resolved.kind == .invalid) state.evaluation_diagnostic = resolved.data.diagnostic + } + } + if (loop.pointer_capture) + state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .comptime_value_escape, loop.span) + flow := TypeFlow.invalid + if (state.evaluation_diagnostic == diagnostics.no_diagnostic) + flow = try execute_type_for(state, &execution, id) + if (flow != .next or state.evaluation_diagnostic != diagnostics.no_diagnostic) { + if (state.evaluation_diagnostic == diagnostics.no_diagnostic) + state.evaluation_diagnostic = try diagnostics.add(state.diagnostics, .comptime_argument_required, loop.span) + invalid :: try ast.add_invalid_stmt(state.builder, loop.span, state.evaluation_diagnostic) + try replace(state, id, invalid) + try refresh(state) + return + } + statements := arraylist.init(ast.NodeId, state.allocator) + defer arraylist.deinit(&statements) + for (blocks.items) |block| { + statement :: try ast.add_block_stmt(state.builder, block) + try arraylist.append(&statements, statement) + } + block :: try ast.add_block(state.builder, loop.span, statements.items) + replacement :: try ast.add_block_stmt(state.builder, block) + try replace(state, id, replacement) + try refresh(state) +} + # select with ordinary checked IR so patterns, overlap and exhaustiveness have # the same rules as runtime matching, without checking unexecuted type bodies. @hide:file @@ -2373,6 +2528,13 @@ type_assignment_value proc( @hide:file execute_type_block proc(state @mut State, execution @mut TypeExecution, block ast.NodeId) TypeFlow ! Error { + if (block == execution.inline_body) { + snapshot :: try type_snapshot(state, execution, block) + flow :: try inline_block(state, snapshot) + if (flow == .invalid) return flow + try arraylist.append(execution.inline_blocks?, snapshot) + return flow + } saved :: execution.locals.items.len defer execution.locals.items = execution.locals.items[..saved] for (0..ast.block_statement_count(&state.tree, block)) |i| { @@ -3284,8 +3446,15 @@ expression_type proc(state @mut State, id ast.NodeId, depth usize) ast.NodeId ! .array_literal: { count :: ast.array_element_count(&state.tree, id) if (count == 0) return no_node - element :: try expression_type(state, ast.array_element(&state.tree, id, 0), depth + 1) + element := try expression_type(state, ast.array_element(&state.tree, id, 0), depth + 1) if (element == no_node) return no_node + # inferred values have concrete default widths even when a cast or + # procedure result used the int/float annotation shorthand. + scalar :: binding(state, element) + if (scalar.kind == .builtin) { + if (scalar.data.builtin == .int) element = try builtin(state, element, .i64) + if (scalar.data.builtin == .float) element = try builtin(state, element, .f64) + } length :: try ast.add_integer(state.builder, ast.node_span(&state.tree, id), u64(count)) try refresh(state) result :: try ast.add_array_type(state.builder, ast.node_span(&state.tree, id), length, element, false) diff --git a/tests/native.py b/tests/native.py index 2ab577a..207fe6a 100644 --- a/tests/native.py +++ b/tests/native.py @@ -11,6 +11,327 @@ HONEY = ROOT / "build/honey" # Explicit expectations guard against frontend bugs shared by both execution paths. CASES = { + "inline_for_range_runtime_calls_and_constant_branches": ( + """ + add proc(value i16, bias i16) i16 { return value + bias } + answer proc(bias i16) i16 { + total i16 := 0 + inline for (i8(0)..i8(3)) |index| { + if (index == 0) { + total = add(total, bias) + } else { + total = add(total, i16(index)) + } + if (index > 9) return false + } + return total + } + main proc() i16 { return answer(39) } + """, b"42\n", None, + ), + "inline_for_inferred_arrays_concretize_scalar_shorthand": ( + """ + signed proc(value i64) i64 { return value } + real proc(value f64) f64 { return value } + integer proc() int { return 23 } + floating proc() float { return 22.5 } + main proc() i64 { + total i64 := 0 + inline for [int(19), integer()] |value| { total = total + signed(value) } + fraction f64 := 0.0 + inline for [float(19.5), floating()] |value| { fraction = fraction + real(value) } + return total + i64(fraction) - 42 + } + """, b"42\n", None, + ), + "inline_for_array_constants_and_usize_indices": ( + """ + data proc() [2]i16 { return [19, 22] } + at proc(index usize, value i16) i16 { return value + i16(index) } + main proc() i16 { + total i16 := 0 + inline for data() |value, index| { + if (value == 19) { + total = total + at(index, value) + } else if (value == 22) { + total = total + at(index, value) + } else return false + } + return total + } + """, b"42\n", None, + ), + "inline_for_string_bytes_embedded_nul_and_usize_indices": ( + r""" + text proc() string { return "a\0b" } + at proc(index usize, byte u8) i16 { + if (byte == 0) return i16(index) + 39 + return 1 + } + main proc() i16 { + total i16 := 0 + inline for text() |byte, index| { + if (index == 1) { + if (byte != 0) return false + } else if (byte == 0) return false + total = total + at(index, byte) + } + return total + } + """, b"42\n", None, + ), + "inline_for_type_array_specializes_each_runtime_body": ( + """ + types proc() [2]type { return [i16, bool] } + add proc(value i16) i16 { return value + 1 } + main proc() i16 { + total i16 := 0 + inline for types() |T, index| { + if (T == i16) { + value T :: 39 + total = total + add(value) + } else { + value T :: true + if (value) total = total + i16(index) + 1 + } + } + return total + } + """, b"42\n", None, + ), + "inline_for_narrow_signed_and_unsigned_inclusive_maxima": ( + """ + signed proc(value i8) i16 { return i16(value) } + unsigned proc(value u8) i16 { return i16(value) } + main proc() [3]i16 { + negative i16 := 0 + inline for (i8(-2)..i8(0)) |value| { negative = negative + signed(value) } + high i16 := 0 + inline for (i8(126)..=i8(127)) |value| { high = high + signed(value) } + top i16 := 0 + inline for (u8(254)..=u8(255)) |value| { top = top + unsigned(value) } + return [negative, high, top] + } + """, b"[-3, 253, 509]\n", None, + ), + "inline_for_nested_inline_and_ordinary_loops": ( + """ + answer proc(length usize) i16 { + total i16 := 0 + inline for [i16(10), i16(11)] |base| { + inline for (i8(0)..i8(2)) |offset| { + if (offset == 0) continue + for (0..length) |index| { + if (index == 0) continue + total = total + base + break + } + } + } + return total + } + main proc() i16 { return answer(3) + answer(2) } + """, b"42\n", None, + ), + "inline_for_static_break_continue_run_iteration_defers": ( + """ + walk proc(cleanups @mut i16) i16 { + total i16 := 0 + inline for (i8(0)..i8(5)) |index| { + defer cleanups^ = cleanups^ + 1 + if (index < 1) continue + if (index == 3) break + total = total + i16(index) + } + return total + } + main proc() i16 { + cleanups i16 := 0 + total :: walk(&cleanups) + return total + cleanups + 35 + } + """, b"42\n", None, + ), + "inline_for_static_transfers_skip_invalid_tail_and_later_iterations": ( + """ + main proc() i16 { + inline for (i8(0)..i8(3)) |index| { + if (index == 0) continue + if (index == 1) break + return false + } + return 42 + } + """, b"42\n", None, + ), + "inline_for_tagged_match_transfers_prune_tail_and_payloads": ( + """ + Choice :: union(enum) { number i16, stop void } + choices proc() [4]Choice { return [.number{0}, .number{39}, .stop, .number{99}] } + main proc() i16 { + total i16 := 0 + inline for choices() |choice| { + defer total = total + 1 + match choice { + .number |value|: { + if (value == 0) continue + total = total + value + continue + }, + .stop: break, + } + return false + } + return total + } + """, b"42\n", None, + ), + "inline_for_rejects_runtime_break": ( + """ + walk proc(stop bool) i16 { + inline for (i8(0)..i8(2)) |_| { + if (stop) break + } + return 42 + } + main proc() i16 { return walk(true) } + """, b"", b"compile-time-resolvable", + ), + "inline_for_rejects_runtime_continue": ( + """ + walk proc(skip bool) i16 { + inline for [i16(1), i16(2)] |_| { + if (skip) continue + } + return 42 + } + main proc() i16 { return walk(true) } + """, b"", b"compile-time-resolvable", + ), + "inline_for_runtime_return_runs_iteration_defer": ( + """ + find proc(take bool, cleanups @mut i16) i16 { + inline for [i16(1), i16(2), i16(3)] |value| { + defer cleanups^ = cleanups^ + 1 + if (take and value == 2) return 40 + } + return 0 + } + main proc() i16 { + cleanups i16 := 0 + value :: find(true, &cleanups) + return value + cleanups + } + """, b"42\n", None, + ), + "inline_for_runtime_error_return_runs_errdefer_only_on_error": ( + """ + Error :: enum { failed } + walk proc(fail bool, cleanups @mut i16) i16 ! Error { + inline for (i8(0)..i8(3)) |index| { + defer cleanups^ = cleanups^ + 1 + errdefer cleanups^ = cleanups^ + 10 + if (fail and index == 1) return .failed + } + return 39 + } + main proc() [2]i16 { + failed i16 := 0 + success i16 := 0 + left :: walk(true, &failed) catch 30 + right :: walk(false, &success) catch 0 + return [left + failed, right + success] + } + """, b"[42, 42]\n", None, + ), + "inline_for_empty_ranges_arrays_strings_skip_invalid_typing": ( + """ + empty proc() [0]type { return [] } + main proc() i16 { + inline for (i8(2)..i8(-2)) |_| { return false } + inline for (i8(1)..i8(1)) |_| { return i8(128) } + inline for empty() |_| { return false } + inline for "" |_| { return i8(128) } + return 42 + } + """, b"42\n", None, + ), + "inline_for_capture_keeps_narrow_arithmetic_overflow": ( + """ + main proc() i16 { + inline for (i8(127)..=i8(127)) |value| { next :: value + 1 } + return 42 + } + """, b"", b"integer arithmetic overflow", + ), + "inline_for_capture_is_immutable": ( + """ + main proc() i16 { + inline for [i16(1)] |value| { value = 42 } + return 0 + } + """, b"", b"assignment requires a mutable local binding", + ), + "inline_for_index_is_immutable": ( + """ + main proc() i16 { + inline for "a" |_, index| { index = 42 } + return 0 + } + """, b"", b"assignment requires a mutable local binding", + ), + "inline_for_rejects_runtime_range_bound": ( + """ + walk proc(length usize) i16 { + inline for (0..length) |_| {} + return 42 + } + main proc() i16 { return walk(2) } + """, b"", b"compile-time value", + ), + "inline_for_rejects_runtime_array": ( + """ + walk proc(values [2]i16) i16 { + inline for values |_| {} + return 42 + } + main proc() i16 { return walk([19, 23]) } + """, b"", b"compile-time value", + ), + "inline_for_rejects_runtime_string": ( + """ + walk proc(text string) i16 { + inline for text |_| {} + return 42 + } + main proc() i16 { return walk("ok") } + """, b"", b"compile-time value", + ), + "inline_for_rejects_pointer_capture": ( + """ + main proc() i16 { + inline for [i16(42)] |@value| { return value^ } + return 0 + } + """, b"", b"escape", + ), + "inline_for_rejects_borrowed_element_escape": ( + """ + data proc() [1][]u8 { return ["borrowed"] } + main proc() usize { + inline for data() |value| { return value.len } + return 0 + } + """, b"", b"escape", + ), + "inline_for_expansion_is_bounded": ( + """ + main proc() i16 { + inline for (0..usize(100000)) |_| {} + return 42 + } + """, b"", b"step limit", + ), "struct_type_literal_fields_contextual_defaults_and_required_null": ( """ present proc() ??i16 { return 2 } @@ -6386,7 +6707,48 @@ def check_specialization_packages(temporary, executable): interpreted = run(str(HONEY), "run", str(app)) assert interpreted.stderr.startswith(f"{sizing.resolve()}:5:".encode()), outcome(interpreted) assert len(interpreted.stderr.splitlines()) == 1, outcome(interpreted) - return 20 + sizing.write_text(textwrap.dedent(""" + values proc() [2]i16 { return [19, 22] } + types proc() [2]type { return [i16, bool] } + """).strip() + "\n") + source.write_text(textwrap.dedent(""" + lib :: import "../library" + add proc(value i16, index usize) i16 { return value + i16(index) } + main proc() i16 { + total i16 := 0 + inline for lib.values() |value, index| { total = total + add(value, index) } + inline for lib.types() |T| { + if (T == i16) { + value T :: total + total = value + } else { + value T :: true + if (!value) return 0 + } + } + return total + } + """).strip() + "\n") + check(app, executable, (b"42\n", None)) + sizing.write_text(textwrap.dedent(""" + values proc() [1]i16 { + value i8 :: 127 + overflow :: value + 1 + return [i16(overflow)] + } + """).strip() + "\n") + source.write_text(textwrap.dedent(""" + lib :: import "../library" + main proc() i16 { + inline for lib.values() |value| { return value } + return 0 + } + """).strip() + "\n") + check(app, executable, (b"", b"integer arithmetic overflow")) + interpreted = run(str(HONEY), "run", str(app)) + assert interpreted.stderr.startswith(f"{sizing.resolve()}:3:".encode()), outcome(interpreted) + assert len(interpreted.stderr.splitlines()) == 1, outcome(interpreted) + return 22 def check_dependency_provenance(temporary, executable): @@ -6436,6 +6798,14 @@ def check_dependency_provenance(temporary, executable): return 9223372036854775807 + 1 } """, 4, 12, False), + (""" + fail proc() int { + inline for [int(9223372036854775807)] |value| { + return value + 1 + } + return 42 + } + """, 5, 16, False), (""" fail proc() int { a :: [42]