feat: resolve file-local import namespaces

This commit is contained in:
2026-10-25 18:16:30 +01:00
parent abd85e531b
commit d834ff9587
14 changed files with 388 additions and 45 deletions
+4 -2
View File
@@ -20,7 +20,9 @@ run a source file with the interpreter:
`run`, `emit-llvm` and `build` also accept a directory. directory inputs compile its `.hon` and `.bro` files in bytewise filename order, excluding `.test.hon`/`.test.bro` and non-file entries; a file input still compiles only that file, not its siblings. declarations share a scope within each package, while dependencies have separate scopes and cannot supply the root package's `main`. `run`, `emit-llvm` and `build` also accept a directory. directory inputs compile its `.hon` and `.bro` files in bytewise filename order, excluding `.test.hon`/`.test.bro` and non-file entries; a file input still compiles only that file, not its siblings. declarations share a scope within each package, while dependencies have separate scopes and cannot supply the root package's `main`.
bare `import "../library"` and aliased `library :: import "@library"` declarations discover dependencies. relative paths start at the importing package directory; `@` starts at the nearest ancestor containing `build.hon` or `build.bro`, falling back to the entry package directory. package directories are canonicalized through symlinks and loaded once. cycles, missing/empty dependencies and invalid import paths produce source diagnostics. absolute paths and empty, escaped or control-containing import strings are rejected. production loading ignores test-only imports. namespace aliases are not bound yet, and qualified calls such as `library.add(...)` remain unsupported. bare `import "../library"` and aliased `library :: import "@library"` declarations load dependencies. relative paths start at the importing package directory; `@` starts at the nearest ancestor containing `build.hon` or `build.bro`, falling back to the entry package directory. package directories are canonicalized through symlinks and loaded once. cycles, missing/empty dependencies and invalid import paths produce source diagnostics. absolute paths and empty, escaped or control-containing import strings are rejected. production loading ignores test-only imports.
imports bind file-local namespaces, never dependency declarations in the importing package. bare imports use the canonical package directory's final name; directories without a valid identifier name require an explicit alias. sibling files can use the same alias for different packages, but an alias cannot conflict with a package declaration, builtin type or another import in its own file. local shadowing remains forbidden. `library.add(...)` selects a public declaration and uses ordinary signature checking, function values and IR calls without a runtime namespace value. imported namespaces are not re-exported; missing members and non-namespace selections are diagnosed. general record fields, imported types and visibility attributes are not implemented; unsupported attributes produce syntax diagnostics.
```hon ```hon
add proc(left, right int) int { add proc(left, right int) int {
@@ -105,7 +107,7 @@ identifier interning is compilation-owned: initialize a `strpool.StringPool`, pa
resolver scopes use `@std/hashmap.AutoHashMap(K, V)`, which shares `init`, `put`, `get` and `deinit` with custom `HashMap` and `StringHashMap`. automatic keys support integers, booleans, enums, distinct types, fixed arrays and records recursively; record padding is ignored. slices, floating-point values, pointers, optionals and unions require explicit hashing and equality through `HashMap` (`StringHashMap` supplies content-based byte-string operations). keys and values are stored by value; referenced resources remain caller-owned. the builtin declaration count is inferred, and builtin ID storage derives its fixed stack-array length from that declaration array rather than repeating a count or initializer. resolver scopes use `@std/hashmap.AutoHashMap(K, V)`, which shares `init`, `put`, `get` and `deinit` with custom `HashMap` and `StringHashMap`. automatic keys support integers, booleans, enums, distinct types, fixed arrays and records recursively; record padding is ignored. slices, floating-point values, pointers, optionals and unions require explicit hashing and equality through `HashMap` (`StringHashMap` supplies content-based byte-string operations). keys and values are stored by value; referenced resources remain caller-owned. the builtin declaration count is inferred, and builtin ID storage derives its fixed stack-array length from that declaration array rather than repeating a count or initializer.
`sema.resolve` returns an owned binding table indexed by AST `NodeId`, without modifying the AST. `sema.binding` identifies a declaration, builtin type, or invalid binding carrying its diagnostic ID; nodes with no binding return `.none`. release the result with `sema.deinit`. unknown names and duplicate declarations add source-spanned diagnostics; invalid syntax retains its existing diagnostic identity. same-scope duplicates retain the first declaration for recovery, while forbidden inner declarations invalidate that name's uses rather than silently binding them to an outer declaration. `sema.resolve` returns an owned binding table indexed by AST `NodeId`, without modifying the AST. `sema.binding` identifies a declaration, package namespace, builtin type, or invalid binding carrying its diagnostic ID; nodes with no binding return `.none`. `sema.resolve_packages` additionally borrows package roots and import links, predeclares every package before resolving qualified uses, and keeps import aliases in their originating source file. selected declarations have the same binding identity as unqualified declarations. release the result with `sema.deinit`. unknown names and duplicate declarations add source-spanned diagnostics; invalid syntax and failed imports retain their existing diagnostic identity. same-scope duplicates retain the first declaration for recovery, while forbidden inner declarations invalidate that name's uses rather than silently binding them to an outer declaration.
type checking (`@source/sema/check`) runs after name resolution. `check.check` borrows the AST and binding table and returns owned per-node types and interned array descriptors; query with `check.type_of` and `check.array_type`, and release with `check.deinit`. declarations infer their initializer types, including forward references and function aliases; function types identify the declaration supplying the callable signature. annotations support builtin types and fixed arrays; type names are not ordinary values. `void` is valid as a function result, not as a parameter or stored value. type checking (`@source/sema/check`) runs after name resolution. `check.check` borrows the AST and binding table and returns owned per-node types and interned array descriptors; query with `check.type_of` and `check.array_type`, and release with `check.deinit`. declarations infer their initializer types, including forward references and function aliases; function types identify the declaration supplying the callable signature. annotations support builtin types and fixed arrays; type names are not ordinary values. `void` is valid as a function result, not as a parameter or stored value.
+6 -6
View File
@@ -4,10 +4,10 @@
## current boundary ## current boundary
directory loading and import declaration discovery are implemented; qualified directory packages support file-local import namespaces and postfix declaration
namespace selection and the richer types/control flow used by the compiler selection through the interpreter and native backend. the richer types/control
remain unsupported. the driver retains frontend storage and source provenance flow used by the compiler remain unsupported. the driver retains frontend storage
through `@source/compilation`, with compilation-wide AST node identities. and source provenance through `@source/compilation`, with compilation-wide AST node identities.
file commands retain single-file behavior; directory commands compile the file commands retain single-file behavior; directory commands compile the
package's production files and load its dependencies in separate scopes. package's production files and load its dependencies in separate scopes.
@@ -20,8 +20,8 @@ needs records, pointers, generics and C calls.
- [x] retain source/package identity and source ownership across the compilation; keep one identifier pool and unambiguous declaration IDs. - [x] retain source/package identity and source ownership across the compilation; keep one identifier pool and unambiguous declaration IDs.
- [x] load directory packages once per canonical path, with deterministic file discovery, package scopes and root-only entry selection. follow Bro's existing relative and project-root `@` import rules; diagnose cycles rather than recurse indefinitely. - [x] load directory packages once per canonical path, with deterministic file discovery, package scopes and root-only entry selection. follow Bro's existing relative and project-root `@` import rules; diagnose cycles rather than recurse indefinitely.
- [ ] finish postfix namespace selection and resolve file-local import aliases without injecting dependency names into the importing package; bare and aliased import declarations are already parsed for dependency discovery. - [x] finish postfix namespace selection and resolve file-local import aliases without injecting dependency names into the importing package; bare and aliased import declarations are already parsed for dependency discovery.
- [ ] reuse ordinary scalar signature checking, function values and IR calls for selected declarations; namespaces have no runtime representation. - [x] reuse ordinary scalar signature checking, function values and IR calls for selected declarations; namespaces have no runtime representation.
- [ ] preserve originating filenames and spans in dependency diagnostics and native runtime traps; retain demand-driven diagnostic recovery. - [ ] preserve originating filenames and spans in dependency diagnostics and native runtime traps; retain demand-driven diagnostic recovery.
- [ ] verify sibling source files, transitive/shared imports, same-spelled declarations in different packages, alias conflicts, missing packages/members, argument errors and dependency-file diagnostics. preserve existing single-file commands. - [ ] verify sibling source files, transitive/shared imports, same-spelled declarations in different packages, alias conflicts, missing packages/members, argument errors and dependency-file diagnostics. preserve existing single-file commands.
+33 -2
View File
@@ -3,13 +3,13 @@
#! * binding ... -> :: | := #! * binding ... -> :: | :=
#! * literal .. -> int | float | string | 'true' | 'false' #! * literal .. -> int | float | string | 'true' | 'false'
#! * op ....... -> '+' | '-' | '*' | '/' | '==' | '!=' | '<' | '<=' | '>' | '>=' | 'and' | 'or' #! * op ....... -> '+' | '-' | '*' | '/' | '==' | '!=' | '<' | '<=' | '>' | '>=' | 'and' | 'or'
#! * expr ..... -> identifier | literal | type | '[' [expr {',' expr} [',']] ']' | '(' expr ')' | ('-' | '!') expr | expr op expr | expr '(' [expr {',' expr} [',']] ')' | expr '[' expr ']' | expr ('..' | '..=') expr #! * expr ..... -> identifier | literal | type | '[' [expr {',' expr} [',']] ']' | '(' expr ')' | ('-' | '!') expr | expr op expr | expr '(' [expr {',' expr} [',']] ')' | expr '[' expr ']' | expr '.' identifier | expr ('..' | '..=') expr
#! * decl ..... -> identifier [type] binding expr | ['test'] [identifier '::'] 'import' string #! * decl ..... -> identifier [type] binding expr | ['test'] [identifier '::'] 'import' string
#! * block .... -> '{' {stmt} '}' #! * block .... -> '{' {stmt} '}'
#! * arm ...... -> block | stmt (single statements become canonical blocks) #! * arm ...... -> block | stmt (single statements become canonical blocks)
#! * stmt ..... -> decl | expr | expr '=' expr | 'return' [expr] | 'break' | 'continue' | 'if' expr arm ['else' arm] | 'while' expr arm | 'for' expr '|' ['@'] identifier [',' identifier] '|' arm #! * stmt ..... -> decl | expr | expr '=' expr | 'return' [expr] | 'break' | 'continue' | 'if' expr arm ['else' arm] | 'while' expr arm | 'for' expr '|' ['@'] identifier [',' identifier] '|' arm
#! braceless guards/iterables require parentheses unless they are calls; else binds to the nearest if. #! braceless guards/iterables require parentheses unless they are calls; else binds to the nearest if.
#! precedence (tightest first): calls/indices, unary -/!, multiplication/division, addition/subtraction, relational comparisons, equality, and, or, ranges. compound range endpoints require their own parentheses. #! precedence (tightest first): calls/indices/selections, unary -/!, multiplication/division, addition/subtraction, relational comparisons, equality, and, or, ranges. compound range endpoints require their own parentheses.
#! binary operators associate left; unary operators associate right. #! binary operators associate left; unary operators associate right.
import "@std/debug" import "@std/debug"
@@ -44,6 +44,7 @@ ExprKind :: enum {
array_type array_type
array_literal array_literal
index index
selection
invalid invalid
} }
@@ -96,6 +97,7 @@ NodeKind :: enum {
expr_array_type expr_array_type
expr_array_literal expr_array_literal
expr_index expr_index
expr_selection
stmt_expr stmt_expr
stmt_decl stmt_decl
stmt_return stmt_return
@@ -215,6 +217,12 @@ IndexView :: struct {
index NodeId index NodeId
} }
SelectionView :: struct {
span Span
base NodeId
member NodeId
}
ReturnView :: struct { ReturnView :: struct {
span Span span Span
value ?NodeId value ?NodeId
@@ -450,6 +458,18 @@ add_index proc(builder @mut Builder, span Span, base, index NodeId) NodeId ! mem
}, span) }, span)
} }
add_selection proc(
builder @mut Builder,
span Span,
base, member NodeId,
) NodeId ! mem.AllocError {
return try add_node(builder, Node{
kind = .expr_selection,
data0 = NodeData{ node_id = base },
data1 = NodeData{ node_id = member },
}, span)
}
add_expression_stmt proc(builder @mut Builder, expression NodeId) NodeId ! mem.AllocError { add_expression_stmt proc(builder @mut Builder, expression NodeId) NodeId ! mem.AllocError {
return try add_node(builder, Node{ return try add_node(builder, Node{
kind = .stmt_expr, kind = .stmt_expr,
@@ -903,6 +923,7 @@ expression_kind proc(kind NodeKind) ExprKind {
.expr_array_type: .array_type .expr_array_type: .array_type
.expr_array_literal: .array_literal .expr_array_literal: .array_literal
.expr_index: .index .expr_index: .index
.expr_selection: .selection
.invalid_expr: .invalid .invalid_expr: .invalid
else: unreachable else: unreachable
} }
@@ -996,6 +1017,16 @@ index_expr proc(tree @Tree, id NodeId) IndexView {
} }
} }
selection_expr proc(tree @Tree, id NodeId) SelectionView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_selection)
return SelectionView{
span = get_span(&tree.storage, id),
base = node.data0.node_id,
member = node.data1.node_id,
}
}
node_span proc(tree @Tree, id NodeId) Span { node_span proc(tree @Tree, id NodeId) Span {
return get_span(&tree.storage, id) return get_span(&tree.storage, id)
} }
+42 -1
View File
@@ -348,8 +348,49 @@ compile proc(
if (compilation.module_root == null) { if (compilation.module_root == null) {
compilation.module_root = try ast.add_package(&compilation.builder, diagnostics.Span{ start = 0, end = 0 }, declarations.items) compilation.module_root = try ast.add_package(&compilation.builder, diagnostics.Span{ start = 0, end = 0 }, declarations.items)
} }
imports := arraylist.init(sema.Import, compilation.allocator)
defer arraylist.deinit(&imports)
tree :: ast.view(&compilation.builder, compilation.module_root?) tree :: ast.view(&compilation.builder, compilation.module_root?)
resolution := try sema.resolve_packages(compilation.allocator, &tree, roots.items, &compilation.strings, diagnostic_store) for (compilation.imports.items) |link| {
imported :: ast.import_decl(&tree, link.node)
name := strpool.no_str
failure ?diagnostics.DiagnosticId := null
for (diagnostic_store.list.items) |diagnostic, index| {
if (diagnostic.span.source == imported.path.source
and diagnostic.span.start == imported.path.start
and diagnostic.span.end == imported.path.end
and (diagnostic.code == .package_unavailable
or diagnostic.code == .invalid_import_path
or diagnostic.code == .import_cycle)) {
failure = diagnostics.DiagnosticId(index)
break
}
}
if (imported.namespace) |explicit| {
name = explicit
} else {
path :: if (link.target) |target|
compilation.packages.items[usize(target)].path
else
ast.source_text(&tree, imported.path)
start := path.len
while (start > 0 and path[start - 1] != '/') start -= 1
candidate :: path[start..]
valid := candidate.len > 0
for (candidate) |byte, index| {
if (!(byte >= 'a' and byte <= 'z') and !(byte >= 'A' and byte <= 'Z')
and byte != '_' and !(index > 0 and byte >= '0' and byte <= '9')) valid = false
}
if (valid) {
name = try strpool.intern(&compilation.strings, candidate)
} else if (failure == null) {
failure = try diagnostics.add(diagnostic_store, .invalid_import_alias, imported.path)
}
}
target ?ast.NodeId :: if (link.target) |package| roots.items[usize(package)] else null
try arraylist.append(&imports, sema.Import{ node = link.node, name = name, target = target, diagnostic = failure })
}
resolution := try sema.resolve_packages(compilation.allocator, &tree, roots.items, imports.items, &compilation.strings, diagnostic_store)
defer sema.deinit(&resolution) defer sema.deinit(&resolution)
checked := try typecheck.check(compilation.allocator, &tree, &resolution, diagnostic_store) checked := try typecheck.check(compilation.allocator, &tree, &resolution, diagnostic_store)
defer typecheck.deinit(&checked) defer typecheck.deinit(&checked)
+6
View File
@@ -66,6 +66,9 @@ Code :: enum {
import_cycle import_cycle
invalid_import_path invalid_import_path
package_unavailable package_unavailable
namespace_not_value
not_namespace
invalid_import_alias
} }
Diagnostic :: struct { Diagnostic :: struct {
@@ -153,5 +156,8 @@ messages :: $enummap.init(Code, []u8, {
import_cycle = "package import cycle", import_cycle = "package import cycle",
invalid_import_path = "invalid import path", invalid_import_path = "invalid import path",
package_unavailable = "cannot load package directory", package_unavailable = "cannot load package directory",
namespace_not_value = "package namespace cannot be used as a value",
not_namespace = "member selection requires a package namespace",
invalid_import_alias = "import requires an explicit valid identifier alias",
}) })
+8
View File
@@ -160,6 +160,14 @@ scan proc(
cursor += width cursor += width
continue continue
} }
if (char == '.') {
try add_token(state, Token{
span = make_span(source, cursor, cursor + 1),
kind = .dot,
})
cursor += 1
continue
}
if char == ':' { if char == ':' {
if cursor + 1 < program.len and program[cursor + 1] == ':' { if cursor + 1 < program.len and program[cursor + 1] == ':' {
+2 -2
View File
@@ -112,7 +112,7 @@ scans_ranges_without_consuming_float_dots test {
expected [19]TokenKind :: [ expected [19]TokenKind :: [
.int, .range, .int, .int, .range_inclusive, .int, .int, .range, .int, .int, .range_inclusive, .int,
.float, .range, .float, .pipe, .at, .ident, .comma, .ident, .pipe, .float, .range, .float, .pipe, .at, .ident, .comma, .ident, .pipe,
.range, .invalid, .invalid, .eof, .range, .dot, .invalid, .eof,
] ]
try testing.expect_equal(expected.len, state.tokens.items.len) try testing.expect_equal(expected.len, state.tokens.items.len)
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind) for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
@@ -120,7 +120,7 @@ scans_ranges_without_consuming_float_dots test {
try testing.expect_equal(u32(3), state.tokens.items[1].span.end) try testing.expect_equal(u32(3), state.tokens.items[1].span.end)
try testing.expect_equal(u32(7), state.tokens.items[4].span.start) try testing.expect_equal(u32(7), state.tokens.items[4].span.start)
try testing.expect_equal(u32(10), state.tokens.items[4].span.end) try testing.expect_equal(u32(10), state.tokens.items[4].span.end)
try testing.expect_equal(2, diagnostics.count(&diagnostic_store)) try testing.expect_equal(1, diagnostics.count(&diagnostic_store))
} }
scans_array_delimiters_and_mut_keyword test { scans_array_delimiters_and_mut_keyword test {
+1 -1
View File
@@ -59,7 +59,7 @@ TokenKind :: enum {
open_bracket, close_bracket open_bracket, close_bracket
open_curly, close_curly open_curly, close_curly
comma comma, dot
colon colon
newline newline
+1 -1
View File
@@ -760,7 +760,7 @@ lower_expression proc(state @mut State, id ast.NodeId) ir.ValueId ! (mem.AllocEr
if (trap_type(state, id)) return ir.no_value if (trap_type(state, id)) return ir.no_value
return try lower_string(state, id) return try lower_string(state, id)
} }
.identifier: { .identifier, .selection: {
if (trap_binding(state, id) or trap_type(state, id)) return ir.no_value if (trap_binding(state, id) or trap_type(state, id)) return ir.no_value
binding :: sema.binding(state.resolution, id) binding :: sema.binding(state.resolution, id)
location :: state.storage[usize(binding.data.declaration)] location :: state.storage[usize(binding.data.declaration)]
+16
View File
@@ -658,6 +658,22 @@ parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! Par
left_grouped = false left_grouped = false
continue continue
} }
if (token.kind == .dot) {
state.next_token += 1
member :: try consume(state, .ident)
member_id :: try ast.add_identifier(state.builder, member.span, member.data.str_id)
left = try ast.add_selection(
state.builder,
ast.Span{
start = state.tokens[expression_start].span.start,
end = member.span.end,
source = member.span.source,
},
left, member_id,
)
left_grouped = false
continue
}
if (token.kind == .range or token.kind == .range_inclusive) { if (token.kind == .range or token.kind == .range_inclusive) {
if (min_bp > BindingPowerScalar(0)) break if (min_bp > BindingPowerScalar(0)) break
left = try validate_range_bound(state, left, left_grouped) left = try validate_range_bound(state, left, left_grouped)
+26
View File
@@ -250,6 +250,32 @@ parses_call_statements_and_postfix_precedence test {
) )
} }
parses_namespace_selection_with_calls_indices_and_prefixes test {
strings := strpool.init(mem.c_allocator)
defer strpool.deinit(&strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source :: "main proc() int { return -library.factory()[0].member(1) + 2 }"
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &strings, &diagnostic_store)
tree := try parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
function :: ast.function_decl(&tree, ast.package_decl(&tree, 0))
returned :: ast.return_stmt(&tree, ast.block_statement(&tree, function.body, 0)).value?
binary :: ast.binary_expr(&tree, returned)
try testing.expect_equal(ast.BinaryOperator.add, binary.operator)
call :: ast.unary_expr(&tree, binary.left).operand
selected :: ast.selection_expr(&tree, ast.call_expr(&tree, call).callee)
try testing.expect_equal("member", ast.source_text(&tree, ast.node_span(&tree, selected.member)))
index :: ast.index_expr(&tree, selected.base)
factory :: ast.selection_expr(&tree, ast.call_expr(&tree, index.base).callee)
try testing.expect_equal("library", ast.source_text(&tree, ast.node_span(&tree, factory.base)))
try testing.expect_equal("factory", ast.source_text(&tree, ast.node_span(&tree, factory.member)))
try testing.expect_equal("library.factory()[0].member", ast.source_text(&tree, selected.span))
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
}
recovers_malformed_calls_and_preserves_siblings test { recovers_malformed_calls_and_preserves_siblings test {
strings := strpool.init(mem.c_allocator) strings := strpool.init(mem.c_allocator)
defer strpool.deinit(&strings) defer strpool.deinit(&strings)
+26 -10
View File
@@ -171,7 +171,7 @@ check proc(
for 0..count |i| { for 0..count |i| {
if (cells[i].role != .assignment_stmt) continue if (cells[i].role != .assignment_stmt) continue
target :: ast.assignment_stmt(tree, ast.NodeId(i)).target target :: ast.assignment_stmt(tree, ast.NodeId(i)).target
if (ast.expr_kind(tree, target) != .identifier) continue if (!declaration_reference(tree, target)) continue
binding :: sema.binding(resolution, target) binding :: sema.binding(resolution, target)
if (binding.kind == .invalid) { if (binding.kind == .invalid) {
cells[i].type = invalid_type(binding.data.diagnostic) cells[i].type = invalid_type(binding.data.diagnostic)
@@ -185,7 +185,7 @@ check proc(
# aliases share a type variable; explicit annotations and parameters are boundaries. # aliases share a type variable; explicit annotations and parameters are boundaries.
for (0..count) |i| { for (0..count) |i| {
id :: ast.NodeId(i) id :: ast.NodeId(i)
if (cells[i].role != .expression or ast.expr_kind(tree, id) != .identifier) continue if (cells[i].role != .expression or !declaration_reference(tree, id)) continue
binding :: sema.binding(resolution, id) binding :: sema.binding(resolution, id)
if (binding.kind != .declaration) continue if (binding.kind != .declaration) continue
target :: binding.data.declaration target :: binding.data.declaration
@@ -197,7 +197,7 @@ check proc(
if (cells[i].role == .declaration and !cells[i].fixed) { if (cells[i].role == .declaration and !cells[i].fixed) {
join(&state, id, ast.statement_decl(tree, id).initializer) join(&state, id, ast.statement_decl(tree, id).initializer)
} }
if (cells[i].role != .expression or ast.expr_kind(tree, id) != .identifier) continue if (cells[i].role != .expression or !declaration_reference(tree, id)) continue
binding :: sema.binding(resolution, id) binding :: sema.binding(resolution, id)
if (binding.kind == .declaration and !cells[i].assignment_value and !cells[usize(binding.data.declaration)].fixed) { if (binding.kind == .declaration and !cells[i].assignment_value and !cells[usize(binding.data.declaration)].fixed) {
join(&state, id, binding.data.declaration) join(&state, id, binding.data.declaration)
@@ -212,7 +212,10 @@ check proc(
if (assignment_values) { if (assignment_values) {
for (0..count) |i| { for (0..count) |i| {
id :: ast.NodeId(i) id :: ast.NodeId(i)
if (!cells[i].assignment_value or cells[i].role != .expression or ast.expr_kind(tree, id) != .identifier) continue if (!cells[i].assignment_value
or cells[i].role != .expression
or !declaration_reference(tree, id)
) continue
binding :: sema.binding(resolution, id) binding :: sema.binding(resolution, id)
if (binding.kind == .declaration) cells[i].type = types[usize(binding.data.declaration)] if (binding.kind == .declaration) cells[i].type = types[usize(binding.data.declaration)]
} }
@@ -677,7 +680,7 @@ collect_block proc(state @mut State, id, return_annotation ast.NodeId, in_loop b
state.has_assignments = true state.has_assignments = true
state.cells[usize(statement)].type = Type{ kind = .void } state.cells[usize(statement)].type = Type{ kind = .void }
binding :: sema.binding(state.resolution, assignment.target) binding :: sema.binding(state.resolution, assignment.target)
if (binding.kind == .builtin) { if (binding.kind == .builtin or binding.kind == .namespace) {
state.cells[usize(assignment.target)].role = .expression state.cells[usize(assignment.target)].role = .expression
state.cells[usize(assignment.target)].type = invalid_type(try diagnostics.add( state.cells[usize(assignment.target)].type = invalid_type(try diagnostics.add(
state.diagnostics, .invalid_assignment_target, ast.node_span(tree, assignment.target), state.diagnostics, .invalid_assignment_target, ast.node_span(tree, assignment.target),
@@ -760,11 +763,12 @@ collect_expression proc(state @mut State, id ast.NodeId) void ! (mem.AllocError
.literal_float: state.cells[usize(id)].type = Type{ kind = .float } .literal_float: state.cells[usize(id)].type = Type{ kind = .float }
.literal_bool: state.cells[usize(id)].type = Type{ kind = .bool } .literal_bool: state.cells[usize(id)].type = Type{ kind = .bool }
.literal_string: state.cells[usize(id)].type = Type{ kind = .string } .literal_string: state.cells[usize(id)].type = Type{ kind = .string }
.identifier: { .identifier, .selection: {
binding :: sema.binding(state.resolution, id) binding :: sema.binding(state.resolution, id)
match binding.kind { match binding.kind {
.invalid: state.cells[usize(id)].type = invalid_type(binding.data.diagnostic) .invalid: state.cells[usize(id)].type = invalid_type(binding.data.diagnostic)
.builtin: state.cells[usize(id)].type = invalid_type(try diagnostics.add(state.diagnostics, .type_not_value, ast.node_span(tree, id))) .builtin: state.cells[usize(id)].type = invalid_type(try diagnostics.add(state.diagnostics, .type_not_value, ast.node_span(tree, id)))
.namespace: state.cells[usize(id)].type = invalid_type(try diagnostics.add(state.diagnostics, .namespace_not_value, ast.node_span(tree, id)))
else: {} else: {}
} }
} }
@@ -839,6 +843,12 @@ collect_expression proc(state @mut State, id ast.NodeId) void ! (mem.AllocError
} }
} }
@hide:file
declaration_reference proc(tree @ast.Tree, id ast.NodeId) bool {
kind :: ast.expr_kind(tree, id)
return kind == .identifier or kind == .selection
}
@hide:file @hide:file
numeric_kind proc(kind TypeKind) bool { numeric_kind proc(kind TypeKind) bool {
return kind == .int or kind == .float or kind == .usize return kind == .int or kind == .float or kind == .usize
@@ -1200,7 +1210,7 @@ constant_integer proc(state @mut State, id ast.NodeId) ?ConstantInteger {
} }
return ConstantInteger{ magnitude = value } return ConstantInteger{ magnitude = value }
} }
.identifier: { .identifier, .selection: {
binding :: sema.binding(state.resolution, id) binding :: sema.binding(state.resolution, id)
if (binding.kind != .declaration) return null if (binding.kind != .declaration) return null
target :: binding.data.declaration target :: binding.data.declaration
@@ -1328,7 +1338,7 @@ infer_shape proc(state @mut State, id ast.NodeId, defaults bool) Type ! (mem.All
if (state.cells[index].type.kind == .invalid) return state.cells[index].type if (state.cells[index].type.kind == .invalid) return state.cells[index].type
state.cells[index].visiting = true state.cells[index].visiting = true
defer state.cells[index].visiting = false defer state.cells[index].visiting = false
if (state.cells[usize(id)].role == .expression and ast.expr_kind(state.tree, id) == .identifier) { if (state.cells[usize(id)].role == .expression and declaration_reference(state.tree, id)) {
binding :: sema.binding(state.resolution, id) binding :: sema.binding(state.resolution, id)
if (binding.kind == .declaration) { if (binding.kind == .declaration) {
target :: binding.data.declaration target :: binding.data.declaration
@@ -1486,7 +1496,10 @@ prepare_ranges proc(state @mut State, assignment_values, constrain bool) void !
} }
for 0..(state.cells.len) |i| { for 0..(state.cells.len) |i| {
id :: ast.NodeId(i) id :: ast.NodeId(i)
if (state.cells[i].role != .expression or state.cells[i].assignment_value or ast.expr_kind(tree, id) != .identifier) continue if (state.cells[i].role != .expression
or state.cells[i].assignment_value
or !declaration_reference(tree, id)
) continue
binding :: sema.binding(state.resolution, id) binding :: sema.binding(state.resolution, id)
if (binding.kind == .declaration and state.cells[usize(binding.data.declaration)].role == .loop_capture) { if (binding.kind == .declaration and state.cells[usize(binding.data.declaration)].role == .loop_capture) {
seed(state, id, group_type(state, binding.data.declaration)) seed(state, id, group_type(state, binding.data.declaration))
@@ -1544,7 +1557,10 @@ specialize_annotations proc(state @mut State) void ! (mem.AllocError | diagnosti
} }
for 0..(state.cells.len) |i| { for 0..(state.cells.len) |i| {
id :: ast.NodeId(i) id :: ast.NodeId(i)
if (state.cells[i].role != .expression or state.cells[i].assignment_value or ast.expr_kind(state.tree, id) != .identifier) continue if (state.cells[i].role != .expression
or state.cells[i].assignment_value
or !declaration_reference(state.tree, id)
) continue
binding :: sema.binding(state.resolution, id) binding :: sema.binding(state.resolution, id)
if (binding.kind == .declaration and state.cells[usize(binding.data.declaration)].fixed) { if (binding.kind == .declaration and state.cells[usize(binding.data.declaration)].fixed) {
seed(state, id, state.cells[usize(binding.data.declaration)].type) seed(state, id, state.cells[usize(binding.data.declaration)].type)
+123 -20
View File
@@ -42,10 +42,11 @@ intern_builtins proc(
return ids return ids
} }
BindingKind :: enum { none, declaration, builtin, invalid } BindingKind :: enum { none, declaration, namespace, builtin, invalid }
BindingData :: union { BindingData :: union {
declaration ast.NodeId declaration ast.NodeId
namespace ast.NodeId
builtin BuiltinType builtin BuiltinType
diagnostic diagnostics.DiagnosticId diagnostic diagnostics.DiagnosticId
} }
@@ -55,6 +56,14 @@ Binding :: struct {
data BindingData = BindingData{ diagnostic = diagnostics.no_diagnostic } data BindingData = BindingData{ diagnostic = diagnostics.no_diagnostic }
} }
#! links a file-local import to a package root; failed loads retain their diagnostic.
Import :: struct {
node ast.NodeId
name strpool.StringId
target ?ast.NodeId
diagnostic ?diagnostics.DiagnosticId = null
}
@hide:file @hide:file
Storage :: struct { Storage :: struct {
bindings []mut Binding bindings []mut Binding
@@ -78,6 +87,7 @@ State :: struct {
bindings []mut Binding bindings []mut Binding
builtin_ids []strpool.StringId builtin_ids []strpool.StringId
allocator mem.Allocator allocator mem.Allocator
namespaces hashmap.AutoHashMap(ast.NodeId, Scope)
} }
#! borrows `tree`, its identifier `pool`, and `diagnostic_store` during resolution. #! borrows `tree`, its identifier `pool`, and `diagnostic_store` during resolution.
@@ -88,9 +98,20 @@ resolve proc(
tree @ast.Tree, tree @ast.Tree,
pool @mut strpool.StringPool, pool @mut strpool.StringPool,
diagnostic_store @mut diagnostics.Store, diagnostic_store @mut diagnostics.Store,
) Resolution ! (mem.AllocError | strpool.InternError | diagnostics.CapacityError) { ) Resolution ! (
mem.AllocError
| strpool.InternError
| diagnostics.CapacityError
) {
roots [1]ast.NodeId :: [tree.root] roots [1]ast.NodeId :: [tree.root]
return try resolve_packages(allocator, tree, roots[..], pool, diagnostic_store) return try resolve_packages(
allocator,
tree,
roots[..],
&[],
pool,
diagnostic_store,
)
} }
#! resolves each package root in its own namespace; AST IDs share the compilation arena. #! resolves each package root in its own namespace; AST IDs share the compilation arena.
@@ -98,6 +119,7 @@ resolve_packages proc(
allocator mem.Allocator, allocator mem.Allocator,
tree @ast.Tree, tree @ast.Tree,
roots []ast.NodeId, roots []ast.NodeId,
imports []Import,
pool @mut strpool.StringPool, pool @mut strpool.StringPool,
diagnostic_store @mut diagnostics.Store, diagnostic_store @mut diagnostics.Store,
) Resolution ! (mem.AllocError | strpool.InternError | diagnostics.CapacityError) { ) Resolution ! (mem.AllocError | strpool.InternError | diagnostics.CapacityError) {
@@ -112,9 +134,45 @@ resolve_packages proc(
bindings = bindings, bindings = bindings,
builtin_ids = builtin_ids[..], builtin_ids = builtin_ids[..],
allocator = allocator, allocator = allocator,
namespaces = hashmap.init(allocator),
}
defer {
for (state.namespaces.entries) |entry| {
if (entry.hash == 0) continue
scope := entry.value
hashmap.deinit(&scope.names)
}
hashmap.deinit(&state.namespaces)
} }
for (roots) |root| try resolve_package(&state, root) package_scopes :: try mem.alloc(Scope, allocator, roots.len)
defer mem.free(allocator, package_scopes)
file_scopes :: try mem.alloc(Scope, allocator, ast.source_files(tree).len)
defer mem.free(allocator, file_scopes)
for (file_scopes) |@scope| scope^ = Scope{ names = hashmap.init(allocator), parent = null }
defer { for (file_scopes) |@scope| hashmap.deinit(&scope.names) }
for (roots) |root, index| {
package_scopes[index] = Scope{ names = hashmap.init(allocator), parent = null }
scope := &package_scopes[index]
errdefer hashmap.deinit(&scope.names)
for (0..ast.package_decl_count_at(tree, root)) |i| {
id :: ast.package_decl_at(tree, root, i)
file_scopes[usize(ast.node_span(tree, id).source)].parent = scope
match ast.decl_kind(tree, id) {
.statement: try predeclare(&state, scope, id, ast.statement_decl(tree, id).name)
.function: try predeclare(&state, scope, id, ast.function_decl(tree, id).name)
.import, .invalid: {}
}
}
hashmap.put(&state.namespaces, root, scope^) catch |err| {
match err {
.key_exists: unreachable
else: return err
}
}
}
for (imports) |imported| try resolve_import(&state, &file_scopes[usize(ast.node_span(tree, imported.node).source)], imported)
for (roots) |root| try resolve_package(&state, root, file_scopes)
return Resolution{ storage = Storage{ return Resolution{ storage = Storage{
bindings = bindings, bindings = bindings,
@@ -123,32 +181,53 @@ resolve_packages proc(
} }
@hide:file @hide:file
resolve_package proc(state @mut State, root ast.NodeId) void ! (mem.AllocError | diagnostics.CapacityError) { resolve_package proc(
state @mut State, root ast.NodeId, file_scopes []Scope,
) void ! (mem.AllocError | diagnostics.CapacityError) {
tree := state.tree^ tree := state.tree^
tree.root = root tree.root = root
package_scope := Scope{
names = hashmap.init(state.allocator),
parent = null,
}
defer hashmap.deinit(&package_scope.names)
for (0..ast.package_decl_count(&tree)) |i| { for (0..ast.package_decl_count(&tree)) |i| {
id :: ast.package_decl(&tree, i) id :: ast.package_decl(&tree, i)
scope :: &file_scopes[usize(ast.node_span(&tree, id).source)]
match ast.decl_kind(&tree, id) { match ast.decl_kind(&tree, id) {
.statement: try predeclare(state, &package_scope, id, ast.statement_decl(&tree, id).name) .statement: try resolve_statement_decl(state, scope, id)
.function: try predeclare(state, &package_scope, id, ast.function_decl(&tree, id).name) .function: try resolve_function(state, scope, id)
.import, .invalid: {}
}
}
for (0..ast.package_decl_count(&tree)) |i| {
id :: ast.package_decl(&tree, i)
match ast.decl_kind(&tree, id) {
.statement: try resolve_statement_decl(state, &package_scope, id)
.function: try resolve_function(state, &package_scope, id)
.invalid: resolve_invalid(state, id) .invalid: resolve_invalid(state, id)
.import: {} .import: {}
} }
} }
}
@hide:file
resolve_import proc(
state @mut State, scope @mut Scope, imported Import,
) void ! (mem.AllocError | diagnostics.CapacityError) {
span :: ast.import_decl(state.tree, imported.node).span
value := if (imported.diagnostic) |diagnostic|
Binding{ kind = .invalid, data = BindingData{ diagnostic = diagnostic } }
else if (imported.target) |target|
Binding{ kind = .namespace, data = BindingData{ namespace = target } }
else
Binding{ kind = .invalid, data = BindingData{
diagnostic = try diagnostics.add(state.diagnostics, .package_unavailable, span),
} }
if (imported.name == strpool.no_str) {
state.bindings[usize(imported.node)] = value
return
}
if hashmap.get(&scope.names, imported.name) |_| {
state.bindings[usize(imported.node)] = Binding{ kind = .invalid, data = BindingData{
diagnostic = try diagnostics.add(state.diagnostics, .duplicate_name, span),
} }
return
}
if lookup(state, scope.parent, imported.name) |_| {
value = Binding{ kind = .invalid, data = BindingData{
diagnostic = try diagnostics.add(state.diagnostics, .duplicate_name, span),
} }
}
try put_name(scope, imported.name, value)
state.bindings[usize(imported.node)] = value
} }
#! releases binding storage with the allocator retained by resolve. #! releases binding storage with the allocator retained by resolve.
@@ -349,6 +428,30 @@ resolve_expr proc(
} }
} }
} }
.selection: {
expression :: ast.selection_expr(tree, id)
try resolve_expr(state, scope, expression.base)
base :: state.bindings[usize(expression.base)]
value := base
if (base.kind != .invalid) {
if (base.kind == .namespace) {
namespace :: hashmap.get(&state.namespaces, base.data.namespace)?
if hashmap.get(&namespace.names, ast.name_id(tree, expression.member)) |found| {
value = found
} else {
value = Binding{ kind = .invalid, data = BindingData{
diagnostic = try diagnostics.add(state.diagnostics, .unknown_name, ast.node_span(tree, expression.member)),
} }
}
} else {
value = Binding{ kind = .invalid, data = BindingData{
diagnostic = try diagnostics.add(state.diagnostics, .not_namespace, ast.node_span(tree, expression.base)),
} }
}
}
state.bindings[usize(id)] = value
state.bindings[usize(expression.member)] = value
}
.unary: try resolve_expr(state, scope, ast.unary_expr(tree, id).operand) .unary: try resolve_expr(state, scope, ast.unary_expr(tree, id).operand)
.binary: { .binary: {
expression :: ast.binary_expr(tree, id) expression :: ast.binary_expr(tree, id)
+94
View File
@@ -336,6 +336,99 @@ def check_packages(temporary, executable):
return 15 return 15
def check_namespace_packages(temporary, executable):
project = temporary / "namespaces"
def put(path, text):
target = project / path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(text)
return target
put("build.hon", "# project root\n")
main = put("app/main.hon", """
import "../left"
right :: import "@right"
import "../shared-link"
main proc() int {
functions :: [left.helper, right.helper]
selected := left.helper
selected = right.helper
return sibling(18) + functions[0]() + selected() + shared.helper()
}
""")
sibling = put("app/sibling.hon", """
right :: import "../left"
sibling proc(n int) int { return n * 2 + right.helper() - 1 }
""")
put("left/main.hon", """
import "../shared"
helper proc() int { return shared.helper() - 2 }
main proc() int { return 99 }
""")
put("right/main.hon", """
import "../shared-link"
helper proc() int { return shared.helper() - 1 }
""")
dependency = put("shared/main.hon", """
value :: 3
helper proc() int { return value }
identity proc(n f64) f64 { return n }
fail proc() int { return 9223372036854775807 + 1 }
""")
(project / "shared-link").symlink_to(project / "shared", target_is_directory=True)
app = main.parent
check(app, executable, (b"42\n", None))
emitted = run(str(HONEY), "emit-llvm", str(app))
assert emitted.returncode == 0 and emitted.stderr == b"", outcome(emitted)
sibling.write_text("sibling proc(n int) int { return n * 2 }\n")
cases = [
('import "../left"\nmain proc() int { return left.helper() }', b"1\n", None),
('lib :: import "../shared"\nmain proc() f64 { f :: lib.identity\nreturn f(1.5) }', b"1.5\n", None),
('lib :: import "../shared"\nmain proc() int { return lib.value }', b"3\n", None),
('lib :: import "../shared"\nmain proc() int { return lib.absent() }', b"", b"unknown name"),
('lib :: import "../shared"\nmain proc() int { return helper() }', b"", b"unknown name"),
('import "../left"\nmain proc() int { return left.shared.helper() }', b"", b"unknown name"),
('import "../shared"\nmain proc() int { return shared.helper(1) }', b"", b"argument count"),
('import "../shared"\nmain proc() f64 { return shared.identity(true) }', b"", b"incompatible value type"),
('import "../shared"\nmain proc() int { shared\nreturn 42 }', b"", b"namespace cannot be used as a value"),
('import "../shared"\nmain proc() int { return shared.value.member }', b"", b"requires a package namespace"),
('lib :: import "../absent"\nmain proc() int { return lib.helper() }', b"", b"cannot load package directory"),
('lib :: import "../left"\nlib :: import "../right"\nmain proc() int { return lib.helper() }', b"1\n", b"duplicate name"),
('lib :: import "../shared"\nlib :: 42\nmain proc() int { return lib.helper() }', b"", b"duplicate name"),
('lib :: import "../shared"\nmain proc() int { lib :: 42\nreturn lib }', b"", b"duplicate name"),
('int :: import "../shared"\nmain proc() int { return 42 }', b"", b"duplicate name"),
('import "../shared"\nmain proc() int { shared.value = 42\nreturn 0 }', b"", b"mutable local binding"),
('import "../shared"\nmain proc() int { shared = 42\nreturn 0 }', b"", b"mutable local binding"),
('import "../shared"\nunused proc() int { return shared.absent() }\nmain proc() int { return 42 }', b"42\n", b"unknown name"),
('import "../shared"\nmain proc() int { return shared.fail() }', b"", b"integer arithmetic overflow"),
('import "../shared"\nmain proc() int { return shared. }', b"", b"unexpected token"),
]
for text, stdout, error in cases:
main.write_text(text + "\n")
check(app, executable, (stdout, error))
# Imports in a sibling file are invisible even though its declarations are shared.
main.write_text("main proc() int { return local.helper() }\n")
sibling.write_text('local :: import "../shared"\nsibling proc(n int) int { return n }\n')
check(app, executable, (b"", b"unknown name"))
# A missing package selection reuses the loader diagnostic instead of cascading.
main.write_text('lib :: import "../absent"\nmain proc() int { return lib.helper() }\n')
diagnosed = run(str(HONEY), "run", str(app))
assert diagnosed.stderr.count(b"cannot load package directory") == 1, outcome(diagnosed)
assert b"unknown name" not in diagnosed.stderr, outcome(diagnosed)
# Dependency diagnostics and reached traps preserve their original source file.
main.write_text('import "../shared"\nmain proc() int { return shared.helper() }\n')
dependency.write_text("helper proc() int { return missing }\n")
check(app, executable, (b"", b"unknown name"))
diagnosed = run(str(HONEY), "run", str(app))
assert str(dependency.resolve()).encode() + b":1:" in diagnosed.stderr, outcome(diagnosed)
dependency.write_text("\nhelper proc() int { return 9223372036854775807 + 1 }\n")
check(app, executable, (b"", b"integer arithmetic overflow"))
trapped = run(str(HONEY), "run", str(app))
assert str(dependency.resolve()).encode() + b":2:" in trapped.stderr, outcome(trapped)
return len(cases) + 4
def main(): def main():
with tempfile.TemporaryDirectory(prefix="honey native ' ; $") as directory: with tempfile.TemporaryDirectory(prefix="honey native ' ; $") as directory:
temporary = Path(directory) temporary = Path(directory)
@@ -350,6 +443,7 @@ def main():
check(source, executable, (stdout, error)) check(source, executable, (stdout, error))
count += 1 count += 1
count += check_packages(temporary, executable) count += check_packages(temporary, executable)
count += check_namespace_packages(temporary, executable)
source = ROOT / "programs/factorial.hon" source = ROOT / "programs/factorial.hon"
emitted = run(str(HONEY), "emit-llvm", str(source)) emitted = run(str(HONEY), "emit-llvm", str(source))
assert emitted.returncode == 0 and emitted.stderr == b"", outcome(emitted) assert emitted.returncode == 0 and emitted.stderr == b"", outcome(emitted)