Compare commits

...

21 Commits

Author SHA1 Message Date
hl-valdemar 0593786553 feat: parse floating-point text through the standard library 2026-10-24 04:08:58 +02:00
hl-valdemar 5fd8d9eb89 fix: reject inexact integer literals in float context 2026-10-24 03:55:53 +02:00
hl-valdemar 9c1bd2a20e fix: use composed enum keys for driver diagnostics (EnumMap) 2026-10-24 03:54:23 +02:00
hl-valdemar d2cec370fa perf: separate checker union-find links from node metadata 2026-10-24 03:48:27 +02:00
hl-valdemar c6a56b7e79 feat: execute fixed arrays and indexed access 2026-10-24 03:39:25 +02:00
hl-valdemar cf39481bfc feat: execute Bro-compatible integer range loops 2026-10-24 03:27:52 +02:00
hl-valdemar d51ca57c9e feat: execute break and continue in while loops 2026-10-24 03:22:05 +02:00
hl-valdemar 0add5ab187 fix: allow multiline parenthesized expressions in every context 2026-10-24 03:07:51 +02:00
hl-valdemar 5f845b9520 feat: execute short-circuit boolean expressions 2026-10-24 03:01:47 +02:00
hl-valdemar 7551ce5f6d refactor: query diagnostic message table directly 2026-10-24 02:50:00 +02:00
hl-valdemar e9478dcf3f refactor: store diagnostic messages in a static enum map 2026-10-24 02:40:15 +02:00
hl-valdemar b35a68dff8 feat: execute local assignments and while loops 2026-10-24 02:34:58 +02:00
hl-valdemar 021cbecac8 refactor: name recovery boundaries and compact IR strings 2026-10-24 02:15:50 +02:00
hl-valdemar 9932750e21 feat: execute conditionals through typed IR
Use typed singleton arrays for braceless arm blocks: &[statement] compiled but trapped under the Bro bootstrap.
2026-10-24 01:47:13 +02:00
hl-valdemar 0821b84bc3 refactor: index preprocessed process arguments directly 2026-10-23 12:11:42 +02:00
hl-valdemar c1e03632ed refactor: obtain arguments through process initialization 2026-10-23 11:39:27 +02:00
hl-valdemar 877ee21579 feat: restore sentinel-slice file path interfaces 2026-10-23 11:22:41 +02:00
hl-valdemar 31c2cad9f9 feat: add bounded read-to-end and compatible file paths 2026-10-23 11:02:11 +02:00
hl-valdemar 98f3a04daf feat: lower typed programs and interpret IR 2026-10-22 22:20:50 +02:00
hl-valdemar bc318a29ad fix: infer float context from division 2026-10-22 11:53:04 +02:00
hl-valdemar bbf39c16c5 feat: type-check expressions and function calls 2026-10-09 22:35:25 +02:00
38 changed files with 13063 additions and 140 deletions
+74 -5
View File
@@ -1,6 +1,6 @@
# Honey
Honey is currently bootstrapped with the Bro compiler. the expected development workspace places the Bro project at [`../brolang`](../brolang); consult that project for compiler behavior and documentation.
Bro and Honey implement the same language; Bro is the prototype compiler used to bootstrap the self-hosted Honey compiler. the features below describe Honey's currently implemented subset. the expected development workspace places Bro at [`../brolang`](../brolang); consult that project for the language grammar and documentation.
style: see the [language guidelines](docs/style.md).
@@ -10,10 +10,79 @@ build Honey from this directory:
bro build
```
the parser supports arithmetic expressions with nested parenthesized groups, such as `a + (b - c) * 2 / 3`. groups control precedence without adding AST nodes; inner expression spans exclude the grouping parentheses.
run a source file with the interpreter:
function calls are postfix expressions: `f()`, `f(a, b)`, and `factory()(x)`. calls bind tighter than unary and binary arithmetic and can appear in initializers or as expression statements inside function bodies. argument lists allow a trailing comma; newlines remain statement boundaries. argument/type checking belongs to semantic analysis and is not yet implemented.
```sh
./build/honey run example.hon
```
name resolution (`@source/sema`) runs after parsing. it predeclares package names and function-body locals, allowing forward references and mutually recursive functions. parameters and locals share a function scope; signatures resolve names in the enclosing package scope. shadowing is forbidden, including reuse of builtin type names (`int`, `float`, `string`, `void`); sibling function scopes may independently reuse names.
```hon
add proc(left, right int) int {
return left + right
}
`sema.resolve` returns an owned binding table indexed by AST `NodeId`, without modifying the AST. `sema.binding` identifies a declaration, builtin type, or invalid binding carrying its diagnostic ID; nodes with no binding return `.none`. release the result with `sema.deinit`. unknown names and duplicate declarations add source-spanned diagnostics; invalid syntax retains its existing diagnostic identity. same-scope duplicates retain the first declaration for recovery, while forbidden inner declarations invalidate that name's uses rather than silently binding them to an outer declaration. type checking and read-before-initialization checks remain separate, unimplemented stages.
main proc() int {
answer :: add(19, 23)
return answer
}
```
this prints `42`. interpreted `main` must be a top-level function with no parameters; nonvoid results are printed, not used as process exit statuses.
the native bootstrap accepts either a no-argument `main` or `main proc(init process.Init)`. it supplies `init.io` and `init.args` from native startup `argc`/`argv`. startup scans each argument once into a heap-allocated array of borrowed zero-terminated slices (`process.Args.values: [][;0]u8`), without copying strings. the descriptor array and borrowed strings remain valid until `main` returns; startup then frees the descriptors, including on a fallible return. allocation failure exits with status `2`. `init.args.values.len` includes the executable name, and `init.args.values[index]` directly accesses an argument after a length guard. callers can supply their own `process.Args { values = slices[..] }` for testing.
source loading uses the supplied `io.Io` for opening, reading, and closing files. `io.open` accepts a borrowed zero-terminated slice (`[;0]u8`), including `init.args.values[index]` without copying its storage. the bootstrap supports checked sentinel-slice construction with `pointer[..length;0]`. `io.read_all(allocator, reader, maximum)` reads to EOF into a caller-owned `ArrayList(u8)`; release it with `arraylist.deinit`. it frees partial storage on error and consumes one extra byte to distinguish an exact-size input from a limit violation. the driver maps `limit_exceeded` to `source_too_large` at its `u32` source-size limit.
a recursive example is in [programs/factorial.hon](programs/factorial.hon); `./build/honey run programs/factorial.hon` prints `120`.
an iterative example is in [programs/iterative_factorial.hon](programs/iterative_factorial.hon); `./build/honey run programs/iterative_factorial.hon` also prints `120`.
the parser supports arithmetic expressions with nested parenthesized groups, such as `a + (b - c) * 2 / 3`. groups control precedence without adding AST nodes; inner expression spans exclude the grouping parentheses. groups may span lines in any expression position, including returns, initializers, arguments and guards. parsing accepts integer division syntax, but type checking requires float operands for `/`.
function calls are postfix expressions: `f()`, `f(a, b)`, and `factory()(x)`. calls bind tighter than unary and binary arithmetic and can appear in initializers or as expression statements inside function bodies. argument lists allow newlines and a trailing comma; outside parentheses, newlines remain statement boundaries. type checking validates callable values, argument counts and argument types.
`true` and `false` are boolean literals; `bool` is available for parameters, storage and results. `==`, `!=`, `<`, `<=`, `>` and `>=` compare compatible numeric operands and produce `bool`; `==` and `!=` also accept boolean operands. comparisons preserve contextual numeric inference without making their operands boolean. arithmetic binds tighter than relational comparisons, which bind tighter than equality; comparisons associate left to right. boolean arithmetic/ordering and string/function/void comparisons are invalid; there is no implicit numeric-to-boolean conversion.
`!`, `and` and `or` require boolean operands and produce `bool`. prefix `!` and `-` bind tighter than arithmetic; equality binds tighter than `and`, which binds tighter than `or`. binary operators associate left to right, while prefixes nest right to left. operands evaluate left to right: `false and rhs` and `true or rhs` skip the right operand, including calls, lazy global initialization and runtime traps. both operands are still parsed, resolved and type-checked; errors in a skipped operand remain source diagnostics without forcing a runtime trap. this applies in initializers, assignments, returns, call arguments and conditions. [programs/boolean_conditions.hon](programs/boolean_conditions.hon) demonstrates combined predicates; `./build/honey run programs/boolean_conditions.hon` prints `true`.
`if`/`else` supports braced arms, single statements and `else if` chains. conditions must have type `bool`. a braceless guard requires parentheses unless it is a function call: `if (n <= 1) return 1` or `if ready() return`. `else` binds to the nearest `if`; after a braced arm it may start on the next line. branches are represented as AST blocks even when braceless.
`name = expression` updates a local `:=` binding. `::` bindings, parameters, named functions, builtin types and globals are not assignment targets. the established binding type is preserved: assignment inputs do not widen or infer that type, though numeric literals can adopt its float context. function-valued locals accept replacement functions with the same parameter and result types.
fixed arrays use `[N]T` for readonly elements and `[N]mut T` for mutable elements; literals use `[a, b]`, including empty, nested and multiline forms with trailing commas. `values [4]mut i64 := [1, 2, 3, 4]` permits `values[2] = 10`. writable elements require a mutable local binding and `mut` at every traversed array level; parameters and globals remain readonly. whole-array assignment only requires a mutable binding, but source and destination types must match recursively, including counts and element mutability. initializations, assignments, arguments and returns copy arrays by value, including nested rows.
array lengths currently support nonnegative integer literals, unary signs, `+`, `-`, `*` and package-level immutable integer constants; local constants and other compile-time expressions are not implemented. `[_]` infers a binding's array length from its initializer. element annotations require concrete supported value types: use `i64` or `f64` rather than the `int` or `float` constraints. an unannotated array literal has readonly elements.
indices require `usize`; integer literals and flexible inferred values receive this context, while fixed signed integers do not convert implicitly. out-of-bounds access traps before loading or storing, including for empty arrays. reads evaluate the base before the index; indexed assignment evaluates its right-hand side before the destination, each once. leading nested assignment targets follow Bro's declaration ambiguity: write `(matrix[1])[0] = 9`, not `matrix[1][0] = 9`. [programs/array_sum.hon](programs/array_sum.hon) uses a `usize` range bound to index, mutate and sum an array; `./build/honey run programs/array_sum.hon` prints `17`.
`while` repeats a block or single statement while its `bool` condition holds, re-evaluating the condition on every iteration. guards follow the same braced/braceless convention as `if`. loop bodies introduce child scopes; their locals, including locals in nested conditional arms, begin uninitialized on each scope entry.
`for 0..10 |i| { ... }` iterates an ascending, upper-exclusive integer range; `..=` includes the upper bound. a capture is required: `|i|` binds an immutable value with the bounds' integer family (`int` or `usize`) in the body scope, while `|_|` discards it. bounds resolve in the enclosing scope and are evaluated left-to-right once, so changing a bound variable in the body does not change the range. reversed ranges are empty; equal bounds produce zero iterations with `..` and one with `..=`. inclusive ranges terminate safely at the maximum integer.
range bounds that are compound or unary expressions require their own parentheses, such as `(-2)..(limit + 1)`. a braceless body also requires parentheses around the entire iterable: `for (0..10) |i| total = total + i`. current execution supports direct integer ranges only; first-class range values, collection iteration, pointer captures and index captures remain unimplemented. [programs/range_loop.hon](programs/range_loop.hon) skips `5` while summing `0..10`; `./build/honey run programs/range_loop.hon` prints `40`.
`break` exits the nearest enclosing `for` or `while`; `continue` restarts a `while` condition or advances a `for` range through its guarded update. both are operand-free statements permitted within loop bodies and nested conditional arms; labels and values are unsupported. controls outside loops produce keyword-spanned diagnostics, including in untaken or unreachable code, and trap only if reached. code after a transfer is still resolved and type-checked but does not execute on that path. body locals are reset on re-entry after `continue`; loop transfers do not guarantee a function return. both controls lower to existing IR jumps, without new interpreter instructions or loop frames. [programs/loop_control.hon](programs/loop_control.hon) skips an iteration and exits early; `./build/honey run programs/loop_control.hon` prints `12`.
name resolution (`@source/sema`) runs after parsing. it predeclares package names and each block's direct locals, allowing forward references and mutually recursive functions. parameters and top-level function-body locals share a scope; each conditional arm and loop body introduces a child scope whose names do not escape it. signatures resolve names in the enclosing package scope. shadowing is forbidden, including reuse of builtin type names (`int`, `float`, `i64`, `f64`, `usize`, `bool`, `string`, `void`); sibling scopes may independently reuse names.
`sema.resolve` returns an owned binding table indexed by AST `NodeId`, without modifying the AST. `sema.binding` identifies a declaration, builtin type, or invalid binding carrying its diagnostic ID; nodes with no binding return `.none`. release the result with `sema.deinit`. unknown names and duplicate declarations add source-spanned diagnostics; invalid syntax retains its existing diagnostic identity. same-scope duplicates retain the first declaration for recovery, while forbidden inner declarations invalidate that name's uses rather than silently binding them to an outer declaration.
type checking (`@source/sema/check`) runs after name resolution. `check.check` borrows the AST and binding table and returns owned per-node types and interned array descriptors; query with `check.type_of` and `check.array_type`, and release with `check.deinit`. declarations infer their initializer types, including forward references and function aliases; function types identify the declaration supplying the callable signature. annotations support builtin types and fixed arrays; type names are not ordinary values. `void` is valid as a function result, not as a parameter or stored value.
the checker keeps union-find parent links and group sizes in a compact array separate from per-node inference metadata. root walks and path compression access only that link storage; both temporary arrays are released before `check.check` returns.
numeric typing follows contextual rules within Honey's current `int` and `float` families. integer literals can adopt a supported numeric family only when their value is exactly representable; float literals remain float and opt into ordinary IEEE rounding. numeric demands propagate through arithmetic and inferred bindings before undemanded integers default to `int`. conflicting demands are errors; fixed `int`/`i64`, `usize`, and `float`/`f64` values do not implicitly convert between families. `+`, `-`, `*` and unary `-` require compatible numeric operands; `/` requires and supplies float context. `3 / 2`, `3.0 / 2`, and `ratio float :: 3 / 2` are valid. `value f64 :: 9007199254740993` is diagnosed, including when float context arrives through aliases, division, or array elements; `9007199254740993.0` explicitly permits rounding. exactly representable integers beyond the signed/unsigned integer ranges remain valid in float context. explicit numeric casts are not yet implemented.
the conversion policy is value-preserving by default: any future implicit widening must preserve every value of the source type, not merely the value observed at one use. narrowing, potentially inexact integer-to-float conversion, float-to-integer conversion, and signedness changes that cannot preserve the full source range require an explicit operation. integer arithmetic remains checked rather than wrapping; floating arithmetic retains IEEE rounding.
`@std/strconv.parse_float(allocator, text)` parses a complete borrowed `[]u8` decimal into `f64`. it accepts an optional sign, decimal point, and exponent; whitespace, embedded NUL bytes, trailing input, hexadecimal, infinities, and NaNs return `invalid_number`. finite subnormals and signed zero are preserved; overflow and nonzero values rounded to zero return `out_of_range`. allocation failures propagate as `out_of_memory`, and temporary storage is always released. the Darwin backend uses libc's C-locale conversion without changing the process locale; rounding follows the caller's floating-point rounding mode. numeric lowering uses this interface, keeping exact integer-to-float validation compiler-specific.
type errors carry source spans and diagnostic identities; invalid operands propagate their existing identity without duplicate errors, while independent expressions, arguments and nested bodies are still checked. inference cycles without a type anchor are diagnosed. function signatures are checked independently of their bodies. bare `return` and same-line `return expression` are supported inside function bodies and must match the declared result type. nonvoid return-path checking is structural: every path must return or terminate in a diagnostic trap, and both conditional successors are considered without constant folding. a missing `else` or a loop permits fallthrough, even for a literal `true` condition; a later return can close that path. statements after a return and untaken branches or loops are still type-checked but are not executed.
`lower.lower` turns the checked AST and binding table into an owned `@source/ir` module. functions, globals, local slots and instruction-result temporaries have explicit IDs; typed instructions execute in source evaluation order and basic blocks end in jump, conditional-branch, return or diagnostic-trap terminators. conditional arms join only when execution can continue; loops use condition/body/exit blocks and backedges. short-circuit expressions use branches and synthetic boolean local slots to merge results; call arguments are packed in the block reached after evaluating them. local slots and temporaries are function-wide, without SSA or phi nodes. literals are parsed and strings decoded during lowering; the module retains no AST, binding-table or source-buffer references. release it with `ir.deinit`; keep and reuse the diagnostic store used during lowering so diagnostic IDs remain valid. the IR is target-independent and is intended to feed later native backends as well as the current interpreter.
array IR retains recursive element types, counts and mutability. typed aggregate construction, element addresses and load/store instructions use flattened storage reserved per invocation; nested copies and indexed accesses reuse that storage without per-iteration allocation. addresses are internal IR values, not source-language pointers.
`interpreter.run` consumes only IR and returns an `Outcome` containing a value or diagnostic trap. returned arrays own their escaped cell storage; release every outcome with `interpreter.deinit` before releasing its module. string values carry a module-local `ir.StringId`, not a slice; resolve bytes with `ir.string_bytes(&module, id)` before releasing the module. decoded bytes and their full backing allocations remain module-owned, without copying strings during calls or returns. array descriptors also borrow the module. heap-backed invocation frames track their current block and resume it after calls or global initialization; loop iterations reuse the invocation's storage without allocating new frames. parameters are initialized on entry and local declarations execute in statement order. premature local reads and assignments trap; assignment evaluates its right-hand side before checking the destination's initialization. globals initialize lazily on their first read, cache their result for that run, and trap on initialization cycles. `int`/`i64` use signed 64-bit storage, `usize` uses unsigned 64-bit storage, both with checked arithmetic; `float`/`f64` use 64-bit IEEE arithmetic and comparisons, including NaN behavior. `usize` supports `+`, `-`, `*` and comparisons, but not unary negation or division. numeric literal overflow and float underflow to zero are diagnosed; representable subnormals are accepted. strings support `\\`, `\"`, `\n`, `\r`, `\t`, `\0` and `\xNN` escapes.
recoverable source errors do not stop execution of unaffected code. unused invalid functions, globals and nested bodies retain diagnostic traps; reaching invalid code reports its original diagnostic identity. missing-return traps affect only paths that fall through, not paths that already return or trap. diagnostics include the filename, line and byte column. exit status is `0` for clean execution, `1` for source diagnostics or a runtime trap, and `2` for usage or infrastructure failure. native code generation and compound assignment remain unimplemented.
View File
+2 -2
View File
@@ -1,3 +1,3 @@
# compile through recoverable source errors
# execute through recoverable source errors
Honey produces an executable whenever source errors can be represented safely: structurally valid syntax trees may contain typed invalid nodes carrying source spans and diagnostic identities, and semantic checking or lowering turns demanded invalid code into diagnostic runtime traps. clean compilation returns status `0`, compilation that emits an executable with source diagnostics returns `1`, and unrecoverable infrastructure or backend failure returns `2`. this deliberately favors testing unaffected program paths over conventional fail-fast compilation; parser recovery must preserve valid surrounding syntax, while compact syntax-tree storage remains private behind typed views.
Honey lowers recoverable source errors into typed, target-independent IR with source spans and diagnostic-bearing trap terminators; the interpreter executes that IR first, and future native backends reuse the same lowering rather than duplicating frontend semantics. invalid code traps only when execution reaches it, allowing unaffected program paths to run; parser recovery preserves surrounding syntax and compact syntax-tree storage remains private behind typed views. clean execution returns status `0`, source diagnostics or runtime traps return `1` even when an unaffected entry point finishes successfully, and usage or unrecoverable infrastructure failure returns `2`.
+12
View File
@@ -0,0 +1,12 @@
main proc() int {
values [4]mut i64 := [1, 2, 3, 4]
values[2] = 10
total := 0
count usize :: 4
for 0..count |index| {
total = total + values[index]
}
return total
}
+11
View File
@@ -0,0 +1,11 @@
in_range proc(value, low, high int) bool {
return value >= low and value < high
}
main proc() bool {
return (
in_range(5, 0, 10)
and !in_range(12, 0, 10)
or in_range(20, 0, 10)
)
}
+8
View File
@@ -0,0 +1,8 @@
factorial proc(n int) int {
if (n <= 1) return 1
return n * factorial(n - 1)
}
main proc() int {
return factorial(5)
}
+11
View File
@@ -0,0 +1,11 @@
main proc() int {
product := 1
remaining := 5
while remaining > 1 {
product = product * remaining
remaining = remaining - 1
}
return product
}
+13
View File
@@ -0,0 +1,13 @@
main proc() int {
number := 0
total := 0
while true {
number = number + 1
if (number == 3) continue
if (number > 5) break
total = total + number
}
return total
}
+10
View File
@@ -0,0 +1,10 @@
main proc() int {
total := 0
for 0..10 |i| {
if (i == 5) continue
total = total + i
}
return total
}
+6
View File
@@ -0,0 +1,6 @@
to_fahrenheit proc(celsius float) float { return celsius * 9 / 5 + 32 }
main proc() float {
temperature :: 20
return to_fahrenheit(temperature)
}
+345 -12
View File
@@ -1,11 +1,16 @@
#! grammar:
#! * type ..... -> identifier
#! * assignment -> :: | :=
#! * literal .. -> int | float | string
#! * op ....... -> '+' | '-' | '*' | '/'
#! * expr ..... -> identifier | literal | '(' expr ')' | '-' expr | expr op expr | expr '(' [expr {',' expr} [',']] ')'
#! * decl ..... -> identifier [type] assignment expr
#! * stmt ..... -> decl | expr
#! * type ..... -> identifier | '[' (expr | '_') ']' ['mut'] type
#! * binding ... -> :: | :=
#! * literal .. -> int | float | string | 'true' | 'false'
#! * op ....... -> '+' | '-' | '*' | '/' | '==' | '!=' | '<' | '<=' | '>' | '>=' | 'and' | 'or'
#! * expr ..... -> identifier | literal | type | '[' [expr {',' expr} [',']] ']' | '(' expr ')' | ('-' | '!') expr | expr op expr | expr '(' [expr {',' expr} [',']] ')' | expr '[' expr ']' | expr ('..' | '..=') expr
#! * decl ..... -> identifier [type] binding expr
#! * block .... -> '{' {stmt} '}'
#! * arm ...... -> block | stmt (single statements become canonical blocks)
#! * stmt ..... -> decl | expr | expr '=' expr | 'return' [expr] | 'break' | 'continue' | 'if' expr arm ['else' arm] | 'while' expr arm | 'for' expr '|' ['@'] identifier [',' identifier] '|' arm
#! braceless guards/iterables require parentheses unless they are calls; else binds to the nearest if.
#! precedence (tightest first): calls/indices, unary -/!, multiplication/division, addition/subtraction, relational comparisons, equality, and, or, ranges. compound range endpoints require their own parentheses.
#! binary operators associate left; unary operators associate right.
import "@std/debug"
import "@std/mem"
@@ -22,17 +27,22 @@ NodeId :: distinct u32
no_node :: maxval!(NodeId)
LiteralKind :: enum {
int, float, string
int, float, string, bool
}
ExprKind :: enum {
literal_int
literal_float
literal_string
literal_bool
identifier
unary
binary
call
range
array_type
array_literal
index
invalid
}
@@ -41,17 +51,31 @@ DeclKind :: enum {
}
StmtKind :: enum {
declaration, expression, invalid
declaration,
expression,
assignment_stmt,
return_stmt,
break_stmt,
continue_stmt,
if_stmt,
while_stmt,
for_stmt,
invalid
}
Mutability :: enum {
constant, variable
}
UnaryOperator :: enum { negate }
UnaryOperator :: enum { negate, logical_not }
BinaryOperator :: enum {
add, subtract, multiply, divide
add, subtract,
multiply, divide
equal, not_equal,
less, less_equal,
greater, greater_equal
logical_and, logical_or
}
@hide:file
@@ -59,12 +83,24 @@ NodeKind :: enum {
literal_int,
literal_float,
literal_string,
literal_bool,
expr_identifier,
expr_unary,
expr_binary,
expr_call,
expr_range,
expr_array_type,
expr_array_literal,
expr_index,
stmt_expr,
stmt_decl,
stmt_return,
stmt_break,
stmt_continue,
stmt_if,
stmt_assignment,
stmt_while,
stmt_for,
func_decl,
param_decl,
block,
@@ -151,6 +187,60 @@ CallExprView :: struct {
callee NodeId
}
ArrayTypeView :: struct {
span Span
count ?NodeId
element NodeId
mutable bool
}
IndexView :: struct {
span Span
base NodeId
index NodeId
}
ReturnView :: struct {
span Span
value ?NodeId
}
IfView :: struct {
span Span
condition NodeId
then_block NodeId
else_block ?NodeId
}
AssignmentView :: struct {
span Span
target NodeId
value NodeId
}
WhileView :: struct {
span Span
condition NodeId
body NodeId
}
RangeView :: struct {
span Span
start NodeId
end NodeId
inclusive bool
}
ForView :: struct {
span Span
iterable NodeId
capture ?NodeId
index ?NodeId
pointer_capture bool
index_capture bool
body NodeId
}
InvalidView :: struct {
span Span
diagnostic DiagnosticId
@@ -200,6 +290,7 @@ add_literal proc(builder @mut Builder, kind LiteralKind, span Span) NodeId ! mem
.int: .literal_int
.float: .literal_float
.string: .literal_string
.bool: .literal_bool
}
return try add_node(builder, Node{ kind = node_kind }, span)
}
@@ -236,6 +327,18 @@ add_binary proc(
}, span)
}
add_range proc(
builder @mut Builder, span Span, start NodeId, end NodeId, inclusive bool,
) NodeId ! mem.AllocError {
flag :: if (inclusive) u32(1) else u32(0)
return try add_node(builder, Node{
kind = .expr_range,
main = NodeData{ extra = flag },
data0 = NodeData{ node_id = start },
data1 = NodeData{ node_id = end },
}, span)
}
add_call proc(
builder @mut Builder,
span Span,
@@ -252,6 +355,37 @@ add_call proc(
}, span)
}
add_array_type proc(
builder @mut Builder, span Span, count ?NodeId, element NodeId, mutable bool,
) NodeId ! mem.AllocError {
raw_count :: if (count) |id| id else no_node
flag :: if (mutable) u32(1) else u32(0)
return try add_node(builder, Node{
kind = .expr_array_type,
main = NodeData{ extra = flag },
data0 = NodeData{ node_id = raw_count },
data1 = NodeData{ node_id = element },
}, span)
}
add_array_literal proc(builder @mut Builder, span Span, elements []NodeId) NodeId ! mem.AllocError {
start :: builder.storage.extra.items.len
for (elements) |element| _ = try add_extra(builder, u32(element))
return try add_node(builder, Node{
kind = .expr_array_literal,
data0 = NodeData{ extra = u32(start) },
data1 = NodeData{ extra = u32(builder.storage.extra.items.len) },
}, span)
}
add_index proc(builder @mut Builder, span Span, base, index NodeId) NodeId ! mem.AllocError {
return try add_node(builder, Node{
kind = .expr_index,
data0 = NodeData{ node_id = base },
data1 = NodeData{ node_id = index },
}, span)
}
add_expression_stmt proc(builder @mut Builder, expression NodeId) NodeId ! mem.AllocError {
return try add_node(builder, Node{
kind = .stmt_expr,
@@ -259,6 +393,84 @@ add_expression_stmt proc(builder @mut Builder, expression NodeId) NodeId ! mem.A
}, builder_span(builder, expression))
}
add_return_stmt proc(builder @mut Builder, span Span, value ?NodeId) NodeId ! mem.AllocError {
raw_value :: if (value) |id| id else no_node
return try add_node(builder, Node{
kind = .stmt_return,
main = NodeData{ node_id = raw_value },
}, span)
}
add_break_stmt proc(builder @mut Builder, span Span) NodeId ! mem.AllocError {
return try add_node(builder, Node{ kind = .stmt_break }, span)
}
add_continue_stmt proc(builder @mut Builder, span Span) NodeId ! mem.AllocError {
return try add_node(builder, Node{ kind = .stmt_continue }, span)
}
add_if_stmt proc(
builder @mut Builder,
span Span,
condition NodeId,
then_block NodeId,
else_block ?NodeId,
) NodeId ! mem.AllocError {
alternative :: if (else_block) |id| id else no_node
return try add_node(builder, Node{
kind = .stmt_if,
main = NodeData{ node_id = condition },
data0 = NodeData{ node_id = then_block },
data1 = NodeData{ node_id = alternative },
}, span)
}
add_assignment_stmt proc(
builder @mut Builder,
span Span,
target NodeId,
value NodeId,
) NodeId ! mem.AllocError {
return try add_node(builder, Node{
kind = .stmt_assignment,
main = NodeData{ node_id = target },
data0 = NodeData{ node_id = value },
}, span)
}
add_while_stmt proc(
builder @mut Builder,
span Span,
condition NodeId,
body NodeId,
) NodeId ! mem.AllocError {
return try add_node(builder, Node{
kind = .stmt_while,
main = NodeData{ node_id = condition },
data0 = NodeData{ node_id = body },
}, span)
}
add_for_stmt proc(
builder @mut Builder, span Span, iterable NodeId, capture, index ?NodeId,
pointer_capture, index_capture bool, body NodeId,
) NodeId ! mem.AllocError {
raw_capture :: if (capture) |id| u32(id) else u32(no_node)
raw_index :: if (index) |id| u32(id) else u32(no_node)
pointer_flag :: if (pointer_capture) u32(1) else u32(0)
index_flag :: if (index_capture) u32(1) else u32(0)
extra :: try add_extra(builder, raw_capture)
_ = try add_extra(builder, raw_index)
_ = try add_extra(builder, pointer_flag)
_ = try add_extra(builder, index_flag)
return try add_node(builder, Node{
kind = .stmt_for,
main = NodeData{ node_id = iterable },
data0 = NodeData{ node_id = body },
data1 = NodeData{ extra = extra },
}, span)
}
add_statement_decl proc(
builder @mut Builder,
span Span,
@@ -393,6 +605,13 @@ stmt_kind proc(tree @Tree, id NodeId) StmtKind {
return match get_node(&tree.storage, id).kind {
.stmt_decl: .declaration
.stmt_expr: .expression
.stmt_return: .return_stmt
.stmt_break: .break_stmt
.stmt_continue: .continue_stmt
.stmt_if: .if_stmt
.stmt_assignment: .assignment_stmt
.stmt_while: .while_stmt
.stmt_for: .for_stmt
.invalid_stmt: .invalid
else: unreachable
}
@@ -404,6 +623,61 @@ statement_expr proc(tree @Tree, id NodeId) NodeId {
return node.main.node_id
}
return_stmt proc(tree @Tree, id NodeId) ReturnView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .stmt_return)
return ReturnView{
span = get_span(&tree.storage, id),
value = maybe_node_id(u32(node.main.node_id)),
}
}
if_stmt proc(tree @Tree, id NodeId) IfView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .stmt_if)
return IfView{
span = get_span(&tree.storage, id),
condition = node.main.node_id,
then_block = node.data0.node_id,
else_block = maybe_node_id(u32(node.data1.node_id)),
}
}
assignment_stmt proc(tree @Tree, id NodeId) AssignmentView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .stmt_assignment)
return AssignmentView{
span = get_span(&tree.storage, id),
target = node.main.node_id,
value = node.data0.node_id,
}
}
while_stmt proc(tree @Tree, id NodeId) WhileView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .stmt_while)
return WhileView{
span = get_span(&tree.storage, id),
condition = node.main.node_id,
body = node.data0.node_id,
}
}
for_stmt proc(tree @Tree, id NodeId) ForView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .stmt_for)
extra :: usize(node.data1.extra)
return ForView{
span = get_span(&tree.storage, id),
iterable = node.main.node_id,
capture = maybe_node_id(tree.storage.extra.items[extra]),
index = maybe_node_id(tree.storage.extra.items[extra + 1]),
pointer_capture = tree.storage.extra.items[extra + 2] != 0,
index_capture = tree.storage.extra.items[extra + 3] != 0,
body = node.data0.node_id,
}
}
statement_decl proc(tree @Tree, id NodeId) StatementDeclView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .stmt_decl)
@@ -467,14 +741,28 @@ block_statement proc(tree @Tree, id NodeId, index usize) NodeId {
}
expr_kind proc(tree @Tree, id NodeId) ExprKind {
return match get_node(&tree.storage, id).kind {
return expression_kind(get_node(&tree.storage, id).kind)
}
builder_expr_kind proc(builder @Builder, id NodeId) ExprKind {
return expression_kind(get_node(&builder.storage, id).kind)
}
@hide:file
expression_kind proc(kind NodeKind) ExprKind {
return match kind {
.literal_int: .literal_int
.literal_float: .literal_float
.literal_string: .literal_string
.literal_bool: .literal_bool
.expr_identifier: .identifier
.expr_unary: .unary
.expr_binary: .binary
.expr_call: .call
.expr_range: .range
.expr_array_type: .array_type
.expr_array_literal: .array_literal
.expr_index: .index
.invalid_expr: .invalid
else: unreachable
}
@@ -501,6 +789,17 @@ binary_expr proc(tree @Tree, id NodeId) BinaryExprView {
}
}
range_expr proc(tree @Tree, id NodeId) RangeView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_range)
return RangeView{
span = get_span(&tree.storage, id),
start = node.data0.node_id,
end = node.data1.node_id,
inclusive = node.main.extra != 0,
}
}
call_expr proc(tree @Tree, id NodeId) CallExprView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_call)
@@ -523,6 +822,40 @@ call_arg proc(tree @Tree, id NodeId, index usize) NodeId {
return NodeId(tree.storage.extra.items[usize(node.data0.extra) + index])
}
array_type proc(tree @Tree, id NodeId) ArrayTypeView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_array_type)
return ArrayTypeView{
span = get_span(&tree.storage, id),
count = maybe_node_id(u32(node.data0.node_id)),
element = node.data1.node_id,
mutable = node.main.extra != 0,
}
}
array_element_count proc(tree @Tree, id NodeId) usize {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_array_literal)
return usize(node.data1.extra - node.data0.extra)
}
array_element proc(tree @Tree, id NodeId, index usize) NodeId {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_array_literal)
debug.assert(index < usize(node.data1.extra - node.data0.extra))
return NodeId(tree.storage.extra.items[usize(node.data0.extra) + index])
}
index_expr proc(tree @Tree, id NodeId) IndexView {
node :: get_node(&tree.storage, id)
debug.assert(node.kind == .expr_index)
return IndexView{
span = get_span(&tree.storage, id),
base = node.data0.node_id,
index = node.data1.node_id,
}
}
node_span proc(tree @Tree, id NodeId) Span {
return get_span(&tree.storage, id)
}
+32
View File
@@ -32,3 +32,35 @@ exposes_typed_declaration_views test {
try testing.expect_equal(ExprKind.literal_int, expr_kind(&tree, statement.initializer))
try testing.expect_equal("value", source_text(&tree, statement.name))
}
exposes_array_type_literal_and_index_views test {
builder := init_builder(mem.c_allocator, "")
defer deinit_builder(&builder)
span :: Span{ start = 0, end = 0 }
count :: try add_literal(&builder, .int, span)
element :: try add_identifier(&builder, span)
row :: try add_array_type(&builder, span, count, element, false)
matrix :: try add_array_type(&builder, span, null, row, true)
elements [2]NodeId :: [count, element]
literal :: try add_array_literal(&builder, span, elements[..])
empty :: try add_array_literal(&builder, span, &[])
indexed :: try add_index(&builder, span, literal, count)
package :: try add_package(&builder, span, &[])
tree := finish(&builder, package)
defer deinit(&tree)
try testing.expect_equal(ExprKind.array_type, expr_kind(&tree, matrix))
try testing.expect_equal(null, array_type(&tree, matrix).count)
try testing.expect(array_type(&tree, matrix).mutable)
try testing.expect_equal(row, array_type(&tree, matrix).element)
try testing.expect_equal(count, array_type(&tree, row).count?)
try testing.expect(!array_type(&tree, row).mutable)
try testing.expect_equal(element, array_type(&tree, row).element)
try testing.expect_equal(ExprKind.array_literal, expr_kind(&tree, literal))
try testing.expect_equal(2, array_element_count(&tree, literal))
try testing.expect_equal(count, array_element(&tree, literal, 0))
try testing.expect_equal(element, array_element(&tree, literal, 1))
try testing.expect_equal(0, array_element_count(&tree, empty))
try testing.expect_equal(ExprKind.index, expr_kind(&tree, indexed))
try testing.expect_equal(literal, index_expr(&tree, indexed).base)
try testing.expect_equal(count, index_expr(&tree, indexed).index)
}
+76 -5
View File
@@ -41,17 +41,82 @@ render_function proc(
try append_text(renderer, " function ")
try append_text(renderer, ast.source_text(tree, function.name))
try append_text(renderer, "\n")
for 0..ast.block_statement_count(tree, function.body) |index| {
statement :: ast.block_statement(tree, function.body, index)
try render_block(renderer, tree, function.body, 2)
}
@hide:file
render_block proc(renderer @mut Renderer, tree @ast.Tree, block ast.NodeId, depth usize) void ! mem.AllocError {
for 0..ast.block_statement_count(tree, block) |index| {
statement :: ast.block_statement(tree, block, index)
for 0..depth |_| try append_text(renderer, " ")
match ast.stmt_kind(tree, statement) {
.declaration: try render_statement(renderer, tree, statement, " ")
.declaration: try render_statement(renderer, tree, statement, "")
.expression: {
expression :: ast.statement_expr(tree, statement)
try append_text(renderer, " expression ")
try append_text(renderer, "expression ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, expression)))
try append_text(renderer, "\n")
}
.invalid: try append_text(renderer, " invalid statement\n")
.break_stmt: try append_text(renderer, "break\n")
.continue_stmt: try append_text(renderer, "continue\n")
.return_stmt: {
try append_text(renderer, "return")
if (ast.return_stmt(tree, statement).value) |value| {
try append_text(renderer, " ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, value)))
}
try append_text(renderer, "\n")
}
.if_stmt: {
conditional :: ast.if_stmt(tree, statement)
try append_text(renderer, "if ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, conditional.condition)))
try append_text(renderer, "\n")
try render_block(renderer, tree, conditional.then_block, depth + 1)
if (conditional.else_block) |alternative| {
for 0..depth |_| try append_text(renderer, " ")
try append_text(renderer, "else\n")
try render_block(renderer, tree, alternative, depth + 1)
}
}
.assignment_stmt: {
assignment :: ast.assignment_stmt(tree, statement)
try append_text(renderer, "assignment ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, assignment.target)))
try append_text(renderer, " = ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, assignment.value)))
try append_text(renderer, "\n")
}
.while_stmt: {
loop :: ast.while_stmt(tree, statement)
try append_text(renderer, "while ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, loop.condition)))
try append_text(renderer, "\n")
try render_block(renderer, tree, loop.body, depth + 1)
}
.for_stmt: {
loop :: ast.for_stmt(tree, statement)
try append_text(renderer, "for ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, loop.iterable)))
try append_text(renderer, " |")
if (loop.pointer_capture) try append_text(renderer, "@")
if (loop.capture) |capture| {
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, capture)))
} else {
try append_text(renderer, "_")
}
if (loop.index_capture) {
try append_text(renderer, ", ")
if (loop.index) |index| {
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, index)))
} else {
try append_text(renderer, "_")
}
}
try append_text(renderer, "|\n")
try render_block(renderer, tree, loop.body, depth + 1)
}
.invalid: try append_text(renderer, "invalid statement\n")
}
}
}
@@ -67,6 +132,12 @@ render_statement proc(
try append_text(renderer, indent)
try append_text(renderer, "declaration ")
try append_text(renderer, ast.source_text(tree, statement.name))
if (statement.type) |type| {
try append_text(renderer, " ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, type)))
}
try append_text(renderer, if (statement.mutability == .constant) " :: " else " := ")
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, statement.initializer)))
try append_text(renderer, "\n")
}
+24 -18
View File
@@ -1,27 +1,33 @@
import "@std/mem"
import "@std/testing"
import "@source/diagnostics"
import "@source/strpool"
import "@source/lexer"
import "@source/parser"
import "@source/ast"
renders_semantic_debug_view test {
builder := ast.init_builder(mem.c_allocator, "value :: 42")
defer ast.deinit_builder(&builder)
value :: try ast.add_literal(&builder, .int, ast.Span{ start = 9, end = 11 })
declaration :: try ast.add_statement_decl(
&builder,
ast.Span{ start = 0, end = 11 },
ast.Span{ start = 0, end = 5 },
null,
.constant,
value,
)
declarations [1]ast.NodeId :: [declaration]
package :: try ast.add_package(&builder, ast.Span{ start = 0, end = 11 }, declarations[..])
tree := ast.finish(&builder, package)
renders_array_annotations_literals_and_indexed_assignments test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`data [2]mut i64 := [1, 2]
`main proc() i64 {
` (data)[0] = 9
` return data[0]
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
renderer := init(mem.c_allocator)
defer deinit(&renderer)
try render(&renderer, &tree)
try testing.expect(mem.eql(u8, "package\n declaration value\n", renderer.bytes.items))
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
try testing.expect_equal(
"package\n declaration data [2]mut i64 := [1, 2]\n function main\n assignment (data)[0] = 9\n return data[0]\n",
renderer.bytes.items,
)
}
+87 -1
View File
@@ -1,5 +1,6 @@
import "@std/mem"
import "@std/arraylist"
import "@std/enummap"
Span :: struct {
start u32
@@ -16,6 +17,43 @@ Code :: enum {
unexpected_token,
duplicate_name,
unknown_name,
expected_type,
type_not_value,
invalid_void_type,
type_mismatch,
invalid_operand,
integer_division,
not_callable,
argument_count_mismatch,
cannot_infer_type,
missing_return,
return_type_mismatch,
condition_not_bool,
invalid_assignment_target,
break_outside_loop,
continue_outside_loop,
invalid_range_bound,
invalid_for_iterable,
range_value_not_supported,
range_pointer_capture,
range_index_capture,
invalid_array_length,
array_length_mismatch,
invalid_array_element_type,
array_size_overflow,
invalid_index_type,
not_indexable,
invalid_index_assignment,
index_out_of_bounds,
missing_entry,
invalid_entry,
literal_out_of_range,
inexact_integer_literal,
invalid_escape,
read_before_initialization,
write_before_initialization,
initialization_cycle,
integer_overflow,
}
Diagnostic :: struct {
@@ -26,7 +64,7 @@ Diagnostic :: struct {
#! owns diagnostic storage and retains its allocator; release with deinit.
Store :: struct { list arraylist.ArrayList(Diagnostic) }
CapacityError :: enum { diagnostic_limit_reached }
CapacityError :: enum { diagnostic_limit_reached, storage_limit_reached }
#! retains `allocator` for added diagnostics; the caller owns the returned Store.
init proc(allocator mem.Allocator) Store {
@@ -54,3 +92,51 @@ get proc(store @Store, id DiagnosticId) ?Diagnostic {
count proc(store @Store) usize {
return store.list.items.len
}
#! constructed at compile time so missing entries fail compilation; message bytes live for the process lifetime.
messages :: $enummap.init(Code, []u8, {
invalid_character = "invalid character",
float_must_end_with_digit = "float literal must end with a digit",
unterminated_string = "unterminated string literal",
unexpected_token = "unexpected token",
duplicate_name = "duplicate name or forbidden shadowing",
unknown_name = "unknown name",
expected_type = "annotation must name a type",
type_not_value = "type name cannot be used as a value",
invalid_void_type = "void cannot be stored or used as a parameter type",
type_mismatch = "incompatible value type",
invalid_operand = "operator requires compatible operand types",
integer_division = "division requires float operands",
not_callable = "value is not callable",
argument_count_mismatch = "argument count does not match the function signature",
cannot_infer_type = "cannot infer a type without a type anchor",
missing_return = "nonvoid function can finish without returning a value",
return_type_mismatch = "return does not match the function result type",
condition_not_bool = "condition must have type bool",
invalid_assignment_target = "assignment requires a mutable local binding",
break_outside_loop = "break requires an enclosing loop",
continue_outside_loop = "continue requires an enclosing loop",
invalid_range_bound = "range bounds must have compatible integer types",
invalid_for_iterable = "for-loop iteration currently requires a direct integer range",
range_value_not_supported = "first-class range values are not implemented",
range_pointer_capture = "range loops do not support pointer captures",
range_index_capture = "range loops do not support index captures",
invalid_array_length = "array length must be a nonnegative supported compile-time integer",
array_length_mismatch = "array literal length does not match its type",
invalid_array_element_type = "array element annotations require a supported concrete value type",
array_size_overflow = "array storage size exceeds the supported address space",
invalid_index_type = "array index must have type usize",
not_indexable = "indexing is currently implemented for fixed arrays only",
invalid_index_assignment = "indexed assignment requires a writable array element",
index_out_of_bounds = "index out of bounds",
missing_entry = "program requires a main function",
invalid_entry = "main must be a function with no parameters",
literal_out_of_range = "numeric literal is outside its runtime type's range",
inexact_integer_literal = "integer literal is not exactly representable as float; use a floating literal to allow rounding",
invalid_escape = "invalid string escape",
read_before_initialization = "local value was read before initialization",
write_before_initialization = "local value was assigned before initialization",
initialization_cycle = "global initialization cycle",
integer_overflow = "integer arithmetic overflow",
})
+526
View File
@@ -0,0 +1,526 @@
import "@std/mem"
import "@std/arraylist"
import "@source/ir"
import "@source/diagnostics"
Outcome :: struct {
value ir.Value = ir.Value{}
trap ?diagnostics.DiagnosticId = null
allocator ?mem.Allocator = null
owned_elements ?[]mut ir.Value = null
}
#! release an escaped aggregate with its retained allocator; scalar and trap outcomes need no allocation.
#! strings and array descriptors borrow the module, which must outlive this outcome.
deinit proc(outcome @mut Outcome) void {
if (outcome.owned_elements) |elements| mem.free(outcome.allocator?, elements)
outcome.owned_elements = null
outcome.allocator = null
outcome.value = ir.Value{}
}
@hide:file
Slot :: struct {
initialized bool = false
value ir.Value = ir.Value{}
}
@hide:file
GlobalStatus :: enum { uninitialized, initializing, ready }
@hide:file
GlobalState :: struct {
status GlobalStatus = .uninitialized
value ir.Value = ir.Value{}
}
@hide:file
Address :: struct {
type ir.Type
array_type ir.ArrayId = ir.no_array
cells []mut ir.Value
scalar ?@mut ir.Value = null
initialized ?@mut bool = null
}
@hide:file
Frame :: struct {
function ir.FunctionId
block ir.BlockId
next_instruction usize = 0
destination ir.ValueId = ir.no_value
global ?usize = null
locals []mut Slot
values []mut ir.Value
aggregate []mut ir.Value
addresses []mut Address
}
#! executes typed IR with invocation/global storage released on every exit.
#! array results own flat storage released by `deinit`; string IDs and shapes resolve through `module`.
run proc(
allocator mem.Allocator,
module @ir.Module,
diagnostic_store @mut diagnostics.Store,
) Outcome ! (mem.AllocError | diagnostics.CapacityError) {
entry ir.FunctionId := ir.FunctionId(0)
if module.entry |function| {
entry = function
} else {
return Outcome{ trap = module.entry_diagnostic }
}
globals :: try mem.alloc(GlobalState, allocator, module.globals.items.len)
defer mem.free(allocator, globals)
mem.set(globals, GlobalState{})
global_cells usize := 0
for (module.globals.items) |global| {
if (global.type != .array) continue
cells :: module.arrays.items[usize(global.array_type)].cells
if (cells > maxval!(usize) - global_cells) return .storage_limit_reached
global_cells += cells
}
aggregate_globals :: try mem.alloc(ir.Value, allocator, global_cells)
defer mem.free(allocator, aggregate_globals)
global_offset usize := 0
for (module.globals.items) |global, i| {
if (global.type != .array) continue
cells :: module.arrays.items[usize(global.array_type)].cells
globals[i].value = array_view(global.array_type, aggregate_globals[global_offset..global_offset + cells])
global_offset += cells
}
frames := arraylist.init(Frame, allocator)
defer {
for (frames.items) |frame| free_frame(allocator, frame)
arraylist.deinit(&frames)
}
try push_frame(allocator, module, &frames, entry, ir.no_value, null)
while frames.items.len != 0 {
frame_index :: frames.items.len - 1
function :: &module.functions.items[usize(frames.items[frame_index].function)]
block :: &function.blocks.items[usize(frames.items[frame_index].block)]
instruction_index :: frames.items[frame_index].next_instruction
if instruction_index == block.instructions.items.len {
terminator :: block.terminator
if (terminator.kind == .trap) return Outcome{ trap = terminator.diagnostic }
if (terminator.kind == .jump or terminator.kind == .branch) {
target := terminator.target
if (terminator.kind == .branch) {
condition :: frames.items[frame_index].values[usize(terminator.value)]
if (!condition.data.boolean) target = terminator.alternative
}
frames.items[frame_index].block = target
frames.items[frame_index].next_instruction = 0
continue
}
value := ir.Value{}
if terminator.value != ir.no_value {
value = frames.items[frame_index].values[usize(terminator.value)]
}
completed :: frames.items[frame_index]
# transfer leaves while the completed frame still owns its backing.
if completed.global |global_index| {
copy_value(&globals[global_index].value, value)
globals[global_index].status = .ready
}
if (frames.items.len == 1) {
if (value.type == .array) {
escaped :: try mem.alloc(ir.Value, allocator, value.data.elements.len)
memcopy!(escaped, value.data.elements)
value.data.elements = escaped
return Outcome{ value = value, allocator = allocator, owned_elements = escaped }
}
return Outcome{ value = value }
}
if completed.destination != ir.no_value {
copy_value(&frames.items[frame_index - 1].values[usize(completed.destination)], value)
}
_ = arraylist.pop(&frames)
free_frame(allocator, completed)
continue
}
instruction :: &block.instructions.items[instruction_index]
frames.items[frame_index].next_instruction += 1
value := ir.Value{}
match instruction.op {
.constant: value = instruction.constant
.load_local: {
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
if !local.initialized {
return try runtime_trap(diagnostic_store, .read_before_initialization, instruction.span)
}
value = local.value
}
.store_local: {
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
copy_value(&local.value, frames.items[frame_index].values[usize(instruction.left)])
local.initialized = true
continue
}
.assign_local: {
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
if (!local.initialized) {
return try runtime_trap(diagnostic_store, .write_before_initialization, instruction.span)
}
copy_value(&local.value, frames.items[frame_index].values[usize(instruction.left)])
continue
}
.reset_local: {
frames.items[frame_index].locals[usize(instruction.slot)].initialized = false
continue
}
.load_global: {
global_index :: usize(instruction.slot)
match globals[global_index].status {
.ready: value = globals[global_index].value
.initializing: {
return try runtime_trap(diagnostic_store, .initialization_cycle, instruction.span)
}
.uninitialized: {
try push_frame(
allocator,
module,
&frames,
module.globals.items[global_index].initializer,
instruction.destination,
global_index,
)
globals[global_index].status = .initializing
continue
}
}
}
.call: {
callee :: frames.items[frame_index].values[usize(instruction.left)].data.function
try push_frame(allocator, module, &frames, callee, instruction.destination, null)
# frame indices survive stack growth; pointers into frames.items do not.
for (0..instruction.arguments_count) |argument_index| {
argument :: block.arguments.items[instruction.arguments_start + argument_index]
local :: &frames.items[frame_index + 1].locals[argument_index]
copy_value(&local.value, frames.items[frame_index].values[usize(argument)])
local.initialized = true
}
continue
}
.array_literal: {
destination :: &frames.items[frame_index].values[usize(instruction.destination)]
offset usize := 0
for (0..instruction.arguments_count) |i| {
operand :: block.arguments.items[instruction.arguments_start + i]
element :: frames.items[frame_index].values[usize(operand)]
if (element.type == .array) {
count :: element.data.elements.len
memcopy!(destination.data.elements[offset..offset + count], element.data.elements)
offset += count
} else {
destination.data.elements[offset] = element
offset += 1
}
}
continue
}
.index: {
base :: frames.items[frame_index].values[usize(instruction.left)]
index :: frames.items[frame_index].values[usize(instruction.right)].data.unsigned
shape :: module.arrays.items[usize(base.array_type)]
if (index >= u64(shape.len)) {
return try runtime_trap(diagnostic_store, .index_out_of_bounds, instruction.span)
}
value = indexed_value(module, base.array_type, base.data.elements, usize(index))
}
.address_local: {
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
address := Address{
type = instruction.pointee, array_type = instruction.array_type,
cells = mem.empty(ir.Value), initialized = &local.initialized,
}
if (address.type == .array) address.cells = local.value.data.elements
else address.scalar = &local.value
frames.items[frame_index].addresses[usize(instruction.destination)] = address
value = ir.Value{ type = .address, data = ir.ValueData{ address = u32(instruction.destination) } }
}
.address_global: {
global_index :: usize(instruction.slot)
match globals[global_index].status {
.initializing: return try runtime_trap(diagnostic_store, .initialization_cycle, instruction.span)
.uninitialized: {
frames.items[frame_index].next_instruction -= 1
try push_frame(allocator, module, &frames, module.globals.items[global_index].initializer, ir.no_value, global_index)
globals[global_index].status = .initializing
continue
}
.ready: {}
}
global :: &globals[global_index]
address := Address{
type = instruction.pointee, array_type = instruction.array_type,
cells = mem.empty(ir.Value),
}
if (address.type == .array) address.cells = global.value.data.elements
else address.scalar = &global.value
frames.items[frame_index].addresses[usize(instruction.destination)] = address
value = ir.Value{ type = .address, data = ir.ValueData{ address = u32(instruction.destination) } }
}
.index_address: {
base_id :: frames.items[frame_index].values[usize(instruction.left)].data.address
base :: frames.items[frame_index].addresses[usize(base_id)]
index :: frames.items[frame_index].values[usize(instruction.right)].data.unsigned
shape :: module.arrays.items[usize(base.array_type)]
if (index >= u64(shape.len)) {
return try runtime_trap(diagnostic_store, .index_out_of_bounds, instruction.span)
}
width :: element_cells(module, shape)
start :: usize(index) * width
address := Address{
type = instruction.pointee, array_type = instruction.array_type,
cells = base.cells[start..start + width], initialized = base.initialized,
}
if (address.type != .array) address.scalar = &address.cells[0]
frames.items[frame_index].addresses[usize(instruction.destination)] = address
value = ir.Value{ type = .address, data = ir.ValueData{ address = u32(instruction.destination) } }
}
.load_address, .store_address: {
id :: frames.items[frame_index].values[usize(instruction.left)].data.address
address :: frames.items[frame_index].addresses[usize(id)]
if address.initialized |initialized| {
if (!initialized^) {
code :: if (instruction.op == .load_address) diagnostics.Code.read_before_initialization else diagnostics.Code.write_before_initialization
return try runtime_trap(diagnostic_store, code, instruction.span)
}
}
if (instruction.op == .load_address) {
value = if (address.type == .array) array_view(address.array_type, address.cells) else address.scalar?^
} else {
source :: frames.items[frame_index].values[usize(instruction.right)]
if (address.type == .array) memcopy!(address.cells, source.data.elements)
else address.scalar?^ = source
continue
}
}
.negate: {
operand :: frames.items[frame_index].values[usize(instruction.left)]
if instruction.type == .float {
value = ir.Value{ type = .float, data = ir.ValueData{ real = -operand.data.real } }
} else {
if operand.data.integer == minval!(i64) {
return try runtime_trap(diagnostic_store, .integer_overflow, instruction.span)
}
value = ir.Value{ type = .int, data = ir.ValueData{ integer = -operand.data.integer } }
}
}
.logical_not: {
operand :: frames.items[frame_index].values[usize(instruction.left)]
value = ir.Value{ type = .bool, data = ir.ValueData{ boolean = !operand.data.boolean } }
}
.add, .subtract, .multiply, .divide: {
left :: frames.items[frame_index].values[usize(instruction.left)]
right :: frames.items[frame_index].values[usize(instruction.right)]
if instruction.type == .float {
result f64 := 0
match instruction.op {
.add: result = left.data.real + right.data.real
.subtract: result = left.data.real - right.data.real
.multiply: result = left.data.real * right.data.real
.divide: result = left.data.real / right.data.real
else: unreachable
}
value = ir.Value{ type = .float, data = ir.ValueData{ real = result } }
} else if instruction.type == .usize {
result :: checked_unsigned(instruction.op, left.data.unsigned, right.data.unsigned)
if result |unsigned| {
value = ir.Value{ type = .usize, data = ir.ValueData{ unsigned = unsigned } }
} else {
return try runtime_trap(diagnostic_store, .integer_overflow, instruction.span)
}
} else {
result :: checked_integer(instruction.op, left.data.integer, right.data.integer)
if result |result_integer| {
value = ir.Value{ type = .int, data = ir.ValueData{ integer = result_integer } }
} else {
return try runtime_trap(diagnostic_store, .integer_overflow, instruction.span)
}
}
}
.equal, .not_equal, .less, .less_equal, .greater, .greater_equal: {
left :: frames.items[frame_index].values[usize(instruction.left)]
right :: frames.items[frame_index].values[usize(instruction.right)]
result := false
match left.type {
.int: result = match instruction.op {
.equal: left.data.integer == right.data.integer
.not_equal: left.data.integer != right.data.integer
.less: left.data.integer < right.data.integer
.less_equal: left.data.integer <= right.data.integer
.greater: left.data.integer > right.data.integer
.greater_equal: left.data.integer >= right.data.integer
else: unreachable
}
.usize: result = match instruction.op {
.equal: left.data.unsigned == right.data.unsigned
.not_equal: left.data.unsigned != right.data.unsigned
.less: left.data.unsigned < right.data.unsigned
.less_equal: left.data.unsigned <= right.data.unsigned
.greater: left.data.unsigned > right.data.unsigned
.greater_equal: left.data.unsigned >= right.data.unsigned
else: unreachable
}
.float: result = match instruction.op {
.equal: left.data.real == right.data.real
.not_equal: left.data.real != right.data.real
.less: left.data.real < right.data.real
.less_equal: left.data.real <= right.data.real
.greater: left.data.real > right.data.real
.greater_equal: left.data.real >= right.data.real
else: unreachable
}
.bool: result = match instruction.op {
.equal: left.data.boolean == right.data.boolean
.not_equal: left.data.boolean != right.data.boolean
else: unreachable
}
else: unreachable
}
value = ir.Value{ type = .bool, data = ir.ValueData{ boolean = result } }
}
}
if instruction.destination != ir.no_value {
copy_value(&frames.items[frame_index].values[usize(instruction.destination)], value)
}
}
unreachable
}
@hide:file
push_frame proc(
allocator mem.Allocator,
module @ir.Module,
frames @mut arraylist.ArrayList(Frame),
function ir.FunctionId,
destination ir.ValueId,
global ?usize,
) void ! mem.AllocError {
signature :: &module.functions.items[usize(function)]
locals :: try mem.alloc(Slot, allocator, signature.locals.items.len)
errdefer mem.free(allocator, locals)
mem.set(locals, Slot{})
values :: try mem.alloc(ir.Value, allocator, signature.value_count)
errdefer mem.free(allocator, values)
aggregate :: try mem.alloc(ir.Value, allocator, signature.aggregate_cells)
errdefer mem.free(allocator, aggregate)
address_count usize :: if (signature.has_addresses) signature.value_count else 0
addresses :: try mem.alloc(Address, allocator, address_count)
errdefer mem.free(allocator, addresses)
for (signature.array_storage.items) |storage| {
cells :: module.arrays.items[usize(storage.array_type)].cells
value :: array_view(storage.array_type, aggregate[storage.offset..storage.offset + cells])
if (storage.local) locals[usize(storage.index)].value = value
else values[usize(storage.index)] = value
}
# typed IR assigns each temporary before use; only local slots need initialization flags.
try arraylist.append(frames, Frame{
function = function,
block = signature.entry,
destination = destination,
global = global,
locals = locals,
values = values,
aggregate = aggregate,
addresses = addresses,
})
}
@hide:file
free_frame proc(allocator mem.Allocator, frame Frame) void {
mem.free(allocator, frame.locals)
mem.free(allocator, frame.values)
mem.free(allocator, frame.aggregate)
mem.free(allocator, frame.addresses)
}
@hide:file
runtime_trap proc(
diagnostic_store @mut diagnostics.Store,
code diagnostics.Code,
span diagnostics.Span,
) Outcome ! (mem.AllocError | diagnostics.CapacityError) {
return Outcome{ trap = try diagnostics.add(diagnostic_store, code, span) }
}
@hide:file
checked_integer proc(op ir.Op, left i64, right i64) ?i64 {
minimum :: minval!(i64)
maximum :: maxval!(i64)
match op {
.add: {
if (right > 0 and left > maximum - right) return null
if (right < 0 and left < minimum - right) return null
return left + right
}
.subtract: {
if (right > 0 and left < minimum + right) return null
if (right < 0 and left > maximum + right) return null
return left - right
}
.multiply: {
if left > 0 {
if (right > 0 and left > divtrunc!(maximum, right)) return null
if (right < 0 and right < divtrunc!(minimum, left)) return null
} else if left < 0 {
if (right > 0 and left < divtrunc!(minimum, right)) return null
if (right < 0 and left < divtrunc!(maximum, right)) return null
}
return left * right
}
else: unreachable
}
}
@hide:file
array_view proc(shape ir.ArrayId, cells []mut ir.Value) ir.Value {
return ir.Value{ type = .array, array_type = shape, data = ir.ValueData{ elements = cells } }
}
@hide:file
copy_value proc(destination @mut ir.Value, source ir.Value) void {
if (source.type == .array) {
# aggregate destinations already own disjoint fixed segments; never install an alias.
memcopy!(destination.data.elements, source.data.elements)
} else {
destination^ = source
}
}
@hide:file
element_cells proc(module @ir.Module, shape ir.ArrayType) usize {
return if (shape.element == .array) module.arrays.items[usize(shape.element_array)].cells else 1
}
@hide:file
indexed_value proc(module @ir.Module, shape_id ir.ArrayId, cells []mut ir.Value, index usize) ir.Value {
shape :: module.arrays.items[usize(shape_id)]
if (shape.element != .array) return cells[index]
width :: element_cells(module, shape)
start :: index * width
return array_view(shape.element_array, cells[start..start + width])
}
@hide:file
checked_unsigned proc(op ir.Op, left u64, right u64) ?u64 {
match op {
.add: {
if (left > maxval!(u64) - right) return null
return left + right
}
.subtract: {
if (left < right) return null
return left - right
}
.multiply: {
if (right != 0 and left > divtrunc!(maxval!(u64), right)) return null
return left * right
}
else: unreachable
}
}
File diff suppressed because it is too large Load Diff
+219
View File
@@ -0,0 +1,219 @@
import "@std/mem"
import "@std/arraylist"
import "@source/diagnostics"
FunctionId :: distinct u32
GlobalId :: distinct u32
LocalId :: distinct u32
ValueId :: distinct u32
BlockId :: distinct u32
StringId :: distinct u32
ArrayId :: distinct u32
no_array :: maxval!(ArrayId)
no_value :: maxval!(ValueId)
Type :: enum {
invalid,
int,
float,
bool,
string,
void,
function,
array,
usize,
address,
}
ValueData :: union {
integer i64
unsigned u64
real f64
boolean bool
string StringId
function FunctionId
elements []mut Value
address u32
}
Value :: struct {
type Type = .void
array_type ArrayId = no_array
data ValueData = ValueData{ integer = 0 }
}
#! fixed recursive shape; `cells` counts flattened scalar leaves, including zero-sized children.
ArrayType :: struct {
element Type
element_array ArrayId = no_array
len usize
cells usize
mutable bool
}
#! maps a local slot or temporary to its preallocated, by-value frame segment.
ArrayStorage :: struct {
index u32
array_type ArrayId
offset usize
local bool
}
Op :: enum {
constant
load_local
store_local
assign_local
reset_local
load_global
array_literal
index
address_local
address_global
index_address
load_address
store_address
negate
logical_not
add
subtract
multiply
divide
call
equal
not_equal
less
less_equal
greater
greater_equal
}
Instruction :: struct {
op Op
type Type = .void
array_type ArrayId = no_array
pointee Type = .invalid
span diagnostics.Span = diagnostics.Span{ start = 0, end = 0 }
destination ValueId = no_value
left ValueId = no_value
right ValueId = no_value
slot u32 = 0
constant Value = Value{}
arguments_start usize = 0
arguments_count usize = 0
}
TerminatorKind :: enum {
return_value,
trap,
jump,
branch,
}
Terminator :: struct {
kind TerminatorKind = .return_value
value ValueId = no_value
target BlockId = BlockId(0)
alternative BlockId = BlockId(0)
diagnostic diagnostics.DiagnosticId = diagnostics.no_diagnostic
span diagnostics.Span = diagnostics.Span{ start = 0, end = 0 }
}
Block :: struct {
instructions arraylist.ArrayList(Instruction)
arguments arraylist.ArrayList(ValueId)
terminator Terminator = Terminator{}
}
Function :: struct {
parameters arraylist.ArrayList(Type)
locals arraylist.ArrayList(Type)
result Type = .void
result_array_type ArrayId = no_array
array_storage arraylist.ArrayList(ArrayStorage)
aggregate_cells usize = 0
has_addresses bool = false
blocks arraylist.ArrayList(Block)
entry BlockId = BlockId(0)
value_count usize = 0
span diagnostics.Span = diagnostics.Span{ start = 0, end = 0 }
}
Global :: struct {
type Type
array_type ArrayId = no_array
initializer FunctionId
span diagnostics.Span
}
#! retains the original allocation for freeing while exposing only decoded bytes.
StringStorage :: struct {
allocation []mut u8
len usize
}
#! owns executable storage, including decoded strings; no AST or binding table is retained.
#! spans and diagnostic IDs refer to the caller's source and diagnostic store.
Module :: struct {
allocator mem.Allocator
functions arraylist.ArrayList(Function)
globals arraylist.ArrayList(Global)
strings arraylist.ArrayList(StringStorage)
arrays arraylist.ArrayList(ArrayType)
entry ?FunctionId = null
entry_diagnostic diagnostics.DiagnosticId = diagnostics.no_diagnostic
}
init proc(allocator mem.Allocator) Module {
return Module{
allocator = allocator,
functions = arraylist.init(allocator),
globals = arraylist.init(allocator),
strings = arraylist.init(allocator),
arrays = arraylist.init(allocator),
}
}
init_function proc(allocator mem.Allocator) Function {
return Function{
parameters = arraylist.init(allocator),
locals = arraylist.init(allocator),
blocks = arraylist.init(allocator),
array_storage = arraylist.init(allocator),
}
}
init_block proc(allocator mem.Allocator) Block {
return Block{
instructions = arraylist.init(allocator),
arguments = arraylist.init(allocator),
}
}
#! resolves a module-local ID to bytes borrowed until module deinit.
string_bytes proc(module @Module, id StringId) []u8 {
string :: module.strings.items[usize(id)]
return string.allocation[..string.len]
}
#! releases all executable storage using the retained allocator.
deinit proc(module @mut Module) void {
for (0..(module.functions.items.len)) |i| {
function :: &module.functions.items[i]
for (0..(function.blocks.items.len)) |j| {
block :: &function.blocks.items[j]
arraylist.deinit(&block.instructions)
arraylist.deinit(&block.arguments)
}
arraylist.deinit(&function.parameters)
arraylist.deinit(&function.locals)
arraylist.deinit(&function.blocks)
arraylist.deinit(&function.array_storage)
}
for (module.strings.items) |string|
mem.free(module.allocator, string.allocation)
arraylist.deinit(&module.functions)
arraylist.deinit(&module.globals)
arraylist.deinit(&module.strings)
arraylist.deinit(&module.arrays)
}
+46 -1
View File
@@ -22,10 +22,17 @@ SourceError :: enum { source_too_large }
keywords strmap.StaticStringMap(TokenKind) :: strmap.init([
{ "proc", .proc },
{ "return", .return },
{ "break", .break },
{ "continue", .continue },
{ "if", .if },
{ "for", .for },
{ "else", .else },
{ "while", .while },
{ "true", .true },
{ "false", .false },
{ "and", .and },
{ "or", .or },
{ "mut", .mut },
])
#! owns token storage and retains its allocator; release with deinit.
@@ -128,6 +135,18 @@ scan proc(
continue
}
if (char == '.' and cursor + 1 < program.len and program[cursor + 1] == '.') {
inclusive :: cursor + 2 < program.len and program[cursor + 2] == '='
width :: if (inclusive) 3 else 2
range_kind :: if (inclusive) .range_inclusive else .range
try add_token(state, Token{
span = make_span(cursor, cursor + width),
kind = range_kind,
})
cursor += width
continue
}
if char == ':' {
if cursor + 1 < program.len and program[cursor + 1] == ':' {
try add_token(state, Token{
@@ -203,9 +222,32 @@ scan proc(
continue
}
if (cursor + 1 < program.len and program[cursor + 1] == '=') {
comparison ?TokenKind :: match char {
'=': .equal_equal
'!': .not_equal
'<': .less_equal
'>': .greater_equal
else: null
}
if (comparison) |value| {
try add_token(state, Token{
span = make_span(cursor, cursor + 2),
kind = value,
})
cursor += 2
continue
}
}
kind ?TokenKind :: match char {
'=': .equal
'!': .bang
'<': .less
'>': .greater
',': .comma
'|': .pipe
'@': .at
'(': .open_paren
')': .close_paren
'[': .open_bracket
@@ -277,7 +319,10 @@ scan_number proc(start usize, program []u8) ScanNumResult ! ScanError {
cursor := start
has_decimal := false
while (cursor < program.len and is_digit(program[cursor])) cursor += 1
if cursor < program.len and program[cursor] == '.' {
if (
cursor < program.len and program[cursor] == '.'
and !(cursor + 1 < program.len and program[cursor + 1] == '.')
) {
has_decimal = true
cursor += 1
}
+112 -1
View File
@@ -11,7 +11,7 @@ reports_invalid_input_and_keeps_scanning test {
state := init(mem.c_allocator)
defer deinit(&state)
try scan(&state, "if name # comment\n@1. 2 3.5 \"hi\"", &diagnostic_store)
try scan(&state, "if name # comment\n$1. 2 3.5 \"hi\"", &diagnostic_store)
try testing.expect_equal(8, state.tokens.items.len)
try testing.expect_equal(TokenKind.if, state.tokens.items[0].kind)
@@ -30,3 +30,114 @@ reports_invalid_input_and_keeps_scanning test {
try testing.expect_equal(u32(3), state.tokens.items[1].span.start)
try testing.expect_equal(u32(7), state.tokens.items[1].span.end)
}
reserves_boolean_keywords_and_scans_comparisons test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
state := init(mem.c_allocator)
defer deinit(&state)
source :: "true false bool trueish == != < <= > >="
try scan(&state, source, &diagnostic_store)
expected [11]TokenKind :: [
TokenKind.true, .false, .ident, .ident,
.equal_equal, .not_equal, .less, .less_equal, .greater, .greater_equal, .eof,
]
try testing.expect_equal(expected.len, state.tokens.items.len)
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
try testing.expect_equal(u32(0), state.tokens.items[0].span.start)
try testing.expect_equal(u32(4), state.tokens.items[0].span.end)
try testing.expect_equal(u32(5), state.tokens.items[1].span.start)
try testing.expect_equal(u32(10), state.tokens.items[1].span.end)
try testing.expect_equal("bool", strpool.get_str(&strpool.strings, state.tokens.items[2].data.str_id)?)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
}
scans_logical_operators_without_splitting_identifiers test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
state := init(mem.c_allocator)
defer deinit(&state)
source :: "and or android orbit and_1 or2 ! != !! !true"
try scan(&state, source, &diagnostic_store)
expected [13]TokenKind :: [
TokenKind.and, .or, .ident, .ident, .ident, .ident,
.bang, .not_equal, .bang, .bang, .bang, .true, .eof,
]
try testing.expect_equal(expected.len, state.tokens.items.len)
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
try testing.expect_equal("android", strpool.get_str(&strpool.strings, state.tokens.items[2].data.str_id)?)
try testing.expect_equal("orbit", strpool.get_str(&strpool.strings, state.tokens.items[3].data.str_id)?)
try testing.expect_equal("and_1", strpool.get_str(&strpool.strings, state.tokens.items[4].data.str_id)?)
try testing.expect_equal("or2", strpool.get_str(&strpool.strings, state.tokens.items[5].data.str_id)?)
try testing.expect_equal(u32(31), state.tokens.items[6].span.start)
try testing.expect_equal(u32(32), state.tokens.items[6].span.end)
try testing.expect_equal(u32(33), state.tokens.items[7].span.start)
try testing.expect_equal(u32(35), state.tokens.items[7].span.end)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
}
scans_loop_controls_with_exact_keyword_boundaries test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
state := init(mem.c_allocator)
defer deinit(&state)
source :: "break continue breakfast continued break_1 continue2"
try scan(&state, source, &diagnostic_store)
expected [7]TokenKind :: [.break, .continue, .ident, .ident, .ident, .ident, .eof]
try testing.expect_equal(expected.len, state.tokens.items.len)
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
try testing.expect_equal(u32(0), state.tokens.items[0].span.start)
try testing.expect_equal(u32(5), state.tokens.items[0].span.end)
try testing.expect_equal(u32(6), state.tokens.items[1].span.start)
try testing.expect_equal(u32(14), state.tokens.items[1].span.end)
try testing.expect_equal("breakfast", strpool.get_str(&strpool.strings, state.tokens.items[2].data.str_id)?)
try testing.expect_equal("continued", strpool.get_str(&strpool.strings, state.tokens.items[3].data.str_id)?)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
}
scans_ranges_without_consuming_float_dots test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
state := init(mem.c_allocator)
defer deinit(&state)
try scan(&state, "0..10 0..=10 1.5..2.5 |@item, _| ... 1.", &diagnostic_store)
expected [19]TokenKind :: [
.int, .range, .int, .int, .range_inclusive, .int,
.float, .range, .float, .pipe, .at, .ident, .comma, .ident, .pipe,
.range, .invalid, .invalid, .eof,
]
try testing.expect_equal(expected.len, state.tokens.items.len)
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
try testing.expect_equal(u32(1), state.tokens.items[1].span.start)
try testing.expect_equal(u32(3), state.tokens.items[1].span.end)
try testing.expect_equal(u32(7), state.tokens.items[4].span.start)
try testing.expect_equal(u32(10), state.tokens.items[4].span.end)
try testing.expect_equal(2, diagnostics.count(&diagnostic_store))
}
scans_array_delimiters_and_mut_keyword test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
state := init(mem.c_allocator)
defer deinit(&state)
try scan(&state, "[_]mut [2]i64 [1,2] [] mutable mut_1", &diagnostic_store)
expected [18]TokenKind :: [
.open_bracket, .ident, .close_bracket, .mut,
.open_bracket, .int, .close_bracket, .ident,
.open_bracket, .int, .comma, .int, .close_bracket,
.open_bracket, .close_bracket, .ident, .ident, .eof,
]
try testing.expect_equal(expected.len, state.tokens.items.len)
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
}
+10
View File
@@ -21,6 +21,10 @@ TokenKind :: enum {
if, else
for, while
proc, return
break, continue
true, false
and, or
mut
# literals
ident, string
@@ -39,6 +43,12 @@ TokenKind :: enum {
plus, minus
star, slash
range, range_inclusive
pipe, at
# logical
bang
# assignment & arithmetic
plus_equal, minus_equal
star_equal, slash_equal
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+201 -47
View File
@@ -1,5 +1,8 @@
import "@std/debug"
import "@std/mem"
import "@std/arraylist"
import "@std/enummap"
import "@std/io"
import "@std/process"
import "@source/diagnostics"
import "@source/strpool"
@@ -7,8 +10,11 @@ import "@source/lexer"
import "@source/ast"
import "@source/parser"
import "@source/sema"
import "@source/ir"
import "@source/lower"
import "@source/interpreter"
ast_renderer :: import "@source/ast/renderer"
typecheck :: import "@source/sema/check"
test import "@std/mem"
test import "@std/enummap"
@@ -16,58 +22,206 @@ test import "@std/arraylist"
test import "@std/hashmap"
test import "@std/strmap"
test import "@std/meta"
test import "@std/io"
test import "@std/process"
test import "@std/strconv"
test import "@source/diagnostics"
test import "@source/strpool"
test import "@source/lexer"
test import "@source/ast"
test import "@source/ast/renderer"
test import "@source/parser"
test import "@source/sema"
test import "@source/sema/check"
test import "@source/lower"
test import "@source/interpreter"
@hide:file
program ::
`# literals
`x float :: 123.9
`y int := 43
`z :: "hello"
`
`a := 1 + 2 * 3
`b :: -3 / 2 + 1
`
`grouped proc(left, right int, extra float) void {
` value int :: later + left
` later int :: right
` empty()
` grouped(value, later, extra)
`}
`empty proc() void {}
DriverError :: alias (
mem.AllocError
| diagnostics.CapacityError
| lexer.SourceError
| strpool.InternError
| io.OpenError
| io.CloseError
| io.ReadError
| io.WriteError
)
main proc() void! {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, program, &diagnostic_store)
tree := try parser.parse(
mem.c_allocator,
scan_state.tokens.items,
program,
&diagnostic_store,
)
defer ast.deinit(&tree)
resolution := try sema.resolve(mem.c_allocator, &tree, &diagnostic_store)
defer sema.deinit(&resolution)
renderer := ast_renderer.init(mem.c_allocator)
defer ast_renderer.deinit(&renderer)
try ast_renderer.render(&renderer, &tree)
debug.print("{s}", {renderer.bytes.items})
debug.print("diagnostics: {}\n", {diagnostics.count(&diagnostic_store)})
main proc(init process.Init) i32 {
return command(mem.c_allocator, init.io, init.args) catch |err| {
if (init.args.values.len > 2) {
io.print("{s}: infrastructure error: {s}\n", io.stderr(init.io), {
init.args.values[2], enummap.get(&infrastructure_messages, err)?,
}) catch |_| {}
} else {
io.print("honey: infrastructure error: {s}\n", io.stderr(init.io), {
enummap.get(&infrastructure_messages, err)?,
}) catch |_| {}
}
return 2
}
}
@hide:file
infrastructure_messages :: $enummap.init(DriverError, []u8, {
out_of_memory = "out of memory",
diagnostic_limit_reached = "diagnostic limit reached",
storage_limit_reached = "aggregate storage limit reached",
source_too_large = "source file is too large",
out_of_space = "identifier storage is full",
open_failed = "cannot open source file",
close_failed = "cannot close source file",
not_open_for_reading = "source file is not readable",
read_failed = "cannot read source file",
not_open_for_writing = "output is not writable",
write_failed = "cannot write output",
no_progress = "output writer made no progress",
})
@hide:file
command proc(allocator mem.Allocator, runtime io.Io, args process.Args) i32 ! DriverError {
if (args.values.len != 3) {
try io.print("usage: honey run <file>\n", io.stderr(runtime), {})
return 2
}
if (!mem.eql(args.values[1], "run")) {
try io.print("usage: honey run <file>\n", io.stderr(runtime), {})
return 2
}
return try run_file(allocator, runtime, args.values[2])
}
@hide:file
read_source proc(allocator mem.Allocator, runtime io.Io, path [;0]u8) arraylist.ArrayList(u8) ! DriverError {
file :: try io.open(runtime, path, .read_only)
closing := false
defer {
if (!closing) io.close(file) catch |_| {}
}
bytes := io.read_all(allocator, io.reader(file), usize(maxval!(u32))) catch |err| {
match err {
.limit_exceeded: return .source_too_large
else: return err
}
}
errdefer arraylist.deinit(&bytes)
closing = true
try io.close(file)
return bytes
}
@hide:file
compile proc(allocator mem.Allocator, source []u8, diagnostic_store @mut diagnostics.Store) ir.Module ! DriverError {
strpool.strings = strpool.init(allocator)
defer strpool.deinit(&strpool.strings)
scan_state := lexer.init(allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, diagnostic_store)
tree := try parser.parse(allocator, scan_state.tokens.items, source, diagnostic_store)
defer ast.deinit(&tree)
resolution := try sema.resolve(allocator, &tree, diagnostic_store)
defer sema.deinit(&resolution)
checked := try typecheck.check(allocator, &tree, &resolution, diagnostic_store)
defer typecheck.deinit(&checked)
return try lower.lower(allocator, &tree, &resolution, &checked, diagnostic_store)
}
@hide:file
run_file proc(allocator mem.Allocator, runtime io.Io, path [;0]u8) i32 ! DriverError {
source := try read_source(allocator, runtime, path)
defer arraylist.deinit(&source)
diagnostic_store := diagnostics.init(allocator)
defer diagnostics.deinit(&diagnostic_store)
module := try compile(allocator, source.items, &diagnostic_store)
defer ir.deinit(&module)
outcome := try interpreter.run(allocator, &module, &diagnostic_store)
defer interpreter.deinit(&outcome)
if (outcome.trap == null) try print_result(io.stdout(runtime), &module, outcome.value)
try report_diagnostics(allocator, io.stderr(runtime), path, source.items, &diagnostic_store, outcome.trap)
if (outcome.trap != null or diagnostics.count(&diagnostic_store) != 0) return 1
return 0
}
@hide:file
print_result proc(output io.Writer, module @ir.Module, value ir.Value) void ! io.WriteError {
if (value.type == .void) return
try print_value(output, module, value)
try io.print("\n", output, {})
}
@hide:file
print_value proc(output io.Writer, module @ir.Module, value ir.Value) void ! io.WriteError {
match value.type {
.int: try io.print("{}", output, {value.data.integer})
.usize: try io.print("{}", output, {value.data.unsigned})
.float: try io.print("{d}", output, {value.data.real})
.bool: try io.print("{}", output, {value.data.boolean})
.string: try io.print("{s}", output, {ir.string_bytes(module, value.data.string)})
.array: {
shape :: module.arrays.items[usize(value.array_type)]
stride usize :: if (shape.element == .array)
module.arrays.items[usize(shape.element_array)].cells
else
1
try io.print("[", output, {})
for 0..(shape.len) |index| {
if (index != 0) try io.print(", ", output, {})
element := ir.Value{}
if shape.element == .array {
start :: index * stride
element = ir.Value{
type = .array,
array_type = shape.element_array,
data = ir.ValueData{ elements = value.data.elements[start..(start + stride)] },
}
} else {
element = value.data.elements[index]
}
try print_value(output, module, element)
}
try io.print("]", output, {})
}
else: unreachable
}
}
@hide:file
report_diagnostics proc(
allocator mem.Allocator,
output io.Writer,
path []u8,
source []u8,
diagnostic_store @diagnostics.Store,
trap ?diagnostics.DiagnosticId,
) void ! (mem.AllocError | io.WriteError) {
if (diagnostics.count(diagnostic_store) == 0) return
starts := arraylist.init(usize, allocator)
defer arraylist.deinit(&starts)
try arraylist.append(&starts, usize(0))
for (source) |byte, index| {
if (byte == '\n') try arraylist.append(&starts, index + 1)
}
for (0..diagnostics.count(diagnostic_store)) |index| {
if (diagnostics.get(diagnostic_store, diagnostics.DiagnosticId(index))) |diagnostic| {
offset :: usize(diagnostic.span.start)
line := usize(0)
end := starts.items.len
while (line + 1 < end) {
middle :: line + divtrunc!(end - line, 2)
if (starts.items[middle] <= offset) line = middle
else end = middle
}
label []u8 := "error"
if (trap) |id| {
if (id == diagnostics.DiagnosticId(index)) label = "runtime trap"
}
try io.print("{s}:{}:{}: {s}: {s}\n", output, {
path,
line + 1,
offset - starts.items[line] + 1,
label,
enummap.get(diagnostics.Code, []u8, &diagnostics.messages, diagnostic.code)?,
})
}
}
}
+606 -34
View File
@@ -27,6 +27,7 @@ State :: struct {
tokens []lexer.Token
allocator mem.Allocator
next_token usize
paren_depth usize = 0
}
#! borrows `tokens` and `source` while parsing; the returned Tree borrows `source` until ast.deinit.
@@ -102,7 +103,7 @@ parse_func_decl proc(state @mut State) ast.NodeId ! ParseError {
while true {
token :: try consume(state, .ident)
try arraylist.append(&names, token.span)
if next_is(state, .ident) break
if (next_is(state, .ident) or next_is(state, .open_bracket)) break
_ = try consume(state, .comma)
}
@@ -143,7 +144,7 @@ parse_block proc(state @mut State) ast.NodeId ! ParseError {
skip_newlines(state)
while !next_is(state, .close_curly) and !next_is(state, .eof) {
start :: state.next_token
statement :: parse_statement(state) catch |err| {
statement :: parse_statement(state, false) catch |err| {
match err {
.unexpected_token: {
recovered :: try recover_statement(state, start)
@@ -167,11 +168,12 @@ parse_block proc(state @mut State) ast.NodeId ! ParseError {
}
@hide:file
parse_statement proc(state @mut State) ast.NodeId ! ParseError {
parse_statement proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
if (
next_is(state, .ident)
and (
next_n_is(state, 1, .ident)
or (next_n_is(state, 1, .open_bracket) and array_starts_type(state))
or next_n_is(state, 1, .double_colon)
or next_n_is(state, 1, .colon_equal)
)
@@ -179,18 +181,293 @@ parse_statement proc(state @mut State) ast.NodeId ! ParseError {
return try parse_statement_decl(state)
}
match current(state).kind {
.ident, .int, .float, .string, .minus, .open_paren: {
.if: return try parse_if(state)
.while: return try parse_while(state, allow_else)
.for: return try parse_for(state, allow_else)
.return: return try parse_return(state, allow_else)
.break, .continue: return try parse_loop_control(state, allow_else)
.ident, .int, .float, .string, .true, .false, .minus, .bang, .open_paren, .open_bracket: {
expression :: try parse_expr(state)
if next_is(state, .equal) return try parse_assignment(state, expression, allow_else)
return try ast.add_expression_stmt(&state.builder, expression)
}
else: return .unexpected_token
}
}
@hide:file
parse_return proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
keyword :: try consume(state, .return)
if (
next_is(state, .newline) or next_is(state, .close_curly) or next_is(state, .eof)
or (allow_else and next_is(state, .else))
) {
return try ast.add_return_stmt(&state.builder, keyword.span, null)
}
value :: try parse_expr(state)
if (
!next_is(state, .newline) and !next_is(state, .close_curly) and !next_is(state, .eof)
and !(allow_else and next_is(state, .else))
) {
return .unexpected_token
}
# include consumed grouping punctuation, which has no separate expression node.
end :: state.tokens[state.next_token - 1].span.end
return try ast.add_return_stmt(
&state.builder,
ast.Span{ start = keyword.span.start, end = end },
value,
)
}
@hide:file
parse_loop_control proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
keyword :: current(state)
state.next_token += 1
if (
!next_is(state, .newline)
and !next_is(state, .close_curly)
and !next_is(state, .eof)
and !(allow_else and next_is(state, .else))
) return .unexpected_token
if (keyword.kind == .break)
return try ast.add_break_stmt(&state.builder, keyword.span)
return try ast.add_continue_stmt(&state.builder, keyword.span)
}
@hide:file
parse_assignment proc(state @mut State, target ast.NodeId, allow_else bool) ast.NodeId ! ParseError {
_ = try consume(state, .equal)
value :: try parse_expr(state)
if (
!next_is(state, .newline)
and !next_is(state, .close_curly)
and !next_is(state, .eof)
and !(allow_else and next_is(state, .else))
) {
return .unexpected_token
}
end :: state.tokens[state.next_token - 1].span.end
return try ast.add_assignment_stmt(
&state.builder,
ast.Span{ start = ast.builder_span(&state.builder, target).start, end = end },
target,
value,
)
}
@hide:file
parse_if proc(state @mut State) ast.NodeId ! ParseError {
keyword :: try consume(state, .if)
condition :: try parse_guard(state)
braced :: next_is(state, .open_curly)
then_block :: try parse_arm(state, true)
# only a braced arm permits its else on a later line.
after_then :: state.next_token
if (braced) skip_newlines(state)
alternative ?ast.NodeId := null
if next_is(state, .else) {
state.next_token += 1
alternative = try parse_arm(state, true)
} else {
state.next_token = after_then
}
end :: if (alternative) |block|
ast.builder_span(&state.builder, block).end
else
ast.builder_span(&state.builder, then_block).end
return try ast.add_if_stmt(
&state.builder,
ast.Span{ start = keyword.span.start, end = end },
condition,
then_block,
alternative,
)
}
@hide:file
parse_while proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
keyword :: try consume(state, .while)
condition :: try parse_guard(state)
body :: try parse_arm(state, allow_else)
return try ast.add_while_stmt(
&state.builder,
ast.Span{ start = keyword.span.start, end = ast.builder_span(&state.builder, body).end },
condition,
body,
)
}
@hide:file
CaptureList :: struct {
item ?ast.NodeId = null
index ?ast.NodeId = null
pointer bool = false
indexed bool = false
}
@hide:file
parse_capture_name proc(state @mut State) ?ast.NodeId ! ParseError {
name :: try consume(state, .ident)
capture ?ast.NodeId := null
if (
name.span.end != name.span.start + 1
or state.builder.source[usize(name.span.start)] != '_'
) capture = try ast.add_identifier(&state.builder, name.span)
return capture
}
@hide:file
parse_captures proc(state @mut State) CaptureList ! ParseError {
_ = try consume(state, .pipe)
captures := CaptureList{}
if next_is(state, .at) {
captures.pointer = true
state.next_token += 1
}
captures.item = try parse_capture_name(state)
if next_is(state, .comma) {
captures.indexed = true
state.next_token += 1
captures.index = try parse_capture_name(state)
}
_ = try consume(state, .pipe)
return captures
}
@hide:file
parse_for proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
keyword :: try consume(state, .for)
skip_newlines(state)
iterable_start :: state.next_token
iterable := try parse_expr(state)
grouped :: parenthesized_guard(state, iterable_start)
skip_newlines(state)
captures :: parse_captures(state) catch |err| {
match err {
.unexpected_token: {}
else: return err
}
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
iterable = try ast.add_invalid_expr(
&state.builder, ast.builder_span(&state.builder, iterable), diagnostic,
)
while (
!next_is(state, .open_curly) and !next_is(state, .close_curly)
and !next_is(state, .newline) and !next_is(state, .eof)
and !next_is(state, .else)
) state.next_token += 1
yield CaptureList{}
}
after_captures :: state.next_token
skip_newlines(state)
if (
!next_is(state, .open_curly) and !grouped
and ast.builder_expr_kind(&state.builder, iterable) != .call
and ast.builder_expr_kind(&state.builder, iterable) != .invalid
) {
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
iterable = try ast.add_invalid_expr(
&state.builder, ast.builder_span(&state.builder, iterable), diagnostic,
)
}
# a malformed capture list must not swallow the next line's sibling.
if (
ast.builder_expr_kind(&state.builder, iterable) == .invalid
and !next_is(state, .open_curly)
) state.next_token = after_captures
body :: try parse_arm(state, allow_else)
return try ast.add_for_stmt(
&state.builder,
ast.Span{ start = keyword.span.start, end = ast.builder_span(&state.builder, body).end },
iterable, captures.item, captures.index, captures.pointer, captures.indexed, body,
)
}
@hide:file
parse_guard proc(state @mut State) ast.NodeId ! ParseError {
condition_start :: state.next_token
condition := parse_expr(state) catch |err| {
match err {
.unexpected_token: {}
else: return err
}
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
yield try ast.add_invalid_expr(&state.builder, current(state).span, diagnostic)
}
if (ast.builder_expr_kind(&state.builder, condition) == .invalid) {
# discard a malformed guard's tail, but leave its body and enclosing block intact.
while (
!next_is(state, .open_curly) and !next_is(state, .close_curly)
and !next_is(state, .newline) and !next_is(state, .eof)
and !next_is(state, .else) and !next_is(state, .return)
and !next_is(state, .if) and !next_is(state, .while)
and !next_is(state, .for)
and !next_is(state, .break) and !next_is(state, .continue)
) {
if (next_is(state, .ident) and next_n_is(state, 1, .equal)) break
state.next_token += 1
}
}
if (
!next_is(state, .open_curly)
and !parenthesized_guard(state, condition_start)
and ast.builder_expr_kind(&state.builder, condition) != .call
and ast.builder_expr_kind(&state.builder, condition) != .invalid
) {
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
condition = try ast.add_invalid_expr(
&state.builder,
ast.builder_span(&state.builder, condition),
diagnostic,
)
}
return condition
}
@hide:file
parenthesized_guard proc(state @State, start usize) bool {
if (state.tokens[start].kind != .open_paren) return false
depth usize := 0
for start..state.next_token |index| {
kind :: state.tokens[index].kind
if (kind == .open_paren) depth += 1
if (kind == .close_paren) {
depth -= 1
if (depth == 0) return index + 1 == state.next_token
}
}
return false
}
@hide:file
parse_arm proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
if next_is(state, .open_curly) return try parse_block(state)
start :: state.next_token
statement :: parse_statement(state, allow_else) catch |err| {
match err {
.unexpected_token: {}
else: return err
}
diagnostic :: try unexpected_diagnostic(state)
span :: synchronize(state, start, true, true, false)
yield try ast.add_invalid_stmt(&state.builder, span, diagnostic)
}
statements [1]ast.NodeId :: [statement]
return try ast.add_block(
&state.builder,
ast.builder_span(&state.builder, statement),
statements[..],
)
}
@hide:file
parse_statement_decl proc(state @mut State) ast.NodeId ! ParseError {
name :: try consume(state, .ident)
type_id ?ast.NodeId :: if next_is(state, .ident)
type_id ?ast.NodeId :: if (next_is(state, .ident) or next_is(state, .open_bracket))
try parse_type(state)
else
null
@@ -211,9 +488,29 @@ parse_statement_decl proc(state @mut State) ast.NodeId ! ParseError {
@hide:file
parse_type proc(state @mut State) ast.NodeId ! ParseError {
if (!next_is(state, .ident)) return .unexpected_token
token :: current(state)
if next_is(state, .open_bracket) {
open :: try consume(state, .open_bracket)
count ?ast.NodeId := null
if (
next_is(state, .ident)
and current(state).span.end == current(state).span.start + 1
and state.builder.source[usize(current(state).span.start)] == '_'
) {
state.next_token += 1
} else {
count = try parse_expr(state)
}
_ = try consume(state, .close_bracket)
mutable :: next_is(state, .mut)
if (mutable) state.next_token += 1
element :: try parse_type(state)
return try ast.add_array_type(
&state.builder,
ast.Span{ start = open.span.start, end = ast.builder_span(&state.builder, element).end },
count, element, mutable,
)
}
token :: try consume(state, .ident)
return try ast.add_identifier(&state.builder, token.span)
}
@@ -224,25 +521,54 @@ parse_expr proc(state @mut State) ast.NodeId ! ParseError {
@hide:file
parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! ParseError {
skip_operand_newlines(state)
token := current(state)
expression_start :: state.next_token
left := if (prefix_binding_power(token.kind)) |right_bp| {
state.next_token += 1
operand :: try parse_expr_bp(state, right_bp)
operand_span :: ast.builder_span(&state.builder, operand)
operator ast.UnaryOperator :: if (token.kind == .bang) .logical_not else .negate
yield try ast.add_unary(
&state.builder,
ast.Span{ start = token.span.start, end = operand_span.end },
.negate,
operator,
operand,
)
} else {
yield try parse_primary(state)
}
left_grouped := parenthesized_guard(state, expression_start)
while true {
skip_continuation_newlines(state)
token = current(state)
if (token.kind == .open_paren) {
left = try parse_call(state, left)
left = try parse_call(state, left, state.tokens[expression_start].span.start)
left_grouped = false
continue
}
if (token.kind == .open_bracket) {
left = try parse_index(state, left, state.tokens[expression_start].span.start)
left_grouped = false
continue
}
if (token.kind == .range or token.kind == .range_inclusive) {
if (min_bp > BindingPowerScalar(0)) break
left = try validate_range_bound(state, left, left_grouped)
state.next_token += 1
right_start :: state.next_token
right := try parse_expr_bp(state, BindingPowerScalar(1))
right = try validate_range_bound(state, right, parenthesized_guard(state, right_start))
left = try ast.add_range(
&state.builder,
ast.Span{
start = state.tokens[expression_start].span.start,
end = state.tokens[state.next_token - 1].span.end,
},
left, right, token.kind == .range_inclusive,
)
left_grouped = false
continue
}
if (infix_binding_power(token.kind)) |power| {
@@ -258,6 +584,7 @@ parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! Par
left,
right,
)
left_grouped = false
continue
}
break
@@ -266,27 +593,41 @@ parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! Par
}
@hide:file
parse_call proc(state @mut State, callee ast.NodeId) ast.NodeId ! ParseError {
validate_range_bound proc(
state @mut State, bound ast.NodeId, grouped bool,
) ast.NodeId ! (mem.AllocError | diagnostics.CapacityError) {
if (grouped) return bound
match ast.builder_expr_kind(&state.builder, bound) {
.unary, .binary, .range: {
span :: ast.builder_span(&state.builder, bound)
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, span)
return try ast.add_invalid_expr(&state.builder, span, diagnostic)
}
else: return bound
}
}
@hide:file
parse_call proc(state @mut State, callee ast.NodeId, start u32) ast.NodeId ! ParseError {
_ = try consume(state, .open_paren)
state.paren_depth += 1
defer state.paren_depth -= 1
skip_operand_newlines(state)
arguments arraylist.ArrayList(ast.NodeId) := arraylist.init(state.allocator)
defer arraylist.deinit(&arguments)
while (
!next_is(state, .close_paren)
and !next_is(state, .newline)
and !next_is(state, .close_curly)
and !next_is(state, .eof)
) {
while (!is_call_boundary(current(state).kind)) {
argument :: try parse_expr(state)
try arraylist.append(&arguments, argument)
if (!next_is(state, .comma)) break
state.next_token += 1
skip_operand_newlines(state)
}
close :: try consume(state, .close_paren)
return try ast.add_call(
&state.builder,
ast.Span{
start = ast.builder_span(&state.builder, callee).start,
start = start,
end = close.span.end,
},
callee,
@@ -294,12 +635,112 @@ parse_call proc(state @mut State, callee ast.NodeId) ast.NodeId ! ParseError {
)
}
@hide:file
parse_index proc(state @mut State, base ast.NodeId, start u32) ast.NodeId ! ParseError {
_ = try consume(state, .open_bracket)
index :: try parse_expr(state)
if !next_is(state, .close_bracket) {
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
return try ast.add_invalid_expr(
&state.builder,
ast.Span{
start = start,
end = ast.builder_span(&state.builder, index).end,
},
diagnostic,
)
}
close :: try consume(state, .close_bracket)
return try ast.add_index(
&state.builder,
ast.Span{ start = start, end = close.span.end },
base, index,
)
}
@hide:file
array_starts_type proc(state @State) bool {
depth usize := 0
for (state.next_token)..(state.tokens.len) |i| {
kind :: state.tokens[i].kind
if (kind == .open_bracket) depth += 1
if (kind == .close_bracket) {
depth -= 1
if (depth == 0) {
if (i + 1 >= state.tokens.len) return false
next :: state.tokens[i + 1].kind
return next == .ident or next == .mut or next == .open_bracket
}
}
if (kind == .open_curly or kind == .close_curly or kind == .eof) return false
}
return false
}
@hide:file
parse_array_literal proc(state @mut State) ast.NodeId ! ParseError {
open :: try consume(state, .open_bracket)
elements arraylist.ArrayList(ast.NodeId) := arraylist.init(state.allocator)
defer arraylist.deinit(&elements)
after_open :: state.next_token
skip_newlines(state)
if named_binding_starts(state) {
state.next_token = after_open
return try invalid_array_literal(state, open.span.start, open.span.end)
}
while !next_is(state, .close_bracket) {
element :: try parse_expr(state)
try arraylist.append(&elements, element)
after_element :: state.next_token
skip_newlines(state)
if next_is(state, .close_bracket) break
if !next_is(state, .comma) {
state.next_token = after_element
return try invalid_array_literal(state, open.span.start, ast.builder_span(&state.builder, element).end)
}
state.next_token += 1
after_comma :: state.next_token
skip_newlines(state)
if named_binding_starts(state) {
state.next_token = after_comma
return try invalid_array_literal(state, open.span.start, ast.builder_span(&state.builder, element).end)
}
}
close :: try consume(state, .close_bracket)
return try ast.add_array_literal(
&state.builder, ast.Span{ start = open.span.start, end = close.span.end }, elements.items,
)
}
@hide:file
invalid_array_literal proc(state @mut State, start, end u32) ast.NodeId ! ParseError {
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
return try ast.add_invalid_expr(&state.builder, ast.Span{ start = start, end = end }, diagnostic)
}
@hide:file
named_binding_starts proc(state @State) bool {
return (
next_is(state, .ident)
and (
next_n_is(state, 1, .double_colon) or next_n_is(state, 1, .colon_equal)
or next_n_is(state, 1, .ident) or next_n_is(state, 1, .proc)
)
)
}
@hide:file
parse_primary proc(state @mut State) ast.NodeId ! ParseError {
token :: current(state)
match token.kind {
.open_bracket: {
if array_starts_type(state) return try parse_type(state)
return try parse_array_literal(state)
}
.open_paren: {
state.next_token += 1
state.paren_depth += 1
defer state.paren_depth -= 1
expression :: try parse_expr(state)
if next_is(state, .close_paren) {
state.next_token += 1
@@ -327,6 +768,10 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError {
state.next_token += 1
return try ast.add_literal(&state.builder, .string, token.span)
}
.true, .false: {
state.next_token += 1
return try ast.add_literal(&state.builder, .bool, token.span)
}
.ident: {
state.next_token += 1
return try ast.add_identifier(&state.builder, token.span)
@@ -337,13 +782,7 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError {
else
try diagnostics.add(state.diagnostics, .unexpected_token, token.span)
if (
token.kind != .newline
and token.kind != .comma
and token.kind != .close_paren
and token.kind != .close_curly
and token.kind != .eof
) {
if !is_expression_recovery_boundary(token.kind) {
state.next_token += 1
}
@@ -352,13 +791,70 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError {
}
}
@hide:file
is_expression_boundary proc(kind lexer.TokenKind) bool {
return match kind {
.close_paren, .newline, .close_curly, .open_curly, .else, .return, .if, .while, .for, .eof: true
else: false
}
}
@hide:file
is_call_boundary proc(kind lexer.TokenKind) bool {
return (
kind == .break
or kind == .continue
or is_expression_boundary(kind)
)
}
@hide:file
is_expression_recovery_boundary proc(kind lexer.TokenKind) bool {
return (
kind == .comma
or kind == .close_bracket
or kind == .and
or kind == .or
or kind == .pipe
or is_expression_boundary(kind)
)
}
@hide:file
skip_operand_newlines proc(state @mut State) void {
if (state.paren_depth == 0 or !next_is(state, .newline)) return
start :: state.next_token
skip_newlines(state)
kind :: current(state).kind
if (kind != .close_paren and is_expression_boundary(kind)) state.next_token = start
# a missing operand must not consume the declaration or assignment on the next line.
if (
named_binding_starts(state)
or (next_is(state, .ident) and next_n_is(state, 1, .equal))
) state.next_token = start
}
@hide:file
skip_continuation_newlines proc(state @mut State) void {
if (state.paren_depth == 0 or !next_is(state, .newline)) return
start :: state.next_token
skip_newlines(state)
kind :: current(state).kind
if (
kind != .close_paren and kind != .comma and kind != .open_paren
and kind != .close_bracket and kind != .open_bracket
and kind != .range and kind != .range_inclusive
and infix_binding_power(kind) == null
) state.next_token = start
}
@hide:file
recover_decl proc(
state @mut State,
start usize,
) ast.NodeId ! (mem.AllocError | diagnostics.CapacityError) {
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
span :: synchronize(state, start, false)
diagnostic :: try unexpected_diagnostic(state)
span :: synchronize(state, start, false, false, true)
return try ast.add_invalid_decl(&state.builder, span, diagnostic)
}
@@ -367,27 +863,91 @@ recover_statement proc(
state @mut State,
start usize,
) ast.NodeId ! (mem.AllocError | diagnostics.CapacityError) {
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
span :: synchronize(state, start, true)
diagnostic :: try unexpected_diagnostic(state)
span :: synchronize(state, start, true, false, true)
return try ast.add_invalid_stmt(&state.builder, span, diagnostic)
}
@hide:file
synchronize proc(state @mut State, start usize, stop_at_curly bool) ast.Span {
synchronize proc(
state @mut State, start usize, stop_at_curly, allow_else, consume_newline bool,
) ast.Span {
first :: state.tokens[start].span
end := first.end
while !next_is(state, .eof) and !next_is(state, .newline) {
curly usize := 0
paren usize := 0
square usize := 0
for (start..state.next_token) |i| {
kind :: state.tokens[i].kind
if (kind == .open_paren) paren += 1
if (kind == .close_paren and paren > 0) paren -= 1
if (kind == .open_bracket) square += 1
if (kind == .close_bracket and square > 0) square -= 1
}
while !next_is(state, .eof) {
if (curly == 0) {
if (stop_at_curly and next_is(state, .close_curly)) break
if (allow_else and next_is(state, .else)) break
if (
next_is(state, .newline)
and !recovery_closes_delimiters(state, paren, square)
) break
}
kind :: current(state).kind
if (kind == .open_curly) curly += 1
if (kind == .close_curly and curly > 0) curly -= 1
if (kind == .open_paren) paren += 1
if (kind == .close_paren and paren > 0) paren -= 1
if (kind == .open_bracket) square += 1
if (kind == .close_bracket and square > 0) square -= 1
end = current(state).span.end
state.next_token += 1
}
if next_is(state, .newline) {
if (consume_newline and next_is(state, .newline)) {
end = current(state).span.end
state.next_token += 1
}
return ast.Span{ start = first.start, end = end }
}
@hide:file
recovery_closes_delimiters proc(state @State, paren_depth, square_depth usize) bool {
if (paren_depth == 0 and square_depth == 0) return false
paren := paren_depth
square := square_depth
for (state.next_token)..(state.tokens.len) |i| {
kind :: state.tokens[i].kind
match kind {
.open_curly, .close_curly, .eof, .if, .else, .while, .for,
.return, .break, .continue: return false
.ident: {
if (i + 1 < state.tokens.len) {
next :: state.tokens[i + 1].kind
if (next == .double_colon or next == .colon_equal or next == .ident or next == .proc) {
return false
}
}
}
else: {}
}
if (kind == .open_paren) paren += 1
if (kind == .close_paren and paren > 0) paren -= 1
if (kind == .open_bracket) square += 1
if (kind == .close_bracket and square > 0) square -= 1
if (paren == 0 and square == 0) return true
}
return false
}
@hide:file
unexpected_diagnostic proc(state @mut State) diagnostics.DiagnosticId ! (mem.AllocError | diagnostics.CapacityError) {
token :: current(state)
if (token.kind == .invalid and token.data.diagnostic != diagnostics.no_diagnostic) {
return token.data.diagnostic
}
return try diagnostics.add(state.diagnostics, .unexpected_token, token.span)
}
@hide:file
skip_newlines proc(state @mut State) void {
while next_is(state, .newline) state.next_token += 1
@@ -431,7 +991,7 @@ consume_either proc(state @mut State, kinds []lexer.TokenKind) lexer.Token ! Syn
@hide:file
prefix_binding_power proc(kind lexer.TokenKind) ?BindingPowerScalar {
return match kind {
.minus: BindingPowerScalar(30)
.minus, .bang: BindingPowerScalar(50)
else: null
}
}
@@ -439,8 +999,12 @@ prefix_binding_power proc(kind lexer.TokenKind) ?BindingPowerScalar {
@hide:file
infix_binding_power proc(kind lexer.TokenKind) ?BindingPower {
return match kind {
.plus, .minus: BindingPower{ left = BindingPowerScalar(10), right = BindingPowerScalar(11) }
.star, .slash: BindingPower{ left = BindingPowerScalar(20), right = BindingPowerScalar(21) }
.or: BindingPower{ left = BindingPowerScalar(2), right = BindingPowerScalar(3) }
.and: BindingPower{ left = BindingPowerScalar(4), right = BindingPowerScalar(5) }
.equal_equal, .not_equal: BindingPower{ left = BindingPowerScalar(10), right = BindingPowerScalar(11) }
.less, .less_equal, .greater, .greater_equal: BindingPower{ left = BindingPowerScalar(20), right = BindingPowerScalar(21) }
.plus, .minus: BindingPower{ left = BindingPowerScalar(30), right = BindingPowerScalar(31) }
.star, .slash: BindingPower{ left = BindingPowerScalar(40), right = BindingPowerScalar(41) }
else: null
}
}
@@ -448,10 +1012,18 @@ infix_binding_power proc(kind lexer.TokenKind) ?BindingPower {
@hide:file
binary_operator proc(kind lexer.TokenKind) ast.BinaryOperator {
return match kind {
.and: .logical_and
.or: .logical_or
.plus: .add
.minus: .subtract
.star: .multiply
.slash: .divide
.equal_equal: .equal
.not_equal: .not_equal
.less: .less
.less_equal: .less_equal
.greater: .greater
.greater_equal: .greater_equal
else: unreachable
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+86 -12
View File
@@ -6,14 +6,23 @@ import "@std/strmap"
import "@source/ast"
import "@source/diagnostics"
BuiltinType :: enum { int, float, string, void }
BuiltinType :: enum {
int, float,
bool, string,
void, i64,
f64, usize,
}
@hide:file
builtins strmap.StaticStringMap(BuiltinType) :: strmap.init([
{ "int", .int },
{ "float", .float },
{ "bool", .bool },
{ "string", .string },
{ "void", .void },
{ "i64", .i64 },
{ "f64", .f64 },
{ "usize", .usize },
])
BindingKind :: enum { none, declaration, builtin, invalid }
@@ -175,18 +184,63 @@ resolve_function proc(
parameter :: ast.function_param(tree, id, i)
try predeclare(state, &scope, parameter, ast.param(tree, parameter).name)
}
for (0..ast.block_statement_count(tree, function.body)) |i| {
statement :: ast.block_statement(tree, function.body, i)
if ast.stmt_kind(tree, statement) == .declaration {
try predeclare(state, &scope, statement, ast.statement_decl(tree, statement).name)
}
}
try resolve_block(state, &scope, function.body)
}
for (0..ast.block_statement_count(tree, function.body)) |i| {
statement :: ast.block_statement(tree, function.body, i)
@hide:file
resolve_arm proc(state @mut State, parent @Scope, id ast.NodeId) void ! (mem.AllocError | diagnostics.CapacityError) {
scope := Scope{ names = hashmap.init(state.allocator), parent = parent }
defer hashmap.deinit(&scope.names)
try resolve_block(state, &scope, id)
}
@hide:file
resolve_block proc(state @mut State, scope @mut Scope, id ast.NodeId) void ! (mem.AllocError | diagnostics.CapacityError) {
tree :: state.tree
for (0..ast.block_statement_count(tree, id)) |i| {
statement :: ast.block_statement(tree, id, i)
if ast.stmt_kind(tree, statement) == .declaration {
try predeclare(state, scope, statement, ast.statement_decl(tree, statement).name)
}
}
for (0..ast.block_statement_count(tree, id)) |i| {
statement :: ast.block_statement(tree, id, i)
match ast.stmt_kind(tree, statement) {
.declaration: try resolve_statement_decl(state, &scope, statement)
.expression: try resolve_expr(state, &scope, ast.statement_expr(tree, statement))
.declaration: try resolve_statement_decl(state, scope, statement)
.expression: try resolve_expr(state, scope, ast.statement_expr(tree, statement))
.return_stmt: {
if (ast.return_stmt(tree, statement).value) |value| try resolve_expr(state, scope, value)
}
.break_stmt, .continue_stmt: {}
.if_stmt: {
conditional :: ast.if_stmt(tree, statement)
try resolve_expr(state, scope, conditional.condition)
try resolve_arm(state, scope, conditional.then_block)
if (conditional.else_block) |other| try resolve_arm(state, scope, other)
}
.assignment_stmt: {
assignment :: ast.assignment_stmt(tree, statement)
try resolve_expr(state, scope, assignment.target)
try resolve_expr(state, scope, assignment.value)
}
.while_stmt: {
loop :: ast.while_stmt(tree, statement)
try resolve_expr(state, scope, loop.condition)
try resolve_arm(state, scope, loop.body)
}
.for_stmt: {
loop :: ast.for_stmt(tree, statement)
try resolve_expr(state, scope, loop.iterable)
body_scope := Scope{ names = hashmap.init(state.allocator), parent = scope }
defer hashmap.deinit(&body_scope.names)
if (loop.capture) |capture| {
try predeclare(state, &body_scope, capture, ast.node_span(tree, capture))
}
if (loop.index) |index| {
try predeclare(state, &body_scope, index, ast.node_span(tree, index))
}
try resolve_block(state, &body_scope, loop.body)
}
.invalid: resolve_invalid(state, statement)
}
}
@@ -222,7 +276,7 @@ resolve_expr proc(
if (state.bindings[usize(id)].kind != .none) return
tree :: state.tree
match ast.expr_kind(tree, id) {
.literal_int, .literal_float, .literal_string: {}
.literal_int, .literal_float, .literal_bool, .literal_string: {}
.identifier: {
span :: ast.node_span(tree, id)
name :: ast.source_text(tree, span)
@@ -242,12 +296,32 @@ resolve_expr proc(
try resolve_expr(state, scope, expression.left)
try resolve_expr(state, scope, expression.right)
}
.range: {
expression :: ast.range_expr(tree, id)
try resolve_expr(state, scope, expression.start)
try resolve_expr(state, scope, expression.end)
}
.call: {
try resolve_expr(state, scope, ast.call_expr(tree, id).callee)
for (0..ast.call_arg_count(tree, id)) |i| {
try resolve_expr(state, scope, ast.call_arg(tree, id, i))
}
}
.array_type: {
type :: ast.array_type(tree, id)
if (type.count) |count| try resolve_expr(state, scope, count)
try resolve_expr(state, scope, type.element)
}
.array_literal: {
for (0..ast.array_element_count(tree, id)) |i| {
try resolve_expr(state, scope, ast.array_element(tree, id, i))
}
}
.index: {
expression :: ast.index_expr(tree, id)
try resolve_expr(state, scope, expression.base)
try resolve_expr(state, scope, expression.index)
}
.invalid: resolve_invalid(state, id)
}
}
+457
View File
@@ -298,3 +298,460 @@ preserves_syntax_diagnostics_and_resolves_shared_unknown_types_once test {
try testing.expect_equal(later, binding(&resolution, valid_ref).data.declaration)
try testing.expect_equal(syntax_count + 3, diagnostics.count(&diagnostic_store))
}
resolves_conditional_arm_scopes_without_leaks_or_shadowing test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`choose proc(flag bool, value int) void {
` if flag {
` before :: later
` later :: value
` if before == value { nested :: later }
` } else {
` later :: value
` }
` later
` if later { later :: 1 }
` if flag {
` value :: 2
` bool :: false
` if flag { value }
` }
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
function :: ast.package_decl(&tree, 0)
view :: ast.function_decl(&tree, function)
flag :: ast.function_param(&tree, function, 0)
value :: ast.function_param(&tree, function, 1)
try testing.expect_equal(BuiltinType.bool, binding(&resolution, ast.param(&tree, flag).type).data.builtin)
first :: ast.if_stmt(&tree, ast.block_statement(&tree, view.body, 0))
try testing.expect_equal(flag, binding(&resolution, first.condition).data.declaration)
before :: ast.block_statement(&tree, first.then_block, 0)
later :: ast.block_statement(&tree, first.then_block, 1)
try testing.expect_equal(later, binding(&resolution, ast.statement_decl(&tree, before).initializer).data.declaration)
try testing.expect_equal(value, binding(&resolution, ast.statement_decl(&tree, later).initializer).data.declaration)
nested :: ast.if_stmt(&tree, ast.block_statement(&tree, first.then_block, 2))
nested_decl :: ast.block_statement(&tree, nested.then_block, 0)
try testing.expect_equal(later, binding(&resolution, ast.statement_decl(&tree, nested_decl).initializer).data.declaration)
if (first.else_block) |other| {
sibling :: ast.block_statement(&tree, other, 0)
try testing.expect_equal(BindingKind.declaration, binding(&resolution, sibling).kind)
try testing.expect_equal(sibling, binding(&resolution, sibling).data.declaration)
} else {
return .expectation_failed
}
outside :: ast.statement_expr(&tree, ast.block_statement(&tree, view.body, 1))
try testing.expect_equal(BindingKind.invalid, binding(&resolution, outside).kind)
independent :: ast.if_stmt(&tree, ast.block_statement(&tree, view.body, 2))
try testing.expect_equal(BindingKind.invalid, binding(&resolution, independent.condition).kind)
try testing.expect_equal(BindingKind.declaration, binding(&resolution, ast.block_statement(&tree, independent.then_block, 0)).kind)
forbidden :: ast.if_stmt(&tree, ast.block_statement(&tree, view.body, 3))
shadow :: ast.block_statement(&tree, forbidden.then_block, 0)
builtin :: ast.block_statement(&tree, forbidden.then_block, 1)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, shadow).kind)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, builtin).kind)
poisoned :: ast.if_stmt(&tree, ast.block_statement(&tree, forbidden.then_block, 2))
use :: ast.statement_expr(&tree, ast.block_statement(&tree, poisoned.then_block, 0))
try testing.expect_equal(binding(&resolution, shadow).data.diagnostic, binding(&resolution, use).data.diagnostic)
try testing.expect_equal(4, diagnostics.count(&diagnostic_store))
}
resolves_loop_assignments_in_lexical_scopes_with_forward_local_bindings test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`main proc(flag bool) void {
` outer := 2
` while flag {
` later = outer
` later := 1
` while later > 0 {
` outer = later
` inner := later
` }
` inner
` }
` while flag { later := 3 }
` later = unknown_rhs
` while flag {
` outer := 1
` outer = 2
` bool := false
` }
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
function :: ast.package_decl(&tree, 0)
body :: ast.function_decl(&tree, function).body
flag :: ast.function_param(&tree, function, 0)
outer :: ast.block_statement(&tree, body, 0)
first :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 1))
try testing.expect_equal(flag, binding(&resolution, first.condition).data.declaration)
forward :: ast.assignment_stmt(&tree, ast.block_statement(&tree, first.body, 0))
later :: ast.block_statement(&tree, first.body, 1)
try testing.expect_equal(BindingKind.declaration, binding(&resolution, forward.target).kind)
try testing.expect_equal(later, binding(&resolution, forward.target).data.declaration)
try testing.expect_equal(outer, binding(&resolution, forward.value).data.declaration)
nested :: ast.while_stmt(&tree, ast.block_statement(&tree, first.body, 2))
guard :: ast.binary_expr(&tree, nested.condition)
try testing.expect_equal(later, binding(&resolution, guard.left).data.declaration)
update :: ast.assignment_stmt(&tree, ast.block_statement(&tree, nested.body, 0))
try testing.expect_equal(outer, binding(&resolution, update.target).data.declaration)
try testing.expect_equal(later, binding(&resolution, update.value).data.declaration)
inner :: ast.block_statement(&tree, nested.body, 1)
try testing.expect_equal(later, binding(&resolution, ast.statement_decl(&tree, inner).initializer).data.declaration)
escaped_inner :: ast.statement_expr(&tree, ast.block_statement(&tree, first.body, 3))
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped_inner).kind)
sibling :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 2))
sibling_local :: ast.block_statement(&tree, sibling.body, 0)
try testing.expect_equal(BindingKind.declaration, binding(&resolution, sibling_local).kind)
try testing.expect(sibling_local != later)
escaped :: ast.assignment_stmt(&tree, ast.block_statement(&tree, body, 3))
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped.target).kind)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped.value).kind)
try testing.expect(binding(&resolution, escaped.target).data.diagnostic != binding(&resolution, escaped.value).data.diagnostic)
forbidden :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 4))
shadow :: ast.block_statement(&tree, forbidden.body, 0)
poisoned :: ast.assignment_stmt(&tree, ast.block_statement(&tree, forbidden.body, 1))
builtin :: ast.block_statement(&tree, forbidden.body, 2)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, shadow).kind)
try testing.expect_equal(binding(&resolution, shadow).data.diagnostic, binding(&resolution, poisoned.target).data.diagnostic)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, builtin).kind)
try testing.expect_equal(5, diagnostics.count(&diagnostic_store))
}
resolves_all_code_and_preserves_scopes_after_loop_controls test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`main proc() void {
` while (true) {
` break
` before :: later
` continue
` later :: 1
` missing_after_break
` if (true) {
` continue
` private :: later
` int :: 2
` }
` private
` }
` later
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
try testing.expect_equal(4, diagnostics.count(&diagnostic_store))
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
loop :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 0))
before :: ast.block_statement(&tree, loop.body, 1)
later :: ast.block_statement(&tree, loop.body, 3)
reference :: ast.statement_decl(&tree, before).initializer
try testing.expect_equal(BindingKind.declaration, binding(&resolution, reference).kind)
try testing.expect_equal(later, binding(&resolution, reference).data.declaration)
conditional :: ast.if_stmt(&tree, ast.block_statement(&tree, loop.body, 5))
private_decl :: ast.block_statement(&tree, conditional.then_block, 1)
private_ref :: ast.statement_decl(&tree, private_decl).initializer
try testing.expect_equal(later, binding(&resolution, private_ref).data.declaration)
forbidden :: ast.block_statement(&tree, conditional.then_block, 2)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, forbidden).kind)
}
range_bounds_use_outer_scope_and_captures_are_body_declarations test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`run proc() void {
` for lower..upper |item| {
` before :: later
` later :: item
` continue
` item
` }
` for lower..upper |item| { item }
` lower :: 0
` upper :: 2
` item
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 0))
sibling :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 1))
lower :: ast.block_statement(&tree, body, 2)
upper :: ast.block_statement(&tree, body, 3)
bounds :: ast.range_expr(&tree, loop.iterable)
try testing.expect_equal(lower, binding(&resolution, bounds.start).data.declaration)
try testing.expect_equal(upper, binding(&resolution, bounds.end).data.declaration)
if (loop.capture) |capture| {
try testing.expect_equal(capture, binding(&resolution, capture).data.declaration)
later :: ast.block_statement(&tree, loop.body, 1)
before_ref :: ast.statement_decl(&tree, ast.block_statement(&tree, loop.body, 0)).initializer
try testing.expect_equal(later, binding(&resolution, before_ref).data.declaration)
capture_ref :: ast.statement_decl(&tree, later).initializer
try testing.expect_equal(capture, binding(&resolution, capture_ref).data.declaration)
dead_ref :: ast.statement_expr(&tree, ast.block_statement(&tree, loop.body, 3))
try testing.expect_equal(capture, binding(&resolution, dead_ref).data.declaration)
} else return .expectation_failed
if (sibling.capture) |capture| {
try testing.expect_equal(BindingKind.declaration, binding(&resolution, capture).kind)
sibling_ref :: ast.statement_expr(&tree, ast.block_statement(&tree, sibling.body, 0))
try testing.expect_equal(capture, binding(&resolution, sibling_ref).data.declaration)
} else return .expectation_failed
escaped :: ast.statement_expr(&tree, ast.block_statement(&tree, body, 4))
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped).kind)
try testing.expect_equal(1, diagnostics.count(&diagnostic_store))
}
range_captures_preserve_duplicate_builtin_and_discard_errors test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`run proc() void {
` outer :: 0
` for outer..2 |outer| { outer }
` for 0..2 |int| { int }
` for 0..2 |item, item| { item }
` for 0..2 |local| {
` local :: 1
` local
` }
` for 0..2 |_| {
` break
` _
` missing
` }
` for missing_start..missing_end |i| {}
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
for (usize(1)..usize(3)) |index| {
loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, index))
if (loop.capture) |capture| {
try testing.expect_equal(BindingKind.invalid, binding(&resolution, capture).kind)
reference :: ast.statement_expr(&tree, ast.block_statement(&tree, loop.body, 0))
try testing.expect_equal(
binding(&resolution, capture).data.diagnostic,
binding(&resolution, reference).data.diagnostic,
)
} else return .expectation_failed
}
indexed :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 3))
if (indexed.index) |index| {
try testing.expect_equal(BindingKind.invalid, binding(&resolution, index).kind)
} else return .expectation_failed
if (indexed.capture) |capture| {
reference :: ast.statement_expr(&tree, ast.block_statement(&tree, indexed.body, 0))
try testing.expect_equal(capture, binding(&resolution, reference).data.declaration)
} else return .expectation_failed
local_loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 4))
duplicate :: ast.block_statement(&tree, local_loop.body, 0)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, duplicate).kind)
discard :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 5))
try testing.expect(discard.capture == null)
for (usize(1)..usize(3)) |index| {
reference :: ast.statement_expr(&tree, ast.block_statement(&tree, discard.body, index))
try testing.expect_equal(BindingKind.invalid, binding(&resolution, reference).kind)
}
missing_loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 6))
bounds :: ast.range_expr(&tree, missing_loop.iterable)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, bounds.start).kind)
try testing.expect_equal(BindingKind.invalid, binding(&resolution, bounds.end).kind)
try testing.expect_equal(8, diagnostics.count(&diagnostic_store))
}
resolves_array_counts_elements_indices_and_unreachable_targets test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`size usize :: 2
`make proc(first, second [size]mut [2]i64) [size]i64 {
` return first[0]
` dead [local_size]mut [size - 1]f64 := [[later]]
` local_size :: size
` later f64 :: 1.0
` (dead[index])[0] = later
` index usize :: 0
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
size :: ast.package_decl(&tree, 0)
function :: ast.package_decl(&tree, 1)
view :: ast.function_decl(&tree, function)
return_type :: ast.array_type(&tree, view.return_type)
try testing.expect_equal(size, binding(&resolution, return_type.count?).data.declaration)
try testing.expect_equal(BuiltinType.i64, binding(&resolution, return_type.element).data.builtin)
first :: ast.function_param(&tree, function, 0)
second :: ast.function_param(&tree, function, 1)
first_type :: ast.param(&tree, first).type
try testing.expect_equal(first_type, ast.param(&tree, second).type)
parameter_array :: ast.array_type(&tree, first_type)
try testing.expect_equal(size, binding(&resolution, parameter_array.count?).data.declaration)
row :: ast.array_type(&tree, parameter_array.element)
try testing.expect_equal(BuiltinType.i64, binding(&resolution, row.element).data.builtin)
returned :: ast.return_stmt(&tree, ast.block_statement(&tree, view.body, 0)).value?
try testing.expect_equal(first, binding(&resolution, ast.index_expr(&tree, returned).base).data.declaration)
dead :: ast.block_statement(&tree, view.body, 1)
local_size :: ast.block_statement(&tree, view.body, 2)
later :: ast.block_statement(&tree, view.body, 3)
index :: ast.block_statement(&tree, view.body, 5)
declaration :: ast.statement_decl(&tree, dead)
array :: ast.array_type(&tree, declaration.type?)
try testing.expect_equal(local_size, binding(&resolution, array.count?).data.declaration)
inner :: ast.array_type(&tree, array.element)
count_expression :: ast.binary_expr(&tree, inner.count?)
try testing.expect_equal(size, binding(&resolution, count_expression.left).data.declaration)
try testing.expect_equal(BuiltinType.f64, binding(&resolution, inner.element).data.builtin)
literal_row :: ast.array_element(&tree, declaration.initializer, 0)
try testing.expect_equal(later, binding(
&resolution, ast.array_element(&tree, literal_row, 0),
).data.declaration)
assignment :: ast.assignment_stmt(&tree, ast.block_statement(&tree, view.body, 4))
target :: ast.index_expr(&tree, ast.index_expr(&tree, assignment.target).base)
try testing.expect_equal(dead, binding(&resolution, target.base).data.declaration)
try testing.expect_equal(index, binding(&resolution, target.index).data.declaration)
try testing.expect_equal(later, binding(&resolution, assignment.value).data.declaration)
try testing.expect_equal(BuiltinType.usize, binding(
&resolution, ast.statement_decl(&tree, size).type?,
).data.builtin)
}
array_signature_counts_use_package_scope_and_local_counts_see_parameters test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`size :: 2
`inspect proc(n usize, a, b [n]i64) [body_size]i64 {
` body_size :: size
` local [n]i64 :: []
` return []
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
try testing.expect_equal(2, diagnostics.count(&diagnostic_store))
function :: ast.package_decl(&tree, 1)
view :: ast.function_decl(&tree, function)
parameter :: ast.function_param(&tree, function, 0)
grouped_type :: ast.param(&tree, ast.function_param(&tree, function, 1)).type
try testing.expect_equal(grouped_type, ast.param(
&tree, ast.function_param(&tree, function, 2),
).type)
signature_count :: ast.array_type(&tree, grouped_type).count?
try testing.expect_equal(BindingKind.invalid, binding(&resolution, signature_count).kind)
result_count :: ast.array_type(&tree, view.return_type).count?
try testing.expect_equal(BindingKind.invalid, binding(&resolution, result_count).kind)
local :: ast.statement_decl(&tree, ast.block_statement(&tree, view.body, 1))
local_count :: ast.array_type(&tree, local.type?).count?
try testing.expect_equal(BindingKind.declaration, binding(&resolution, local_count).kind)
try testing.expect_equal(parameter, binding(&resolution, local_count).data.declaration)
try testing.expect_equal(BuiltinType.i64, binding(
&resolution, ast.array_type(&tree, local.type?).element,
).data.builtin)
}
resolves_every_array_child_error_and_preserves_lexer_diagnostics test {
strpool.strings = strpool.init(mem.c_allocator)
defer strpool.deinit(&strpool.strings)
diagnostic_store := diagnostics.init(mem.c_allocator)
defer diagnostics.deinit(&diagnostic_store)
source ::
`main proc() void {
` return
` data [missing_count]MissingElement :: [absent, missing_element]
` (missing_base[missing_index])[missing_outer] = missing_rhs
` malformed [$]i64 :: [$]
`}
scan_state := lexer.init(mem.c_allocator)
defer lexer.deinit(&scan_state)
try lexer.scan(&scan_state, source, &diagnostic_store)
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
defer ast.deinit(&tree)
try testing.expect_equal(2, diagnostics.count(&diagnostic_store))
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
defer deinit(&resolution)
try testing.expect_equal(10, diagnostics.count(&diagnostic_store))
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
declaration :: ast.statement_decl(&tree, ast.block_statement(&tree, body, 1))
type :: ast.array_type(&tree, declaration.type?)
assignment :: ast.assignment_stmt(&tree, ast.block_statement(&tree, body, 2))
outer :: ast.index_expr(&tree, assignment.target)
inner :: ast.index_expr(&tree, outer.base)
unknowns [8]ast.NodeId :: [
type.count?, type.element,
ast.array_element(&tree, declaration.initializer, 0),
ast.array_element(&tree, declaration.initializer, 1),
inner.base, inner.index, outer.index, assignment.value,
]
for (unknowns) |id| {
result :: binding(&resolution, id)
try testing.expect_equal(BindingKind.invalid, result.kind)
try testing.expect_equal(diagnostics.Code.unknown_name, diagnostics.get(
&diagnostic_store, result.data.diagnostic,
)?.code)
}
malformed :: ast.statement_decl(&tree, ast.block_statement(&tree, body, 3))
malformed_count :: ast.array_type(&tree, malformed.type?).count?
malformed_element :: ast.array_element(&tree, malformed.initializer, 0)
try testing.expect_equal(diagnostics.DiagnosticId(0), binding(&resolution, malformed_count).data.diagnostic)
try testing.expect_equal(diagnostics.DiagnosticId(1), binding(&resolution, malformed_element).data.diagnostic)
}
+2
View File
@@ -1,5 +1,7 @@
import "@std/meta"
#! keys may be enums or native enum compositions that normalize to an enum, including aliases.
#! shared variant names follow native composition identity; payload-bearing sums are not enum keys.
EnumMap proc($E, $V type) type {
match typeinfo!(E) {
.enum |info|: return struct {
+61
View File
@@ -24,3 +24,64 @@ handles_sparse_enum_get test {
try testing.expect_type(?[]u8, eof)
try testing.expect_equal(null, eof)
}
@hide:file
TestLeftError :: enum {
shared
left
absent_left
}
@hide:file
TestRightError :: enum {
shared
right
absent_right
}
@hide:file
TestNestedError :: alias (TestLeftError | (TestRightError | TestLeftError))
@hide:file
TestComposedError :: alias TestNestedError
@hide:file
composed_values :: $init(TestComposedError, i32, {
shared = 11,
left = 22,
absent_left = 33,
right = 44,
absent_right = 55,
})
handles_composed_enum_keys_at_runtime test {
names EnumMap(TestComposedError, []u8) :: init({
shared = "shared error",
left = "left error",
right = "right error",
})
keys [_]TestComposedError :: [
TestLeftError.shared,
TestLeftError.left,
TestRightError.shared,
TestRightError.right,
TestLeftError.absent_left,
TestRightError.absent_right,
]
expected [_]?[]u8 :: [
"shared error",
"left error",
"shared error",
"right error",
null,
null,
]
expected_values [_]i32 :: [11, 22, 11, 44, 33, 55]
for keys |key, index| {
value :: get(&names, key)
try testing.expect_type(?[]u8, value)
try testing.expect_equal(expected[index], value)
try testing.expect_equal(expected_values[index], get(&composed_values, key)?)
}
}
+1 -1
View File
@@ -19,7 +19,7 @@ CloseError :: enum {
close_failed
}
#! open a caller-owned handle; release it with close.
#! open a caller-owned handle from a borrowed zero-terminated path; release it with close.
open proc(io Io, path [;0]u8, mode FileMode) File ! OpenError {
handle Handle :: try io.vtable.open(io.context, path, mode)
return File{ io = io, handle = handle }
+36 -1
View File
@@ -1,5 +1,6 @@
import "@ffi/c"
import "@std/mem"
import "@std/arraylist"
# typeinfo! requires this import even without a qualified meta reference.
import "@std/meta"
@@ -8,6 +9,12 @@ ReadError :: enum {
read_failed
}
ReadLimitError :: enum {
limit_exceeded
}
ReadAllError :: alias (mem.AllocError | ReadError | ReadLimitError)
WriteError :: enum {
not_open_for_writing
write_failed
@@ -64,6 +71,34 @@ read proc(input Reader, buffer []mut u8) usize ! ReadError {
return count
}
#! read to EOF, accepting at most `maximum` bytes; the reader remains caller-owned.
#! exceeding the limit consumes one extra byte and returns limit_exceeded.
#! returned storage retains `allocator`; release it with arraylist.deinit.
#! all allocated storage is released on error.
read_all proc(allocator mem.Allocator, input Reader, maximum usize) arraylist.ArrayList(u8) ! ReadAllError {
bytes := arraylist.init(u8, allocator)
errdefer arraylist.deinit(&bytes)
while true {
length :: bytes.items.len
if (length == maximum) {
probe [1]mut u8 := undefined
if (try read(input, (&probe)[..]) != 0) return .limit_exceeded
return bytes
}
if (length == bytes.capacity) {
growth := usize(4096)
if (growth > maximum - length) growth = maximum - length
try arraylist.reserve(&bytes, length + growth)
}
storage :: bytes.items.ptr[..bytes.capacity]
end := bytes.capacity
if (end > maximum) end = maximum
count :: try read(input, storage[length..end])
if (count == 0) return bytes
bytes.items = storage[..length + count]
}
}
write proc(output Writer, bytes []u8) usize ! WriteError {
if (bytes.len == 0) return 0
@@ -392,7 +427,7 @@ write_float proc($T type, output Writer, value T, scientific bool) void ! WriteE
match typeinfo!(T) {
.float: {
buffer [64]mut u8 := undefined
count := 0
count c_int := 0
if sizeof!(T) == 4 {
if (scientific) count = c.snprintf(ptrcast!((&buffer).ptr), c_ulong(buffer.len), "%.8e", value)
else count = c.snprintf(ptrcast!((&buffer).ptr), c_ulong(buffer.len), "%.9g", value)
+127
View File
@@ -0,0 +1,127 @@
import "@std/mem"
import "@std/arraylist"
import "@std/testing"
@hide:file
Input :: struct {
bytes []u8
offset usize = 0
chunk usize = 137
fail_at usize = maxval!(usize)
remaining usize = maxval!(usize)
live usize = 0
}
@hide:file
input_state proc(context ?@mut anyopaque) @mut Input {
if (context) |pointer| return ptrcast!(pointer)
unreachable
}
@hide:file
input_read proc(context ?@mut anyopaque, _ Handle, buffer []mut u8) usize ! ReadError {
state :: input_state(context)
if (state.offset >= state.fail_at) return .read_failed
count := state.bytes.len - state.offset
if (count > buffer.len) count = buffer.len
if (count > state.chunk) count = state.chunk
memcopy!(buffer[..count], state.bytes[state.offset..state.offset + count])
state.offset += count
return count
}
@hide:file
input_reader proc(state @mut Input) Reader {
return Reader{ context = state, handle = Handle{ file_desc = -1 }, read = input_read }
}
@hide:file
tracked_alloc proc(context ?@mut anyopaque, size usize, alignment usize) ?*mut u8 {
state :: input_state(context)
if (state.remaining == 0) return null
state.remaining -= 1
memory :: mem.raw_alloc(mem.c_allocator, size, alignment)
if (memory) |_| state.live += 1
return memory
}
@hide:file
tracked_realloc proc(context ?@mut anyopaque, memory ?*mut u8, old_size usize, new_size usize, alignment usize) ?*mut u8 {
state :: input_state(context)
if (state.remaining == 0) return null
state.remaining -= 1
resized :: mem.raw_realloc(mem.c_allocator, memory, old_size, new_size, alignment)
if (resized) |_| {
if (memory) |_| {} else state.live += 1
}
return resized
}
@hide:file
tracked_free proc(context ?@mut anyopaque, memory ?*mut u8, size usize, alignment usize) void {
state :: input_state(context)
if (memory) |_| state.live -= 1
mem.raw_free(mem.c_allocator, memory, size, alignment)
}
@hide:file
tracked_vtable :: mem.AllocatorVTable{ alloc = tracked_alloc, realloc = tracked_realloc, free = tracked_free }
read_all_preserves_short_reads_across_buffer_growth test {
source [9000]mut u8 := undefined
for (0..source.len) |i| source[i] = u8(rem!(i, 251))
state := Input{ bytes = source[..] }
bytes := try read_all(mem.c_allocator, input_reader(&state), maxval!(usize))
defer arraylist.deinit(&bytes)
try testing.expect_equal(source[..], bytes.items)
}
read_all_accepts_exact_limits_and_probes_for_excess test {
for (usize(0)..4) |maximum| {
state := Input{ bytes = "abc"[..maximum], chunk = 1 }
bytes := try read_all(mem.c_allocator, input_reader(&state), maximum)
defer arraylist.deinit(&bytes)
try testing.expect_equal(state.bytes, bytes.items)
excess := Input{ bytes = "abcd", chunk = 1 }
unexpected := read_all(mem.c_allocator, input_reader(&excess), maximum) catch |err| {
try testing.expect(err == .limit_exceeded)
try testing.expect_equal(maximum + 1, excess.offset)
continue
}
arraylist.deinit(&unexpected)
try testing.expect(false)
}
}
read_all_releases_partial_storage_on_read_failure_and_limit_excess test {
for (0..2) |failure| {
state := Input{ bytes = "abcdef", chunk = 2 }
if (failure == 0) state.fail_at = 2
allocator :: mem.Allocator{ context = &state, vtable = &tracked_vtable }
unexpected := read_all(allocator, input_reader(&state), 4) catch |err| {
expected ReadAllError :: if (failure == 0) .read_failed else .limit_exceeded
try testing.expect_equal(expected, err)
try testing.expect_equal(0, state.live)
continue
}
arraylist.deinit(&unexpected)
try testing.expect(false)
}
}
read_all_releases_storage_at_each_failed_growth test {
source [9000]mut u8 := undefined
mem.set(source[..], u8('x'))
for (usize(0)..3) |remaining| {
state := Input{ bytes = source[..], remaining = remaining }
allocator :: mem.Allocator{ context = &state, vtable = &tracked_vtable }
unexpected := read_all(allocator, input_reader(&state), source.len) catch |err| {
try testing.expect(err == .out_of_memory)
try testing.expect_equal(0, state.live)
continue
}
arraylist.deinit(&unexpected)
try testing.expect(false)
}
}
+6
View File
@@ -2,4 +2,10 @@ import "@std/io"
Init :: struct {
io io.Io
args Args
}
#! borrows argument slices and their zero-terminated strings, valid until main returns.
Args :: struct {
values [][;0]u8
}
+67
View File
@@ -0,0 +1,67 @@
import "@ffi/c"
import "@std/mem"
ParseError :: enum {
invalid_number
out_of_range
}
#! parse a complete ASCII decimal with an optional sign, decimal point, and exponent.
#! at least one mantissa digit and, after an exponent marker, one exponent digit are required.
#! whitespace, embedded NUL bytes, trailing bytes, hexadecimal, infinities, and NaNs are invalid.
#! finite subnormals and signed zero succeed; overflow and nonzero values rounded to zero are out of range.
#! temporary storage uses `allocator` and is released before returning; the input is only borrowed.
#! the Darwin backend uses strtod_l with LC_C_LOCALE, independent of the process and thread locales.
#! decimal rounding follows native libc and the caller's floating-point rounding mode.
parse_float proc(allocator mem.Allocator, text []u8) f64 ! (mem.AllocError | ParseError) {
if (text.len == 0) return .invalid_number
cursor := usize(0)
if (text[cursor] == '+' or text[cursor] == '-') cursor += 1
digits := usize(0)
nonzero := false
point := false
while cursor < text.len {
byte :: text[cursor]
if (byte >= '0' and byte <= '9') {
digits += 1
if (byte != '0') nonzero = true
} else if (byte == '.' and !point) {
point = true
} else break
cursor += 1
}
if (digits == 0) return .invalid_number
if (cursor < text.len and (text[cursor] == 'e' or text[cursor] == 'E')) {
cursor += 1
if (cursor < text.len and (text[cursor] == '+' or text[cursor] == '-')) cursor += 1
start :: cursor
while (cursor < text.len and text[cursor] >= '0' and text[cursor] <= '9') cursor += 1
if (cursor == start) return .invalid_number
}
if (cursor != text.len) return .invalid_number
if (text.len == maxval!(usize)) return .out_of_memory
buffer :: try mem.alloc(u8, allocator, text.len + 1)
defer mem.free(allocator, buffer)
memcopy!(buffer[..text.len], text)
buffer[text.len] = 0
c.__error()?^ = 0
value :: f64(strtod_l(ptrcast!(buffer.ptr), null, null))
range_error :: c.__error()?^ == c.ERANGE
# binary64 magnitudes preserve subnormals despite libc's ERANGE; infinity starts at 9218868437227405312.
# ERANGE at magnitude >= 1 also catches overflow saturated by directed rounding.
magnitude :: ptrcast!(@u64, &value)^ & u64(9223372036854775807)
if (
magnitude >= u64(9218868437227405312)
or (magnitude == 0 and nonzero)
or (range_error and magnitude >= u64(4607182418800017408))
) return .out_of_range
return value
}
# Darwin's locale_t is an opaque pointer and LC_C_LOCALE is null (<_locale_posix2008.h>).
@hide:file
strtod_l c_func(text ?*c_char, end ?*mut ?*mut c_char, locale ?*mut anyopaque) c_double
+150
View File
@@ -0,0 +1,150 @@
import "@std/mem"
import "@std/testing"
@hide:file
expect_bits proc(text []u8, expected u64) void ! testing.Error {
value :: parse_float(mem.c_allocator, text) catch |_| { return .expectation_failed }
try testing.expect_equal(expected, ptrcast!(@u64, &value)^)
}
accepts_signed_decimal_forms_and_exponents test {
texts [7][]u8 :: ["1", "+1", "01", "1.", ".1e1", "10e-1", "1E+0"]
for (texts) |text| try testing.expect_equal(f64(1), try parse_float(mem.c_allocator, text))
try testing.expect_equal(f64(-1.25), try parse_float(mem.c_allocator, "-1.25"))
try testing.expect_equal(f64(0.5), try parse_float(mem.c_allocator, "+.5"))
try testing.expect_equal(f64(125), try parse_float(mem.c_allocator, "1.25e+2"))
try testing.expect_equal(f64(-0.125), try parse_float(mem.c_allocator, "-125E-3"))
}
accepts_borrowed_slices_without_a_terminator test {
bytes [5]mut u8 := ['x', '1', '.', '5', 'x']
try testing.expect_equal(f64(1.5), try parse_float(mem.c_allocator, bytes[1..4]))
try testing.expect_equal(u8('x'), bytes[0])
try testing.expect_equal(u8('x'), bytes[4])
}
rejects_malformed_or_partially_consumed_input test {
texts [33][]u8 :: [
"", "+", "-", ".", "+.", "-.", "e1", ".e1",
"1e", "1e+", "1e-", "1ee2", "1e1.0", "1.2.3", "--1", "+-1",
" 1", "1 ", "\t1", "1\n", "1x", "1_0", "1,5", "1\x00", "1\x002",
"0x1", "0x1p0", "inf", "-Infinity", "nan", "NaN(1)", "\xff", "1\xff",
]
for texts |text| {
_ = parse_float(mem.c_allocator, text) catch |error| {
try testing.expect(error == .invalid_number)
continue
}
try testing.expect(false)
}
}
preserves_signed_zero_even_with_extreme_exponents test {
positive [4][]u8 :: ["0", "+0.0", "0e999999", "0e-999999"]
negative [4][]u8 :: ["-0", "-0.0", "-0e999999", "-0e-999999"]
for positive |text| {
try expect_bits(text, u64(0))
}
for negative |text| {
try expect_bits(text, maxval!(u64) - u64(9223372036854775807))
}
}
rounds_decimal_midpoints_to_nearest_even test {
# IEEE-754 binary64 encodings: 0.1, 1, and the next two values above 1.
try expect_bits("0.1", u64(4591870180066957722))
try expect_bits("1.00000000000000011102230246251565404236316680908203125", u64(4607182418800017408))
try expect_bits("1.00000000000000011102230246251565404236316680908203126", u64(4607182418800017409))
try expect_bits("1.00000000000000033306690738754696212708950042724609375", u64(4607182418800017410))
try expect_bits("9007199254740993", u64(4845873199050653696))
}
accepts_finite_representational_boundaries_and_subnormals test {
# smallest subnormal, largest subnormal, smallest normal, and largest finite binary64.
try expect_bits("4.9406564584124654e-324", u64(1))
try expect_bits("2.4703282292062328e-324", u64(1))
try expect_bits("-4.9406564584124654e-324", maxval!(u64) - u64(9223372036854775806))
try expect_bits("2.225073858507201e-308", u64(4503599627370495))
try expect_bits("2.2250738585072014e-308", u64(4503599627370496))
try expect_bits("1.7976931348623157e308", u64(9218868437227405311))
try expect_bits("1.7976931348623158e308", u64(9218868437227405311))
try expect_bits("-1.7976931348623157e308", maxval!(u64) - u64(4503599627370496))
}
rejects_overflow_and_nonzero_underflow_to_zero test {
texts [12][]u8 :: [
"1e309", "-1e309", "1.7976931348623159e308", "-1.7976931348623159e308",
"1e999999", "-1e999999", "1e-999999", "-1e-999999", "1e-324", "-1e-324",
"2.4703282292062327e-324", "-2.4703282292062327e-324",
]
for texts |text| {
_ = parse_float(mem.c_allocator, text) catch |error| {
try testing.expect(error == .out_of_range)
continue
}
try testing.expect(false)
}
}
@hide:file
AllocationState :: struct {
remaining usize = maxval!(usize)
live usize = 0
}
@hide:file
tracked_alloc proc(context ?@mut anyopaque, size usize, alignment usize) ?*mut u8 {
state @mut AllocationState :: ptrcast!(context?)
if (state.remaining == 0) return null
state.remaining -= 1
memory :: mem.raw_alloc(mem.c_allocator, size, alignment)
if (memory) |_| state.live += 1
return memory
}
@hide:file
tracked_realloc proc(context ?@mut anyopaque, memory ?*mut u8, old_size usize, new_size usize, alignment usize) ?*mut u8 {
state @mut AllocationState :: ptrcast!(context?)
if (state.remaining == 0) return null
state.remaining -= 1
resized :: mem.raw_realloc(mem.c_allocator, memory, old_size, new_size, alignment)
if (new_size == 0) {
if (memory) |_| state.live -= 1
} else if (resized) |_| {
if (memory) |_| {} else state.live += 1
}
return resized
}
@hide:file
tracked_free proc(context ?@mut anyopaque, memory ?*mut u8, size usize, alignment usize) void {
state @mut AllocationState :: ptrcast!(context?)
if (memory) |_| state.live -= 1
mem.raw_free(mem.c_allocator, memory, size, alignment)
}
@hide:file
tracked_vtable :: mem.AllocatorVTable{ alloc = tracked_alloc, realloc = tracked_realloc, free = tracked_free }
releases_temporary_storage_and_propagates_allocation_failure test {
state := AllocationState{}
allocator :: mem.Allocator{ context = &state, vtable = &tracked_vtable }
try testing.expect_equal(f64(1.5), try parse_float(allocator, "1.5"))
try testing.expect_equal(usize(0), state.live)
texts [2][]u8 :: ["1e999999", "1e-999999"]
for texts |text| {
_ = parse_float(allocator, text) catch |error| {
try testing.expect(error == .out_of_range)
try testing.expect_equal(usize(0), state.live)
continue
}
try testing.expect(false)
}
state.remaining = 0
_ = parse_float(allocator, "1.5") catch |error| {
try testing.expect(error == .out_of_memory)
try testing.expect_equal(usize(0), state.live)
return
}
try testing.expect(false)
}