Compare commits
21 Commits
cb3f187b54
...
0593786553
| Author | SHA1 | Date | |
|---|---|---|---|
| 0593786553 | |||
| 5fd8d9eb89 | |||
| 9c1bd2a20e | |||
| d2cec370fa | |||
| c6a56b7e79 | |||
| cf39481bfc | |||
| d51ca57c9e | |||
| 0add5ab187 | |||
| 5f845b9520 | |||
| 7551ce5f6d | |||
| e9478dcf3f | |||
| b35a68dff8 | |||
| 021cbecac8 | |||
| 9932750e21 | |||
| 0821b84bc3 | |||
| c1e03632ed | |||
| 877ee21579 | |||
| 31c2cad9f9 | |||
| 98f3a04daf | |||
| bc318a29ad | |||
| bbf39c16c5 |
@@ -1,6 +1,6 @@
|
||||
# Honey
|
||||
|
||||
Honey is currently bootstrapped with the Bro compiler. the expected development workspace places the Bro project at [`../brolang`](../brolang); consult that project for compiler behavior and documentation.
|
||||
Bro and Honey implement the same language; Bro is the prototype compiler used to bootstrap the self-hosted Honey compiler. the features below describe Honey's currently implemented subset. the expected development workspace places Bro at [`../brolang`](../brolang); consult that project for the language grammar and documentation.
|
||||
|
||||
style: see the [language guidelines](docs/style.md).
|
||||
|
||||
@@ -10,10 +10,79 @@ build Honey from this directory:
|
||||
bro build
|
||||
```
|
||||
|
||||
the parser supports arithmetic expressions with nested parenthesized groups, such as `a + (b - c) * 2 / 3`. groups control precedence without adding AST nodes; inner expression spans exclude the grouping parentheses.
|
||||
run a source file with the interpreter:
|
||||
|
||||
function calls are postfix expressions: `f()`, `f(a, b)`, and `factory()(x)`. calls bind tighter than unary and binary arithmetic and can appear in initializers or as expression statements inside function bodies. argument lists allow a trailing comma; newlines remain statement boundaries. argument/type checking belongs to semantic analysis and is not yet implemented.
|
||||
```sh
|
||||
./build/honey run example.hon
|
||||
```
|
||||
|
||||
name resolution (`@source/sema`) runs after parsing. it predeclares package names and function-body locals, allowing forward references and mutually recursive functions. parameters and locals share a function scope; signatures resolve names in the enclosing package scope. shadowing is forbidden, including reuse of builtin type names (`int`, `float`, `string`, `void`); sibling function scopes may independently reuse names.
|
||||
```hon
|
||||
add proc(left, right int) int {
|
||||
return left + right
|
||||
}
|
||||
|
||||
`sema.resolve` returns an owned binding table indexed by AST `NodeId`, without modifying the AST. `sema.binding` identifies a declaration, builtin type, or invalid binding carrying its diagnostic ID; nodes with no binding return `.none`. release the result with `sema.deinit`. unknown names and duplicate declarations add source-spanned diagnostics; invalid syntax retains its existing diagnostic identity. same-scope duplicates retain the first declaration for recovery, while forbidden inner declarations invalidate that name's uses rather than silently binding them to an outer declaration. type checking and read-before-initialization checks remain separate, unimplemented stages.
|
||||
main proc() int {
|
||||
answer :: add(19, 23)
|
||||
return answer
|
||||
}
|
||||
```
|
||||
|
||||
this prints `42`. interpreted `main` must be a top-level function with no parameters; nonvoid results are printed, not used as process exit statuses.
|
||||
|
||||
the native bootstrap accepts either a no-argument `main` or `main proc(init process.Init)`. it supplies `init.io` and `init.args` from native startup `argc`/`argv`. startup scans each argument once into a heap-allocated array of borrowed zero-terminated slices (`process.Args.values: [][;0]u8`), without copying strings. the descriptor array and borrowed strings remain valid until `main` returns; startup then frees the descriptors, including on a fallible return. allocation failure exits with status `2`. `init.args.values.len` includes the executable name, and `init.args.values[index]` directly accesses an argument after a length guard. callers can supply their own `process.Args { values = slices[..] }` for testing.
|
||||
|
||||
source loading uses the supplied `io.Io` for opening, reading, and closing files. `io.open` accepts a borrowed zero-terminated slice (`[;0]u8`), including `init.args.values[index]` without copying its storage. the bootstrap supports checked sentinel-slice construction with `pointer[..length;0]`. `io.read_all(allocator, reader, maximum)` reads to EOF into a caller-owned `ArrayList(u8)`; release it with `arraylist.deinit`. it frees partial storage on error and consumes one extra byte to distinguish an exact-size input from a limit violation. the driver maps `limit_exceeded` to `source_too_large` at its `u32` source-size limit.
|
||||
|
||||
a recursive example is in [programs/factorial.hon](programs/factorial.hon); `./build/honey run programs/factorial.hon` prints `120`.
|
||||
|
||||
an iterative example is in [programs/iterative_factorial.hon](programs/iterative_factorial.hon); `./build/honey run programs/iterative_factorial.hon` also prints `120`.
|
||||
|
||||
the parser supports arithmetic expressions with nested parenthesized groups, such as `a + (b - c) * 2 / 3`. groups control precedence without adding AST nodes; inner expression spans exclude the grouping parentheses. groups may span lines in any expression position, including returns, initializers, arguments and guards. parsing accepts integer division syntax, but type checking requires float operands for `/`.
|
||||
|
||||
function calls are postfix expressions: `f()`, `f(a, b)`, and `factory()(x)`. calls bind tighter than unary and binary arithmetic and can appear in initializers or as expression statements inside function bodies. argument lists allow newlines and a trailing comma; outside parentheses, newlines remain statement boundaries. type checking validates callable values, argument counts and argument types.
|
||||
|
||||
`true` and `false` are boolean literals; `bool` is available for parameters, storage and results. `==`, `!=`, `<`, `<=`, `>` and `>=` compare compatible numeric operands and produce `bool`; `==` and `!=` also accept boolean operands. comparisons preserve contextual numeric inference without making their operands boolean. arithmetic binds tighter than relational comparisons, which bind tighter than equality; comparisons associate left to right. boolean arithmetic/ordering and string/function/void comparisons are invalid; there is no implicit numeric-to-boolean conversion.
|
||||
|
||||
`!`, `and` and `or` require boolean operands and produce `bool`. prefix `!` and `-` bind tighter than arithmetic; equality binds tighter than `and`, which binds tighter than `or`. binary operators associate left to right, while prefixes nest right to left. operands evaluate left to right: `false and rhs` and `true or rhs` skip the right operand, including calls, lazy global initialization and runtime traps. both operands are still parsed, resolved and type-checked; errors in a skipped operand remain source diagnostics without forcing a runtime trap. this applies in initializers, assignments, returns, call arguments and conditions. [programs/boolean_conditions.hon](programs/boolean_conditions.hon) demonstrates combined predicates; `./build/honey run programs/boolean_conditions.hon` prints `true`.
|
||||
|
||||
`if`/`else` supports braced arms, single statements and `else if` chains. conditions must have type `bool`. a braceless guard requires parentheses unless it is a function call: `if (n <= 1) return 1` or `if ready() return`. `else` binds to the nearest `if`; after a braced arm it may start on the next line. branches are represented as AST blocks even when braceless.
|
||||
|
||||
`name = expression` updates a local `:=` binding. `::` bindings, parameters, named functions, builtin types and globals are not assignment targets. the established binding type is preserved: assignment inputs do not widen or infer that type, though numeric literals can adopt its float context. function-valued locals accept replacement functions with the same parameter and result types.
|
||||
|
||||
fixed arrays use `[N]T` for readonly elements and `[N]mut T` for mutable elements; literals use `[a, b]`, including empty, nested and multiline forms with trailing commas. `values [4]mut i64 := [1, 2, 3, 4]` permits `values[2] = 10`. writable elements require a mutable local binding and `mut` at every traversed array level; parameters and globals remain readonly. whole-array assignment only requires a mutable binding, but source and destination types must match recursively, including counts and element mutability. initializations, assignments, arguments and returns copy arrays by value, including nested rows.
|
||||
|
||||
array lengths currently support nonnegative integer literals, unary signs, `+`, `-`, `*` and package-level immutable integer constants; local constants and other compile-time expressions are not implemented. `[_]` infers a binding's array length from its initializer. element annotations require concrete supported value types: use `i64` or `f64` rather than the `int` or `float` constraints. an unannotated array literal has readonly elements.
|
||||
|
||||
indices require `usize`; integer literals and flexible inferred values receive this context, while fixed signed integers do not convert implicitly. out-of-bounds access traps before loading or storing, including for empty arrays. reads evaluate the base before the index; indexed assignment evaluates its right-hand side before the destination, each once. leading nested assignment targets follow Bro's declaration ambiguity: write `(matrix[1])[0] = 9`, not `matrix[1][0] = 9`. [programs/array_sum.hon](programs/array_sum.hon) uses a `usize` range bound to index, mutate and sum an array; `./build/honey run programs/array_sum.hon` prints `17`.
|
||||
|
||||
`while` repeats a block or single statement while its `bool` condition holds, re-evaluating the condition on every iteration. guards follow the same braced/braceless convention as `if`. loop bodies introduce child scopes; their locals, including locals in nested conditional arms, begin uninitialized on each scope entry.
|
||||
|
||||
`for 0..10 |i| { ... }` iterates an ascending, upper-exclusive integer range; `..=` includes the upper bound. a capture is required: `|i|` binds an immutable value with the bounds' integer family (`int` or `usize`) in the body scope, while `|_|` discards it. bounds resolve in the enclosing scope and are evaluated left-to-right once, so changing a bound variable in the body does not change the range. reversed ranges are empty; equal bounds produce zero iterations with `..` and one with `..=`. inclusive ranges terminate safely at the maximum integer.
|
||||
|
||||
range bounds that are compound or unary expressions require their own parentheses, such as `(-2)..(limit + 1)`. a braceless body also requires parentheses around the entire iterable: `for (0..10) |i| total = total + i`. current execution supports direct integer ranges only; first-class range values, collection iteration, pointer captures and index captures remain unimplemented. [programs/range_loop.hon](programs/range_loop.hon) skips `5` while summing `0..10`; `./build/honey run programs/range_loop.hon` prints `40`.
|
||||
|
||||
`break` exits the nearest enclosing `for` or `while`; `continue` restarts a `while` condition or advances a `for` range through its guarded update. both are operand-free statements permitted within loop bodies and nested conditional arms; labels and values are unsupported. controls outside loops produce keyword-spanned diagnostics, including in untaken or unreachable code, and trap only if reached. code after a transfer is still resolved and type-checked but does not execute on that path. body locals are reset on re-entry after `continue`; loop transfers do not guarantee a function return. both controls lower to existing IR jumps, without new interpreter instructions or loop frames. [programs/loop_control.hon](programs/loop_control.hon) skips an iteration and exits early; `./build/honey run programs/loop_control.hon` prints `12`.
|
||||
|
||||
name resolution (`@source/sema`) runs after parsing. it predeclares package names and each block's direct locals, allowing forward references and mutually recursive functions. parameters and top-level function-body locals share a scope; each conditional arm and loop body introduces a child scope whose names do not escape it. signatures resolve names in the enclosing package scope. shadowing is forbidden, including reuse of builtin type names (`int`, `float`, `i64`, `f64`, `usize`, `bool`, `string`, `void`); sibling scopes may independently reuse names.
|
||||
|
||||
`sema.resolve` returns an owned binding table indexed by AST `NodeId`, without modifying the AST. `sema.binding` identifies a declaration, builtin type, or invalid binding carrying its diagnostic ID; nodes with no binding return `.none`. release the result with `sema.deinit`. unknown names and duplicate declarations add source-spanned diagnostics; invalid syntax retains its existing diagnostic identity. same-scope duplicates retain the first declaration for recovery, while forbidden inner declarations invalidate that name's uses rather than silently binding them to an outer declaration.
|
||||
|
||||
type checking (`@source/sema/check`) runs after name resolution. `check.check` borrows the AST and binding table and returns owned per-node types and interned array descriptors; query with `check.type_of` and `check.array_type`, and release with `check.deinit`. declarations infer their initializer types, including forward references and function aliases; function types identify the declaration supplying the callable signature. annotations support builtin types and fixed arrays; type names are not ordinary values. `void` is valid as a function result, not as a parameter or stored value.
|
||||
|
||||
the checker keeps union-find parent links and group sizes in a compact array separate from per-node inference metadata. root walks and path compression access only that link storage; both temporary arrays are released before `check.check` returns.
|
||||
|
||||
numeric typing follows contextual rules within Honey's current `int` and `float` families. integer literals can adopt a supported numeric family only when their value is exactly representable; float literals remain float and opt into ordinary IEEE rounding. numeric demands propagate through arithmetic and inferred bindings before undemanded integers default to `int`. conflicting demands are errors; fixed `int`/`i64`, `usize`, and `float`/`f64` values do not implicitly convert between families. `+`, `-`, `*` and unary `-` require compatible numeric operands; `/` requires and supplies float context. `3 / 2`, `3.0 / 2`, and `ratio float :: 3 / 2` are valid. `value f64 :: 9007199254740993` is diagnosed, including when float context arrives through aliases, division, or array elements; `9007199254740993.0` explicitly permits rounding. exactly representable integers beyond the signed/unsigned integer ranges remain valid in float context. explicit numeric casts are not yet implemented.
|
||||
|
||||
the conversion policy is value-preserving by default: any future implicit widening must preserve every value of the source type, not merely the value observed at one use. narrowing, potentially inexact integer-to-float conversion, float-to-integer conversion, and signedness changes that cannot preserve the full source range require an explicit operation. integer arithmetic remains checked rather than wrapping; floating arithmetic retains IEEE rounding.
|
||||
|
||||
`@std/strconv.parse_float(allocator, text)` parses a complete borrowed `[]u8` decimal into `f64`. it accepts an optional sign, decimal point, and exponent; whitespace, embedded NUL bytes, trailing input, hexadecimal, infinities, and NaNs return `invalid_number`. finite subnormals and signed zero are preserved; overflow and nonzero values rounded to zero return `out_of_range`. allocation failures propagate as `out_of_memory`, and temporary storage is always released. the Darwin backend uses libc's C-locale conversion without changing the process locale; rounding follows the caller's floating-point rounding mode. numeric lowering uses this interface, keeping exact integer-to-float validation compiler-specific.
|
||||
|
||||
type errors carry source spans and diagnostic identities; invalid operands propagate their existing identity without duplicate errors, while independent expressions, arguments and nested bodies are still checked. inference cycles without a type anchor are diagnosed. function signatures are checked independently of their bodies. bare `return` and same-line `return expression` are supported inside function bodies and must match the declared result type. nonvoid return-path checking is structural: every path must return or terminate in a diagnostic trap, and both conditional successors are considered without constant folding. a missing `else` or a loop permits fallthrough, even for a literal `true` condition; a later return can close that path. statements after a return and untaken branches or loops are still type-checked but are not executed.
|
||||
|
||||
`lower.lower` turns the checked AST and binding table into an owned `@source/ir` module. functions, globals, local slots and instruction-result temporaries have explicit IDs; typed instructions execute in source evaluation order and basic blocks end in jump, conditional-branch, return or diagnostic-trap terminators. conditional arms join only when execution can continue; loops use condition/body/exit blocks and backedges. short-circuit expressions use branches and synthetic boolean local slots to merge results; call arguments are packed in the block reached after evaluating them. local slots and temporaries are function-wide, without SSA or phi nodes. literals are parsed and strings decoded during lowering; the module retains no AST, binding-table or source-buffer references. release it with `ir.deinit`; keep and reuse the diagnostic store used during lowering so diagnostic IDs remain valid. the IR is target-independent and is intended to feed later native backends as well as the current interpreter.
|
||||
|
||||
array IR retains recursive element types, counts and mutability. typed aggregate construction, element addresses and load/store instructions use flattened storage reserved per invocation; nested copies and indexed accesses reuse that storage without per-iteration allocation. addresses are internal IR values, not source-language pointers.
|
||||
|
||||
`interpreter.run` consumes only IR and returns an `Outcome` containing a value or diagnostic trap. returned arrays own their escaped cell storage; release every outcome with `interpreter.deinit` before releasing its module. string values carry a module-local `ir.StringId`, not a slice; resolve bytes with `ir.string_bytes(&module, id)` before releasing the module. decoded bytes and their full backing allocations remain module-owned, without copying strings during calls or returns. array descriptors also borrow the module. heap-backed invocation frames track their current block and resume it after calls or global initialization; loop iterations reuse the invocation's storage without allocating new frames. parameters are initialized on entry and local declarations execute in statement order. premature local reads and assignments trap; assignment evaluates its right-hand side before checking the destination's initialization. globals initialize lazily on their first read, cache their result for that run, and trap on initialization cycles. `int`/`i64` use signed 64-bit storage, `usize` uses unsigned 64-bit storage, both with checked arithmetic; `float`/`f64` use 64-bit IEEE arithmetic and comparisons, including NaN behavior. `usize` supports `+`, `-`, `*` and comparisons, but not unary negation or division. numeric literal overflow and float underflow to zero are diagnosed; representable subnormals are accepted. strings support `\\`, `\"`, `\n`, `\r`, `\t`, `\0` and `\xNN` escapes.
|
||||
|
||||
recoverable source errors do not stop execution of unaffected code. unused invalid functions, globals and nested bodies retain diagnostic traps; reaching invalid code reports its original diagnostic identity. missing-return traps affect only paths that fall through, not paths that already return or trap. diagnostics include the filename, line and byte column. exit status is `0` for clean execution, `1` for source diagnostics or a runtime trap, and `2` for usage or infrastructure failure. native code generation and compound assignment remain unimplemented.
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# compile through recoverable source errors
|
||||
# execute through recoverable source errors
|
||||
|
||||
Honey produces an executable whenever source errors can be represented safely: structurally valid syntax trees may contain typed invalid nodes carrying source spans and diagnostic identities, and semantic checking or lowering turns demanded invalid code into diagnostic runtime traps. clean compilation returns status `0`, compilation that emits an executable with source diagnostics returns `1`, and unrecoverable infrastructure or backend failure returns `2`. this deliberately favors testing unaffected program paths over conventional fail-fast compilation; parser recovery must preserve valid surrounding syntax, while compact syntax-tree storage remains private behind typed views.
|
||||
Honey lowers recoverable source errors into typed, target-independent IR with source spans and diagnostic-bearing trap terminators; the interpreter executes that IR first, and future native backends reuse the same lowering rather than duplicating frontend semantics. invalid code traps only when execution reaches it, allowing unaffected program paths to run; parser recovery preserves surrounding syntax and compact syntax-tree storage remains private behind typed views. clean execution returns status `0`, source diagnostics or runtime traps return `1` even when an unaffected entry point finishes successfully, and usage or unrecoverable infrastructure failure returns `2`.
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
main proc() int {
|
||||
values [4]mut i64 := [1, 2, 3, 4]
|
||||
values[2] = 10
|
||||
|
||||
total := 0
|
||||
count usize :: 4
|
||||
for 0..count |index| {
|
||||
total = total + values[index]
|
||||
}
|
||||
|
||||
return total
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
in_range proc(value, low, high int) bool {
|
||||
return value >= low and value < high
|
||||
}
|
||||
|
||||
main proc() bool {
|
||||
return (
|
||||
in_range(5, 0, 10)
|
||||
and !in_range(12, 0, 10)
|
||||
or in_range(20, 0, 10)
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
factorial proc(n int) int {
|
||||
if (n <= 1) return 1
|
||||
return n * factorial(n - 1)
|
||||
}
|
||||
|
||||
main proc() int {
|
||||
return factorial(5)
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
main proc() int {
|
||||
product := 1
|
||||
remaining := 5
|
||||
|
||||
while remaining > 1 {
|
||||
product = product * remaining
|
||||
remaining = remaining - 1
|
||||
}
|
||||
|
||||
return product
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
main proc() int {
|
||||
number := 0
|
||||
total := 0
|
||||
|
||||
while true {
|
||||
number = number + 1
|
||||
if (number == 3) continue
|
||||
if (number > 5) break
|
||||
total = total + number
|
||||
}
|
||||
|
||||
return total
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
main proc() int {
|
||||
total := 0
|
||||
|
||||
for 0..10 |i| {
|
||||
if (i == 5) continue
|
||||
total = total + i
|
||||
}
|
||||
|
||||
return total
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
to_fahrenheit proc(celsius float) float { return celsius * 9 / 5 + 32 }
|
||||
|
||||
main proc() float {
|
||||
temperature :: 20
|
||||
return to_fahrenheit(temperature)
|
||||
}
|
||||
+345
-12
@@ -1,11 +1,16 @@
|
||||
#! grammar:
|
||||
#! * type ..... -> identifier
|
||||
#! * assignment -> :: | :=
|
||||
#! * literal .. -> int | float | string
|
||||
#! * op ....... -> '+' | '-' | '*' | '/'
|
||||
#! * expr ..... -> identifier | literal | '(' expr ')' | '-' expr | expr op expr | expr '(' [expr {',' expr} [',']] ')'
|
||||
#! * decl ..... -> identifier [type] assignment expr
|
||||
#! * stmt ..... -> decl | expr
|
||||
#! * type ..... -> identifier | '[' (expr | '_') ']' ['mut'] type
|
||||
#! * binding ... -> :: | :=
|
||||
#! * literal .. -> int | float | string | 'true' | 'false'
|
||||
#! * op ....... -> '+' | '-' | '*' | '/' | '==' | '!=' | '<' | '<=' | '>' | '>=' | 'and' | 'or'
|
||||
#! * expr ..... -> identifier | literal | type | '[' [expr {',' expr} [',']] ']' | '(' expr ')' | ('-' | '!') expr | expr op expr | expr '(' [expr {',' expr} [',']] ')' | expr '[' expr ']' | expr ('..' | '..=') expr
|
||||
#! * decl ..... -> identifier [type] binding expr
|
||||
#! * block .... -> '{' {stmt} '}'
|
||||
#! * arm ...... -> block | stmt (single statements become canonical blocks)
|
||||
#! * stmt ..... -> decl | expr | expr '=' expr | 'return' [expr] | 'break' | 'continue' | 'if' expr arm ['else' arm] | 'while' expr arm | 'for' expr '|' ['@'] identifier [',' identifier] '|' arm
|
||||
#! braceless guards/iterables require parentheses unless they are calls; else binds to the nearest if.
|
||||
#! precedence (tightest first): calls/indices, unary -/!, multiplication/division, addition/subtraction, relational comparisons, equality, and, or, ranges. compound range endpoints require their own parentheses.
|
||||
#! binary operators associate left; unary operators associate right.
|
||||
|
||||
import "@std/debug"
|
||||
import "@std/mem"
|
||||
@@ -22,17 +27,22 @@ NodeId :: distinct u32
|
||||
no_node :: maxval!(NodeId)
|
||||
|
||||
LiteralKind :: enum {
|
||||
int, float, string
|
||||
int, float, string, bool
|
||||
}
|
||||
|
||||
ExprKind :: enum {
|
||||
literal_int
|
||||
literal_float
|
||||
literal_string
|
||||
literal_bool
|
||||
identifier
|
||||
unary
|
||||
binary
|
||||
call
|
||||
range
|
||||
array_type
|
||||
array_literal
|
||||
index
|
||||
invalid
|
||||
}
|
||||
|
||||
@@ -41,17 +51,31 @@ DeclKind :: enum {
|
||||
}
|
||||
|
||||
StmtKind :: enum {
|
||||
declaration, expression, invalid
|
||||
declaration,
|
||||
expression,
|
||||
assignment_stmt,
|
||||
return_stmt,
|
||||
break_stmt,
|
||||
continue_stmt,
|
||||
if_stmt,
|
||||
while_stmt,
|
||||
for_stmt,
|
||||
invalid
|
||||
}
|
||||
|
||||
Mutability :: enum {
|
||||
constant, variable
|
||||
}
|
||||
|
||||
UnaryOperator :: enum { negate }
|
||||
UnaryOperator :: enum { negate, logical_not }
|
||||
|
||||
BinaryOperator :: enum {
|
||||
add, subtract, multiply, divide
|
||||
add, subtract,
|
||||
multiply, divide
|
||||
equal, not_equal,
|
||||
less, less_equal,
|
||||
greater, greater_equal
|
||||
logical_and, logical_or
|
||||
}
|
||||
|
||||
@hide:file
|
||||
@@ -59,12 +83,24 @@ NodeKind :: enum {
|
||||
literal_int,
|
||||
literal_float,
|
||||
literal_string,
|
||||
literal_bool,
|
||||
expr_identifier,
|
||||
expr_unary,
|
||||
expr_binary,
|
||||
expr_call,
|
||||
expr_range,
|
||||
expr_array_type,
|
||||
expr_array_literal,
|
||||
expr_index,
|
||||
stmt_expr,
|
||||
stmt_decl,
|
||||
stmt_return,
|
||||
stmt_break,
|
||||
stmt_continue,
|
||||
stmt_if,
|
||||
stmt_assignment,
|
||||
stmt_while,
|
||||
stmt_for,
|
||||
func_decl,
|
||||
param_decl,
|
||||
block,
|
||||
@@ -151,6 +187,60 @@ CallExprView :: struct {
|
||||
callee NodeId
|
||||
}
|
||||
|
||||
ArrayTypeView :: struct {
|
||||
span Span
|
||||
count ?NodeId
|
||||
element NodeId
|
||||
mutable bool
|
||||
}
|
||||
|
||||
IndexView :: struct {
|
||||
span Span
|
||||
base NodeId
|
||||
index NodeId
|
||||
}
|
||||
|
||||
ReturnView :: struct {
|
||||
span Span
|
||||
value ?NodeId
|
||||
}
|
||||
|
||||
IfView :: struct {
|
||||
span Span
|
||||
condition NodeId
|
||||
then_block NodeId
|
||||
else_block ?NodeId
|
||||
}
|
||||
|
||||
AssignmentView :: struct {
|
||||
span Span
|
||||
target NodeId
|
||||
value NodeId
|
||||
}
|
||||
|
||||
WhileView :: struct {
|
||||
span Span
|
||||
condition NodeId
|
||||
body NodeId
|
||||
}
|
||||
|
||||
RangeView :: struct {
|
||||
span Span
|
||||
start NodeId
|
||||
end NodeId
|
||||
inclusive bool
|
||||
}
|
||||
|
||||
ForView :: struct {
|
||||
span Span
|
||||
iterable NodeId
|
||||
capture ?NodeId
|
||||
index ?NodeId
|
||||
pointer_capture bool
|
||||
index_capture bool
|
||||
body NodeId
|
||||
}
|
||||
|
||||
InvalidView :: struct {
|
||||
span Span
|
||||
diagnostic DiagnosticId
|
||||
@@ -200,6 +290,7 @@ add_literal proc(builder @mut Builder, kind LiteralKind, span Span) NodeId ! mem
|
||||
.int: .literal_int
|
||||
.float: .literal_float
|
||||
.string: .literal_string
|
||||
.bool: .literal_bool
|
||||
}
|
||||
return try add_node(builder, Node{ kind = node_kind }, span)
|
||||
}
|
||||
@@ -236,6 +327,18 @@ add_binary proc(
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_range proc(
|
||||
builder @mut Builder, span Span, start NodeId, end NodeId, inclusive bool,
|
||||
) NodeId ! mem.AllocError {
|
||||
flag :: if (inclusive) u32(1) else u32(0)
|
||||
return try add_node(builder, Node{
|
||||
kind = .expr_range,
|
||||
main = NodeData{ extra = flag },
|
||||
data0 = NodeData{ node_id = start },
|
||||
data1 = NodeData{ node_id = end },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_call proc(
|
||||
builder @mut Builder,
|
||||
span Span,
|
||||
@@ -252,6 +355,37 @@ add_call proc(
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_array_type proc(
|
||||
builder @mut Builder, span Span, count ?NodeId, element NodeId, mutable bool,
|
||||
) NodeId ! mem.AllocError {
|
||||
raw_count :: if (count) |id| id else no_node
|
||||
flag :: if (mutable) u32(1) else u32(0)
|
||||
return try add_node(builder, Node{
|
||||
kind = .expr_array_type,
|
||||
main = NodeData{ extra = flag },
|
||||
data0 = NodeData{ node_id = raw_count },
|
||||
data1 = NodeData{ node_id = element },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_array_literal proc(builder @mut Builder, span Span, elements []NodeId) NodeId ! mem.AllocError {
|
||||
start :: builder.storage.extra.items.len
|
||||
for (elements) |element| _ = try add_extra(builder, u32(element))
|
||||
return try add_node(builder, Node{
|
||||
kind = .expr_array_literal,
|
||||
data0 = NodeData{ extra = u32(start) },
|
||||
data1 = NodeData{ extra = u32(builder.storage.extra.items.len) },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_index proc(builder @mut Builder, span Span, base, index NodeId) NodeId ! mem.AllocError {
|
||||
return try add_node(builder, Node{
|
||||
kind = .expr_index,
|
||||
data0 = NodeData{ node_id = base },
|
||||
data1 = NodeData{ node_id = index },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_expression_stmt proc(builder @mut Builder, expression NodeId) NodeId ! mem.AllocError {
|
||||
return try add_node(builder, Node{
|
||||
kind = .stmt_expr,
|
||||
@@ -259,6 +393,84 @@ add_expression_stmt proc(builder @mut Builder, expression NodeId) NodeId ! mem.A
|
||||
}, builder_span(builder, expression))
|
||||
}
|
||||
|
||||
add_return_stmt proc(builder @mut Builder, span Span, value ?NodeId) NodeId ! mem.AllocError {
|
||||
raw_value :: if (value) |id| id else no_node
|
||||
return try add_node(builder, Node{
|
||||
kind = .stmt_return,
|
||||
main = NodeData{ node_id = raw_value },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_break_stmt proc(builder @mut Builder, span Span) NodeId ! mem.AllocError {
|
||||
return try add_node(builder, Node{ kind = .stmt_break }, span)
|
||||
}
|
||||
|
||||
add_continue_stmt proc(builder @mut Builder, span Span) NodeId ! mem.AllocError {
|
||||
return try add_node(builder, Node{ kind = .stmt_continue }, span)
|
||||
}
|
||||
|
||||
add_if_stmt proc(
|
||||
builder @mut Builder,
|
||||
span Span,
|
||||
condition NodeId,
|
||||
then_block NodeId,
|
||||
else_block ?NodeId,
|
||||
) NodeId ! mem.AllocError {
|
||||
alternative :: if (else_block) |id| id else no_node
|
||||
return try add_node(builder, Node{
|
||||
kind = .stmt_if,
|
||||
main = NodeData{ node_id = condition },
|
||||
data0 = NodeData{ node_id = then_block },
|
||||
data1 = NodeData{ node_id = alternative },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_assignment_stmt proc(
|
||||
builder @mut Builder,
|
||||
span Span,
|
||||
target NodeId,
|
||||
value NodeId,
|
||||
) NodeId ! mem.AllocError {
|
||||
return try add_node(builder, Node{
|
||||
kind = .stmt_assignment,
|
||||
main = NodeData{ node_id = target },
|
||||
data0 = NodeData{ node_id = value },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_while_stmt proc(
|
||||
builder @mut Builder,
|
||||
span Span,
|
||||
condition NodeId,
|
||||
body NodeId,
|
||||
) NodeId ! mem.AllocError {
|
||||
return try add_node(builder, Node{
|
||||
kind = .stmt_while,
|
||||
main = NodeData{ node_id = condition },
|
||||
data0 = NodeData{ node_id = body },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_for_stmt proc(
|
||||
builder @mut Builder, span Span, iterable NodeId, capture, index ?NodeId,
|
||||
pointer_capture, index_capture bool, body NodeId,
|
||||
) NodeId ! mem.AllocError {
|
||||
raw_capture :: if (capture) |id| u32(id) else u32(no_node)
|
||||
raw_index :: if (index) |id| u32(id) else u32(no_node)
|
||||
pointer_flag :: if (pointer_capture) u32(1) else u32(0)
|
||||
index_flag :: if (index_capture) u32(1) else u32(0)
|
||||
extra :: try add_extra(builder, raw_capture)
|
||||
_ = try add_extra(builder, raw_index)
|
||||
_ = try add_extra(builder, pointer_flag)
|
||||
_ = try add_extra(builder, index_flag)
|
||||
return try add_node(builder, Node{
|
||||
kind = .stmt_for,
|
||||
main = NodeData{ node_id = iterable },
|
||||
data0 = NodeData{ node_id = body },
|
||||
data1 = NodeData{ extra = extra },
|
||||
}, span)
|
||||
}
|
||||
|
||||
add_statement_decl proc(
|
||||
builder @mut Builder,
|
||||
span Span,
|
||||
@@ -393,6 +605,13 @@ stmt_kind proc(tree @Tree, id NodeId) StmtKind {
|
||||
return match get_node(&tree.storage, id).kind {
|
||||
.stmt_decl: .declaration
|
||||
.stmt_expr: .expression
|
||||
.stmt_return: .return_stmt
|
||||
.stmt_break: .break_stmt
|
||||
.stmt_continue: .continue_stmt
|
||||
.stmt_if: .if_stmt
|
||||
.stmt_assignment: .assignment_stmt
|
||||
.stmt_while: .while_stmt
|
||||
.stmt_for: .for_stmt
|
||||
.invalid_stmt: .invalid
|
||||
else: unreachable
|
||||
}
|
||||
@@ -404,6 +623,61 @@ statement_expr proc(tree @Tree, id NodeId) NodeId {
|
||||
return node.main.node_id
|
||||
}
|
||||
|
||||
return_stmt proc(tree @Tree, id NodeId) ReturnView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .stmt_return)
|
||||
return ReturnView{
|
||||
span = get_span(&tree.storage, id),
|
||||
value = maybe_node_id(u32(node.main.node_id)),
|
||||
}
|
||||
}
|
||||
|
||||
if_stmt proc(tree @Tree, id NodeId) IfView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .stmt_if)
|
||||
return IfView{
|
||||
span = get_span(&tree.storage, id),
|
||||
condition = node.main.node_id,
|
||||
then_block = node.data0.node_id,
|
||||
else_block = maybe_node_id(u32(node.data1.node_id)),
|
||||
}
|
||||
}
|
||||
|
||||
assignment_stmt proc(tree @Tree, id NodeId) AssignmentView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .stmt_assignment)
|
||||
return AssignmentView{
|
||||
span = get_span(&tree.storage, id),
|
||||
target = node.main.node_id,
|
||||
value = node.data0.node_id,
|
||||
}
|
||||
}
|
||||
|
||||
while_stmt proc(tree @Tree, id NodeId) WhileView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .stmt_while)
|
||||
return WhileView{
|
||||
span = get_span(&tree.storage, id),
|
||||
condition = node.main.node_id,
|
||||
body = node.data0.node_id,
|
||||
}
|
||||
}
|
||||
|
||||
for_stmt proc(tree @Tree, id NodeId) ForView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .stmt_for)
|
||||
extra :: usize(node.data1.extra)
|
||||
return ForView{
|
||||
span = get_span(&tree.storage, id),
|
||||
iterable = node.main.node_id,
|
||||
capture = maybe_node_id(tree.storage.extra.items[extra]),
|
||||
index = maybe_node_id(tree.storage.extra.items[extra + 1]),
|
||||
pointer_capture = tree.storage.extra.items[extra + 2] != 0,
|
||||
index_capture = tree.storage.extra.items[extra + 3] != 0,
|
||||
body = node.data0.node_id,
|
||||
}
|
||||
}
|
||||
|
||||
statement_decl proc(tree @Tree, id NodeId) StatementDeclView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .stmt_decl)
|
||||
@@ -467,14 +741,28 @@ block_statement proc(tree @Tree, id NodeId, index usize) NodeId {
|
||||
}
|
||||
|
||||
expr_kind proc(tree @Tree, id NodeId) ExprKind {
|
||||
return match get_node(&tree.storage, id).kind {
|
||||
return expression_kind(get_node(&tree.storage, id).kind)
|
||||
}
|
||||
|
||||
builder_expr_kind proc(builder @Builder, id NodeId) ExprKind {
|
||||
return expression_kind(get_node(&builder.storage, id).kind)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
expression_kind proc(kind NodeKind) ExprKind {
|
||||
return match kind {
|
||||
.literal_int: .literal_int
|
||||
.literal_float: .literal_float
|
||||
.literal_string: .literal_string
|
||||
.literal_bool: .literal_bool
|
||||
.expr_identifier: .identifier
|
||||
.expr_unary: .unary
|
||||
.expr_binary: .binary
|
||||
.expr_call: .call
|
||||
.expr_range: .range
|
||||
.expr_array_type: .array_type
|
||||
.expr_array_literal: .array_literal
|
||||
.expr_index: .index
|
||||
.invalid_expr: .invalid
|
||||
else: unreachable
|
||||
}
|
||||
@@ -501,6 +789,17 @@ binary_expr proc(tree @Tree, id NodeId) BinaryExprView {
|
||||
}
|
||||
}
|
||||
|
||||
range_expr proc(tree @Tree, id NodeId) RangeView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .expr_range)
|
||||
return RangeView{
|
||||
span = get_span(&tree.storage, id),
|
||||
start = node.data0.node_id,
|
||||
end = node.data1.node_id,
|
||||
inclusive = node.main.extra != 0,
|
||||
}
|
||||
}
|
||||
|
||||
call_expr proc(tree @Tree, id NodeId) CallExprView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .expr_call)
|
||||
@@ -523,6 +822,40 @@ call_arg proc(tree @Tree, id NodeId, index usize) NodeId {
|
||||
return NodeId(tree.storage.extra.items[usize(node.data0.extra) + index])
|
||||
}
|
||||
|
||||
array_type proc(tree @Tree, id NodeId) ArrayTypeView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .expr_array_type)
|
||||
return ArrayTypeView{
|
||||
span = get_span(&tree.storage, id),
|
||||
count = maybe_node_id(u32(node.data0.node_id)),
|
||||
element = node.data1.node_id,
|
||||
mutable = node.main.extra != 0,
|
||||
}
|
||||
}
|
||||
|
||||
array_element_count proc(tree @Tree, id NodeId) usize {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .expr_array_literal)
|
||||
return usize(node.data1.extra - node.data0.extra)
|
||||
}
|
||||
|
||||
array_element proc(tree @Tree, id NodeId, index usize) NodeId {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .expr_array_literal)
|
||||
debug.assert(index < usize(node.data1.extra - node.data0.extra))
|
||||
return NodeId(tree.storage.extra.items[usize(node.data0.extra) + index])
|
||||
}
|
||||
|
||||
index_expr proc(tree @Tree, id NodeId) IndexView {
|
||||
node :: get_node(&tree.storage, id)
|
||||
debug.assert(node.kind == .expr_index)
|
||||
return IndexView{
|
||||
span = get_span(&tree.storage, id),
|
||||
base = node.data0.node_id,
|
||||
index = node.data1.node_id,
|
||||
}
|
||||
}
|
||||
|
||||
node_span proc(tree @Tree, id NodeId) Span {
|
||||
return get_span(&tree.storage, id)
|
||||
}
|
||||
|
||||
@@ -32,3 +32,35 @@ exposes_typed_declaration_views test {
|
||||
try testing.expect_equal(ExprKind.literal_int, expr_kind(&tree, statement.initializer))
|
||||
try testing.expect_equal("value", source_text(&tree, statement.name))
|
||||
}
|
||||
|
||||
exposes_array_type_literal_and_index_views test {
|
||||
builder := init_builder(mem.c_allocator, "")
|
||||
defer deinit_builder(&builder)
|
||||
span :: Span{ start = 0, end = 0 }
|
||||
count :: try add_literal(&builder, .int, span)
|
||||
element :: try add_identifier(&builder, span)
|
||||
row :: try add_array_type(&builder, span, count, element, false)
|
||||
matrix :: try add_array_type(&builder, span, null, row, true)
|
||||
elements [2]NodeId :: [count, element]
|
||||
literal :: try add_array_literal(&builder, span, elements[..])
|
||||
empty :: try add_array_literal(&builder, span, &[])
|
||||
indexed :: try add_index(&builder, span, literal, count)
|
||||
package :: try add_package(&builder, span, &[])
|
||||
tree := finish(&builder, package)
|
||||
defer deinit(&tree)
|
||||
try testing.expect_equal(ExprKind.array_type, expr_kind(&tree, matrix))
|
||||
try testing.expect_equal(null, array_type(&tree, matrix).count)
|
||||
try testing.expect(array_type(&tree, matrix).mutable)
|
||||
try testing.expect_equal(row, array_type(&tree, matrix).element)
|
||||
try testing.expect_equal(count, array_type(&tree, row).count?)
|
||||
try testing.expect(!array_type(&tree, row).mutable)
|
||||
try testing.expect_equal(element, array_type(&tree, row).element)
|
||||
try testing.expect_equal(ExprKind.array_literal, expr_kind(&tree, literal))
|
||||
try testing.expect_equal(2, array_element_count(&tree, literal))
|
||||
try testing.expect_equal(count, array_element(&tree, literal, 0))
|
||||
try testing.expect_equal(element, array_element(&tree, literal, 1))
|
||||
try testing.expect_equal(0, array_element_count(&tree, empty))
|
||||
try testing.expect_equal(ExprKind.index, expr_kind(&tree, indexed))
|
||||
try testing.expect_equal(literal, index_expr(&tree, indexed).base)
|
||||
try testing.expect_equal(count, index_expr(&tree, indexed).index)
|
||||
}
|
||||
|
||||
@@ -41,17 +41,82 @@ render_function proc(
|
||||
try append_text(renderer, " function ")
|
||||
try append_text(renderer, ast.source_text(tree, function.name))
|
||||
try append_text(renderer, "\n")
|
||||
for 0..ast.block_statement_count(tree, function.body) |index| {
|
||||
statement :: ast.block_statement(tree, function.body, index)
|
||||
try render_block(renderer, tree, function.body, 2)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
render_block proc(renderer @mut Renderer, tree @ast.Tree, block ast.NodeId, depth usize) void ! mem.AllocError {
|
||||
for 0..ast.block_statement_count(tree, block) |index| {
|
||||
statement :: ast.block_statement(tree, block, index)
|
||||
for 0..depth |_| try append_text(renderer, " ")
|
||||
match ast.stmt_kind(tree, statement) {
|
||||
.declaration: try render_statement(renderer, tree, statement, " ")
|
||||
.declaration: try render_statement(renderer, tree, statement, "")
|
||||
.expression: {
|
||||
expression :: ast.statement_expr(tree, statement)
|
||||
try append_text(renderer, " expression ")
|
||||
try append_text(renderer, "expression ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, expression)))
|
||||
try append_text(renderer, "\n")
|
||||
}
|
||||
.invalid: try append_text(renderer, " invalid statement\n")
|
||||
.break_stmt: try append_text(renderer, "break\n")
|
||||
.continue_stmt: try append_text(renderer, "continue\n")
|
||||
.return_stmt: {
|
||||
try append_text(renderer, "return")
|
||||
if (ast.return_stmt(tree, statement).value) |value| {
|
||||
try append_text(renderer, " ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, value)))
|
||||
}
|
||||
try append_text(renderer, "\n")
|
||||
}
|
||||
.if_stmt: {
|
||||
conditional :: ast.if_stmt(tree, statement)
|
||||
try append_text(renderer, "if ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, conditional.condition)))
|
||||
try append_text(renderer, "\n")
|
||||
try render_block(renderer, tree, conditional.then_block, depth + 1)
|
||||
if (conditional.else_block) |alternative| {
|
||||
for 0..depth |_| try append_text(renderer, " ")
|
||||
try append_text(renderer, "else\n")
|
||||
try render_block(renderer, tree, alternative, depth + 1)
|
||||
}
|
||||
}
|
||||
.assignment_stmt: {
|
||||
assignment :: ast.assignment_stmt(tree, statement)
|
||||
try append_text(renderer, "assignment ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, assignment.target)))
|
||||
try append_text(renderer, " = ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, assignment.value)))
|
||||
try append_text(renderer, "\n")
|
||||
}
|
||||
.while_stmt: {
|
||||
loop :: ast.while_stmt(tree, statement)
|
||||
try append_text(renderer, "while ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, loop.condition)))
|
||||
try append_text(renderer, "\n")
|
||||
try render_block(renderer, tree, loop.body, depth + 1)
|
||||
}
|
||||
.for_stmt: {
|
||||
loop :: ast.for_stmt(tree, statement)
|
||||
try append_text(renderer, "for ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, loop.iterable)))
|
||||
try append_text(renderer, " |")
|
||||
if (loop.pointer_capture) try append_text(renderer, "@")
|
||||
if (loop.capture) |capture| {
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, capture)))
|
||||
} else {
|
||||
try append_text(renderer, "_")
|
||||
}
|
||||
if (loop.index_capture) {
|
||||
try append_text(renderer, ", ")
|
||||
if (loop.index) |index| {
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, index)))
|
||||
} else {
|
||||
try append_text(renderer, "_")
|
||||
}
|
||||
}
|
||||
try append_text(renderer, "|\n")
|
||||
try render_block(renderer, tree, loop.body, depth + 1)
|
||||
}
|
||||
.invalid: try append_text(renderer, "invalid statement\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -67,6 +132,12 @@ render_statement proc(
|
||||
try append_text(renderer, indent)
|
||||
try append_text(renderer, "declaration ")
|
||||
try append_text(renderer, ast.source_text(tree, statement.name))
|
||||
if (statement.type) |type| {
|
||||
try append_text(renderer, " ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, type)))
|
||||
}
|
||||
try append_text(renderer, if (statement.mutability == .constant) " :: " else " := ")
|
||||
try append_text(renderer, ast.source_text(tree, ast.node_span(tree, statement.initializer)))
|
||||
try append_text(renderer, "\n")
|
||||
}
|
||||
|
||||
|
||||
@@ -1,27 +1,33 @@
|
||||
import "@std/mem"
|
||||
import "@std/testing"
|
||||
import "@source/diagnostics"
|
||||
import "@source/strpool"
|
||||
import "@source/lexer"
|
||||
import "@source/parser"
|
||||
import "@source/ast"
|
||||
|
||||
renders_semantic_debug_view test {
|
||||
builder := ast.init_builder(mem.c_allocator, "value :: 42")
|
||||
defer ast.deinit_builder(&builder)
|
||||
value :: try ast.add_literal(&builder, .int, ast.Span{ start = 9, end = 11 })
|
||||
declaration :: try ast.add_statement_decl(
|
||||
&builder,
|
||||
ast.Span{ start = 0, end = 11 },
|
||||
ast.Span{ start = 0, end = 5 },
|
||||
null,
|
||||
.constant,
|
||||
value,
|
||||
)
|
||||
declarations [1]ast.NodeId :: [declaration]
|
||||
package :: try ast.add_package(&builder, ast.Span{ start = 0, end = 11 }, declarations[..])
|
||||
tree := ast.finish(&builder, package)
|
||||
renders_array_annotations_literals_and_indexed_assignments test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`data [2]mut i64 := [1, 2]
|
||||
`main proc() i64 {
|
||||
` (data)[0] = 9
|
||||
` return data[0]
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
renderer := init(mem.c_allocator)
|
||||
defer deinit(&renderer)
|
||||
|
||||
try render(&renderer, &tree)
|
||||
|
||||
try testing.expect(mem.eql(u8, "package\n declaration value\n", renderer.bytes.items))
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
try testing.expect_equal(
|
||||
"package\n declaration data [2]mut i64 := [1, 2]\n function main\n assignment (data)[0] = 9\n return data[0]\n",
|
||||
renderer.bytes.items,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import "@std/mem"
|
||||
import "@std/arraylist"
|
||||
import "@std/enummap"
|
||||
|
||||
Span :: struct {
|
||||
start u32
|
||||
@@ -16,6 +17,43 @@ Code :: enum {
|
||||
unexpected_token,
|
||||
duplicate_name,
|
||||
unknown_name,
|
||||
expected_type,
|
||||
type_not_value,
|
||||
invalid_void_type,
|
||||
type_mismatch,
|
||||
invalid_operand,
|
||||
integer_division,
|
||||
not_callable,
|
||||
argument_count_mismatch,
|
||||
cannot_infer_type,
|
||||
missing_return,
|
||||
return_type_mismatch,
|
||||
condition_not_bool,
|
||||
invalid_assignment_target,
|
||||
break_outside_loop,
|
||||
continue_outside_loop,
|
||||
invalid_range_bound,
|
||||
invalid_for_iterable,
|
||||
range_value_not_supported,
|
||||
range_pointer_capture,
|
||||
range_index_capture,
|
||||
invalid_array_length,
|
||||
array_length_mismatch,
|
||||
invalid_array_element_type,
|
||||
array_size_overflow,
|
||||
invalid_index_type,
|
||||
not_indexable,
|
||||
invalid_index_assignment,
|
||||
index_out_of_bounds,
|
||||
missing_entry,
|
||||
invalid_entry,
|
||||
literal_out_of_range,
|
||||
inexact_integer_literal,
|
||||
invalid_escape,
|
||||
read_before_initialization,
|
||||
write_before_initialization,
|
||||
initialization_cycle,
|
||||
integer_overflow,
|
||||
}
|
||||
|
||||
Diagnostic :: struct {
|
||||
@@ -26,7 +64,7 @@ Diagnostic :: struct {
|
||||
#! owns diagnostic storage and retains its allocator; release with deinit.
|
||||
Store :: struct { list arraylist.ArrayList(Diagnostic) }
|
||||
|
||||
CapacityError :: enum { diagnostic_limit_reached }
|
||||
CapacityError :: enum { diagnostic_limit_reached, storage_limit_reached }
|
||||
|
||||
#! retains `allocator` for added diagnostics; the caller owns the returned Store.
|
||||
init proc(allocator mem.Allocator) Store {
|
||||
@@ -54,3 +92,51 @@ get proc(store @Store, id DiagnosticId) ?Diagnostic {
|
||||
count proc(store @Store) usize {
|
||||
return store.list.items.len
|
||||
}
|
||||
|
||||
#! constructed at compile time so missing entries fail compilation; message bytes live for the process lifetime.
|
||||
messages :: $enummap.init(Code, []u8, {
|
||||
invalid_character = "invalid character",
|
||||
float_must_end_with_digit = "float literal must end with a digit",
|
||||
unterminated_string = "unterminated string literal",
|
||||
unexpected_token = "unexpected token",
|
||||
duplicate_name = "duplicate name or forbidden shadowing",
|
||||
unknown_name = "unknown name",
|
||||
expected_type = "annotation must name a type",
|
||||
type_not_value = "type name cannot be used as a value",
|
||||
invalid_void_type = "void cannot be stored or used as a parameter type",
|
||||
type_mismatch = "incompatible value type",
|
||||
invalid_operand = "operator requires compatible operand types",
|
||||
integer_division = "division requires float operands",
|
||||
not_callable = "value is not callable",
|
||||
argument_count_mismatch = "argument count does not match the function signature",
|
||||
cannot_infer_type = "cannot infer a type without a type anchor",
|
||||
missing_return = "nonvoid function can finish without returning a value",
|
||||
return_type_mismatch = "return does not match the function result type",
|
||||
condition_not_bool = "condition must have type bool",
|
||||
invalid_assignment_target = "assignment requires a mutable local binding",
|
||||
break_outside_loop = "break requires an enclosing loop",
|
||||
continue_outside_loop = "continue requires an enclosing loop",
|
||||
invalid_range_bound = "range bounds must have compatible integer types",
|
||||
invalid_for_iterable = "for-loop iteration currently requires a direct integer range",
|
||||
range_value_not_supported = "first-class range values are not implemented",
|
||||
range_pointer_capture = "range loops do not support pointer captures",
|
||||
range_index_capture = "range loops do not support index captures",
|
||||
invalid_array_length = "array length must be a nonnegative supported compile-time integer",
|
||||
array_length_mismatch = "array literal length does not match its type",
|
||||
invalid_array_element_type = "array element annotations require a supported concrete value type",
|
||||
array_size_overflow = "array storage size exceeds the supported address space",
|
||||
invalid_index_type = "array index must have type usize",
|
||||
not_indexable = "indexing is currently implemented for fixed arrays only",
|
||||
invalid_index_assignment = "indexed assignment requires a writable array element",
|
||||
index_out_of_bounds = "index out of bounds",
|
||||
missing_entry = "program requires a main function",
|
||||
invalid_entry = "main must be a function with no parameters",
|
||||
literal_out_of_range = "numeric literal is outside its runtime type's range",
|
||||
inexact_integer_literal = "integer literal is not exactly representable as float; use a floating literal to allow rounding",
|
||||
invalid_escape = "invalid string escape",
|
||||
read_before_initialization = "local value was read before initialization",
|
||||
write_before_initialization = "local value was assigned before initialization",
|
||||
initialization_cycle = "global initialization cycle",
|
||||
integer_overflow = "integer arithmetic overflow",
|
||||
})
|
||||
|
||||
|
||||
@@ -0,0 +1,526 @@
|
||||
import "@std/mem"
|
||||
import "@std/arraylist"
|
||||
import "@source/ir"
|
||||
import "@source/diagnostics"
|
||||
|
||||
Outcome :: struct {
|
||||
value ir.Value = ir.Value{}
|
||||
trap ?diagnostics.DiagnosticId = null
|
||||
allocator ?mem.Allocator = null
|
||||
owned_elements ?[]mut ir.Value = null
|
||||
}
|
||||
|
||||
#! release an escaped aggregate with its retained allocator; scalar and trap outcomes need no allocation.
|
||||
#! strings and array descriptors borrow the module, which must outlive this outcome.
|
||||
deinit proc(outcome @mut Outcome) void {
|
||||
if (outcome.owned_elements) |elements| mem.free(outcome.allocator?, elements)
|
||||
outcome.owned_elements = null
|
||||
outcome.allocator = null
|
||||
outcome.value = ir.Value{}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
Slot :: struct {
|
||||
initialized bool = false
|
||||
value ir.Value = ir.Value{}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
GlobalStatus :: enum { uninitialized, initializing, ready }
|
||||
|
||||
@hide:file
|
||||
GlobalState :: struct {
|
||||
status GlobalStatus = .uninitialized
|
||||
value ir.Value = ir.Value{}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
Address :: struct {
|
||||
type ir.Type
|
||||
array_type ir.ArrayId = ir.no_array
|
||||
cells []mut ir.Value
|
||||
scalar ?@mut ir.Value = null
|
||||
initialized ?@mut bool = null
|
||||
}
|
||||
|
||||
@hide:file
|
||||
Frame :: struct {
|
||||
function ir.FunctionId
|
||||
block ir.BlockId
|
||||
next_instruction usize = 0
|
||||
destination ir.ValueId = ir.no_value
|
||||
global ?usize = null
|
||||
locals []mut Slot
|
||||
values []mut ir.Value
|
||||
aggregate []mut ir.Value
|
||||
addresses []mut Address
|
||||
}
|
||||
|
||||
#! executes typed IR with invocation/global storage released on every exit.
|
||||
#! array results own flat storage released by `deinit`; string IDs and shapes resolve through `module`.
|
||||
run proc(
|
||||
allocator mem.Allocator,
|
||||
module @ir.Module,
|
||||
diagnostic_store @mut diagnostics.Store,
|
||||
) Outcome ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
entry ir.FunctionId := ir.FunctionId(0)
|
||||
if module.entry |function| {
|
||||
entry = function
|
||||
} else {
|
||||
return Outcome{ trap = module.entry_diagnostic }
|
||||
}
|
||||
|
||||
globals :: try mem.alloc(GlobalState, allocator, module.globals.items.len)
|
||||
defer mem.free(allocator, globals)
|
||||
mem.set(globals, GlobalState{})
|
||||
global_cells usize := 0
|
||||
for (module.globals.items) |global| {
|
||||
if (global.type != .array) continue
|
||||
cells :: module.arrays.items[usize(global.array_type)].cells
|
||||
if (cells > maxval!(usize) - global_cells) return .storage_limit_reached
|
||||
global_cells += cells
|
||||
}
|
||||
aggregate_globals :: try mem.alloc(ir.Value, allocator, global_cells)
|
||||
defer mem.free(allocator, aggregate_globals)
|
||||
global_offset usize := 0
|
||||
for (module.globals.items) |global, i| {
|
||||
if (global.type != .array) continue
|
||||
cells :: module.arrays.items[usize(global.array_type)].cells
|
||||
globals[i].value = array_view(global.array_type, aggregate_globals[global_offset..global_offset + cells])
|
||||
global_offset += cells
|
||||
}
|
||||
frames := arraylist.init(Frame, allocator)
|
||||
defer {
|
||||
for (frames.items) |frame| free_frame(allocator, frame)
|
||||
arraylist.deinit(&frames)
|
||||
}
|
||||
try push_frame(allocator, module, &frames, entry, ir.no_value, null)
|
||||
|
||||
while frames.items.len != 0 {
|
||||
frame_index :: frames.items.len - 1
|
||||
function :: &module.functions.items[usize(frames.items[frame_index].function)]
|
||||
block :: &function.blocks.items[usize(frames.items[frame_index].block)]
|
||||
instruction_index :: frames.items[frame_index].next_instruction
|
||||
if instruction_index == block.instructions.items.len {
|
||||
terminator :: block.terminator
|
||||
if (terminator.kind == .trap) return Outcome{ trap = terminator.diagnostic }
|
||||
if (terminator.kind == .jump or terminator.kind == .branch) {
|
||||
target := terminator.target
|
||||
if (terminator.kind == .branch) {
|
||||
condition :: frames.items[frame_index].values[usize(terminator.value)]
|
||||
if (!condition.data.boolean) target = terminator.alternative
|
||||
}
|
||||
frames.items[frame_index].block = target
|
||||
frames.items[frame_index].next_instruction = 0
|
||||
continue
|
||||
}
|
||||
value := ir.Value{}
|
||||
if terminator.value != ir.no_value {
|
||||
value = frames.items[frame_index].values[usize(terminator.value)]
|
||||
}
|
||||
completed :: frames.items[frame_index]
|
||||
# transfer leaves while the completed frame still owns its backing.
|
||||
if completed.global |global_index| {
|
||||
copy_value(&globals[global_index].value, value)
|
||||
globals[global_index].status = .ready
|
||||
}
|
||||
if (frames.items.len == 1) {
|
||||
if (value.type == .array) {
|
||||
escaped :: try mem.alloc(ir.Value, allocator, value.data.elements.len)
|
||||
memcopy!(escaped, value.data.elements)
|
||||
value.data.elements = escaped
|
||||
return Outcome{ value = value, allocator = allocator, owned_elements = escaped }
|
||||
}
|
||||
return Outcome{ value = value }
|
||||
}
|
||||
if completed.destination != ir.no_value {
|
||||
copy_value(&frames.items[frame_index - 1].values[usize(completed.destination)], value)
|
||||
}
|
||||
_ = arraylist.pop(&frames)
|
||||
free_frame(allocator, completed)
|
||||
continue
|
||||
}
|
||||
|
||||
instruction :: &block.instructions.items[instruction_index]
|
||||
frames.items[frame_index].next_instruction += 1
|
||||
value := ir.Value{}
|
||||
match instruction.op {
|
||||
.constant: value = instruction.constant
|
||||
.load_local: {
|
||||
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
|
||||
if !local.initialized {
|
||||
return try runtime_trap(diagnostic_store, .read_before_initialization, instruction.span)
|
||||
}
|
||||
value = local.value
|
||||
}
|
||||
.store_local: {
|
||||
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
|
||||
copy_value(&local.value, frames.items[frame_index].values[usize(instruction.left)])
|
||||
local.initialized = true
|
||||
continue
|
||||
}
|
||||
.assign_local: {
|
||||
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
|
||||
if (!local.initialized) {
|
||||
return try runtime_trap(diagnostic_store, .write_before_initialization, instruction.span)
|
||||
}
|
||||
copy_value(&local.value, frames.items[frame_index].values[usize(instruction.left)])
|
||||
continue
|
||||
}
|
||||
.reset_local: {
|
||||
frames.items[frame_index].locals[usize(instruction.slot)].initialized = false
|
||||
continue
|
||||
}
|
||||
.load_global: {
|
||||
global_index :: usize(instruction.slot)
|
||||
match globals[global_index].status {
|
||||
.ready: value = globals[global_index].value
|
||||
.initializing: {
|
||||
return try runtime_trap(diagnostic_store, .initialization_cycle, instruction.span)
|
||||
}
|
||||
.uninitialized: {
|
||||
try push_frame(
|
||||
allocator,
|
||||
module,
|
||||
&frames,
|
||||
module.globals.items[global_index].initializer,
|
||||
instruction.destination,
|
||||
global_index,
|
||||
)
|
||||
globals[global_index].status = .initializing
|
||||
continue
|
||||
}
|
||||
}
|
||||
}
|
||||
.call: {
|
||||
callee :: frames.items[frame_index].values[usize(instruction.left)].data.function
|
||||
try push_frame(allocator, module, &frames, callee, instruction.destination, null)
|
||||
# frame indices survive stack growth; pointers into frames.items do not.
|
||||
for (0..instruction.arguments_count) |argument_index| {
|
||||
argument :: block.arguments.items[instruction.arguments_start + argument_index]
|
||||
local :: &frames.items[frame_index + 1].locals[argument_index]
|
||||
copy_value(&local.value, frames.items[frame_index].values[usize(argument)])
|
||||
local.initialized = true
|
||||
}
|
||||
continue
|
||||
}
|
||||
.array_literal: {
|
||||
destination :: &frames.items[frame_index].values[usize(instruction.destination)]
|
||||
offset usize := 0
|
||||
for (0..instruction.arguments_count) |i| {
|
||||
operand :: block.arguments.items[instruction.arguments_start + i]
|
||||
element :: frames.items[frame_index].values[usize(operand)]
|
||||
if (element.type == .array) {
|
||||
count :: element.data.elements.len
|
||||
memcopy!(destination.data.elements[offset..offset + count], element.data.elements)
|
||||
offset += count
|
||||
} else {
|
||||
destination.data.elements[offset] = element
|
||||
offset += 1
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
.index: {
|
||||
base :: frames.items[frame_index].values[usize(instruction.left)]
|
||||
index :: frames.items[frame_index].values[usize(instruction.right)].data.unsigned
|
||||
shape :: module.arrays.items[usize(base.array_type)]
|
||||
if (index >= u64(shape.len)) {
|
||||
return try runtime_trap(diagnostic_store, .index_out_of_bounds, instruction.span)
|
||||
}
|
||||
value = indexed_value(module, base.array_type, base.data.elements, usize(index))
|
||||
}
|
||||
.address_local: {
|
||||
local :: &frames.items[frame_index].locals[usize(instruction.slot)]
|
||||
address := Address{
|
||||
type = instruction.pointee, array_type = instruction.array_type,
|
||||
cells = mem.empty(ir.Value), initialized = &local.initialized,
|
||||
}
|
||||
if (address.type == .array) address.cells = local.value.data.elements
|
||||
else address.scalar = &local.value
|
||||
frames.items[frame_index].addresses[usize(instruction.destination)] = address
|
||||
value = ir.Value{ type = .address, data = ir.ValueData{ address = u32(instruction.destination) } }
|
||||
}
|
||||
.address_global: {
|
||||
global_index :: usize(instruction.slot)
|
||||
match globals[global_index].status {
|
||||
.initializing: return try runtime_trap(diagnostic_store, .initialization_cycle, instruction.span)
|
||||
.uninitialized: {
|
||||
frames.items[frame_index].next_instruction -= 1
|
||||
try push_frame(allocator, module, &frames, module.globals.items[global_index].initializer, ir.no_value, global_index)
|
||||
globals[global_index].status = .initializing
|
||||
continue
|
||||
}
|
||||
.ready: {}
|
||||
}
|
||||
global :: &globals[global_index]
|
||||
address := Address{
|
||||
type = instruction.pointee, array_type = instruction.array_type,
|
||||
cells = mem.empty(ir.Value),
|
||||
}
|
||||
if (address.type == .array) address.cells = global.value.data.elements
|
||||
else address.scalar = &global.value
|
||||
frames.items[frame_index].addresses[usize(instruction.destination)] = address
|
||||
value = ir.Value{ type = .address, data = ir.ValueData{ address = u32(instruction.destination) } }
|
||||
}
|
||||
.index_address: {
|
||||
base_id :: frames.items[frame_index].values[usize(instruction.left)].data.address
|
||||
base :: frames.items[frame_index].addresses[usize(base_id)]
|
||||
index :: frames.items[frame_index].values[usize(instruction.right)].data.unsigned
|
||||
shape :: module.arrays.items[usize(base.array_type)]
|
||||
if (index >= u64(shape.len)) {
|
||||
return try runtime_trap(diagnostic_store, .index_out_of_bounds, instruction.span)
|
||||
}
|
||||
width :: element_cells(module, shape)
|
||||
start :: usize(index) * width
|
||||
address := Address{
|
||||
type = instruction.pointee, array_type = instruction.array_type,
|
||||
cells = base.cells[start..start + width], initialized = base.initialized,
|
||||
}
|
||||
if (address.type != .array) address.scalar = &address.cells[0]
|
||||
frames.items[frame_index].addresses[usize(instruction.destination)] = address
|
||||
value = ir.Value{ type = .address, data = ir.ValueData{ address = u32(instruction.destination) } }
|
||||
}
|
||||
.load_address, .store_address: {
|
||||
id :: frames.items[frame_index].values[usize(instruction.left)].data.address
|
||||
address :: frames.items[frame_index].addresses[usize(id)]
|
||||
if address.initialized |initialized| {
|
||||
if (!initialized^) {
|
||||
code :: if (instruction.op == .load_address) diagnostics.Code.read_before_initialization else diagnostics.Code.write_before_initialization
|
||||
return try runtime_trap(diagnostic_store, code, instruction.span)
|
||||
}
|
||||
}
|
||||
if (instruction.op == .load_address) {
|
||||
value = if (address.type == .array) array_view(address.array_type, address.cells) else address.scalar?^
|
||||
} else {
|
||||
source :: frames.items[frame_index].values[usize(instruction.right)]
|
||||
if (address.type == .array) memcopy!(address.cells, source.data.elements)
|
||||
else address.scalar?^ = source
|
||||
continue
|
||||
}
|
||||
}
|
||||
.negate: {
|
||||
operand :: frames.items[frame_index].values[usize(instruction.left)]
|
||||
if instruction.type == .float {
|
||||
value = ir.Value{ type = .float, data = ir.ValueData{ real = -operand.data.real } }
|
||||
} else {
|
||||
if operand.data.integer == minval!(i64) {
|
||||
return try runtime_trap(diagnostic_store, .integer_overflow, instruction.span)
|
||||
}
|
||||
value = ir.Value{ type = .int, data = ir.ValueData{ integer = -operand.data.integer } }
|
||||
}
|
||||
}
|
||||
.logical_not: {
|
||||
operand :: frames.items[frame_index].values[usize(instruction.left)]
|
||||
value = ir.Value{ type = .bool, data = ir.ValueData{ boolean = !operand.data.boolean } }
|
||||
}
|
||||
.add, .subtract, .multiply, .divide: {
|
||||
left :: frames.items[frame_index].values[usize(instruction.left)]
|
||||
right :: frames.items[frame_index].values[usize(instruction.right)]
|
||||
if instruction.type == .float {
|
||||
result f64 := 0
|
||||
match instruction.op {
|
||||
.add: result = left.data.real + right.data.real
|
||||
.subtract: result = left.data.real - right.data.real
|
||||
.multiply: result = left.data.real * right.data.real
|
||||
.divide: result = left.data.real / right.data.real
|
||||
else: unreachable
|
||||
}
|
||||
value = ir.Value{ type = .float, data = ir.ValueData{ real = result } }
|
||||
} else if instruction.type == .usize {
|
||||
result :: checked_unsigned(instruction.op, left.data.unsigned, right.data.unsigned)
|
||||
if result |unsigned| {
|
||||
value = ir.Value{ type = .usize, data = ir.ValueData{ unsigned = unsigned } }
|
||||
} else {
|
||||
return try runtime_trap(diagnostic_store, .integer_overflow, instruction.span)
|
||||
}
|
||||
} else {
|
||||
result :: checked_integer(instruction.op, left.data.integer, right.data.integer)
|
||||
if result |result_integer| {
|
||||
value = ir.Value{ type = .int, data = ir.ValueData{ integer = result_integer } }
|
||||
} else {
|
||||
return try runtime_trap(diagnostic_store, .integer_overflow, instruction.span)
|
||||
}
|
||||
}
|
||||
}
|
||||
.equal, .not_equal, .less, .less_equal, .greater, .greater_equal: {
|
||||
left :: frames.items[frame_index].values[usize(instruction.left)]
|
||||
right :: frames.items[frame_index].values[usize(instruction.right)]
|
||||
result := false
|
||||
match left.type {
|
||||
.int: result = match instruction.op {
|
||||
.equal: left.data.integer == right.data.integer
|
||||
.not_equal: left.data.integer != right.data.integer
|
||||
.less: left.data.integer < right.data.integer
|
||||
.less_equal: left.data.integer <= right.data.integer
|
||||
.greater: left.data.integer > right.data.integer
|
||||
.greater_equal: left.data.integer >= right.data.integer
|
||||
else: unreachable
|
||||
}
|
||||
.usize: result = match instruction.op {
|
||||
.equal: left.data.unsigned == right.data.unsigned
|
||||
.not_equal: left.data.unsigned != right.data.unsigned
|
||||
.less: left.data.unsigned < right.data.unsigned
|
||||
.less_equal: left.data.unsigned <= right.data.unsigned
|
||||
.greater: left.data.unsigned > right.data.unsigned
|
||||
.greater_equal: left.data.unsigned >= right.data.unsigned
|
||||
else: unreachable
|
||||
}
|
||||
.float: result = match instruction.op {
|
||||
.equal: left.data.real == right.data.real
|
||||
.not_equal: left.data.real != right.data.real
|
||||
.less: left.data.real < right.data.real
|
||||
.less_equal: left.data.real <= right.data.real
|
||||
.greater: left.data.real > right.data.real
|
||||
.greater_equal: left.data.real >= right.data.real
|
||||
else: unreachable
|
||||
}
|
||||
.bool: result = match instruction.op {
|
||||
.equal: left.data.boolean == right.data.boolean
|
||||
.not_equal: left.data.boolean != right.data.boolean
|
||||
else: unreachable
|
||||
}
|
||||
else: unreachable
|
||||
}
|
||||
value = ir.Value{ type = .bool, data = ir.ValueData{ boolean = result } }
|
||||
}
|
||||
}
|
||||
if instruction.destination != ir.no_value {
|
||||
copy_value(&frames.items[frame_index].values[usize(instruction.destination)], value)
|
||||
}
|
||||
}
|
||||
unreachable
|
||||
}
|
||||
|
||||
@hide:file
|
||||
push_frame proc(
|
||||
allocator mem.Allocator,
|
||||
module @ir.Module,
|
||||
frames @mut arraylist.ArrayList(Frame),
|
||||
function ir.FunctionId,
|
||||
destination ir.ValueId,
|
||||
global ?usize,
|
||||
) void ! mem.AllocError {
|
||||
signature :: &module.functions.items[usize(function)]
|
||||
locals :: try mem.alloc(Slot, allocator, signature.locals.items.len)
|
||||
errdefer mem.free(allocator, locals)
|
||||
mem.set(locals, Slot{})
|
||||
values :: try mem.alloc(ir.Value, allocator, signature.value_count)
|
||||
errdefer mem.free(allocator, values)
|
||||
aggregate :: try mem.alloc(ir.Value, allocator, signature.aggregate_cells)
|
||||
errdefer mem.free(allocator, aggregate)
|
||||
address_count usize :: if (signature.has_addresses) signature.value_count else 0
|
||||
addresses :: try mem.alloc(Address, allocator, address_count)
|
||||
errdefer mem.free(allocator, addresses)
|
||||
for (signature.array_storage.items) |storage| {
|
||||
cells :: module.arrays.items[usize(storage.array_type)].cells
|
||||
value :: array_view(storage.array_type, aggregate[storage.offset..storage.offset + cells])
|
||||
if (storage.local) locals[usize(storage.index)].value = value
|
||||
else values[usize(storage.index)] = value
|
||||
}
|
||||
# typed IR assigns each temporary before use; only local slots need initialization flags.
|
||||
try arraylist.append(frames, Frame{
|
||||
function = function,
|
||||
block = signature.entry,
|
||||
destination = destination,
|
||||
global = global,
|
||||
locals = locals,
|
||||
values = values,
|
||||
aggregate = aggregate,
|
||||
addresses = addresses,
|
||||
})
|
||||
}
|
||||
|
||||
@hide:file
|
||||
free_frame proc(allocator mem.Allocator, frame Frame) void {
|
||||
mem.free(allocator, frame.locals)
|
||||
mem.free(allocator, frame.values)
|
||||
mem.free(allocator, frame.aggregate)
|
||||
mem.free(allocator, frame.addresses)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
runtime_trap proc(
|
||||
diagnostic_store @mut diagnostics.Store,
|
||||
code diagnostics.Code,
|
||||
span diagnostics.Span,
|
||||
) Outcome ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
return Outcome{ trap = try diagnostics.add(diagnostic_store, code, span) }
|
||||
}
|
||||
|
||||
@hide:file
|
||||
checked_integer proc(op ir.Op, left i64, right i64) ?i64 {
|
||||
minimum :: minval!(i64)
|
||||
maximum :: maxval!(i64)
|
||||
match op {
|
||||
.add: {
|
||||
if (right > 0 and left > maximum - right) return null
|
||||
if (right < 0 and left < minimum - right) return null
|
||||
return left + right
|
||||
}
|
||||
.subtract: {
|
||||
if (right > 0 and left < minimum + right) return null
|
||||
if (right < 0 and left > maximum + right) return null
|
||||
return left - right
|
||||
}
|
||||
.multiply: {
|
||||
if left > 0 {
|
||||
if (right > 0 and left > divtrunc!(maximum, right)) return null
|
||||
if (right < 0 and right < divtrunc!(minimum, left)) return null
|
||||
} else if left < 0 {
|
||||
if (right > 0 and left < divtrunc!(minimum, right)) return null
|
||||
if (right < 0 and left < divtrunc!(maximum, right)) return null
|
||||
}
|
||||
return left * right
|
||||
}
|
||||
else: unreachable
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
array_view proc(shape ir.ArrayId, cells []mut ir.Value) ir.Value {
|
||||
return ir.Value{ type = .array, array_type = shape, data = ir.ValueData{ elements = cells } }
|
||||
}
|
||||
|
||||
@hide:file
|
||||
copy_value proc(destination @mut ir.Value, source ir.Value) void {
|
||||
if (source.type == .array) {
|
||||
# aggregate destinations already own disjoint fixed segments; never install an alias.
|
||||
memcopy!(destination.data.elements, source.data.elements)
|
||||
} else {
|
||||
destination^ = source
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
element_cells proc(module @ir.Module, shape ir.ArrayType) usize {
|
||||
return if (shape.element == .array) module.arrays.items[usize(shape.element_array)].cells else 1
|
||||
}
|
||||
|
||||
@hide:file
|
||||
indexed_value proc(module @ir.Module, shape_id ir.ArrayId, cells []mut ir.Value, index usize) ir.Value {
|
||||
shape :: module.arrays.items[usize(shape_id)]
|
||||
if (shape.element != .array) return cells[index]
|
||||
width :: element_cells(module, shape)
|
||||
start :: index * width
|
||||
return array_view(shape.element_array, cells[start..start + width])
|
||||
}
|
||||
|
||||
@hide:file
|
||||
checked_unsigned proc(op ir.Op, left u64, right u64) ?u64 {
|
||||
match op {
|
||||
.add: {
|
||||
if (left > maxval!(u64) - right) return null
|
||||
return left + right
|
||||
}
|
||||
.subtract: {
|
||||
if (left < right) return null
|
||||
return left - right
|
||||
}
|
||||
.multiply: {
|
||||
if (right != 0 and left > divtrunc!(maxval!(u64), right)) return null
|
||||
return left * right
|
||||
}
|
||||
else: unreachable
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,219 @@
|
||||
import "@std/mem"
|
||||
import "@std/arraylist"
|
||||
import "@source/diagnostics"
|
||||
|
||||
FunctionId :: distinct u32
|
||||
GlobalId :: distinct u32
|
||||
LocalId :: distinct u32
|
||||
ValueId :: distinct u32
|
||||
BlockId :: distinct u32
|
||||
StringId :: distinct u32
|
||||
ArrayId :: distinct u32
|
||||
no_array :: maxval!(ArrayId)
|
||||
no_value :: maxval!(ValueId)
|
||||
|
||||
Type :: enum {
|
||||
invalid,
|
||||
int,
|
||||
float,
|
||||
bool,
|
||||
string,
|
||||
void,
|
||||
function,
|
||||
array,
|
||||
usize,
|
||||
address,
|
||||
}
|
||||
|
||||
ValueData :: union {
|
||||
integer i64
|
||||
unsigned u64
|
||||
real f64
|
||||
boolean bool
|
||||
string StringId
|
||||
function FunctionId
|
||||
elements []mut Value
|
||||
address u32
|
||||
}
|
||||
Value :: struct {
|
||||
type Type = .void
|
||||
array_type ArrayId = no_array
|
||||
data ValueData = ValueData{ integer = 0 }
|
||||
}
|
||||
|
||||
#! fixed recursive shape; `cells` counts flattened scalar leaves, including zero-sized children.
|
||||
ArrayType :: struct {
|
||||
element Type
|
||||
element_array ArrayId = no_array
|
||||
len usize
|
||||
cells usize
|
||||
mutable bool
|
||||
}
|
||||
|
||||
#! maps a local slot or temporary to its preallocated, by-value frame segment.
|
||||
ArrayStorage :: struct {
|
||||
index u32
|
||||
array_type ArrayId
|
||||
offset usize
|
||||
local bool
|
||||
}
|
||||
|
||||
Op :: enum {
|
||||
constant
|
||||
load_local
|
||||
store_local
|
||||
assign_local
|
||||
reset_local
|
||||
load_global
|
||||
array_literal
|
||||
index
|
||||
address_local
|
||||
address_global
|
||||
index_address
|
||||
load_address
|
||||
store_address
|
||||
negate
|
||||
logical_not
|
||||
add
|
||||
subtract
|
||||
multiply
|
||||
divide
|
||||
call
|
||||
equal
|
||||
not_equal
|
||||
less
|
||||
less_equal
|
||||
greater
|
||||
greater_equal
|
||||
}
|
||||
|
||||
Instruction :: struct {
|
||||
op Op
|
||||
type Type = .void
|
||||
array_type ArrayId = no_array
|
||||
pointee Type = .invalid
|
||||
span diagnostics.Span = diagnostics.Span{ start = 0, end = 0 }
|
||||
destination ValueId = no_value
|
||||
left ValueId = no_value
|
||||
right ValueId = no_value
|
||||
slot u32 = 0
|
||||
constant Value = Value{}
|
||||
arguments_start usize = 0
|
||||
arguments_count usize = 0
|
||||
}
|
||||
|
||||
TerminatorKind :: enum {
|
||||
return_value,
|
||||
trap,
|
||||
jump,
|
||||
branch,
|
||||
}
|
||||
|
||||
Terminator :: struct {
|
||||
kind TerminatorKind = .return_value
|
||||
value ValueId = no_value
|
||||
target BlockId = BlockId(0)
|
||||
alternative BlockId = BlockId(0)
|
||||
diagnostic diagnostics.DiagnosticId = diagnostics.no_diagnostic
|
||||
span diagnostics.Span = diagnostics.Span{ start = 0, end = 0 }
|
||||
}
|
||||
|
||||
Block :: struct {
|
||||
instructions arraylist.ArrayList(Instruction)
|
||||
arguments arraylist.ArrayList(ValueId)
|
||||
terminator Terminator = Terminator{}
|
||||
}
|
||||
|
||||
Function :: struct {
|
||||
parameters arraylist.ArrayList(Type)
|
||||
locals arraylist.ArrayList(Type)
|
||||
result Type = .void
|
||||
result_array_type ArrayId = no_array
|
||||
array_storage arraylist.ArrayList(ArrayStorage)
|
||||
aggregate_cells usize = 0
|
||||
has_addresses bool = false
|
||||
blocks arraylist.ArrayList(Block)
|
||||
entry BlockId = BlockId(0)
|
||||
value_count usize = 0
|
||||
span diagnostics.Span = diagnostics.Span{ start = 0, end = 0 }
|
||||
}
|
||||
|
||||
Global :: struct {
|
||||
type Type
|
||||
array_type ArrayId = no_array
|
||||
initializer FunctionId
|
||||
span diagnostics.Span
|
||||
}
|
||||
|
||||
#! retains the original allocation for freeing while exposing only decoded bytes.
|
||||
StringStorage :: struct {
|
||||
allocation []mut u8
|
||||
len usize
|
||||
}
|
||||
|
||||
#! owns executable storage, including decoded strings; no AST or binding table is retained.
|
||||
#! spans and diagnostic IDs refer to the caller's source and diagnostic store.
|
||||
Module :: struct {
|
||||
allocator mem.Allocator
|
||||
functions arraylist.ArrayList(Function)
|
||||
globals arraylist.ArrayList(Global)
|
||||
strings arraylist.ArrayList(StringStorage)
|
||||
arrays arraylist.ArrayList(ArrayType)
|
||||
entry ?FunctionId = null
|
||||
entry_diagnostic diagnostics.DiagnosticId = diagnostics.no_diagnostic
|
||||
}
|
||||
|
||||
init proc(allocator mem.Allocator) Module {
|
||||
return Module{
|
||||
allocator = allocator,
|
||||
functions = arraylist.init(allocator),
|
||||
globals = arraylist.init(allocator),
|
||||
strings = arraylist.init(allocator),
|
||||
arrays = arraylist.init(allocator),
|
||||
}
|
||||
}
|
||||
|
||||
init_function proc(allocator mem.Allocator) Function {
|
||||
return Function{
|
||||
parameters = arraylist.init(allocator),
|
||||
locals = arraylist.init(allocator),
|
||||
blocks = arraylist.init(allocator),
|
||||
array_storage = arraylist.init(allocator),
|
||||
}
|
||||
}
|
||||
|
||||
init_block proc(allocator mem.Allocator) Block {
|
||||
return Block{
|
||||
instructions = arraylist.init(allocator),
|
||||
arguments = arraylist.init(allocator),
|
||||
}
|
||||
}
|
||||
|
||||
#! resolves a module-local ID to bytes borrowed until module deinit.
|
||||
string_bytes proc(module @Module, id StringId) []u8 {
|
||||
string :: module.strings.items[usize(id)]
|
||||
return string.allocation[..string.len]
|
||||
}
|
||||
|
||||
#! releases all executable storage using the retained allocator.
|
||||
deinit proc(module @mut Module) void {
|
||||
for (0..(module.functions.items.len)) |i| {
|
||||
function :: &module.functions.items[i]
|
||||
for (0..(function.blocks.items.len)) |j| {
|
||||
block :: &function.blocks.items[j]
|
||||
arraylist.deinit(&block.instructions)
|
||||
arraylist.deinit(&block.arguments)
|
||||
}
|
||||
arraylist.deinit(&function.parameters)
|
||||
arraylist.deinit(&function.locals)
|
||||
arraylist.deinit(&function.blocks)
|
||||
arraylist.deinit(&function.array_storage)
|
||||
}
|
||||
for (module.strings.items) |string|
|
||||
mem.free(module.allocator, string.allocation)
|
||||
|
||||
arraylist.deinit(&module.functions)
|
||||
arraylist.deinit(&module.globals)
|
||||
arraylist.deinit(&module.strings)
|
||||
arraylist.deinit(&module.arrays)
|
||||
}
|
||||
+46
-1
@@ -22,10 +22,17 @@ SourceError :: enum { source_too_large }
|
||||
keywords strmap.StaticStringMap(TokenKind) :: strmap.init([
|
||||
{ "proc", .proc },
|
||||
{ "return", .return },
|
||||
{ "break", .break },
|
||||
{ "continue", .continue },
|
||||
{ "if", .if },
|
||||
{ "for", .for },
|
||||
{ "else", .else },
|
||||
{ "while", .while },
|
||||
{ "true", .true },
|
||||
{ "false", .false },
|
||||
{ "and", .and },
|
||||
{ "or", .or },
|
||||
{ "mut", .mut },
|
||||
])
|
||||
|
||||
#! owns token storage and retains its allocator; release with deinit.
|
||||
@@ -128,6 +135,18 @@ scan proc(
|
||||
continue
|
||||
}
|
||||
|
||||
if (char == '.' and cursor + 1 < program.len and program[cursor + 1] == '.') {
|
||||
inclusive :: cursor + 2 < program.len and program[cursor + 2] == '='
|
||||
width :: if (inclusive) 3 else 2
|
||||
range_kind :: if (inclusive) .range_inclusive else .range
|
||||
try add_token(state, Token{
|
||||
span = make_span(cursor, cursor + width),
|
||||
kind = range_kind,
|
||||
})
|
||||
cursor += width
|
||||
continue
|
||||
}
|
||||
|
||||
if char == ':' {
|
||||
if cursor + 1 < program.len and program[cursor + 1] == ':' {
|
||||
try add_token(state, Token{
|
||||
@@ -203,9 +222,32 @@ scan proc(
|
||||
continue
|
||||
}
|
||||
|
||||
if (cursor + 1 < program.len and program[cursor + 1] == '=') {
|
||||
comparison ?TokenKind :: match char {
|
||||
'=': .equal_equal
|
||||
'!': .not_equal
|
||||
'<': .less_equal
|
||||
'>': .greater_equal
|
||||
else: null
|
||||
}
|
||||
if (comparison) |value| {
|
||||
try add_token(state, Token{
|
||||
span = make_span(cursor, cursor + 2),
|
||||
kind = value,
|
||||
})
|
||||
cursor += 2
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
kind ?TokenKind :: match char {
|
||||
'=': .equal
|
||||
'!': .bang
|
||||
'<': .less
|
||||
'>': .greater
|
||||
',': .comma
|
||||
'|': .pipe
|
||||
'@': .at
|
||||
'(': .open_paren
|
||||
')': .close_paren
|
||||
'[': .open_bracket
|
||||
@@ -277,7 +319,10 @@ scan_number proc(start usize, program []u8) ScanNumResult ! ScanError {
|
||||
cursor := start
|
||||
has_decimal := false
|
||||
while (cursor < program.len and is_digit(program[cursor])) cursor += 1
|
||||
if cursor < program.len and program[cursor] == '.' {
|
||||
if (
|
||||
cursor < program.len and program[cursor] == '.'
|
||||
and !(cursor + 1 < program.len and program[cursor + 1] == '.')
|
||||
) {
|
||||
has_decimal = true
|
||||
cursor += 1
|
||||
}
|
||||
|
||||
+112
-1
@@ -11,7 +11,7 @@ reports_invalid_input_and_keeps_scanning test {
|
||||
state := init(mem.c_allocator)
|
||||
defer deinit(&state)
|
||||
|
||||
try scan(&state, "if name # comment\n@1. 2 3.5 \"hi\"", &diagnostic_store)
|
||||
try scan(&state, "if name # comment\n$1. 2 3.5 \"hi\"", &diagnostic_store)
|
||||
|
||||
try testing.expect_equal(8, state.tokens.items.len)
|
||||
try testing.expect_equal(TokenKind.if, state.tokens.items[0].kind)
|
||||
@@ -30,3 +30,114 @@ reports_invalid_input_and_keeps_scanning test {
|
||||
try testing.expect_equal(u32(3), state.tokens.items[1].span.start)
|
||||
try testing.expect_equal(u32(7), state.tokens.items[1].span.end)
|
||||
}
|
||||
|
||||
reserves_boolean_keywords_and_scans_comparisons test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
state := init(mem.c_allocator)
|
||||
defer deinit(&state)
|
||||
source :: "true false bool trueish == != < <= > >="
|
||||
try scan(&state, source, &diagnostic_store)
|
||||
expected [11]TokenKind :: [
|
||||
TokenKind.true, .false, .ident, .ident,
|
||||
.equal_equal, .not_equal, .less, .less_equal, .greater, .greater_equal, .eof,
|
||||
]
|
||||
try testing.expect_equal(expected.len, state.tokens.items.len)
|
||||
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
|
||||
try testing.expect_equal(u32(0), state.tokens.items[0].span.start)
|
||||
try testing.expect_equal(u32(4), state.tokens.items[0].span.end)
|
||||
try testing.expect_equal(u32(5), state.tokens.items[1].span.start)
|
||||
try testing.expect_equal(u32(10), state.tokens.items[1].span.end)
|
||||
try testing.expect_equal("bool", strpool.get_str(&strpool.strings, state.tokens.items[2].data.str_id)?)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
scans_logical_operators_without_splitting_identifiers test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
state := init(mem.c_allocator)
|
||||
defer deinit(&state)
|
||||
source :: "and or android orbit and_1 or2 ! != !! !true"
|
||||
try scan(&state, source, &diagnostic_store)
|
||||
expected [13]TokenKind :: [
|
||||
TokenKind.and, .or, .ident, .ident, .ident, .ident,
|
||||
.bang, .not_equal, .bang, .bang, .bang, .true, .eof,
|
||||
]
|
||||
try testing.expect_equal(expected.len, state.tokens.items.len)
|
||||
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
|
||||
try testing.expect_equal("android", strpool.get_str(&strpool.strings, state.tokens.items[2].data.str_id)?)
|
||||
try testing.expect_equal("orbit", strpool.get_str(&strpool.strings, state.tokens.items[3].data.str_id)?)
|
||||
try testing.expect_equal("and_1", strpool.get_str(&strpool.strings, state.tokens.items[4].data.str_id)?)
|
||||
try testing.expect_equal("or2", strpool.get_str(&strpool.strings, state.tokens.items[5].data.str_id)?)
|
||||
try testing.expect_equal(u32(31), state.tokens.items[6].span.start)
|
||||
try testing.expect_equal(u32(32), state.tokens.items[6].span.end)
|
||||
try testing.expect_equal(u32(33), state.tokens.items[7].span.start)
|
||||
try testing.expect_equal(u32(35), state.tokens.items[7].span.end)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
scans_loop_controls_with_exact_keyword_boundaries test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
state := init(mem.c_allocator)
|
||||
defer deinit(&state)
|
||||
source :: "break continue breakfast continued break_1 continue2"
|
||||
try scan(&state, source, &diagnostic_store)
|
||||
expected [7]TokenKind :: [.break, .continue, .ident, .ident, .ident, .ident, .eof]
|
||||
try testing.expect_equal(expected.len, state.tokens.items.len)
|
||||
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
|
||||
try testing.expect_equal(u32(0), state.tokens.items[0].span.start)
|
||||
try testing.expect_equal(u32(5), state.tokens.items[0].span.end)
|
||||
try testing.expect_equal(u32(6), state.tokens.items[1].span.start)
|
||||
try testing.expect_equal(u32(14), state.tokens.items[1].span.end)
|
||||
try testing.expect_equal("breakfast", strpool.get_str(&strpool.strings, state.tokens.items[2].data.str_id)?)
|
||||
try testing.expect_equal("continued", strpool.get_str(&strpool.strings, state.tokens.items[3].data.str_id)?)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
scans_ranges_without_consuming_float_dots test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
state := init(mem.c_allocator)
|
||||
defer deinit(&state)
|
||||
try scan(&state, "0..10 0..=10 1.5..2.5 |@item, _| ... 1.", &diagnostic_store)
|
||||
expected [19]TokenKind :: [
|
||||
.int, .range, .int, .int, .range_inclusive, .int,
|
||||
.float, .range, .float, .pipe, .at, .ident, .comma, .ident, .pipe,
|
||||
.range, .invalid, .invalid, .eof,
|
||||
]
|
||||
try testing.expect_equal(expected.len, state.tokens.items.len)
|
||||
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
|
||||
try testing.expect_equal(u32(1), state.tokens.items[1].span.start)
|
||||
try testing.expect_equal(u32(3), state.tokens.items[1].span.end)
|
||||
try testing.expect_equal(u32(7), state.tokens.items[4].span.start)
|
||||
try testing.expect_equal(u32(10), state.tokens.items[4].span.end)
|
||||
try testing.expect_equal(2, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
scans_array_delimiters_and_mut_keyword test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
state := init(mem.c_allocator)
|
||||
defer deinit(&state)
|
||||
try scan(&state, "[_]mut [2]i64 [1,2] [] mutable mut_1", &diagnostic_store)
|
||||
expected [18]TokenKind :: [
|
||||
.open_bracket, .ident, .close_bracket, .mut,
|
||||
.open_bracket, .int, .close_bracket, .ident,
|
||||
.open_bracket, .int, .comma, .int, .close_bracket,
|
||||
.open_bracket, .close_bracket, .ident, .ident, .eof,
|
||||
]
|
||||
try testing.expect_equal(expected.len, state.tokens.items.len)
|
||||
for (expected) |kind, index| try testing.expect_equal(kind, state.tokens.items[index].kind)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
@@ -21,6 +21,10 @@ TokenKind :: enum {
|
||||
if, else
|
||||
for, while
|
||||
proc, return
|
||||
break, continue
|
||||
true, false
|
||||
and, or
|
||||
mut
|
||||
|
||||
# literals
|
||||
ident, string
|
||||
@@ -39,6 +43,12 @@ TokenKind :: enum {
|
||||
plus, minus
|
||||
star, slash
|
||||
|
||||
range, range_inclusive
|
||||
pipe, at
|
||||
|
||||
# logical
|
||||
bang
|
||||
|
||||
# assignment & arithmetic
|
||||
plus_equal, minus_equal
|
||||
star_equal, slash_equal
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+201
-47
@@ -1,5 +1,8 @@
|
||||
import "@std/debug"
|
||||
import "@std/mem"
|
||||
import "@std/arraylist"
|
||||
import "@std/enummap"
|
||||
import "@std/io"
|
||||
import "@std/process"
|
||||
|
||||
import "@source/diagnostics"
|
||||
import "@source/strpool"
|
||||
@@ -7,8 +10,11 @@ import "@source/lexer"
|
||||
import "@source/ast"
|
||||
import "@source/parser"
|
||||
import "@source/sema"
|
||||
import "@source/ir"
|
||||
import "@source/lower"
|
||||
import "@source/interpreter"
|
||||
|
||||
ast_renderer :: import "@source/ast/renderer"
|
||||
typecheck :: import "@source/sema/check"
|
||||
|
||||
test import "@std/mem"
|
||||
test import "@std/enummap"
|
||||
@@ -16,58 +22,206 @@ test import "@std/arraylist"
|
||||
test import "@std/hashmap"
|
||||
test import "@std/strmap"
|
||||
test import "@std/meta"
|
||||
test import "@std/io"
|
||||
test import "@std/process"
|
||||
test import "@std/strconv"
|
||||
test import "@source/diagnostics"
|
||||
test import "@source/strpool"
|
||||
test import "@source/lexer"
|
||||
test import "@source/ast"
|
||||
test import "@source/ast/renderer"
|
||||
test import "@source/parser"
|
||||
test import "@source/sema"
|
||||
test import "@source/sema/check"
|
||||
test import "@source/lower"
|
||||
test import "@source/interpreter"
|
||||
|
||||
@hide:file
|
||||
program ::
|
||||
`# literals
|
||||
`x float :: 123.9
|
||||
`y int := 43
|
||||
`z :: "hello"
|
||||
`
|
||||
`a := 1 + 2 * 3
|
||||
`b :: -3 / 2 + 1
|
||||
`
|
||||
`grouped proc(left, right int, extra float) void {
|
||||
` value int :: later + left
|
||||
` later int :: right
|
||||
` empty()
|
||||
` grouped(value, later, extra)
|
||||
`}
|
||||
`empty proc() void {}
|
||||
DriverError :: alias (
|
||||
mem.AllocError
|
||||
| diagnostics.CapacityError
|
||||
| lexer.SourceError
|
||||
| strpool.InternError
|
||||
| io.OpenError
|
||||
| io.CloseError
|
||||
| io.ReadError
|
||||
| io.WriteError
|
||||
)
|
||||
|
||||
main proc() void! {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, program, &diagnostic_store)
|
||||
|
||||
tree := try parser.parse(
|
||||
mem.c_allocator,
|
||||
scan_state.tokens.items,
|
||||
program,
|
||||
&diagnostic_store,
|
||||
)
|
||||
defer ast.deinit(&tree)
|
||||
|
||||
resolution := try sema.resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer sema.deinit(&resolution)
|
||||
|
||||
renderer := ast_renderer.init(mem.c_allocator)
|
||||
defer ast_renderer.deinit(&renderer)
|
||||
try ast_renderer.render(&renderer, &tree)
|
||||
|
||||
debug.print("{s}", {renderer.bytes.items})
|
||||
debug.print("diagnostics: {}\n", {diagnostics.count(&diagnostic_store)})
|
||||
main proc(init process.Init) i32 {
|
||||
return command(mem.c_allocator, init.io, init.args) catch |err| {
|
||||
if (init.args.values.len > 2) {
|
||||
io.print("{s}: infrastructure error: {s}\n", io.stderr(init.io), {
|
||||
init.args.values[2], enummap.get(&infrastructure_messages, err)?,
|
||||
}) catch |_| {}
|
||||
} else {
|
||||
io.print("honey: infrastructure error: {s}\n", io.stderr(init.io), {
|
||||
enummap.get(&infrastructure_messages, err)?,
|
||||
}) catch |_| {}
|
||||
}
|
||||
return 2
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
infrastructure_messages :: $enummap.init(DriverError, []u8, {
|
||||
out_of_memory = "out of memory",
|
||||
diagnostic_limit_reached = "diagnostic limit reached",
|
||||
storage_limit_reached = "aggregate storage limit reached",
|
||||
source_too_large = "source file is too large",
|
||||
out_of_space = "identifier storage is full",
|
||||
open_failed = "cannot open source file",
|
||||
close_failed = "cannot close source file",
|
||||
not_open_for_reading = "source file is not readable",
|
||||
read_failed = "cannot read source file",
|
||||
not_open_for_writing = "output is not writable",
|
||||
write_failed = "cannot write output",
|
||||
no_progress = "output writer made no progress",
|
||||
})
|
||||
|
||||
@hide:file
|
||||
command proc(allocator mem.Allocator, runtime io.Io, args process.Args) i32 ! DriverError {
|
||||
if (args.values.len != 3) {
|
||||
try io.print("usage: honey run <file>\n", io.stderr(runtime), {})
|
||||
return 2
|
||||
}
|
||||
if (!mem.eql(args.values[1], "run")) {
|
||||
try io.print("usage: honey run <file>\n", io.stderr(runtime), {})
|
||||
return 2
|
||||
}
|
||||
return try run_file(allocator, runtime, args.values[2])
|
||||
}
|
||||
|
||||
@hide:file
|
||||
read_source proc(allocator mem.Allocator, runtime io.Io, path [;0]u8) arraylist.ArrayList(u8) ! DriverError {
|
||||
file :: try io.open(runtime, path, .read_only)
|
||||
closing := false
|
||||
defer {
|
||||
if (!closing) io.close(file) catch |_| {}
|
||||
}
|
||||
bytes := io.read_all(allocator, io.reader(file), usize(maxval!(u32))) catch |err| {
|
||||
match err {
|
||||
.limit_exceeded: return .source_too_large
|
||||
else: return err
|
||||
}
|
||||
}
|
||||
errdefer arraylist.deinit(&bytes)
|
||||
closing = true
|
||||
try io.close(file)
|
||||
return bytes
|
||||
}
|
||||
|
||||
@hide:file
|
||||
compile proc(allocator mem.Allocator, source []u8, diagnostic_store @mut diagnostics.Store) ir.Module ! DriverError {
|
||||
strpool.strings = strpool.init(allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
scan_state := lexer.init(allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, diagnostic_store)
|
||||
tree := try parser.parse(allocator, scan_state.tokens.items, source, diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
resolution := try sema.resolve(allocator, &tree, diagnostic_store)
|
||||
defer sema.deinit(&resolution)
|
||||
checked := try typecheck.check(allocator, &tree, &resolution, diagnostic_store)
|
||||
defer typecheck.deinit(&checked)
|
||||
return try lower.lower(allocator, &tree, &resolution, &checked, diagnostic_store)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
run_file proc(allocator mem.Allocator, runtime io.Io, path [;0]u8) i32 ! DriverError {
|
||||
source := try read_source(allocator, runtime, path)
|
||||
defer arraylist.deinit(&source)
|
||||
diagnostic_store := diagnostics.init(allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
module := try compile(allocator, source.items, &diagnostic_store)
|
||||
defer ir.deinit(&module)
|
||||
outcome := try interpreter.run(allocator, &module, &diagnostic_store)
|
||||
defer interpreter.deinit(&outcome)
|
||||
if (outcome.trap == null) try print_result(io.stdout(runtime), &module, outcome.value)
|
||||
try report_diagnostics(allocator, io.stderr(runtime), path, source.items, &diagnostic_store, outcome.trap)
|
||||
if (outcome.trap != null or diagnostics.count(&diagnostic_store) != 0) return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
@hide:file
|
||||
print_result proc(output io.Writer, module @ir.Module, value ir.Value) void ! io.WriteError {
|
||||
if (value.type == .void) return
|
||||
try print_value(output, module, value)
|
||||
try io.print("\n", output, {})
|
||||
}
|
||||
|
||||
@hide:file
|
||||
print_value proc(output io.Writer, module @ir.Module, value ir.Value) void ! io.WriteError {
|
||||
match value.type {
|
||||
.int: try io.print("{}", output, {value.data.integer})
|
||||
.usize: try io.print("{}", output, {value.data.unsigned})
|
||||
.float: try io.print("{d}", output, {value.data.real})
|
||||
.bool: try io.print("{}", output, {value.data.boolean})
|
||||
.string: try io.print("{s}", output, {ir.string_bytes(module, value.data.string)})
|
||||
.array: {
|
||||
shape :: module.arrays.items[usize(value.array_type)]
|
||||
stride usize :: if (shape.element == .array)
|
||||
module.arrays.items[usize(shape.element_array)].cells
|
||||
else
|
||||
1
|
||||
try io.print("[", output, {})
|
||||
for 0..(shape.len) |index| {
|
||||
if (index != 0) try io.print(", ", output, {})
|
||||
element := ir.Value{}
|
||||
if shape.element == .array {
|
||||
start :: index * stride
|
||||
element = ir.Value{
|
||||
type = .array,
|
||||
array_type = shape.element_array,
|
||||
data = ir.ValueData{ elements = value.data.elements[start..(start + stride)] },
|
||||
}
|
||||
} else {
|
||||
element = value.data.elements[index]
|
||||
}
|
||||
try print_value(output, module, element)
|
||||
}
|
||||
try io.print("]", output, {})
|
||||
}
|
||||
else: unreachable
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
report_diagnostics proc(
|
||||
allocator mem.Allocator,
|
||||
output io.Writer,
|
||||
path []u8,
|
||||
source []u8,
|
||||
diagnostic_store @diagnostics.Store,
|
||||
trap ?diagnostics.DiagnosticId,
|
||||
) void ! (mem.AllocError | io.WriteError) {
|
||||
if (diagnostics.count(diagnostic_store) == 0) return
|
||||
starts := arraylist.init(usize, allocator)
|
||||
defer arraylist.deinit(&starts)
|
||||
try arraylist.append(&starts, usize(0))
|
||||
for (source) |byte, index| {
|
||||
if (byte == '\n') try arraylist.append(&starts, index + 1)
|
||||
}
|
||||
for (0..diagnostics.count(diagnostic_store)) |index| {
|
||||
if (diagnostics.get(diagnostic_store, diagnostics.DiagnosticId(index))) |diagnostic| {
|
||||
offset :: usize(diagnostic.span.start)
|
||||
line := usize(0)
|
||||
end := starts.items.len
|
||||
while (line + 1 < end) {
|
||||
middle :: line + divtrunc!(end - line, 2)
|
||||
if (starts.items[middle] <= offset) line = middle
|
||||
else end = middle
|
||||
}
|
||||
label []u8 := "error"
|
||||
if (trap) |id| {
|
||||
if (id == diagnostics.DiagnosticId(index)) label = "runtime trap"
|
||||
}
|
||||
try io.print("{s}:{}:{}: {s}: {s}\n", output, {
|
||||
path,
|
||||
line + 1,
|
||||
offset - starts.items[line] + 1,
|
||||
label,
|
||||
enummap.get(diagnostics.Code, []u8, &diagnostics.messages, diagnostic.code)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+608
-36
@@ -27,6 +27,7 @@ State :: struct {
|
||||
tokens []lexer.Token
|
||||
allocator mem.Allocator
|
||||
next_token usize
|
||||
paren_depth usize = 0
|
||||
}
|
||||
|
||||
#! borrows `tokens` and `source` while parsing; the returned Tree borrows `source` until ast.deinit.
|
||||
@@ -102,7 +103,7 @@ parse_func_decl proc(state @mut State) ast.NodeId ! ParseError {
|
||||
while true {
|
||||
token :: try consume(state, .ident)
|
||||
try arraylist.append(&names, token.span)
|
||||
if next_is(state, .ident) break
|
||||
if (next_is(state, .ident) or next_is(state, .open_bracket)) break
|
||||
_ = try consume(state, .comma)
|
||||
}
|
||||
|
||||
@@ -143,7 +144,7 @@ parse_block proc(state @mut State) ast.NodeId ! ParseError {
|
||||
skip_newlines(state)
|
||||
while !next_is(state, .close_curly) and !next_is(state, .eof) {
|
||||
start :: state.next_token
|
||||
statement :: parse_statement(state) catch |err| {
|
||||
statement :: parse_statement(state, false) catch |err| {
|
||||
match err {
|
||||
.unexpected_token: {
|
||||
recovered :: try recover_statement(state, start)
|
||||
@@ -167,11 +168,12 @@ parse_block proc(state @mut State) ast.NodeId ! ParseError {
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_statement proc(state @mut State) ast.NodeId ! ParseError {
|
||||
parse_statement proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
|
||||
if (
|
||||
next_is(state, .ident)
|
||||
and (
|
||||
next_n_is(state, 1, .ident)
|
||||
or (next_n_is(state, 1, .open_bracket) and array_starts_type(state))
|
||||
or next_n_is(state, 1, .double_colon)
|
||||
or next_n_is(state, 1, .colon_equal)
|
||||
)
|
||||
@@ -179,18 +181,293 @@ parse_statement proc(state @mut State) ast.NodeId ! ParseError {
|
||||
return try parse_statement_decl(state)
|
||||
}
|
||||
match current(state).kind {
|
||||
.ident, .int, .float, .string, .minus, .open_paren: {
|
||||
.if: return try parse_if(state)
|
||||
.while: return try parse_while(state, allow_else)
|
||||
.for: return try parse_for(state, allow_else)
|
||||
.return: return try parse_return(state, allow_else)
|
||||
.break, .continue: return try parse_loop_control(state, allow_else)
|
||||
.ident, .int, .float, .string, .true, .false, .minus, .bang, .open_paren, .open_bracket: {
|
||||
expression :: try parse_expr(state)
|
||||
if next_is(state, .equal) return try parse_assignment(state, expression, allow_else)
|
||||
return try ast.add_expression_stmt(&state.builder, expression)
|
||||
}
|
||||
else: return .unexpected_token
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_return proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
|
||||
keyword :: try consume(state, .return)
|
||||
if (
|
||||
next_is(state, .newline) or next_is(state, .close_curly) or next_is(state, .eof)
|
||||
or (allow_else and next_is(state, .else))
|
||||
) {
|
||||
return try ast.add_return_stmt(&state.builder, keyword.span, null)
|
||||
}
|
||||
value :: try parse_expr(state)
|
||||
if (
|
||||
!next_is(state, .newline) and !next_is(state, .close_curly) and !next_is(state, .eof)
|
||||
and !(allow_else and next_is(state, .else))
|
||||
) {
|
||||
return .unexpected_token
|
||||
}
|
||||
# include consumed grouping punctuation, which has no separate expression node.
|
||||
end :: state.tokens[state.next_token - 1].span.end
|
||||
return try ast.add_return_stmt(
|
||||
&state.builder,
|
||||
ast.Span{ start = keyword.span.start, end = end },
|
||||
value,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_loop_control proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
|
||||
keyword :: current(state)
|
||||
state.next_token += 1
|
||||
if (
|
||||
!next_is(state, .newline)
|
||||
and !next_is(state, .close_curly)
|
||||
and !next_is(state, .eof)
|
||||
and !(allow_else and next_is(state, .else))
|
||||
) return .unexpected_token
|
||||
|
||||
if (keyword.kind == .break)
|
||||
return try ast.add_break_stmt(&state.builder, keyword.span)
|
||||
|
||||
return try ast.add_continue_stmt(&state.builder, keyword.span)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_assignment proc(state @mut State, target ast.NodeId, allow_else bool) ast.NodeId ! ParseError {
|
||||
_ = try consume(state, .equal)
|
||||
value :: try parse_expr(state)
|
||||
if (
|
||||
!next_is(state, .newline)
|
||||
and !next_is(state, .close_curly)
|
||||
and !next_is(state, .eof)
|
||||
and !(allow_else and next_is(state, .else))
|
||||
) {
|
||||
return .unexpected_token
|
||||
}
|
||||
end :: state.tokens[state.next_token - 1].span.end
|
||||
return try ast.add_assignment_stmt(
|
||||
&state.builder,
|
||||
ast.Span{ start = ast.builder_span(&state.builder, target).start, end = end },
|
||||
target,
|
||||
value,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_if proc(state @mut State) ast.NodeId ! ParseError {
|
||||
keyword :: try consume(state, .if)
|
||||
condition :: try parse_guard(state)
|
||||
braced :: next_is(state, .open_curly)
|
||||
then_block :: try parse_arm(state, true)
|
||||
# only a braced arm permits its else on a later line.
|
||||
after_then :: state.next_token
|
||||
if (braced) skip_newlines(state)
|
||||
alternative ?ast.NodeId := null
|
||||
if next_is(state, .else) {
|
||||
state.next_token += 1
|
||||
alternative = try parse_arm(state, true)
|
||||
} else {
|
||||
state.next_token = after_then
|
||||
}
|
||||
end :: if (alternative) |block|
|
||||
ast.builder_span(&state.builder, block).end
|
||||
else
|
||||
ast.builder_span(&state.builder, then_block).end
|
||||
return try ast.add_if_stmt(
|
||||
&state.builder,
|
||||
ast.Span{ start = keyword.span.start, end = end },
|
||||
condition,
|
||||
then_block,
|
||||
alternative,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_while proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
|
||||
keyword :: try consume(state, .while)
|
||||
condition :: try parse_guard(state)
|
||||
body :: try parse_arm(state, allow_else)
|
||||
return try ast.add_while_stmt(
|
||||
&state.builder,
|
||||
ast.Span{ start = keyword.span.start, end = ast.builder_span(&state.builder, body).end },
|
||||
condition,
|
||||
body,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
CaptureList :: struct {
|
||||
item ?ast.NodeId = null
|
||||
index ?ast.NodeId = null
|
||||
pointer bool = false
|
||||
indexed bool = false
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_capture_name proc(state @mut State) ?ast.NodeId ! ParseError {
|
||||
name :: try consume(state, .ident)
|
||||
capture ?ast.NodeId := null
|
||||
if (
|
||||
name.span.end != name.span.start + 1
|
||||
or state.builder.source[usize(name.span.start)] != '_'
|
||||
) capture = try ast.add_identifier(&state.builder, name.span)
|
||||
return capture
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_captures proc(state @mut State) CaptureList ! ParseError {
|
||||
_ = try consume(state, .pipe)
|
||||
captures := CaptureList{}
|
||||
if next_is(state, .at) {
|
||||
captures.pointer = true
|
||||
state.next_token += 1
|
||||
}
|
||||
captures.item = try parse_capture_name(state)
|
||||
if next_is(state, .comma) {
|
||||
captures.indexed = true
|
||||
state.next_token += 1
|
||||
captures.index = try parse_capture_name(state)
|
||||
}
|
||||
_ = try consume(state, .pipe)
|
||||
return captures
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_for proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
|
||||
keyword :: try consume(state, .for)
|
||||
skip_newlines(state)
|
||||
iterable_start :: state.next_token
|
||||
iterable := try parse_expr(state)
|
||||
grouped :: parenthesized_guard(state, iterable_start)
|
||||
skip_newlines(state)
|
||||
captures :: parse_captures(state) catch |err| {
|
||||
match err {
|
||||
.unexpected_token: {}
|
||||
else: return err
|
||||
}
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
iterable = try ast.add_invalid_expr(
|
||||
&state.builder, ast.builder_span(&state.builder, iterable), diagnostic,
|
||||
)
|
||||
while (
|
||||
!next_is(state, .open_curly) and !next_is(state, .close_curly)
|
||||
and !next_is(state, .newline) and !next_is(state, .eof)
|
||||
and !next_is(state, .else)
|
||||
) state.next_token += 1
|
||||
yield CaptureList{}
|
||||
}
|
||||
after_captures :: state.next_token
|
||||
skip_newlines(state)
|
||||
if (
|
||||
!next_is(state, .open_curly) and !grouped
|
||||
and ast.builder_expr_kind(&state.builder, iterable) != .call
|
||||
and ast.builder_expr_kind(&state.builder, iterable) != .invalid
|
||||
) {
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
iterable = try ast.add_invalid_expr(
|
||||
&state.builder, ast.builder_span(&state.builder, iterable), diagnostic,
|
||||
)
|
||||
}
|
||||
# a malformed capture list must not swallow the next line's sibling.
|
||||
if (
|
||||
ast.builder_expr_kind(&state.builder, iterable) == .invalid
|
||||
and !next_is(state, .open_curly)
|
||||
) state.next_token = after_captures
|
||||
body :: try parse_arm(state, allow_else)
|
||||
return try ast.add_for_stmt(
|
||||
&state.builder,
|
||||
ast.Span{ start = keyword.span.start, end = ast.builder_span(&state.builder, body).end },
|
||||
iterable, captures.item, captures.index, captures.pointer, captures.indexed, body,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_guard proc(state @mut State) ast.NodeId ! ParseError {
|
||||
condition_start :: state.next_token
|
||||
condition := parse_expr(state) catch |err| {
|
||||
match err {
|
||||
.unexpected_token: {}
|
||||
else: return err
|
||||
}
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
yield try ast.add_invalid_expr(&state.builder, current(state).span, diagnostic)
|
||||
}
|
||||
if (ast.builder_expr_kind(&state.builder, condition) == .invalid) {
|
||||
# discard a malformed guard's tail, but leave its body and enclosing block intact.
|
||||
while (
|
||||
!next_is(state, .open_curly) and !next_is(state, .close_curly)
|
||||
and !next_is(state, .newline) and !next_is(state, .eof)
|
||||
and !next_is(state, .else) and !next_is(state, .return)
|
||||
and !next_is(state, .if) and !next_is(state, .while)
|
||||
and !next_is(state, .for)
|
||||
and !next_is(state, .break) and !next_is(state, .continue)
|
||||
) {
|
||||
if (next_is(state, .ident) and next_n_is(state, 1, .equal)) break
|
||||
state.next_token += 1
|
||||
}
|
||||
}
|
||||
if (
|
||||
!next_is(state, .open_curly)
|
||||
and !parenthesized_guard(state, condition_start)
|
||||
and ast.builder_expr_kind(&state.builder, condition) != .call
|
||||
and ast.builder_expr_kind(&state.builder, condition) != .invalid
|
||||
) {
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
condition = try ast.add_invalid_expr(
|
||||
&state.builder,
|
||||
ast.builder_span(&state.builder, condition),
|
||||
diagnostic,
|
||||
)
|
||||
}
|
||||
return condition
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parenthesized_guard proc(state @State, start usize) bool {
|
||||
if (state.tokens[start].kind != .open_paren) return false
|
||||
depth usize := 0
|
||||
for start..state.next_token |index| {
|
||||
kind :: state.tokens[index].kind
|
||||
if (kind == .open_paren) depth += 1
|
||||
if (kind == .close_paren) {
|
||||
depth -= 1
|
||||
if (depth == 0) return index + 1 == state.next_token
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_arm proc(state @mut State, allow_else bool) ast.NodeId ! ParseError {
|
||||
if next_is(state, .open_curly) return try parse_block(state)
|
||||
start :: state.next_token
|
||||
statement :: parse_statement(state, allow_else) catch |err| {
|
||||
match err {
|
||||
.unexpected_token: {}
|
||||
else: return err
|
||||
}
|
||||
diagnostic :: try unexpected_diagnostic(state)
|
||||
span :: synchronize(state, start, true, true, false)
|
||||
yield try ast.add_invalid_stmt(&state.builder, span, diagnostic)
|
||||
}
|
||||
statements [1]ast.NodeId :: [statement]
|
||||
return try ast.add_block(
|
||||
&state.builder,
|
||||
ast.builder_span(&state.builder, statement),
|
||||
statements[..],
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@hide:file
|
||||
parse_statement_decl proc(state @mut State) ast.NodeId ! ParseError {
|
||||
name :: try consume(state, .ident)
|
||||
type_id ?ast.NodeId :: if next_is(state, .ident)
|
||||
type_id ?ast.NodeId :: if (next_is(state, .ident) or next_is(state, .open_bracket))
|
||||
try parse_type(state)
|
||||
else
|
||||
null
|
||||
@@ -211,9 +488,29 @@ parse_statement_decl proc(state @mut State) ast.NodeId ! ParseError {
|
||||
|
||||
@hide:file
|
||||
parse_type proc(state @mut State) ast.NodeId ! ParseError {
|
||||
if (!next_is(state, .ident)) return .unexpected_token
|
||||
token :: current(state)
|
||||
state.next_token += 1
|
||||
if next_is(state, .open_bracket) {
|
||||
open :: try consume(state, .open_bracket)
|
||||
count ?ast.NodeId := null
|
||||
if (
|
||||
next_is(state, .ident)
|
||||
and current(state).span.end == current(state).span.start + 1
|
||||
and state.builder.source[usize(current(state).span.start)] == '_'
|
||||
) {
|
||||
state.next_token += 1
|
||||
} else {
|
||||
count = try parse_expr(state)
|
||||
}
|
||||
_ = try consume(state, .close_bracket)
|
||||
mutable :: next_is(state, .mut)
|
||||
if (mutable) state.next_token += 1
|
||||
element :: try parse_type(state)
|
||||
return try ast.add_array_type(
|
||||
&state.builder,
|
||||
ast.Span{ start = open.span.start, end = ast.builder_span(&state.builder, element).end },
|
||||
count, element, mutable,
|
||||
)
|
||||
}
|
||||
token :: try consume(state, .ident)
|
||||
return try ast.add_identifier(&state.builder, token.span)
|
||||
}
|
||||
|
||||
@@ -224,25 +521,54 @@ parse_expr proc(state @mut State) ast.NodeId ! ParseError {
|
||||
|
||||
@hide:file
|
||||
parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! ParseError {
|
||||
skip_operand_newlines(state)
|
||||
token := current(state)
|
||||
expression_start :: state.next_token
|
||||
left := if (prefix_binding_power(token.kind)) |right_bp| {
|
||||
state.next_token += 1
|
||||
operand :: try parse_expr_bp(state, right_bp)
|
||||
operand_span :: ast.builder_span(&state.builder, operand)
|
||||
operator ast.UnaryOperator :: if (token.kind == .bang) .logical_not else .negate
|
||||
yield try ast.add_unary(
|
||||
&state.builder,
|
||||
ast.Span{ start = token.span.start, end = operand_span.end },
|
||||
.negate,
|
||||
operator,
|
||||
operand,
|
||||
)
|
||||
} else {
|
||||
yield try parse_primary(state)
|
||||
}
|
||||
left_grouped := parenthesized_guard(state, expression_start)
|
||||
|
||||
while true {
|
||||
skip_continuation_newlines(state)
|
||||
token = current(state)
|
||||
if (token.kind == .open_paren) {
|
||||
left = try parse_call(state, left)
|
||||
left = try parse_call(state, left, state.tokens[expression_start].span.start)
|
||||
left_grouped = false
|
||||
continue
|
||||
}
|
||||
if (token.kind == .open_bracket) {
|
||||
left = try parse_index(state, left, state.tokens[expression_start].span.start)
|
||||
left_grouped = false
|
||||
continue
|
||||
}
|
||||
if (token.kind == .range or token.kind == .range_inclusive) {
|
||||
if (min_bp > BindingPowerScalar(0)) break
|
||||
left = try validate_range_bound(state, left, left_grouped)
|
||||
state.next_token += 1
|
||||
right_start :: state.next_token
|
||||
right := try parse_expr_bp(state, BindingPowerScalar(1))
|
||||
right = try validate_range_bound(state, right, parenthesized_guard(state, right_start))
|
||||
left = try ast.add_range(
|
||||
&state.builder,
|
||||
ast.Span{
|
||||
start = state.tokens[expression_start].span.start,
|
||||
end = state.tokens[state.next_token - 1].span.end,
|
||||
},
|
||||
left, right, token.kind == .range_inclusive,
|
||||
)
|
||||
left_grouped = false
|
||||
continue
|
||||
}
|
||||
if (infix_binding_power(token.kind)) |power| {
|
||||
@@ -258,6 +584,7 @@ parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! Par
|
||||
left,
|
||||
right,
|
||||
)
|
||||
left_grouped = false
|
||||
continue
|
||||
}
|
||||
break
|
||||
@@ -266,27 +593,41 @@ parse_expr_bp proc(state @mut State, min_bp BindingPowerScalar) ast.NodeId ! Par
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_call proc(state @mut State, callee ast.NodeId) ast.NodeId ! ParseError {
|
||||
validate_range_bound proc(
|
||||
state @mut State, bound ast.NodeId, grouped bool,
|
||||
) ast.NodeId ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
if (grouped) return bound
|
||||
match ast.builder_expr_kind(&state.builder, bound) {
|
||||
.unary, .binary, .range: {
|
||||
span :: ast.builder_span(&state.builder, bound)
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, span)
|
||||
return try ast.add_invalid_expr(&state.builder, span, diagnostic)
|
||||
}
|
||||
else: return bound
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_call proc(state @mut State, callee ast.NodeId, start u32) ast.NodeId ! ParseError {
|
||||
_ = try consume(state, .open_paren)
|
||||
state.paren_depth += 1
|
||||
defer state.paren_depth -= 1
|
||||
skip_operand_newlines(state)
|
||||
arguments arraylist.ArrayList(ast.NodeId) := arraylist.init(state.allocator)
|
||||
defer arraylist.deinit(&arguments)
|
||||
|
||||
while (
|
||||
!next_is(state, .close_paren)
|
||||
and !next_is(state, .newline)
|
||||
and !next_is(state, .close_curly)
|
||||
and !next_is(state, .eof)
|
||||
) {
|
||||
while (!is_call_boundary(current(state).kind)) {
|
||||
argument :: try parse_expr(state)
|
||||
try arraylist.append(&arguments, argument)
|
||||
if (!next_is(state, .comma)) break
|
||||
state.next_token += 1
|
||||
skip_operand_newlines(state)
|
||||
}
|
||||
close :: try consume(state, .close_paren)
|
||||
return try ast.add_call(
|
||||
&state.builder,
|
||||
ast.Span{
|
||||
start = ast.builder_span(&state.builder, callee).start,
|
||||
start = start,
|
||||
end = close.span.end,
|
||||
},
|
||||
callee,
|
||||
@@ -294,12 +635,112 @@ parse_call proc(state @mut State, callee ast.NodeId) ast.NodeId ! ParseError {
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_index proc(state @mut State, base ast.NodeId, start u32) ast.NodeId ! ParseError {
|
||||
_ = try consume(state, .open_bracket)
|
||||
index :: try parse_expr(state)
|
||||
if !next_is(state, .close_bracket) {
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
return try ast.add_invalid_expr(
|
||||
&state.builder,
|
||||
ast.Span{
|
||||
start = start,
|
||||
end = ast.builder_span(&state.builder, index).end,
|
||||
},
|
||||
diagnostic,
|
||||
)
|
||||
}
|
||||
close :: try consume(state, .close_bracket)
|
||||
return try ast.add_index(
|
||||
&state.builder,
|
||||
ast.Span{ start = start, end = close.span.end },
|
||||
base, index,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
array_starts_type proc(state @State) bool {
|
||||
depth usize := 0
|
||||
for (state.next_token)..(state.tokens.len) |i| {
|
||||
kind :: state.tokens[i].kind
|
||||
if (kind == .open_bracket) depth += 1
|
||||
if (kind == .close_bracket) {
|
||||
depth -= 1
|
||||
if (depth == 0) {
|
||||
if (i + 1 >= state.tokens.len) return false
|
||||
next :: state.tokens[i + 1].kind
|
||||
return next == .ident or next == .mut or next == .open_bracket
|
||||
}
|
||||
}
|
||||
if (kind == .open_curly or kind == .close_curly or kind == .eof) return false
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_array_literal proc(state @mut State) ast.NodeId ! ParseError {
|
||||
open :: try consume(state, .open_bracket)
|
||||
elements arraylist.ArrayList(ast.NodeId) := arraylist.init(state.allocator)
|
||||
defer arraylist.deinit(&elements)
|
||||
after_open :: state.next_token
|
||||
skip_newlines(state)
|
||||
if named_binding_starts(state) {
|
||||
state.next_token = after_open
|
||||
return try invalid_array_literal(state, open.span.start, open.span.end)
|
||||
}
|
||||
while !next_is(state, .close_bracket) {
|
||||
element :: try parse_expr(state)
|
||||
try arraylist.append(&elements, element)
|
||||
after_element :: state.next_token
|
||||
skip_newlines(state)
|
||||
if next_is(state, .close_bracket) break
|
||||
if !next_is(state, .comma) {
|
||||
state.next_token = after_element
|
||||
return try invalid_array_literal(state, open.span.start, ast.builder_span(&state.builder, element).end)
|
||||
}
|
||||
state.next_token += 1
|
||||
after_comma :: state.next_token
|
||||
skip_newlines(state)
|
||||
if named_binding_starts(state) {
|
||||
state.next_token = after_comma
|
||||
return try invalid_array_literal(state, open.span.start, ast.builder_span(&state.builder, element).end)
|
||||
}
|
||||
}
|
||||
close :: try consume(state, .close_bracket)
|
||||
return try ast.add_array_literal(
|
||||
&state.builder, ast.Span{ start = open.span.start, end = close.span.end }, elements.items,
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
invalid_array_literal proc(state @mut State, start, end u32) ast.NodeId ! ParseError {
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
return try ast.add_invalid_expr(&state.builder, ast.Span{ start = start, end = end }, diagnostic)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
named_binding_starts proc(state @State) bool {
|
||||
return (
|
||||
next_is(state, .ident)
|
||||
and (
|
||||
next_n_is(state, 1, .double_colon) or next_n_is(state, 1, .colon_equal)
|
||||
or next_n_is(state, 1, .ident) or next_n_is(state, 1, .proc)
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
parse_primary proc(state @mut State) ast.NodeId ! ParseError {
|
||||
token :: current(state)
|
||||
match token.kind {
|
||||
.open_bracket: {
|
||||
if array_starts_type(state) return try parse_type(state)
|
||||
return try parse_array_literal(state)
|
||||
}
|
||||
.open_paren: {
|
||||
state.next_token += 1
|
||||
state.paren_depth += 1
|
||||
defer state.paren_depth -= 1
|
||||
expression :: try parse_expr(state)
|
||||
if next_is(state, .close_paren) {
|
||||
state.next_token += 1
|
||||
@@ -327,6 +768,10 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError {
|
||||
state.next_token += 1
|
||||
return try ast.add_literal(&state.builder, .string, token.span)
|
||||
}
|
||||
.true, .false: {
|
||||
state.next_token += 1
|
||||
return try ast.add_literal(&state.builder, .bool, token.span)
|
||||
}
|
||||
.ident: {
|
||||
state.next_token += 1
|
||||
return try ast.add_identifier(&state.builder, token.span)
|
||||
@@ -337,13 +782,7 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError {
|
||||
else
|
||||
try diagnostics.add(state.diagnostics, .unexpected_token, token.span)
|
||||
|
||||
if (
|
||||
token.kind != .newline
|
||||
and token.kind != .comma
|
||||
and token.kind != .close_paren
|
||||
and token.kind != .close_curly
|
||||
and token.kind != .eof
|
||||
) {
|
||||
if !is_expression_recovery_boundary(token.kind) {
|
||||
state.next_token += 1
|
||||
}
|
||||
|
||||
@@ -352,13 +791,70 @@ parse_primary proc(state @mut State) ast.NodeId ! ParseError {
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
is_expression_boundary proc(kind lexer.TokenKind) bool {
|
||||
return match kind {
|
||||
.close_paren, .newline, .close_curly, .open_curly, .else, .return, .if, .while, .for, .eof: true
|
||||
else: false
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
is_call_boundary proc(kind lexer.TokenKind) bool {
|
||||
return (
|
||||
kind == .break
|
||||
or kind == .continue
|
||||
or is_expression_boundary(kind)
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
is_expression_recovery_boundary proc(kind lexer.TokenKind) bool {
|
||||
return (
|
||||
kind == .comma
|
||||
or kind == .close_bracket
|
||||
or kind == .and
|
||||
or kind == .or
|
||||
or kind == .pipe
|
||||
or is_expression_boundary(kind)
|
||||
)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
skip_operand_newlines proc(state @mut State) void {
|
||||
if (state.paren_depth == 0 or !next_is(state, .newline)) return
|
||||
start :: state.next_token
|
||||
skip_newlines(state)
|
||||
kind :: current(state).kind
|
||||
if (kind != .close_paren and is_expression_boundary(kind)) state.next_token = start
|
||||
# a missing operand must not consume the declaration or assignment on the next line.
|
||||
if (
|
||||
named_binding_starts(state)
|
||||
or (next_is(state, .ident) and next_n_is(state, 1, .equal))
|
||||
) state.next_token = start
|
||||
}
|
||||
|
||||
@hide:file
|
||||
skip_continuation_newlines proc(state @mut State) void {
|
||||
if (state.paren_depth == 0 or !next_is(state, .newline)) return
|
||||
start :: state.next_token
|
||||
skip_newlines(state)
|
||||
kind :: current(state).kind
|
||||
if (
|
||||
kind != .close_paren and kind != .comma and kind != .open_paren
|
||||
and kind != .close_bracket and kind != .open_bracket
|
||||
and kind != .range and kind != .range_inclusive
|
||||
and infix_binding_power(kind) == null
|
||||
) state.next_token = start
|
||||
}
|
||||
|
||||
@hide:file
|
||||
recover_decl proc(
|
||||
state @mut State,
|
||||
start usize,
|
||||
) ast.NodeId ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
span :: synchronize(state, start, false)
|
||||
diagnostic :: try unexpected_diagnostic(state)
|
||||
span :: synchronize(state, start, false, false, true)
|
||||
return try ast.add_invalid_decl(&state.builder, span, diagnostic)
|
||||
}
|
||||
|
||||
@@ -367,27 +863,91 @@ recover_statement proc(
|
||||
state @mut State,
|
||||
start usize,
|
||||
) ast.NodeId ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
diagnostic :: try diagnostics.add(state.diagnostics, .unexpected_token, current(state).span)
|
||||
span :: synchronize(state, start, true)
|
||||
diagnostic :: try unexpected_diagnostic(state)
|
||||
span :: synchronize(state, start, true, false, true)
|
||||
return try ast.add_invalid_stmt(&state.builder, span, diagnostic)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
synchronize proc(state @mut State, start usize, stop_at_curly bool) ast.Span {
|
||||
synchronize proc(
|
||||
state @mut State, start usize, stop_at_curly, allow_else, consume_newline bool,
|
||||
) ast.Span {
|
||||
first :: state.tokens[start].span
|
||||
end := first.end
|
||||
while !next_is(state, .eof) and !next_is(state, .newline) {
|
||||
if (stop_at_curly and next_is(state, .close_curly)) break
|
||||
curly usize := 0
|
||||
paren usize := 0
|
||||
square usize := 0
|
||||
for (start..state.next_token) |i| {
|
||||
kind :: state.tokens[i].kind
|
||||
if (kind == .open_paren) paren += 1
|
||||
if (kind == .close_paren and paren > 0) paren -= 1
|
||||
if (kind == .open_bracket) square += 1
|
||||
if (kind == .close_bracket and square > 0) square -= 1
|
||||
}
|
||||
while !next_is(state, .eof) {
|
||||
if (curly == 0) {
|
||||
if (stop_at_curly and next_is(state, .close_curly)) break
|
||||
if (allow_else and next_is(state, .else)) break
|
||||
if (
|
||||
next_is(state, .newline)
|
||||
and !recovery_closes_delimiters(state, paren, square)
|
||||
) break
|
||||
}
|
||||
kind :: current(state).kind
|
||||
if (kind == .open_curly) curly += 1
|
||||
if (kind == .close_curly and curly > 0) curly -= 1
|
||||
if (kind == .open_paren) paren += 1
|
||||
if (kind == .close_paren and paren > 0) paren -= 1
|
||||
if (kind == .open_bracket) square += 1
|
||||
if (kind == .close_bracket and square > 0) square -= 1
|
||||
end = current(state).span.end
|
||||
state.next_token += 1
|
||||
}
|
||||
if next_is(state, .newline) {
|
||||
if (consume_newline and next_is(state, .newline)) {
|
||||
end = current(state).span.end
|
||||
state.next_token += 1
|
||||
}
|
||||
return ast.Span{ start = first.start, end = end }
|
||||
}
|
||||
|
||||
@hide:file
|
||||
recovery_closes_delimiters proc(state @State, paren_depth, square_depth usize) bool {
|
||||
if (paren_depth == 0 and square_depth == 0) return false
|
||||
paren := paren_depth
|
||||
square := square_depth
|
||||
for (state.next_token)..(state.tokens.len) |i| {
|
||||
kind :: state.tokens[i].kind
|
||||
match kind {
|
||||
.open_curly, .close_curly, .eof, .if, .else, .while, .for,
|
||||
.return, .break, .continue: return false
|
||||
.ident: {
|
||||
if (i + 1 < state.tokens.len) {
|
||||
next :: state.tokens[i + 1].kind
|
||||
if (next == .double_colon or next == .colon_equal or next == .ident or next == .proc) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
else: {}
|
||||
}
|
||||
if (kind == .open_paren) paren += 1
|
||||
if (kind == .close_paren and paren > 0) paren -= 1
|
||||
if (kind == .open_bracket) square += 1
|
||||
if (kind == .close_bracket and square > 0) square -= 1
|
||||
if (paren == 0 and square == 0) return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
@hide:file
|
||||
unexpected_diagnostic proc(state @mut State) diagnostics.DiagnosticId ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
token :: current(state)
|
||||
if (token.kind == .invalid and token.data.diagnostic != diagnostics.no_diagnostic) {
|
||||
return token.data.diagnostic
|
||||
}
|
||||
return try diagnostics.add(state.diagnostics, .unexpected_token, token.span)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
skip_newlines proc(state @mut State) void {
|
||||
while next_is(state, .newline) state.next_token += 1
|
||||
@@ -431,7 +991,7 @@ consume_either proc(state @mut State, kinds []lexer.TokenKind) lexer.Token ! Syn
|
||||
@hide:file
|
||||
prefix_binding_power proc(kind lexer.TokenKind) ?BindingPowerScalar {
|
||||
return match kind {
|
||||
.minus: BindingPowerScalar(30)
|
||||
.minus, .bang: BindingPowerScalar(50)
|
||||
else: null
|
||||
}
|
||||
}
|
||||
@@ -439,8 +999,12 @@ prefix_binding_power proc(kind lexer.TokenKind) ?BindingPowerScalar {
|
||||
@hide:file
|
||||
infix_binding_power proc(kind lexer.TokenKind) ?BindingPower {
|
||||
return match kind {
|
||||
.plus, .minus: BindingPower{ left = BindingPowerScalar(10), right = BindingPowerScalar(11) }
|
||||
.star, .slash: BindingPower{ left = BindingPowerScalar(20), right = BindingPowerScalar(21) }
|
||||
.or: BindingPower{ left = BindingPowerScalar(2), right = BindingPowerScalar(3) }
|
||||
.and: BindingPower{ left = BindingPowerScalar(4), right = BindingPowerScalar(5) }
|
||||
.equal_equal, .not_equal: BindingPower{ left = BindingPowerScalar(10), right = BindingPowerScalar(11) }
|
||||
.less, .less_equal, .greater, .greater_equal: BindingPower{ left = BindingPowerScalar(20), right = BindingPowerScalar(21) }
|
||||
.plus, .minus: BindingPower{ left = BindingPowerScalar(30), right = BindingPowerScalar(31) }
|
||||
.star, .slash: BindingPower{ left = BindingPowerScalar(40), right = BindingPowerScalar(41) }
|
||||
else: null
|
||||
}
|
||||
}
|
||||
@@ -448,10 +1012,18 @@ infix_binding_power proc(kind lexer.TokenKind) ?BindingPower {
|
||||
@hide:file
|
||||
binary_operator proc(kind lexer.TokenKind) ast.BinaryOperator {
|
||||
return match kind {
|
||||
.and: .logical_and
|
||||
.or: .logical_or
|
||||
.plus: .add
|
||||
.minus: .subtract
|
||||
.star: .multiply
|
||||
.slash: .divide
|
||||
.equal_equal: .equal
|
||||
.not_equal: .not_equal
|
||||
.less: .less
|
||||
.less_equal: .less_equal
|
||||
.greater: .greater
|
||||
.greater_equal: .greater_equal
|
||||
else: unreachable
|
||||
}
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+84
-10
@@ -6,14 +6,23 @@ import "@std/strmap"
|
||||
import "@source/ast"
|
||||
import "@source/diagnostics"
|
||||
|
||||
BuiltinType :: enum { int, float, string, void }
|
||||
BuiltinType :: enum {
|
||||
int, float,
|
||||
bool, string,
|
||||
void, i64,
|
||||
f64, usize,
|
||||
}
|
||||
|
||||
@hide:file
|
||||
builtins strmap.StaticStringMap(BuiltinType) :: strmap.init([
|
||||
{ "int", .int },
|
||||
{ "float", .float },
|
||||
{ "bool", .bool },
|
||||
{ "string", .string },
|
||||
{ "void", .void },
|
||||
{ "i64", .i64 },
|
||||
{ "f64", .f64 },
|
||||
{ "usize", .usize },
|
||||
])
|
||||
|
||||
BindingKind :: enum { none, declaration, builtin, invalid }
|
||||
@@ -175,18 +184,63 @@ resolve_function proc(
|
||||
parameter :: ast.function_param(tree, id, i)
|
||||
try predeclare(state, &scope, parameter, ast.param(tree, parameter).name)
|
||||
}
|
||||
for (0..ast.block_statement_count(tree, function.body)) |i| {
|
||||
statement :: ast.block_statement(tree, function.body, i)
|
||||
try resolve_block(state, &scope, function.body)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
resolve_arm proc(state @mut State, parent @Scope, id ast.NodeId) void ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
scope := Scope{ names = hashmap.init(state.allocator), parent = parent }
|
||||
defer hashmap.deinit(&scope.names)
|
||||
try resolve_block(state, &scope, id)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
resolve_block proc(state @mut State, scope @mut Scope, id ast.NodeId) void ! (mem.AllocError | diagnostics.CapacityError) {
|
||||
tree :: state.tree
|
||||
for (0..ast.block_statement_count(tree, id)) |i| {
|
||||
statement :: ast.block_statement(tree, id, i)
|
||||
if ast.stmt_kind(tree, statement) == .declaration {
|
||||
try predeclare(state, &scope, statement, ast.statement_decl(tree, statement).name)
|
||||
try predeclare(state, scope, statement, ast.statement_decl(tree, statement).name)
|
||||
}
|
||||
}
|
||||
|
||||
for (0..ast.block_statement_count(tree, function.body)) |i| {
|
||||
statement :: ast.block_statement(tree, function.body, i)
|
||||
for (0..ast.block_statement_count(tree, id)) |i| {
|
||||
statement :: ast.block_statement(tree, id, i)
|
||||
match ast.stmt_kind(tree, statement) {
|
||||
.declaration: try resolve_statement_decl(state, &scope, statement)
|
||||
.expression: try resolve_expr(state, &scope, ast.statement_expr(tree, statement))
|
||||
.declaration: try resolve_statement_decl(state, scope, statement)
|
||||
.expression: try resolve_expr(state, scope, ast.statement_expr(tree, statement))
|
||||
.return_stmt: {
|
||||
if (ast.return_stmt(tree, statement).value) |value| try resolve_expr(state, scope, value)
|
||||
}
|
||||
.break_stmt, .continue_stmt: {}
|
||||
.if_stmt: {
|
||||
conditional :: ast.if_stmt(tree, statement)
|
||||
try resolve_expr(state, scope, conditional.condition)
|
||||
try resolve_arm(state, scope, conditional.then_block)
|
||||
if (conditional.else_block) |other| try resolve_arm(state, scope, other)
|
||||
}
|
||||
.assignment_stmt: {
|
||||
assignment :: ast.assignment_stmt(tree, statement)
|
||||
try resolve_expr(state, scope, assignment.target)
|
||||
try resolve_expr(state, scope, assignment.value)
|
||||
}
|
||||
.while_stmt: {
|
||||
loop :: ast.while_stmt(tree, statement)
|
||||
try resolve_expr(state, scope, loop.condition)
|
||||
try resolve_arm(state, scope, loop.body)
|
||||
}
|
||||
.for_stmt: {
|
||||
loop :: ast.for_stmt(tree, statement)
|
||||
try resolve_expr(state, scope, loop.iterable)
|
||||
body_scope := Scope{ names = hashmap.init(state.allocator), parent = scope }
|
||||
defer hashmap.deinit(&body_scope.names)
|
||||
if (loop.capture) |capture| {
|
||||
try predeclare(state, &body_scope, capture, ast.node_span(tree, capture))
|
||||
}
|
||||
if (loop.index) |index| {
|
||||
try predeclare(state, &body_scope, index, ast.node_span(tree, index))
|
||||
}
|
||||
try resolve_block(state, &body_scope, loop.body)
|
||||
}
|
||||
.invalid: resolve_invalid(state, statement)
|
||||
}
|
||||
}
|
||||
@@ -222,7 +276,7 @@ resolve_expr proc(
|
||||
if (state.bindings[usize(id)].kind != .none) return
|
||||
tree :: state.tree
|
||||
match ast.expr_kind(tree, id) {
|
||||
.literal_int, .literal_float, .literal_string: {}
|
||||
.literal_int, .literal_float, .literal_bool, .literal_string: {}
|
||||
.identifier: {
|
||||
span :: ast.node_span(tree, id)
|
||||
name :: ast.source_text(tree, span)
|
||||
@@ -242,12 +296,32 @@ resolve_expr proc(
|
||||
try resolve_expr(state, scope, expression.left)
|
||||
try resolve_expr(state, scope, expression.right)
|
||||
}
|
||||
.range: {
|
||||
expression :: ast.range_expr(tree, id)
|
||||
try resolve_expr(state, scope, expression.start)
|
||||
try resolve_expr(state, scope, expression.end)
|
||||
}
|
||||
.call: {
|
||||
try resolve_expr(state, scope, ast.call_expr(tree, id).callee)
|
||||
for (0..ast.call_arg_count(tree, id)) |i| {
|
||||
try resolve_expr(state, scope, ast.call_arg(tree, id, i))
|
||||
}
|
||||
}
|
||||
.array_type: {
|
||||
type :: ast.array_type(tree, id)
|
||||
if (type.count) |count| try resolve_expr(state, scope, count)
|
||||
try resolve_expr(state, scope, type.element)
|
||||
}
|
||||
.array_literal: {
|
||||
for (0..ast.array_element_count(tree, id)) |i| {
|
||||
try resolve_expr(state, scope, ast.array_element(tree, id, i))
|
||||
}
|
||||
}
|
||||
.index: {
|
||||
expression :: ast.index_expr(tree, id)
|
||||
try resolve_expr(state, scope, expression.base)
|
||||
try resolve_expr(state, scope, expression.index)
|
||||
}
|
||||
.invalid: resolve_invalid(state, id)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -298,3 +298,460 @@ preserves_syntax_diagnostics_and_resolves_shared_unknown_types_once test {
|
||||
try testing.expect_equal(later, binding(&resolution, valid_ref).data.declaration)
|
||||
try testing.expect_equal(syntax_count + 3, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
resolves_conditional_arm_scopes_without_leaks_or_shadowing test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`choose proc(flag bool, value int) void {
|
||||
` if flag {
|
||||
` before :: later
|
||||
` later :: value
|
||||
` if before == value { nested :: later }
|
||||
` } else {
|
||||
` later :: value
|
||||
` }
|
||||
` later
|
||||
` if later { later :: 1 }
|
||||
` if flag {
|
||||
` value :: 2
|
||||
` bool :: false
|
||||
` if flag { value }
|
||||
` }
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
|
||||
function :: ast.package_decl(&tree, 0)
|
||||
view :: ast.function_decl(&tree, function)
|
||||
flag :: ast.function_param(&tree, function, 0)
|
||||
value :: ast.function_param(&tree, function, 1)
|
||||
try testing.expect_equal(BuiltinType.bool, binding(&resolution, ast.param(&tree, flag).type).data.builtin)
|
||||
first :: ast.if_stmt(&tree, ast.block_statement(&tree, view.body, 0))
|
||||
try testing.expect_equal(flag, binding(&resolution, first.condition).data.declaration)
|
||||
before :: ast.block_statement(&tree, first.then_block, 0)
|
||||
later :: ast.block_statement(&tree, first.then_block, 1)
|
||||
try testing.expect_equal(later, binding(&resolution, ast.statement_decl(&tree, before).initializer).data.declaration)
|
||||
try testing.expect_equal(value, binding(&resolution, ast.statement_decl(&tree, later).initializer).data.declaration)
|
||||
nested :: ast.if_stmt(&tree, ast.block_statement(&tree, first.then_block, 2))
|
||||
nested_decl :: ast.block_statement(&tree, nested.then_block, 0)
|
||||
try testing.expect_equal(later, binding(&resolution, ast.statement_decl(&tree, nested_decl).initializer).data.declaration)
|
||||
if (first.else_block) |other| {
|
||||
sibling :: ast.block_statement(&tree, other, 0)
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, sibling).kind)
|
||||
try testing.expect_equal(sibling, binding(&resolution, sibling).data.declaration)
|
||||
} else {
|
||||
return .expectation_failed
|
||||
}
|
||||
outside :: ast.statement_expr(&tree, ast.block_statement(&tree, view.body, 1))
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, outside).kind)
|
||||
independent :: ast.if_stmt(&tree, ast.block_statement(&tree, view.body, 2))
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, independent.condition).kind)
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, ast.block_statement(&tree, independent.then_block, 0)).kind)
|
||||
forbidden :: ast.if_stmt(&tree, ast.block_statement(&tree, view.body, 3))
|
||||
shadow :: ast.block_statement(&tree, forbidden.then_block, 0)
|
||||
builtin :: ast.block_statement(&tree, forbidden.then_block, 1)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, shadow).kind)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, builtin).kind)
|
||||
poisoned :: ast.if_stmt(&tree, ast.block_statement(&tree, forbidden.then_block, 2))
|
||||
use :: ast.statement_expr(&tree, ast.block_statement(&tree, poisoned.then_block, 0))
|
||||
try testing.expect_equal(binding(&resolution, shadow).data.diagnostic, binding(&resolution, use).data.diagnostic)
|
||||
try testing.expect_equal(4, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
resolves_loop_assignments_in_lexical_scopes_with_forward_local_bindings test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`main proc(flag bool) void {
|
||||
` outer := 2
|
||||
` while flag {
|
||||
` later = outer
|
||||
` later := 1
|
||||
` while later > 0 {
|
||||
` outer = later
|
||||
` inner := later
|
||||
` }
|
||||
` inner
|
||||
` }
|
||||
` while flag { later := 3 }
|
||||
` later = unknown_rhs
|
||||
` while flag {
|
||||
` outer := 1
|
||||
` outer = 2
|
||||
` bool := false
|
||||
` }
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
|
||||
function :: ast.package_decl(&tree, 0)
|
||||
body :: ast.function_decl(&tree, function).body
|
||||
flag :: ast.function_param(&tree, function, 0)
|
||||
outer :: ast.block_statement(&tree, body, 0)
|
||||
first :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 1))
|
||||
try testing.expect_equal(flag, binding(&resolution, first.condition).data.declaration)
|
||||
forward :: ast.assignment_stmt(&tree, ast.block_statement(&tree, first.body, 0))
|
||||
later :: ast.block_statement(&tree, first.body, 1)
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, forward.target).kind)
|
||||
try testing.expect_equal(later, binding(&resolution, forward.target).data.declaration)
|
||||
try testing.expect_equal(outer, binding(&resolution, forward.value).data.declaration)
|
||||
nested :: ast.while_stmt(&tree, ast.block_statement(&tree, first.body, 2))
|
||||
guard :: ast.binary_expr(&tree, nested.condition)
|
||||
try testing.expect_equal(later, binding(&resolution, guard.left).data.declaration)
|
||||
update :: ast.assignment_stmt(&tree, ast.block_statement(&tree, nested.body, 0))
|
||||
try testing.expect_equal(outer, binding(&resolution, update.target).data.declaration)
|
||||
try testing.expect_equal(later, binding(&resolution, update.value).data.declaration)
|
||||
inner :: ast.block_statement(&tree, nested.body, 1)
|
||||
try testing.expect_equal(later, binding(&resolution, ast.statement_decl(&tree, inner).initializer).data.declaration)
|
||||
escaped_inner :: ast.statement_expr(&tree, ast.block_statement(&tree, first.body, 3))
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped_inner).kind)
|
||||
sibling :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 2))
|
||||
sibling_local :: ast.block_statement(&tree, sibling.body, 0)
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, sibling_local).kind)
|
||||
try testing.expect(sibling_local != later)
|
||||
escaped :: ast.assignment_stmt(&tree, ast.block_statement(&tree, body, 3))
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped.target).kind)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped.value).kind)
|
||||
try testing.expect(binding(&resolution, escaped.target).data.diagnostic != binding(&resolution, escaped.value).data.diagnostic)
|
||||
forbidden :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 4))
|
||||
shadow :: ast.block_statement(&tree, forbidden.body, 0)
|
||||
poisoned :: ast.assignment_stmt(&tree, ast.block_statement(&tree, forbidden.body, 1))
|
||||
builtin :: ast.block_statement(&tree, forbidden.body, 2)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, shadow).kind)
|
||||
try testing.expect_equal(binding(&resolution, shadow).data.diagnostic, binding(&resolution, poisoned.target).data.diagnostic)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, builtin).kind)
|
||||
try testing.expect_equal(5, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
resolves_all_code_and_preserves_scopes_after_loop_controls test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`main proc() void {
|
||||
` while (true) {
|
||||
` break
|
||||
` before :: later
|
||||
` continue
|
||||
` later :: 1
|
||||
` missing_after_break
|
||||
` if (true) {
|
||||
` continue
|
||||
` private :: later
|
||||
` int :: 2
|
||||
` }
|
||||
` private
|
||||
` }
|
||||
` later
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
try testing.expect_equal(4, diagnostics.count(&diagnostic_store))
|
||||
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
|
||||
loop :: ast.while_stmt(&tree, ast.block_statement(&tree, body, 0))
|
||||
before :: ast.block_statement(&tree, loop.body, 1)
|
||||
later :: ast.block_statement(&tree, loop.body, 3)
|
||||
reference :: ast.statement_decl(&tree, before).initializer
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, reference).kind)
|
||||
try testing.expect_equal(later, binding(&resolution, reference).data.declaration)
|
||||
conditional :: ast.if_stmt(&tree, ast.block_statement(&tree, loop.body, 5))
|
||||
private_decl :: ast.block_statement(&tree, conditional.then_block, 1)
|
||||
private_ref :: ast.statement_decl(&tree, private_decl).initializer
|
||||
try testing.expect_equal(later, binding(&resolution, private_ref).data.declaration)
|
||||
forbidden :: ast.block_statement(&tree, conditional.then_block, 2)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, forbidden).kind)
|
||||
}
|
||||
|
||||
range_bounds_use_outer_scope_and_captures_are_body_declarations test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`run proc() void {
|
||||
` for lower..upper |item| {
|
||||
` before :: later
|
||||
` later :: item
|
||||
` continue
|
||||
` item
|
||||
` }
|
||||
` for lower..upper |item| { item }
|
||||
` lower :: 0
|
||||
` upper :: 2
|
||||
` item
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
|
||||
loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 0))
|
||||
sibling :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 1))
|
||||
lower :: ast.block_statement(&tree, body, 2)
|
||||
upper :: ast.block_statement(&tree, body, 3)
|
||||
bounds :: ast.range_expr(&tree, loop.iterable)
|
||||
try testing.expect_equal(lower, binding(&resolution, bounds.start).data.declaration)
|
||||
try testing.expect_equal(upper, binding(&resolution, bounds.end).data.declaration)
|
||||
if (loop.capture) |capture| {
|
||||
try testing.expect_equal(capture, binding(&resolution, capture).data.declaration)
|
||||
later :: ast.block_statement(&tree, loop.body, 1)
|
||||
before_ref :: ast.statement_decl(&tree, ast.block_statement(&tree, loop.body, 0)).initializer
|
||||
try testing.expect_equal(later, binding(&resolution, before_ref).data.declaration)
|
||||
capture_ref :: ast.statement_decl(&tree, later).initializer
|
||||
try testing.expect_equal(capture, binding(&resolution, capture_ref).data.declaration)
|
||||
dead_ref :: ast.statement_expr(&tree, ast.block_statement(&tree, loop.body, 3))
|
||||
try testing.expect_equal(capture, binding(&resolution, dead_ref).data.declaration)
|
||||
} else return .expectation_failed
|
||||
if (sibling.capture) |capture| {
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, capture).kind)
|
||||
sibling_ref :: ast.statement_expr(&tree, ast.block_statement(&tree, sibling.body, 0))
|
||||
try testing.expect_equal(capture, binding(&resolution, sibling_ref).data.declaration)
|
||||
} else return .expectation_failed
|
||||
escaped :: ast.statement_expr(&tree, ast.block_statement(&tree, body, 4))
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, escaped).kind)
|
||||
try testing.expect_equal(1, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
range_captures_preserve_duplicate_builtin_and_discard_errors test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`run proc() void {
|
||||
` outer :: 0
|
||||
` for outer..2 |outer| { outer }
|
||||
` for 0..2 |int| { int }
|
||||
` for 0..2 |item, item| { item }
|
||||
` for 0..2 |local| {
|
||||
` local :: 1
|
||||
` local
|
||||
` }
|
||||
` for 0..2 |_| {
|
||||
` break
|
||||
` _
|
||||
` missing
|
||||
` }
|
||||
` for missing_start..missing_end |i| {}
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
|
||||
for (usize(1)..usize(3)) |index| {
|
||||
loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, index))
|
||||
if (loop.capture) |capture| {
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, capture).kind)
|
||||
reference :: ast.statement_expr(&tree, ast.block_statement(&tree, loop.body, 0))
|
||||
try testing.expect_equal(
|
||||
binding(&resolution, capture).data.diagnostic,
|
||||
binding(&resolution, reference).data.diagnostic,
|
||||
)
|
||||
} else return .expectation_failed
|
||||
}
|
||||
indexed :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 3))
|
||||
if (indexed.index) |index| {
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, index).kind)
|
||||
} else return .expectation_failed
|
||||
if (indexed.capture) |capture| {
|
||||
reference :: ast.statement_expr(&tree, ast.block_statement(&tree, indexed.body, 0))
|
||||
try testing.expect_equal(capture, binding(&resolution, reference).data.declaration)
|
||||
} else return .expectation_failed
|
||||
local_loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 4))
|
||||
duplicate :: ast.block_statement(&tree, local_loop.body, 0)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, duplicate).kind)
|
||||
discard :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 5))
|
||||
try testing.expect(discard.capture == null)
|
||||
for (usize(1)..usize(3)) |index| {
|
||||
reference :: ast.statement_expr(&tree, ast.block_statement(&tree, discard.body, index))
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, reference).kind)
|
||||
}
|
||||
missing_loop :: ast.for_stmt(&tree, ast.block_statement(&tree, body, 6))
|
||||
bounds :: ast.range_expr(&tree, missing_loop.iterable)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, bounds.start).kind)
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, bounds.end).kind)
|
||||
try testing.expect_equal(8, diagnostics.count(&diagnostic_store))
|
||||
}
|
||||
|
||||
resolves_array_counts_elements_indices_and_unreachable_targets test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`size usize :: 2
|
||||
`make proc(first, second [size]mut [2]i64) [size]i64 {
|
||||
` return first[0]
|
||||
` dead [local_size]mut [size - 1]f64 := [[later]]
|
||||
` local_size :: size
|
||||
` later f64 :: 1.0
|
||||
` (dead[index])[0] = later
|
||||
` index usize :: 0
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
size :: ast.package_decl(&tree, 0)
|
||||
function :: ast.package_decl(&tree, 1)
|
||||
view :: ast.function_decl(&tree, function)
|
||||
return_type :: ast.array_type(&tree, view.return_type)
|
||||
try testing.expect_equal(size, binding(&resolution, return_type.count?).data.declaration)
|
||||
try testing.expect_equal(BuiltinType.i64, binding(&resolution, return_type.element).data.builtin)
|
||||
first :: ast.function_param(&tree, function, 0)
|
||||
second :: ast.function_param(&tree, function, 1)
|
||||
first_type :: ast.param(&tree, first).type
|
||||
try testing.expect_equal(first_type, ast.param(&tree, second).type)
|
||||
parameter_array :: ast.array_type(&tree, first_type)
|
||||
try testing.expect_equal(size, binding(&resolution, parameter_array.count?).data.declaration)
|
||||
row :: ast.array_type(&tree, parameter_array.element)
|
||||
try testing.expect_equal(BuiltinType.i64, binding(&resolution, row.element).data.builtin)
|
||||
returned :: ast.return_stmt(&tree, ast.block_statement(&tree, view.body, 0)).value?
|
||||
try testing.expect_equal(first, binding(&resolution, ast.index_expr(&tree, returned).base).data.declaration)
|
||||
dead :: ast.block_statement(&tree, view.body, 1)
|
||||
local_size :: ast.block_statement(&tree, view.body, 2)
|
||||
later :: ast.block_statement(&tree, view.body, 3)
|
||||
index :: ast.block_statement(&tree, view.body, 5)
|
||||
declaration :: ast.statement_decl(&tree, dead)
|
||||
array :: ast.array_type(&tree, declaration.type?)
|
||||
try testing.expect_equal(local_size, binding(&resolution, array.count?).data.declaration)
|
||||
inner :: ast.array_type(&tree, array.element)
|
||||
count_expression :: ast.binary_expr(&tree, inner.count?)
|
||||
try testing.expect_equal(size, binding(&resolution, count_expression.left).data.declaration)
|
||||
try testing.expect_equal(BuiltinType.f64, binding(&resolution, inner.element).data.builtin)
|
||||
literal_row :: ast.array_element(&tree, declaration.initializer, 0)
|
||||
try testing.expect_equal(later, binding(
|
||||
&resolution, ast.array_element(&tree, literal_row, 0),
|
||||
).data.declaration)
|
||||
assignment :: ast.assignment_stmt(&tree, ast.block_statement(&tree, view.body, 4))
|
||||
target :: ast.index_expr(&tree, ast.index_expr(&tree, assignment.target).base)
|
||||
try testing.expect_equal(dead, binding(&resolution, target.base).data.declaration)
|
||||
try testing.expect_equal(index, binding(&resolution, target.index).data.declaration)
|
||||
try testing.expect_equal(later, binding(&resolution, assignment.value).data.declaration)
|
||||
try testing.expect_equal(BuiltinType.usize, binding(
|
||||
&resolution, ast.statement_decl(&tree, size).type?,
|
||||
).data.builtin)
|
||||
}
|
||||
|
||||
array_signature_counts_use_package_scope_and_local_counts_see_parameters test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`size :: 2
|
||||
`inspect proc(n usize, a, b [n]i64) [body_size]i64 {
|
||||
` body_size :: size
|
||||
` local [n]i64 :: []
|
||||
` return []
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
try testing.expect_equal(0, diagnostics.count(&diagnostic_store))
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
try testing.expect_equal(2, diagnostics.count(&diagnostic_store))
|
||||
function :: ast.package_decl(&tree, 1)
|
||||
view :: ast.function_decl(&tree, function)
|
||||
parameter :: ast.function_param(&tree, function, 0)
|
||||
grouped_type :: ast.param(&tree, ast.function_param(&tree, function, 1)).type
|
||||
try testing.expect_equal(grouped_type, ast.param(
|
||||
&tree, ast.function_param(&tree, function, 2),
|
||||
).type)
|
||||
signature_count :: ast.array_type(&tree, grouped_type).count?
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, signature_count).kind)
|
||||
result_count :: ast.array_type(&tree, view.return_type).count?
|
||||
try testing.expect_equal(BindingKind.invalid, binding(&resolution, result_count).kind)
|
||||
local :: ast.statement_decl(&tree, ast.block_statement(&tree, view.body, 1))
|
||||
local_count :: ast.array_type(&tree, local.type?).count?
|
||||
try testing.expect_equal(BindingKind.declaration, binding(&resolution, local_count).kind)
|
||||
try testing.expect_equal(parameter, binding(&resolution, local_count).data.declaration)
|
||||
try testing.expect_equal(BuiltinType.i64, binding(
|
||||
&resolution, ast.array_type(&tree, local.type?).element,
|
||||
).data.builtin)
|
||||
}
|
||||
|
||||
resolves_every_array_child_error_and_preserves_lexer_diagnostics test {
|
||||
strpool.strings = strpool.init(mem.c_allocator)
|
||||
defer strpool.deinit(&strpool.strings)
|
||||
diagnostic_store := diagnostics.init(mem.c_allocator)
|
||||
defer diagnostics.deinit(&diagnostic_store)
|
||||
source ::
|
||||
`main proc() void {
|
||||
` return
|
||||
` data [missing_count]MissingElement :: [absent, missing_element]
|
||||
` (missing_base[missing_index])[missing_outer] = missing_rhs
|
||||
` malformed [$]i64 :: [$]
|
||||
`}
|
||||
scan_state := lexer.init(mem.c_allocator)
|
||||
defer lexer.deinit(&scan_state)
|
||||
try lexer.scan(&scan_state, source, &diagnostic_store)
|
||||
tree := try parser.parse(mem.c_allocator, scan_state.tokens.items, source, &diagnostic_store)
|
||||
defer ast.deinit(&tree)
|
||||
try testing.expect_equal(2, diagnostics.count(&diagnostic_store))
|
||||
resolution := try resolve(mem.c_allocator, &tree, &diagnostic_store)
|
||||
defer deinit(&resolution)
|
||||
try testing.expect_equal(10, diagnostics.count(&diagnostic_store))
|
||||
body :: ast.function_decl(&tree, ast.package_decl(&tree, 0)).body
|
||||
declaration :: ast.statement_decl(&tree, ast.block_statement(&tree, body, 1))
|
||||
type :: ast.array_type(&tree, declaration.type?)
|
||||
assignment :: ast.assignment_stmt(&tree, ast.block_statement(&tree, body, 2))
|
||||
outer :: ast.index_expr(&tree, assignment.target)
|
||||
inner :: ast.index_expr(&tree, outer.base)
|
||||
unknowns [8]ast.NodeId :: [
|
||||
type.count?, type.element,
|
||||
ast.array_element(&tree, declaration.initializer, 0),
|
||||
ast.array_element(&tree, declaration.initializer, 1),
|
||||
inner.base, inner.index, outer.index, assignment.value,
|
||||
]
|
||||
for (unknowns) |id| {
|
||||
result :: binding(&resolution, id)
|
||||
try testing.expect_equal(BindingKind.invalid, result.kind)
|
||||
try testing.expect_equal(diagnostics.Code.unknown_name, diagnostics.get(
|
||||
&diagnostic_store, result.data.diagnostic,
|
||||
)?.code)
|
||||
}
|
||||
malformed :: ast.statement_decl(&tree, ast.block_statement(&tree, body, 3))
|
||||
malformed_count :: ast.array_type(&tree, malformed.type?).count?
|
||||
malformed_element :: ast.array_element(&tree, malformed.initializer, 0)
|
||||
try testing.expect_equal(diagnostics.DiagnosticId(0), binding(&resolution, malformed_count).data.diagnostic)
|
||||
try testing.expect_equal(diagnostics.DiagnosticId(1), binding(&resolution, malformed_element).data.diagnostic)
|
||||
}
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import "@std/meta"
|
||||
|
||||
#! keys may be enums or native enum compositions that normalize to an enum, including aliases.
|
||||
#! shared variant names follow native composition identity; payload-bearing sums are not enum keys.
|
||||
EnumMap proc($E, $V type) type {
|
||||
match typeinfo!(E) {
|
||||
.enum |info|: return struct {
|
||||
|
||||
@@ -24,3 +24,64 @@ handles_sparse_enum_get test {
|
||||
try testing.expect_type(?[]u8, eof)
|
||||
try testing.expect_equal(null, eof)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
TestLeftError :: enum {
|
||||
shared
|
||||
left
|
||||
absent_left
|
||||
}
|
||||
|
||||
@hide:file
|
||||
TestRightError :: enum {
|
||||
shared
|
||||
right
|
||||
absent_right
|
||||
}
|
||||
|
||||
@hide:file
|
||||
TestNestedError :: alias (TestLeftError | (TestRightError | TestLeftError))
|
||||
|
||||
@hide:file
|
||||
TestComposedError :: alias TestNestedError
|
||||
|
||||
@hide:file
|
||||
composed_values :: $init(TestComposedError, i32, {
|
||||
shared = 11,
|
||||
left = 22,
|
||||
absent_left = 33,
|
||||
right = 44,
|
||||
absent_right = 55,
|
||||
})
|
||||
|
||||
handles_composed_enum_keys_at_runtime test {
|
||||
names EnumMap(TestComposedError, []u8) :: init({
|
||||
shared = "shared error",
|
||||
left = "left error",
|
||||
right = "right error",
|
||||
})
|
||||
keys [_]TestComposedError :: [
|
||||
TestLeftError.shared,
|
||||
TestLeftError.left,
|
||||
TestRightError.shared,
|
||||
TestRightError.right,
|
||||
TestLeftError.absent_left,
|
||||
TestRightError.absent_right,
|
||||
]
|
||||
expected [_]?[]u8 :: [
|
||||
"shared error",
|
||||
"left error",
|
||||
"shared error",
|
||||
"right error",
|
||||
null,
|
||||
null,
|
||||
]
|
||||
expected_values [_]i32 :: [11, 22, 11, 44, 33, 55]
|
||||
|
||||
for keys |key, index| {
|
||||
value :: get(&names, key)
|
||||
try testing.expect_type(?[]u8, value)
|
||||
try testing.expect_equal(expected[index], value)
|
||||
try testing.expect_equal(expected_values[index], get(&composed_values, key)?)
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -19,7 +19,7 @@ CloseError :: enum {
|
||||
close_failed
|
||||
}
|
||||
|
||||
#! open a caller-owned handle; release it with close.
|
||||
#! open a caller-owned handle from a borrowed zero-terminated path; release it with close.
|
||||
open proc(io Io, path [;0]u8, mode FileMode) File ! OpenError {
|
||||
handle Handle :: try io.vtable.open(io.context, path, mode)
|
||||
return File{ io = io, handle = handle }
|
||||
|
||||
+36
-1
@@ -1,5 +1,6 @@
|
||||
import "@ffi/c"
|
||||
import "@std/mem"
|
||||
import "@std/arraylist"
|
||||
# typeinfo! requires this import even without a qualified meta reference.
|
||||
import "@std/meta"
|
||||
|
||||
@@ -8,6 +9,12 @@ ReadError :: enum {
|
||||
read_failed
|
||||
}
|
||||
|
||||
ReadLimitError :: enum {
|
||||
limit_exceeded
|
||||
}
|
||||
|
||||
ReadAllError :: alias (mem.AllocError | ReadError | ReadLimitError)
|
||||
|
||||
WriteError :: enum {
|
||||
not_open_for_writing
|
||||
write_failed
|
||||
@@ -64,6 +71,34 @@ read proc(input Reader, buffer []mut u8) usize ! ReadError {
|
||||
return count
|
||||
}
|
||||
|
||||
#! read to EOF, accepting at most `maximum` bytes; the reader remains caller-owned.
|
||||
#! exceeding the limit consumes one extra byte and returns limit_exceeded.
|
||||
#! returned storage retains `allocator`; release it with arraylist.deinit.
|
||||
#! all allocated storage is released on error.
|
||||
read_all proc(allocator mem.Allocator, input Reader, maximum usize) arraylist.ArrayList(u8) ! ReadAllError {
|
||||
bytes := arraylist.init(u8, allocator)
|
||||
errdefer arraylist.deinit(&bytes)
|
||||
while true {
|
||||
length :: bytes.items.len
|
||||
if (length == maximum) {
|
||||
probe [1]mut u8 := undefined
|
||||
if (try read(input, (&probe)[..]) != 0) return .limit_exceeded
|
||||
return bytes
|
||||
}
|
||||
if (length == bytes.capacity) {
|
||||
growth := usize(4096)
|
||||
if (growth > maximum - length) growth = maximum - length
|
||||
try arraylist.reserve(&bytes, length + growth)
|
||||
}
|
||||
storage :: bytes.items.ptr[..bytes.capacity]
|
||||
end := bytes.capacity
|
||||
if (end > maximum) end = maximum
|
||||
count :: try read(input, storage[length..end])
|
||||
if (count == 0) return bytes
|
||||
bytes.items = storage[..length + count]
|
||||
}
|
||||
}
|
||||
|
||||
write proc(output Writer, bytes []u8) usize ! WriteError {
|
||||
if (bytes.len == 0) return 0
|
||||
|
||||
@@ -392,7 +427,7 @@ write_float proc($T type, output Writer, value T, scientific bool) void ! WriteE
|
||||
match typeinfo!(T) {
|
||||
.float: {
|
||||
buffer [64]mut u8 := undefined
|
||||
count := 0
|
||||
count c_int := 0
|
||||
if sizeof!(T) == 4 {
|
||||
if (scientific) count = c.snprintf(ptrcast!((&buffer).ptr), c_ulong(buffer.len), "%.8e", value)
|
||||
else count = c.snprintf(ptrcast!((&buffer).ptr), c_ulong(buffer.len), "%.9g", value)
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
import "@std/mem"
|
||||
import "@std/arraylist"
|
||||
import "@std/testing"
|
||||
|
||||
@hide:file
|
||||
Input :: struct {
|
||||
bytes []u8
|
||||
offset usize = 0
|
||||
chunk usize = 137
|
||||
fail_at usize = maxval!(usize)
|
||||
remaining usize = maxval!(usize)
|
||||
live usize = 0
|
||||
}
|
||||
|
||||
@hide:file
|
||||
input_state proc(context ?@mut anyopaque) @mut Input {
|
||||
if (context) |pointer| return ptrcast!(pointer)
|
||||
unreachable
|
||||
}
|
||||
|
||||
@hide:file
|
||||
input_read proc(context ?@mut anyopaque, _ Handle, buffer []mut u8) usize ! ReadError {
|
||||
state :: input_state(context)
|
||||
if (state.offset >= state.fail_at) return .read_failed
|
||||
count := state.bytes.len - state.offset
|
||||
if (count > buffer.len) count = buffer.len
|
||||
if (count > state.chunk) count = state.chunk
|
||||
memcopy!(buffer[..count], state.bytes[state.offset..state.offset + count])
|
||||
state.offset += count
|
||||
return count
|
||||
}
|
||||
|
||||
@hide:file
|
||||
input_reader proc(state @mut Input) Reader {
|
||||
return Reader{ context = state, handle = Handle{ file_desc = -1 }, read = input_read }
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_alloc proc(context ?@mut anyopaque, size usize, alignment usize) ?*mut u8 {
|
||||
state :: input_state(context)
|
||||
if (state.remaining == 0) return null
|
||||
state.remaining -= 1
|
||||
memory :: mem.raw_alloc(mem.c_allocator, size, alignment)
|
||||
if (memory) |_| state.live += 1
|
||||
return memory
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_realloc proc(context ?@mut anyopaque, memory ?*mut u8, old_size usize, new_size usize, alignment usize) ?*mut u8 {
|
||||
state :: input_state(context)
|
||||
if (state.remaining == 0) return null
|
||||
state.remaining -= 1
|
||||
resized :: mem.raw_realloc(mem.c_allocator, memory, old_size, new_size, alignment)
|
||||
if (resized) |_| {
|
||||
if (memory) |_| {} else state.live += 1
|
||||
}
|
||||
return resized
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_free proc(context ?@mut anyopaque, memory ?*mut u8, size usize, alignment usize) void {
|
||||
state :: input_state(context)
|
||||
if (memory) |_| state.live -= 1
|
||||
mem.raw_free(mem.c_allocator, memory, size, alignment)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_vtable :: mem.AllocatorVTable{ alloc = tracked_alloc, realloc = tracked_realloc, free = tracked_free }
|
||||
|
||||
read_all_preserves_short_reads_across_buffer_growth test {
|
||||
source [9000]mut u8 := undefined
|
||||
for (0..source.len) |i| source[i] = u8(rem!(i, 251))
|
||||
state := Input{ bytes = source[..] }
|
||||
bytes := try read_all(mem.c_allocator, input_reader(&state), maxval!(usize))
|
||||
defer arraylist.deinit(&bytes)
|
||||
try testing.expect_equal(source[..], bytes.items)
|
||||
}
|
||||
|
||||
read_all_accepts_exact_limits_and_probes_for_excess test {
|
||||
for (usize(0)..4) |maximum| {
|
||||
state := Input{ bytes = "abc"[..maximum], chunk = 1 }
|
||||
bytes := try read_all(mem.c_allocator, input_reader(&state), maximum)
|
||||
defer arraylist.deinit(&bytes)
|
||||
try testing.expect_equal(state.bytes, bytes.items)
|
||||
|
||||
excess := Input{ bytes = "abcd", chunk = 1 }
|
||||
unexpected := read_all(mem.c_allocator, input_reader(&excess), maximum) catch |err| {
|
||||
try testing.expect(err == .limit_exceeded)
|
||||
try testing.expect_equal(maximum + 1, excess.offset)
|
||||
continue
|
||||
}
|
||||
arraylist.deinit(&unexpected)
|
||||
try testing.expect(false)
|
||||
}
|
||||
}
|
||||
|
||||
read_all_releases_partial_storage_on_read_failure_and_limit_excess test {
|
||||
for (0..2) |failure| {
|
||||
state := Input{ bytes = "abcdef", chunk = 2 }
|
||||
if (failure == 0) state.fail_at = 2
|
||||
allocator :: mem.Allocator{ context = &state, vtable = &tracked_vtable }
|
||||
unexpected := read_all(allocator, input_reader(&state), 4) catch |err| {
|
||||
expected ReadAllError :: if (failure == 0) .read_failed else .limit_exceeded
|
||||
try testing.expect_equal(expected, err)
|
||||
try testing.expect_equal(0, state.live)
|
||||
continue
|
||||
}
|
||||
arraylist.deinit(&unexpected)
|
||||
try testing.expect(false)
|
||||
}
|
||||
}
|
||||
|
||||
read_all_releases_storage_at_each_failed_growth test {
|
||||
source [9000]mut u8 := undefined
|
||||
mem.set(source[..], u8('x'))
|
||||
for (usize(0)..3) |remaining| {
|
||||
state := Input{ bytes = source[..], remaining = remaining }
|
||||
allocator :: mem.Allocator{ context = &state, vtable = &tracked_vtable }
|
||||
unexpected := read_all(allocator, input_reader(&state), source.len) catch |err| {
|
||||
try testing.expect(err == .out_of_memory)
|
||||
try testing.expect_equal(0, state.live)
|
||||
continue
|
||||
}
|
||||
arraylist.deinit(&unexpected)
|
||||
try testing.expect(false)
|
||||
}
|
||||
}
|
||||
@@ -2,4 +2,10 @@ import "@std/io"
|
||||
|
||||
Init :: struct {
|
||||
io io.Io
|
||||
args Args
|
||||
}
|
||||
|
||||
#! borrows argument slices and their zero-terminated strings, valid until main returns.
|
||||
Args :: struct {
|
||||
values [][;0]u8
|
||||
}
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import "@ffi/c"
|
||||
import "@std/mem"
|
||||
|
||||
ParseError :: enum {
|
||||
invalid_number
|
||||
out_of_range
|
||||
}
|
||||
|
||||
#! parse a complete ASCII decimal with an optional sign, decimal point, and exponent.
|
||||
#! at least one mantissa digit and, after an exponent marker, one exponent digit are required.
|
||||
#! whitespace, embedded NUL bytes, trailing bytes, hexadecimal, infinities, and NaNs are invalid.
|
||||
#! finite subnormals and signed zero succeed; overflow and nonzero values rounded to zero are out of range.
|
||||
#! temporary storage uses `allocator` and is released before returning; the input is only borrowed.
|
||||
#! the Darwin backend uses strtod_l with LC_C_LOCALE, independent of the process and thread locales.
|
||||
#! decimal rounding follows native libc and the caller's floating-point rounding mode.
|
||||
parse_float proc(allocator mem.Allocator, text []u8) f64 ! (mem.AllocError | ParseError) {
|
||||
if (text.len == 0) return .invalid_number
|
||||
cursor := usize(0)
|
||||
if (text[cursor] == '+' or text[cursor] == '-') cursor += 1
|
||||
|
||||
digits := usize(0)
|
||||
nonzero := false
|
||||
point := false
|
||||
while cursor < text.len {
|
||||
byte :: text[cursor]
|
||||
if (byte >= '0' and byte <= '9') {
|
||||
digits += 1
|
||||
if (byte != '0') nonzero = true
|
||||
} else if (byte == '.' and !point) {
|
||||
point = true
|
||||
} else break
|
||||
cursor += 1
|
||||
}
|
||||
if (digits == 0) return .invalid_number
|
||||
|
||||
if (cursor < text.len and (text[cursor] == 'e' or text[cursor] == 'E')) {
|
||||
cursor += 1
|
||||
if (cursor < text.len and (text[cursor] == '+' or text[cursor] == '-')) cursor += 1
|
||||
start :: cursor
|
||||
while (cursor < text.len and text[cursor] >= '0' and text[cursor] <= '9') cursor += 1
|
||||
if (cursor == start) return .invalid_number
|
||||
}
|
||||
if (cursor != text.len) return .invalid_number
|
||||
if (text.len == maxval!(usize)) return .out_of_memory
|
||||
|
||||
buffer :: try mem.alloc(u8, allocator, text.len + 1)
|
||||
defer mem.free(allocator, buffer)
|
||||
memcopy!(buffer[..text.len], text)
|
||||
buffer[text.len] = 0
|
||||
c.__error()?^ = 0
|
||||
value :: f64(strtod_l(ptrcast!(buffer.ptr), null, null))
|
||||
range_error :: c.__error()?^ == c.ERANGE
|
||||
|
||||
# binary64 magnitudes preserve subnormals despite libc's ERANGE; infinity starts at 9218868437227405312.
|
||||
# ERANGE at magnitude >= 1 also catches overflow saturated by directed rounding.
|
||||
magnitude :: ptrcast!(@u64, &value)^ & u64(9223372036854775807)
|
||||
if (
|
||||
magnitude >= u64(9218868437227405312)
|
||||
or (magnitude == 0 and nonzero)
|
||||
or (range_error and magnitude >= u64(4607182418800017408))
|
||||
) return .out_of_range
|
||||
return value
|
||||
}
|
||||
|
||||
# Darwin's locale_t is an opaque pointer and LC_C_LOCALE is null (<_locale_posix2008.h>).
|
||||
@hide:file
|
||||
strtod_l c_func(text ?*c_char, end ?*mut ?*mut c_char, locale ?*mut anyopaque) c_double
|
||||
@@ -0,0 +1,150 @@
|
||||
import "@std/mem"
|
||||
import "@std/testing"
|
||||
|
||||
@hide:file
|
||||
expect_bits proc(text []u8, expected u64) void ! testing.Error {
|
||||
value :: parse_float(mem.c_allocator, text) catch |_| { return .expectation_failed }
|
||||
try testing.expect_equal(expected, ptrcast!(@u64, &value)^)
|
||||
}
|
||||
|
||||
accepts_signed_decimal_forms_and_exponents test {
|
||||
texts [7][]u8 :: ["1", "+1", "01", "1.", ".1e1", "10e-1", "1E+0"]
|
||||
for (texts) |text| try testing.expect_equal(f64(1), try parse_float(mem.c_allocator, text))
|
||||
try testing.expect_equal(f64(-1.25), try parse_float(mem.c_allocator, "-1.25"))
|
||||
try testing.expect_equal(f64(0.5), try parse_float(mem.c_allocator, "+.5"))
|
||||
try testing.expect_equal(f64(125), try parse_float(mem.c_allocator, "1.25e+2"))
|
||||
try testing.expect_equal(f64(-0.125), try parse_float(mem.c_allocator, "-125E-3"))
|
||||
}
|
||||
|
||||
accepts_borrowed_slices_without_a_terminator test {
|
||||
bytes [5]mut u8 := ['x', '1', '.', '5', 'x']
|
||||
try testing.expect_equal(f64(1.5), try parse_float(mem.c_allocator, bytes[1..4]))
|
||||
try testing.expect_equal(u8('x'), bytes[0])
|
||||
try testing.expect_equal(u8('x'), bytes[4])
|
||||
}
|
||||
|
||||
rejects_malformed_or_partially_consumed_input test {
|
||||
texts [33][]u8 :: [
|
||||
"", "+", "-", ".", "+.", "-.", "e1", ".e1",
|
||||
"1e", "1e+", "1e-", "1ee2", "1e1.0", "1.2.3", "--1", "+-1",
|
||||
" 1", "1 ", "\t1", "1\n", "1x", "1_0", "1,5", "1\x00", "1\x002",
|
||||
"0x1", "0x1p0", "inf", "-Infinity", "nan", "NaN(1)", "\xff", "1\xff",
|
||||
]
|
||||
for texts |text| {
|
||||
_ = parse_float(mem.c_allocator, text) catch |error| {
|
||||
try testing.expect(error == .invalid_number)
|
||||
continue
|
||||
}
|
||||
try testing.expect(false)
|
||||
}
|
||||
}
|
||||
|
||||
preserves_signed_zero_even_with_extreme_exponents test {
|
||||
positive [4][]u8 :: ["0", "+0.0", "0e999999", "0e-999999"]
|
||||
negative [4][]u8 :: ["-0", "-0.0", "-0e999999", "-0e-999999"]
|
||||
for positive |text| {
|
||||
try expect_bits(text, u64(0))
|
||||
}
|
||||
for negative |text| {
|
||||
try expect_bits(text, maxval!(u64) - u64(9223372036854775807))
|
||||
}
|
||||
}
|
||||
|
||||
rounds_decimal_midpoints_to_nearest_even test {
|
||||
# IEEE-754 binary64 encodings: 0.1, 1, and the next two values above 1.
|
||||
try expect_bits("0.1", u64(4591870180066957722))
|
||||
try expect_bits("1.00000000000000011102230246251565404236316680908203125", u64(4607182418800017408))
|
||||
try expect_bits("1.00000000000000011102230246251565404236316680908203126", u64(4607182418800017409))
|
||||
try expect_bits("1.00000000000000033306690738754696212708950042724609375", u64(4607182418800017410))
|
||||
try expect_bits("9007199254740993", u64(4845873199050653696))
|
||||
}
|
||||
|
||||
accepts_finite_representational_boundaries_and_subnormals test {
|
||||
# smallest subnormal, largest subnormal, smallest normal, and largest finite binary64.
|
||||
try expect_bits("4.9406564584124654e-324", u64(1))
|
||||
try expect_bits("2.4703282292062328e-324", u64(1))
|
||||
try expect_bits("-4.9406564584124654e-324", maxval!(u64) - u64(9223372036854775806))
|
||||
try expect_bits("2.225073858507201e-308", u64(4503599627370495))
|
||||
try expect_bits("2.2250738585072014e-308", u64(4503599627370496))
|
||||
try expect_bits("1.7976931348623157e308", u64(9218868437227405311))
|
||||
try expect_bits("1.7976931348623158e308", u64(9218868437227405311))
|
||||
try expect_bits("-1.7976931348623157e308", maxval!(u64) - u64(4503599627370496))
|
||||
}
|
||||
|
||||
rejects_overflow_and_nonzero_underflow_to_zero test {
|
||||
texts [12][]u8 :: [
|
||||
"1e309", "-1e309", "1.7976931348623159e308", "-1.7976931348623159e308",
|
||||
"1e999999", "-1e999999", "1e-999999", "-1e-999999", "1e-324", "-1e-324",
|
||||
"2.4703282292062327e-324", "-2.4703282292062327e-324",
|
||||
]
|
||||
for texts |text| {
|
||||
_ = parse_float(mem.c_allocator, text) catch |error| {
|
||||
try testing.expect(error == .out_of_range)
|
||||
continue
|
||||
}
|
||||
try testing.expect(false)
|
||||
}
|
||||
}
|
||||
|
||||
@hide:file
|
||||
AllocationState :: struct {
|
||||
remaining usize = maxval!(usize)
|
||||
live usize = 0
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_alloc proc(context ?@mut anyopaque, size usize, alignment usize) ?*mut u8 {
|
||||
state @mut AllocationState :: ptrcast!(context?)
|
||||
if (state.remaining == 0) return null
|
||||
state.remaining -= 1
|
||||
memory :: mem.raw_alloc(mem.c_allocator, size, alignment)
|
||||
if (memory) |_| state.live += 1
|
||||
return memory
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_realloc proc(context ?@mut anyopaque, memory ?*mut u8, old_size usize, new_size usize, alignment usize) ?*mut u8 {
|
||||
state @mut AllocationState :: ptrcast!(context?)
|
||||
if (state.remaining == 0) return null
|
||||
state.remaining -= 1
|
||||
resized :: mem.raw_realloc(mem.c_allocator, memory, old_size, new_size, alignment)
|
||||
if (new_size == 0) {
|
||||
if (memory) |_| state.live -= 1
|
||||
} else if (resized) |_| {
|
||||
if (memory) |_| {} else state.live += 1
|
||||
}
|
||||
return resized
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_free proc(context ?@mut anyopaque, memory ?*mut u8, size usize, alignment usize) void {
|
||||
state @mut AllocationState :: ptrcast!(context?)
|
||||
if (memory) |_| state.live -= 1
|
||||
mem.raw_free(mem.c_allocator, memory, size, alignment)
|
||||
}
|
||||
|
||||
@hide:file
|
||||
tracked_vtable :: mem.AllocatorVTable{ alloc = tracked_alloc, realloc = tracked_realloc, free = tracked_free }
|
||||
|
||||
releases_temporary_storage_and_propagates_allocation_failure test {
|
||||
state := AllocationState{}
|
||||
allocator :: mem.Allocator{ context = &state, vtable = &tracked_vtable }
|
||||
try testing.expect_equal(f64(1.5), try parse_float(allocator, "1.5"))
|
||||
try testing.expect_equal(usize(0), state.live)
|
||||
texts [2][]u8 :: ["1e999999", "1e-999999"]
|
||||
for texts |text| {
|
||||
_ = parse_float(allocator, text) catch |error| {
|
||||
try testing.expect(error == .out_of_range)
|
||||
try testing.expect_equal(usize(0), state.live)
|
||||
continue
|
||||
}
|
||||
try testing.expect(false)
|
||||
}
|
||||
state.remaining = 0
|
||||
_ = parse_float(allocator, "1.5") catch |error| {
|
||||
try testing.expect(error == .out_of_memory)
|
||||
try testing.expect_equal(usize(0), state.live)
|
||||
return
|
||||
}
|
||||
try testing.expect(false)
|
||||
}
|
||||
Reference in New Issue
Block a user