feat: support binary octal and hexadecimal integer literals
This commit is contained in:
@@ -30,6 +30,7 @@ roadmap and milestone history.
|
|||||||
- exact-width integers, concrete pointer-sized `isize` / `usize`, `f32`, `f64`, `bool`, `void`, `noreturn`, and `anyopaque`; `noreturn` is a bottom type valid as a native function result and coerces to any expected value type; contextual `int` accepts the whole integer family, while `uint` accepts only unsigned native and target-classified C integers
|
- exact-width integers, concrete pointer-sized `isize` / `usize`, `f32`, `f64`, `bool`, `void`, `noreturn`, and `anyopaque`; `noreturn` is a bottom type valid as a native function result and coerces to any expected value type; contextual `int` accepts the whole integer family, while `uint` accepts only unsigned native and target-classified C integers
|
||||||
- target-dependent C scalar primitives from `c_char` through `c_longdouble`, kept semantically distinct from native scalars
|
- target-dependent C scalar primitives from `c_char` through `c_longdouble`, kept semantically distinct from native scalars
|
||||||
- contextual integer/float/character literals, backward type-demand inference through names and arithmetic/bitwise expressions, and typed compile-time evaluation for numeric constant expressions
|
- contextual integer/float/character literals, backward type-demand inference through names and arithmetic/bitwise expressions, and typed compile-time evaluation for numeric constant expressions
|
||||||
|
- integer literals use decimal digits or explicit binary `0b`, octal `0o`, and hexadecimal `0x` prefixes; uppercase prefixes and hexadecimal digits are also accepted. leading-zero unprefixed literals remain decimal, signs remain unary operators, and tuple ordinals remain canonical decimal. digits must match the base and magnitudes must fit `u64` before contextual range checking; digit separators and nondecimal floating-point literals are not supported.
|
||||||
- strict numeric conversion by default, widening where valid, C scalar coercions at C boundaries, and explicit scalar casts through keywords or transparent aliases, such as `i32(x)`, `c_float(x)`, or `StringId(x)`
|
- strict numeric conversion by default, widening where valid, C scalar coercions at C boundaries, and explicit scalar casts through keywords or transparent aliases, such as `i32(x)`, `c_float(x)`, or `StringId(x)`
|
||||||
- implicit contextual integer constants must be exactly representable in the destination float, including folded expressions and inferred constants; `f32` rejects `16777217` but accepts `16777218`, while `f64` rejects `9007199254740993` but accepts `9007199254740994`; powers of two remain exact
|
- implicit contextual integer constants must be exactly representable in the destination float, including folded expressions and inferred constants; `f32` rejects `16777217` but accepts `16777218`, while `f64` rejects `9007199254740993` but accepts `9007199254740994`; powers of two remain exact
|
||||||
- float literals retain IEEE rounding, and explicit scalar casts such as `f32(16777217)` or `f64(9007199254740993)` deliberately permit rounding; existing typed-value widening and C scalar coercion rules are unchanged
|
- float literals retain IEEE rounding, and explicit scalar casts such as `f32(16777217)` or `f64(9007199254740993)` deliberately permit rounding; existing typed-value widening and C scalar coercion rules are unchanged
|
||||||
|
|||||||
@@ -248,6 +248,7 @@ Current prototype features:
|
|||||||
- String literals as immutable pointers to static zero-terminated byte arrays
|
- String literals as immutable pointers to static zero-terminated byte arrays
|
||||||
- Pointer-preserving `.ptr`/`.len`, pointer-to-array indexing and slicing, postfix pointer dereference and optional unwrap, and keyed struct literals
|
- Pointer-preserving `.ptr`/`.len`, pointer-to-array indexing and slicing, postfix pointer dereference and optional unwrap, and keyed struct literals
|
||||||
- Contextual integer constants and typed compile-time evaluation of arithmetic and Zig-style bitwise expressions
|
- Contextual integer constants and typed compile-time evaluation of arithmetic and Zig-style bitwise expressions
|
||||||
|
- Decimal, binary (`0b101010`), octal (`0o52`), and hexadecimal (`0x2a`) integer literals, with uppercase prefix variants, checked `u64` magnitudes, and ordinary contextual typing; leading zeros alone do not select octal
|
||||||
- Contextual integer-to-float constants require exact representation (`f32` accepts `16777218`, not `16777217`; `f64` accepts `9007199254740994`, not `9007199254740993`), including folded and inferred constants; float literals and explicit scalar casts retain IEEE rounding
|
- Contextual integer-to-float constants require exact representation (`f32` accepts `16777218`, not `16777217`; `f64` accepts `9007199254740994`, not `9007199254740993`), including folded and inferred constants; float literals and explicit scalar casts retain IEEE rounding
|
||||||
- Integer `~`, `&`, `|`, `xor`, guarded `<<` / `>>`, saturating `<<|`, and their compound assignments; postfix `^` remains pointer dereference
|
- Integer `~`, `&`, `|`, `xor`, guarded `<<` / `>>`, saturating `<<|`, and their compound assignments; postfix `^` remains pointer dereference
|
||||||
- Scalar-backed nominal `distinct` types with same-identity runtime/comptime operators, explicit backing extraction casts, integer bounds, reflection, and recursive standard formatting
|
- Scalar-backed nominal `distinct` types with same-identity runtime/comptime operators, explicit backing extraction casts, integer bounds, reflection, and recursive standard formatting
|
||||||
|
|||||||
@@ -411,6 +411,35 @@ lex :: proc(
|
|||||||
case:
|
case:
|
||||||
if value >= '0' && value <= '9' {
|
if value >= '0' && value <= '9' {
|
||||||
start := cursor
|
start := cursor
|
||||||
|
base: byte
|
||||||
|
if value == '0' && cursor+1 < len(bytes) {
|
||||||
|
switch bytes[cursor+1] {
|
||||||
|
case 'b', 'B': base = 2
|
||||||
|
case 'o', 'O': base = 8
|
||||||
|
case 'x', 'X': base = 16
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if base != 0 {
|
||||||
|
cursor += 2
|
||||||
|
digits_start := cursor
|
||||||
|
valid := true
|
||||||
|
for cursor < len(bytes) && is_identifier_continue(bytes[cursor]) {
|
||||||
|
digit := bytes[cursor]
|
||||||
|
valid = valid && is_hex_digit(digit) && (base == 16 || digit-'0' < base)
|
||||||
|
cursor += 1
|
||||||
|
}
|
||||||
|
if !valid || cursor == digits_start {
|
||||||
|
id := source.add(
|
||||||
|
diagnostics,
|
||||||
|
source.Span{file=source_file.id, start=source.Offset(start), end=source.Offset(cursor)},
|
||||||
|
"invalid digits in prefixed integer literal",
|
||||||
|
)
|
||||||
|
append_token(&stream, source_file, .Invalid, start, cursor, diagnostic=id)
|
||||||
|
} else {
|
||||||
|
append_token(&stream, source_file, .Integer, start, cursor)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
for cursor < len(bytes) && bytes[cursor] >= '0' && bytes[cursor] <= '9' {
|
for cursor < len(bytes) && bytes[cursor] >= '0' && bytes[cursor] <= '9' {
|
||||||
cursor += 1
|
cursor += 1
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -239,6 +239,7 @@ parse_type_constant :: proc(parser: ^Parser) -> (u64, bool) {
|
|||||||
advance(parser)
|
advance(parser)
|
||||||
value, ok := parse_integer_magnitude(token_text(parser, tok))
|
value, ok := parse_integer_magnitude(token_text(parser, tok))
|
||||||
if !ok {
|
if !ok {
|
||||||
|
source.add(parser.diagnostics, tok.span, "integer literal magnitude does not fit in u64")
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
if negative {
|
if negative {
|
||||||
@@ -906,21 +907,37 @@ parse_anonymous_struct_type_expr :: proc(parser: ^Parser) -> ast.Expr_Id {
|
|||||||
}
|
}
|
||||||
|
|
||||||
parse_integer_magnitude :: proc(text: string) -> (u64, bool) {
|
parse_integer_magnitude :: proc(text: string) -> (u64, bool) {
|
||||||
|
base := u64(10)
|
||||||
|
digits := text
|
||||||
|
if len(text) >= 2 && text[0] == '0' {
|
||||||
|
switch text[1] {
|
||||||
|
case 'b', 'B': base = 2
|
||||||
|
case 'o', 'O': base = 8
|
||||||
|
case 'x', 'X': base = 16
|
||||||
|
}
|
||||||
|
if base != 10 {
|
||||||
|
digits = text[2:]
|
||||||
|
}
|
||||||
|
}
|
||||||
value: u64
|
value: u64
|
||||||
for byte in transmute([]byte)text {
|
for byte in transmute([]byte)digits {
|
||||||
if byte < '0' || byte > '9' {
|
if !(byte >= '0' && byte <= '9' || byte >= 'a' && byte <= 'f' || byte >= 'A' && byte <= 'F') {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
next, overflow := intrinsics.overflow_mul(value, u64(10))
|
digit := u64(decode_hex_digit(byte))
|
||||||
|
if digit >= base {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
next, overflow := intrinsics.overflow_mul(value, base)
|
||||||
if overflow {
|
if overflow {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
value, overflow = intrinsics.overflow_add(next, u64(byte-'0'))
|
value, overflow = intrinsics.overflow_add(next, digit)
|
||||||
if overflow {
|
if overflow {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return value, len(text) > 0
|
return value, len(digits) > 0
|
||||||
}
|
}
|
||||||
|
|
||||||
parse_primary :: proc(parser: ^Parser, nesting: int) -> ast.Expr_Id {
|
parse_primary :: proc(parser: ^Parser, nesting: int) -> ast.Expr_Id {
|
||||||
|
|||||||
@@ -86,6 +86,137 @@ main func() void { _ = value }
|
|||||||
testing.expect_value(t, module.statements[module.functions[0].body[0]].name, sink_symbol)
|
testing.expect_value(t, module.statements[module.functions[0].body[0]].name, sink_symbol)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@(test)
|
||||||
|
integer_base_literals_compile_and_run :: proc(t: ^testing.T) {
|
||||||
|
directory := "/tmp/brolang-test-integer-bases"
|
||||||
|
output := "/tmp/brolang-test-integer-bases-output"
|
||||||
|
text := `Flags :: enum(u8) { one = 0b0001, two = 0o2, four = 0x04 }
|
||||||
|
maximum u64 :: 0xffffffffffffffff
|
||||||
|
minimum i64 :: -0x8000000000000000
|
||||||
|
Array :: alias [0b10]u8
|
||||||
|
value func() u8 { return 0B101010 }
|
||||||
|
length func($N usize) usize { return N }
|
||||||
|
main func() i32 {
|
||||||
|
if (value() != 42) return 1
|
||||||
|
if (0o52 != 42 or 0x2a != 42 or 0X2A != 42 or 0O52 != 42) return 2
|
||||||
|
if (maximum != maxval!(u64) or minimum != minval!(i64)) return 3
|
||||||
|
binary_max u64 :: 0b1111111111111111111111111111111111111111111111111111111111111111
|
||||||
|
octal_max u64 :: 0o1777777777777777777777
|
||||||
|
if (binary_max != maximum or octal_max != maximum) return 4
|
||||||
|
items Array :: [0x28, 0b10]
|
||||||
|
if (items.len != 2 or items[0] + items[1] != 42) return 5
|
||||||
|
if (length(0x2A) != 42 or 042 != 42) return 6
|
||||||
|
fraction f64 := 42.5
|
||||||
|
fraction += 0.5
|
||||||
|
if (fraction != 43.0) return 10
|
||||||
|
if (u8(Flags.four) != 4) return 7
|
||||||
|
total := 0
|
||||||
|
for 0b1..0x4 |index| { total += index }
|
||||||
|
if (total != 6) return 8
|
||||||
|
tuple :: {0o50, 0b10}
|
||||||
|
if (tuple.0 + tuple.1 != 42) return 9
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
`
|
||||||
|
_ = os2.remove_all(directory)
|
||||||
|
defer _ = os2.remove_all(directory)
|
||||||
|
defer _ = os.remove(output)
|
||||||
|
testing.expect(t, os.make_directory(directory) == nil)
|
||||||
|
testing.expect(t, os.write_entire_file("/tmp/brolang-test-integer-bases/main.bro", transmute([]byte)text))
|
||||||
|
status := compiler_core.compile_package(directory, output)
|
||||||
|
testing.expect_value(t, status, 0)
|
||||||
|
if status != 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
state := run_executable(output)
|
||||||
|
testing.expect_value(t, state.exit_code, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
@(test)
|
||||||
|
integer_base_literals_reject_invalid_digits_without_losing_following_declarations :: proc(t: ^testing.T) {
|
||||||
|
literals := []string{
|
||||||
|
"0b", "0o", "0x", "0B", "0O", "0X",
|
||||||
|
"0b2", "0b102", "0o8", "0o78", "0xg", "0x1G",
|
||||||
|
"0b1name", "0x_1", "0b10_01",
|
||||||
|
}
|
||||||
|
for literal in literals {
|
||||||
|
text := fmt.aprintf("bad :: %s\nvalid :: 42\n", literal)
|
||||||
|
defer delete(text)
|
||||||
|
source_file := source.Source{path="test.bro", text=text}
|
||||||
|
diagnostics := source.init_diagnostics(&source_file)
|
||||||
|
defer source.destroy_diagnostics(&diagnostics)
|
||||||
|
symbols := symbol.init_table()
|
||||||
|
defer symbol.destroy_table(&symbols)
|
||||||
|
stream := lexer.lex(&source_file, &diagnostics, &symbols)
|
||||||
|
defer delete(stream.items)
|
||||||
|
module := parser.parse(&stream, &source_file, &diagnostics)
|
||||||
|
defer ast.destroy_module(&module)
|
||||||
|
testing.expect_value(t, len(diagnostics.items), 1)
|
||||||
|
if len(diagnostics.items) != 1 || len(module.globals) != 2 {
|
||||||
|
testing.expect_value(t, len(module.globals), 2)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
testing.expect_value(t, diagnostics.items[0].span.start, source.Offset(7))
|
||||||
|
testing.expect_value(t, diagnostics.items[0].span.end, source.Offset(7+len(literal)))
|
||||||
|
testing.expect_value(t, module.exprs[module.globals[0].expr].kind, ast.Expr_Kind.Invalid)
|
||||||
|
testing.expect_value(t, module.exprs[module.globals[1].expr].integer, u64(42))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@(test)
|
||||||
|
integer_base_literals_check_magnitude_and_keep_tuple_ordinals_decimal :: proc(t: ^testing.T) {
|
||||||
|
cases := []string{
|
||||||
|
"value u64 :: 0x10000000000000000\n",
|
||||||
|
"value u64 :: 0o2000000000000000000000\n",
|
||||||
|
"value u64 :: 0b10000000000000000000000000000000000000000000000000000000000000000\n",
|
||||||
|
"value :: tuple.0b0\n",
|
||||||
|
"value :: tuple.0o0\n",
|
||||||
|
"value :: tuple.0x0\n",
|
||||||
|
}
|
||||||
|
for text in cases {
|
||||||
|
source_file := source.Source{path="test.bro", text=text}
|
||||||
|
diagnostics := source.init_diagnostics(&source_file)
|
||||||
|
defer source.destroy_diagnostics(&diagnostics)
|
||||||
|
symbols := symbol.init_table()
|
||||||
|
defer symbol.destroy_table(&symbols)
|
||||||
|
stream := lexer.lex(&source_file, &diagnostics, &symbols)
|
||||||
|
defer delete(stream.items)
|
||||||
|
module := parser.parse(&stream, &source_file, &diagnostics)
|
||||||
|
defer ast.destroy_module(&module)
|
||||||
|
testing.expect_value(t, len(diagnostics.items), 1)
|
||||||
|
if len(module.globals) == 1 {
|
||||||
|
testing.expect_value(t, module.exprs[module.globals[0].expr].kind, ast.Expr_Kind.Invalid)
|
||||||
|
} else {
|
||||||
|
testing.expect_value(t, len(module.globals), 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@(test)
|
||||||
|
integer_base_literals_reject_overflow_in_array_counts_and_sentinels :: proc(t: ^testing.T) {
|
||||||
|
text := `Array :: alias [0x10000000000000000]u8
|
||||||
|
Pointer :: alias [*;0o2000000000000000000000]u8
|
||||||
|
Slice :: alias [;0b10000000000000000000000000000000000000000000000000000000000000000]u8
|
||||||
|
`
|
||||||
|
source_file := source.Source{path="test.bro", text=text}
|
||||||
|
diagnostics := source.init_diagnostics(&source_file)
|
||||||
|
defer source.destroy_diagnostics(&diagnostics)
|
||||||
|
symbols := symbol.init_table()
|
||||||
|
defer symbol.destroy_table(&symbols)
|
||||||
|
stream := lexer.lex(&source_file, &diagnostics, &symbols)
|
||||||
|
defer delete(stream.items)
|
||||||
|
module := parser.parse(&stream, &source_file, &diagnostics)
|
||||||
|
defer ast.destroy_module(&module)
|
||||||
|
testing.expect_value(t, len(diagnostics.items), 3)
|
||||||
|
for diagnostic in diagnostics.items {
|
||||||
|
// Overflow must identify the complete literal, not its prefix or delimiter.
|
||||||
|
literal := text[int(diagnostic.span.start):int(diagnostic.span.end)]
|
||||||
|
testing.expect(t, literal == "0x10000000000000000" ||
|
||||||
|
literal == "0o2000000000000000000000" ||
|
||||||
|
literal == "0b10000000000000000000000000000000000000000000000000000000000000000")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@(test)
|
@(test)
|
||||||
parser_marks_only_bang_calls_as_intrinsic :: proc(t: ^testing.T) {
|
parser_marks_only_bang_calls_as_intrinsic :: proc(t: ^testing.T) {
|
||||||
text := `main func() void {
|
text := `main func() void {
|
||||||
|
|||||||
Reference in New Issue
Block a user