[Nix#8605] Tokenizer Bug: Floats with leading 0's wrong tokenized #136

New issue

Open

opened 2024-03-16 06:45:06 +00:00 by lix-bot · 0 comments

lix-bot commented

2024-03-16 06:45:06 +00:00

Member

Upstream-Issue: NixOS/nix#8605

Describe the bug

As described in the Title, nix tokenizes floats with leading zeros differently.
The first part of the float is interpreted as an integer if it starts with a zero.

IEEE doesn't specify this, but usually, I would expect the implementation to take care of leading zeros. and interpret the number as just one number

double checked for similar issues.

Steps To Reproduce

nix repl

nix-repl> [ 01.1e2 ]
[ 1 10 ]

Which comes due to a fault in the tokenizer

01.1e2

Is tokenized into

TOKEN_INTEGER, "01"
TOKEN_FLOAT, ".1e2"

Expected behavior

01.1e2

This should produce only one token

TOKEN_FLOAT, "01.1e2"

Priorities

Add 👍 to issues you find important.

Upstream-Issue: https://git.lix.systems/NixOS/nix/issues/8605 **Describe the bug** As described in the Title, nix tokenizes floats with leading zeros differently. The first part of the float is interpreted as an integer if it starts with a zero. IEEE doesn't specify this, but usually, I would expect the implementation to take care of leading zeros. and interpret the number as just one number - [x] double checked for similar issues. **Steps To Reproduce** `nix repl` ``` nix-repl> [ 01.1e2 ] [ 1 10 ] ``` Which comes due to a fault in the tokenizer ``` 01.1e2 ``` Is tokenized into ``` TOKEN_INTEGER, "01" TOKEN_FLOAT, ".1e2" ``` **Expected behavior** ``` 01.1e2 ``` This should produce only one token ``` TOKEN_FLOAT, "01.1e2" ``` **Priorities** Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).