about summary refs log tree commit diff
path: root/users/tazjin/rlox/src/scanner.rs (follow)
AgeCommit message (Collapse)AuthorFilesLines
2022-02-07 r/3776 fix(users/tazjin): rustfmt code with non-default settingsVincent Ambo1-10/+3
rustfmt only sometimes detects path-based nested config files (probably some kind of race?), so my users folder uses a separate formatting check for rustfmt to avoid flaky CI. Enough flakes around already ... Change-Id: Ifd862f9974f071b3a256643dd8e56c019116156a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5242 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2021-02-27 r/2235 style(tazjin/rlox): Set max_width=80Vincent Ambo1-3/+10
Change-Id: Ib64831c0b97c94fdfbdbae64f4dfccc86879ef73 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2554 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-01-19 r/2132 refactor(tazjin/rlox): Prepare scanner for shared useVincent Ambo1-0/+284
In the book, the clox interpreter has its own scanner which uses a pull-based model for a single pass compiler. I can't be bothered to write another scanner, or amend this one into pull-mode to work with the treewalk interpreter, so instead I will just reuse it and pull from a vector of tokens. The tokens are shared between both interpreters and the scanner is not what I'm interested in here. Change-Id: Ib07e89127fce2b047f9b3e1ff7e9908d798b3b2b Reviewed-on: https://cl.tvl.fyi/c/depot/+/2420 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-01-17 r/2126 chore(tazjin/rlox): Move other modules under treewalk::Vincent Ambo1-283/+0
It's unclear if the second part of the book can reuse anything from the first part (I'm guessing probably the scanner, but I'll move that back if it turns out to be the case). Change-Id: I9411355929e31ac6e953599e51665406b1f48d55 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2415 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-01-14 r/2107 refactor(tazjin/rlox): Let scanner tokens own their lexemeVincent Ambo1-5/+5
This removes the runtime dependency on a borrow into the program source code. It's not yet ideal because there are a lot of tokens where we really don't care about the lexeme, but this is what the book does and I am not going to change that. Change-Id: I888e18f98597766d6f725cbf9241e8eb2bd839e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2394 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-12-06 r/1987 feat(tazjin/rlox): Implement parsing up to unary expressionsVincent Ambo1-5/+5
... with the exception of parenthesised expressions, because error threading is not implemented yet. Change-Id: I8d455d85e647548d5b71cbfd3d078f4970dab7fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/2232 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-30 r/1966 feat(tazjin/rlox): Bootstrap recursive-descent parser for LoxVincent Ambo1-5/+4
... mostly some AST boilerplate and a first top-level rule, plus boilerplate similar to that set up in the Scanner. Change-Id: I605d1de23c47a3b3702ab4f62cd3371bc3988c7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/2194 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-28 r/1956 refactor(tazjin/rlox): Thread through scanner errorsVincent Ambo1-5/+10
... and show them to users, very crudely. Change-Id: If4491b14db1124313f6ab7e5fbfdce9fea501d11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2193 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-28 r/1955 feat(tazjin/rlox): Scan identifiers and keywordsVincent Ambo1-6/+41
Change-Id: Ifec627605c23c25f199d47eaa91e441ed9590208 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2192 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-28 r/1954 feat(tazjin/rlox): Implement number scanningVincent Ambo1-2/+36
Change-Id: Ide0126d1c2274d56903092816ff9cd531c03f513 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2191 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-28 r/1953 feat(tazjin/rlox): Implement string scanningVincent Ambo1-6/+35
Note that Lox does not support escapes, and I don't care about that. Change-Id: Ie848cbc1164c4b005b15e29aad8fe723aaa68d1b Reviewed-on: https://cl.tvl.fyi/c/depot/+/2190 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-28 r/1951 feat(tazjin/rlox): Handle whitespace in scannerVincent Ambo1-0/+6
Change-Id: Iead3efa6aa797af5c5b57c67226205bf7e39e647 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2188 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-28 r/1950 feat(tazjin/rlox): Scan single-line commentsVincent Ambo1-2/+29
Change-Id: I2efafdb1374ac2863a1f8d8a9310123a7dec0900 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2187 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-27 r/1927 chore(tazjin/rlox): Wire scanner to interpreter to reduce warningsVincent Ambo1-0/+13
... they're just noisy at the moment. This isn't complete because it doesn't thread through scanner errors. Change-Id: I0f75d2b20fa3f57be1af5d1d8aa8059856855825 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2162 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-27 r/1926 feat(tazjin/rlox): Implement multi-character scanningVincent Ambo1-0/+16
Change-Id: Ic5a246a7f5834477aeb97824fa30ba74a8422bc1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2161 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-27 r/1925 refactor(tazjin/rlox): Use &[char] instead of &str in scannerVincent Ambo1-7/+3
This makes it easier to work with the Unicode issue. The original string representation can be discarded. Change-Id: I740be4cb9654679ea7950f3899c5c709b1e7a739 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2160 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-23 r/1914 feat(tazjin/rlox): Implement single-character scanningVincent Ambo1-0/+123
... still not that interesting, but at this point slightly divergent from the book: The book embraces mutability for interpreter state, initially for tracking whether an error condition has occured. I avoid this by instead defining an error type and collecting the error values, to be handled later on. Notes: So far nothing special, but this is just the beginning of the book. I like the style it is written in and it has pointed to some interesting resources, such as a 1965 paper titled "The Next 700 Languages". Change-Id: I030b38438fec9eb55372bf547af225138908230a Reviewed-on: https://cl.tvl.fyi/c/depot/+/2144 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI