refactor(tazjin/rlox): Prepare scanner for shared use r/2132

In the book, the clox interpreter has its own scanner which uses a
pull-based model for a single pass compiler.

I can't be bothered to write another scanner, or amend this one into
pull-mode to work with the treewalk interpreter, so instead I will
just reuse it and pull from a vector of tokens.

The tokens are shared between both interpreters and the scanner is not
what I'm interested in here.

Change-Id: Ib07e89127fce2b047f9b3e1ff7e9908d798b3b2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2420
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI

1 files changed, 1 insertions, 0 deletions

diff --git a/users/tazjin/rlox/src/main.rs b/users/tazjin/rlox/src/main.rs
index c9cc96d2e6a9..3a956833c15d 100644
--- a/users/tazjin/rlox/src/main.rs
+++ b/users/tazjin/rlox/src/main.rs
@@ -5,6 +5,7 @@ use std::io::Write;
 use std::process;
 
 mod bytecode;
+mod scanner;
 mod treewalk;
 
 /// Trait for making the different interpreters callable in the same