2023 October Update

While active with Hadron development, I will publish monthly (ish) updates on the Hadron website, cross-posted to the scsynth forum. The hope is to drive engagement and volunteerism.

After hearing the news of their involvement with ICE, I looked into leaving GitHub. However, this recent rebranding of GitHub to center around their Large Language Model product, Copilot, provided a second big push to go.

I’m moving all Hadron-related projects to a self-hosted GitLab instance I set up at Solitary Bees. Please ping me if you’d like me to send you an account invite. We’re still setting up shop regarding continuous integration and other DevOps infrastructure, but I plan to keep Hadron and my other personal projects there for the foreseeable future.

Fuzzing is a vital part of modern testing strategies, and I’m pleased to report we’ve added fuzzing support to this new Rust-based iteration of Hadron. Please read the instructions at docs/fuzzing.md in the Hadron repository, try them out, and file any bug reports if you find them.

A JIT compiler like Hadron needs to parse sclang quickly to allow for rapid feedback to users on input code snippets. Modern language frontends see a lot of invalid and incomplete code due to the rise of fast-feedback development tooling like LSP, popularized by VSCode.

So, not only do parsers need to be fast, but they need to give actionable feedback to users while they are typing. We’d like to “raise the bar” for user feedback with Hadron, providing great error messages with lots of context and helpful hints about what might be going wrong.

I’m writing two parsers, one that parses the input source code in detail and one that only builds an “outline” of the parsed code and is much more robust in the face of errors. If parsing fails using the detailed parser, it can hand off to the outline parser for error recovery and to make better suggestions around the input code. Furthermore, the outline parser will allow for lazy compilation of the class library.

I copied another speed-centric design choice from the Carbon Language Project by representing the parse tree in a linear array, holding the parse nodes in postorder traversal order. This structure keeps the parser inner loops free of memory allocations. This linear structure also makes an iterative parser implementation easier, saving additional computation and memory required by a recursive parser.

• Rust: #1 I’ve broken the detailed parser work into a bunch of handle_ functions in lang/src/toolchain/parser/tree. Please let me know if you’d like to tackle a particular part!
• DevOps: #2 Are you interested in learning how to build a continuous integration pipeline on Google Cloud?
• Docs: #3 The parser has been moving around quite a bit but should now be settled down enough that we could start to pay down some of the RustDoc debt.
• Rust: To get early feedback on the parser, particularly the error messages, I’ll be spinning up a Rust-based LSP implementation targeting WASM.
• TypeScript, VSCode: Given a rust-based WASM LSP crate, add the trimmings for a full-featured sclang VSCode extension.
• Anyone: Suggestions and feedback. Is there something on your mind about sclang and Hadron? Let me know!