hkust-taco/mlscript

This system is the HKMC2 compiler for MLscript, a language implementation that compiles source code into JavaScript and WebAssembly. It provides a full compilation pipeline including lexical analysis, parsing, semantic analysis, and optimization passes like deforestation and tail-call optimization. The compiler supports multi-file imports, effect handlers, and advanced pattern matching, with a web-based demo for interactive testing and debugging.

37 changes.

The project underwent a comprehensive rewrite of the MLscript compiler, replacing legacy components with a new architecture featuring a custom lexer, parser, and block IR code generation pipeline. This period focused on establishing core infrastructure for multi-file compilation, async lowering, and cross-platform support, while simultaneously building an extensive test suite to validate parsing, type checking, and code generation across various language features.

31 changes.

This period focused on establishing a comprehensive diff-test framework and expanding test coverage across the parser, lifter, and pattern matching systems. It also introduced initial WebAssembly code generation, C++ test infrastructure, and a JavaScript-compiled compiler runtime with virtualized file system support.

11 changes.

This period focused on implementing and verifying new compiler optimizations, including a rewritten deforestation pass, dead parameter elimination, and fixed-point pattern compilation. A comprehensive suite of tests was added to validate these changes alongside existing optimizations, type checking, and code generation for targets like WebAssembly.

Vim users can now enjoy syntax highlighting for MLScript source files. A new syntax file (vim/mlscript.vim) has been added to define keywords, operators, strings, comments, and other language constructs, providing proper colorization and structure recognition within the editor.

vim · high confidence

This change introduces the core infrastructure for generating WebAssembly text format output, starting with the \Instructions.scala\ module. This module defines the foundational instruction set required for Wasm execution, including control flow constructs (block, loop, if, call, return, branch), numeric operations for all four Wasm value types (i32, i64, f32, f64), and basic instructions like nop, drop, throw, and unreachable. This provides the low-level building blocks for the new Wasm backend.

hkmc2/shared/src/main/scala/hkmc2/codegen/wasm · high confidence

This change establishes the foundational structure for the MLscript project, introducing the build configuration (SBT options, Nix flake, scalafmt), documentation (README, CONTRIBUTING, AGENTS), and the online web demo interface (HTML, CSS, and JavaScript bindings). It also adds essential repository metadata such as the MIT license, .gitattributes, and .gitignore to support the development workflow and CI.

(repo-wide) · high confidence

A new Compiler.scala file has been added to the hkmc2/js module, exposing a Scala.js-compiled Compiler class and a Paths configuration class to the JavaScript runtime. The Compiler class wraps the internal MLsCompiler, manages path-based diagnostics, and exposes a compile method that accepts a file path and returns structured diagnostic information. The Paths class provides configuration for prelude, runtime, and term file locations, bridging the Scala compiler internals with the JavaScript environment.

hkmc2/js/src/main/scala/hkmc2 · high confidence

The hkmc2 compiler now supports compiling MLscript source files into JavaScript modules via the new MLsCompiler entry point. This release adds AsyncLowering, which transforms async functions and await expressions into JavaScript-compatible code using the runtime's toJsAsync helper. To support multi-file compilation and caching, file system operations are now virtualized through an abstract FileSystem trait and a cross-platform Path abstraction, allowing the compiler to read inputs and write .mjs outputs without direct OS coupling. Additionally, a new opaque Uid type is introduced to manage symbol and result identifiers without relying on global state.

hkmc2/shared/src/main/scala/hkmc2 · high confidence

This change adds a new set of foundational utility classes and extensions to the \hkmc2/shared/src/main/scala/utils\ package, establishing the internal tooling for the compiler. Key additions include \SccAnalysis\ for strongly connected component detection (used for dependency and flow analysis), \Lazy\ and \Identity\ for reference-managed lazy evaluation, \FastParseHelpers\ for efficient source code line/column tracking, and \TraceLogger\ for structured debug output. The package also provides extensive implicit extensions (\package.scala\, \shorthands.scala\) for standard Scala types (such as \Opt\, \Ls\, \Str\, and \Iterable\) to simplify pattern matching, collection manipulation, and functional composition throughout the codebase.

hkmc2/shared/src/main/scala/utils · high confidence

The compiler now uses a unified \Path\ and \RelPath\ interface to handle file system operations, with separate implementations for the JVM (wrapping the \os-lib\ library) and Node.js (wrapping the \path\ module). This change enables the compiler to resolve source files and manage import paths correctly regardless of the runtime environment, ensuring consistent behavior across development and production builds.

repository · high confidence

Adds a new \document\ package providing a pretty-printing library that handles indentation and conditional line-breaking. Users can now build documents using the \doc"..."\ string interpolation syntax by importing \document.\_\, which leverages implicit conversions and \Liftable\ instances for types like \String\ and \Int\. The package also includes utility methods for combining document sequences with custom separators.

hkmc2/shared/src/main/scala/hkmc2/utils/document · high confidence

The compiler now includes a new \ReportFormatter\ that displays errors, warnings, and internal errors with box-drawing characters and optional ANSI colors, including source context lines and caret markers. Additionally, a \DebugPrinter\ utility was added to provide structured, tree-like representations of internal compiler data structures for easier debugging, and a \SymbolSubst\ utility provides identity mappings for various symbol types to support future substitution logic.

hkmc2/shared/src/main/scala/hkmc2/utils · high confidence

The \hkmc2/semantics\ module introduces core implementation files that define how the compiler processes code structure and dependencies. \BlockImpl\ handles block-level statement desugaring, including support for ADT-like class definitions and handler continuations, while \Importer\ manages multi-file compilation by resolving \.mls\ and JavaScript imports and detecting circular dependencies. \SimpleSplit\ provides the internal representation for control-flow constructs like \match\ and \let\, and \SymbolPrinter\ adds debugging capabilities for symbol resolution. These changes collectively enable more robust multi-file compilation and refined semantic analysis for control flow and imports.

hkmc2/shared/src/main/scala/hkmc2/semantics · high confidence

The \Parser\, \Syntax\, \Typing\, and \package\ modules in \src/main/scala/simplesub\ have been deleted. This removes the previous implementation of the simple-sub language, including its Fastparse-based parser, the dual \Type\/\Pos.Type\ ADT representations, and the mutable \TypeShape\-based type inference engine. Users will no longer have access to these specific internal components, which were superseded by the new simplification and hash-consing strategies.

src/main · high confidence

The codegen module has been restructured around a new Block IR, introducing core traversal and transformation classes (BlockTraverser, BlockTransformer) and a comprehensive CompilationPipeline. This pipeline orchestrates a series of optimization passes including LambdaRewriter, Deforest, EtaExpansion, Lifter, HandlerLowering, AsyncLowering, BufferableTransform, FirstClassFunctionTransformer, ClassParamFlattener, DeadParamElim, and TailRecOpt. Additionally, a new BlockChecker validates IR invariants, and a CachedAnalysis trait provides memoization for analysis results.

hkmc2/shared/src/main/scala/hkmc2/codegen · high confidence

The syntax layer for HKMC2 has been completely rewritten, replacing the previous implementation with a new architecture in \hkmc2/syntax\. This change introduces a centralized \Keyword.scala\ file that defines all language keywords (such as \let\, \set\, \class\, \import\, \try\, \finally\, and operators like \\|\, \&\, \=\>\) along with their specific precedence levels and infix behavior. The \Lexer.scala\ module has been updated to support new lexical features, including Unicode escape sequences (\\\uXXXX\ and \\\u{...}\), numeric literals with underscores, and symbolic identifier suffixes. The \Parser.scala\ and \ParseRule.scala\ files now use a new algebraic data type-based parsing DSL (\ParseRule\, \Alt\) to define grammar rules, enabling more robust handling of indentation, blocks, and operator precedence. This rewrite underpins recent language additions like \set\ mutation, \using\ summoning, and refined \new\ expressions.

hkmc2/shared/src/main/scala/hkmc2/syntax · high confidence

The UCS semantic layer now supports a new flattened pattern representation (\FlatPattern\) that compiles complex patterns into constant-time matchable cases, alongside a \TermSynthesizer\ trait that generates runtime calls for pattern matching operations like tuple slicing, string manipulation, and match success/failure handling. This change enables more efficient and unified pattern compilation by separating pattern structure from term generation, allowing the compiler to handle splits and normalizations with optimized runtime support for common operations.

hkmc2/shared/src/main/scala/hkmc2/semantics/ucs · high confidence

A new \PlatformCompilerCache\ class has been added to the \hkmc2.utils\ package to manage cached compiler artifacts. This component initializes specific caches for elaboration results (\Artifact\) and prelude data (\PreludeArtifact\), enabling the system to store and retrieve these intermediate compilation states to avoid redundant processing.

hkmc2/js/src/main/scala/hkmc2/utils · high confidence

The deforestation optimization pass now uses a new \DeforestRewriter\ class to transform the intermediate representation. This rewriter analyzes control flow and data dependencies to fuse function calls with their consumers, eliminating intermediate allocations. It introduces new internal symbols for branch functions and rest functions, allowing the compiler to generate more efficient code by inlining and restructuring match expressions and label-based control flow directly within the deforestation phase.

hkmc2/shared/src/main/scala/hkmc2/codegen/deforest · high confidence

The UPS (Ultimate Pattern Syntax) semantics module now includes a new \Context\ class to track pattern monomorphization, an \Instantiator\ object to handle breadth-first pattern instantiation and substitution, and a \SplitCompiler\ class to compile patterns into matching splits with support for scrutinee reuse and extraction slots. These components replace or supplement previous ad-hoc handling, enabling more robust pattern matching logic including support for class-like pattern constructors, range expansion, and binding management during split generation.

hkmc2/shared/src/main/scala/hkmc2/semantics/ups · high confidence

The JVM test suite now runs compile tests in parallel, significantly reducing test execution time. To support this concurrency, the compiler cache implementation has been updated to handle concurrent access safely, specifically preventing deadlocks that could occur when processing cyclic imports. Additionally, diagnostic output during these parallel tests is synchronized to ensure error messages are not interleaved.

hkmc2/jvm · high confidence

The JavaScript compiler runtime now uses a virtualized abstraction for file system and path operations, replacing direct Node.js dependencies with platform-specific implementations. A new \InMemoryFileSystem\ provides a pure in-memory store for the web compiler and testing, while \VirtualPath\ and \VirtualRelPath\ offer a pure JavaScript path normalization engine that correctly resolves \.\ and \..\ segments to ensure consistent cache keys. The \PlatformFileSystem\ object now delegates to a \NodeFileSystem\ implementation (backed by new Scala.js facades for Node's \fs\ and \path\ modules) for native Node.js environments, ensuring that path identity and normalization behave consistently across both web and Node targets.

hkmc2/js/src/main/scala/hkmc2/io · high confidence

The JavaScript file in the bin directory has been updated to enhance the web demo's output capabilities. Users will now see structured 'Typing Results' and 'Execution Results' sections in the UI, replacing the previous generic output. The demo also features improved error handling, specifically ensuring that unresolved symbols are not reported as errors (as noted in commit \#141), and provides clearer visual feedback for runtime errors and warnings using styled HTML output.

bin · high confidence

Add parsing web demo with interactive editor and syntax visualization; Added C++ test infrastructure for MLScript compilation; Added InvalML test suite for type checking, borrowing, and code generation; Added JavaScript test utilities and test class; Added MLscript compile test infrastructure and standard library modules; Added MLscript type declarations for standard JavaScript APIs; Added Wasm test utilities for Binaryen integration; Added accounting test documentation and placeholder files; Added benchmark test infrastructure and file watcher; Added benchmark tests for FingerTrees, LazySpreads, and nofib suites; Added compiler integration tests for caching and private ABI stability; Added comprehensive code generation tests for MLscript; Added comprehensive diff-test suite for MLscript basics; Added comprehensive test coverage for the Unified Pattern System (UPS); Added comprehensive test suite for the mlscript parser; Added deforestation test suite; Added flow analysis test suite for the MLScript compiler; Added interop tests for MLscript JavaScript integration; Added meta-level tests for MLscript compiler features; Added nofib benchmark test suite for mlscript-compile; Added nofib benchmark tests to the MLscript test suite; Added parser tests for MLscript language constructs; Added parsing test suite with lexer, parser, and rule definitions; Added pattern-matching example tests for the UPS language; Added staged-module compiler tests for IR generation and code printing; Added staging tests for UCS pattern matching and desugaring; Added standard library test coverage for MLscript data structures and utilities; Added syntax tests for UCS pattern matching and control flow; Added test cases for parser and code generation edge cases; Added test coverage for MLscript compiler optimizations; Added test coverage for UCS operator split syntax; Added test coverage for UPS syntax features and error handling; Added test coverage for contextual parameters and type class resolution; Added test coverage for eta-expansion in deforestation; Added test coverage for fixed-point pattern compilation; Added test coverage for tail-call optimization and annotations; Added test examples for CSV parsing and accounting logic; Added test runners for apps, nofib, and wasm compilation tests; Added test suite for UCS language features; Added test suite for UCS normalization edge cases; Added test suite for Unified Case Syntax (UCS) pattern matching; Added test suite for dead parameter elimination; Added test suite for effect handlers and async interop; Added test suite for general UCS pattern matching and code generation; Added test suite for overloaded modules in signatures; Added test suite for the MLscript lifter; Added tests for UCS syntax features and error handling; Added tests for VirtualPath normalization and InMemoryFileSystem path resolution; Added tests for annotation syntax and precedence rules; Added tests for application examples and utility libraries; Added tests for parametric pattern matching in UPS; Added tests for parsing web demo examples; Added tests for pattern-matching hygiene and variable scoping; Added tests for recursive pattern matching in the MLScript compiler; Added tests for the ObjectBuffer feature; Expanded test coverage for the Wasm backend; New diff-test infrastructure and test suite for the hkmc2 compiler; Removed legacy Scala test suite for parser and type inference.

The project now includes a package.json and package-lock.json defining Node.js dependencies: benchmark (^2.1.4), binaryen (^131.0.0), and TypeScript (^4.7.4). The Scala build (build.sbt) has been significantly restructured to support a multi-project layout with Scala 3 (3.8.4) for the hkmc2 module, while retaining Scala 2.13.18 for the root. New dependencies include directory-watcher (0.18.0), ScalaTest (3.2.20), and various LiHaoyi libraries (fansi, sourcecode, os-lib). The build also introduces Scala.js support for hkmc2JS and organizes tests into specific subprojects (hkmc2DiffTests, hkmc2NofibTests, hkmc2AppsTests, hkmc2WasmTests) with dedicated SBT command aliases.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 57.

• Code Health 75
• Architecture 97
• Maturity 64
• Readiness 65
• Security 99
• Accessibility 43

• 300 commits — 246 feature/other, 54 fixes

• hkmc2/shared — 235 commits
• (repo) — 26 commits
• (root) — 20 commits
• hkmc2/js — 8 commits
• hkmc2DiffTests/src — 4 commits
• hkmc2/jvm — 3 commits
• doc/cross-unit-inlining.md — 2 commits
• doc/reference.md — 1 commit
• shared/src — 1 commit

• fix: Add a refocusing-machine compiler for fixed-point patterns (#519)
• fix: Add metadata wrapper for Call and Instantiate IR nodes + fix ClassCtorSymbol logic (#518)
• fix: Add more documentation and fix a possible bug
• fix: Add regression test for UPS bug
• fix: Add small threshold and fix issue in size calculation (#497)
• fix: Codex fix
• fix: Fix BMS leaking through localsWithoutBms; filter BMS from reqSymbols; add :fixme to Debugging.mls
• fix: Fix SymbolRefresher: copy defn to refreshed ClassSymbol and handle pre-mapped ctor symbols
• fix: Fix TailRecOpt to create wrapper for multi-param-list functions
• fix: Fix all Scala warnings in HKMC2 (#503)
• fix: Fix compiled constructor patterns (close #521) (#537)
• fix: Fix constructor function
• fix: Fix cross-unit cache invalidation and path identity
• fix: Fix cross-unit recursion analysis
• fix: Fix dead-param-elim for multi-arg-list calls and split multi-arg-list calls before handler lowering
• fix: Fix duplicate binder detection for nested definitions
• fix: Fix elaboration of bare anonymous classes
• fix: Fix handler loop bug (#542)
• fix: Fix immediate call prefix dataflow
• fix: Fix merge compilation errors from tightened symbol types
• …and 280 more

• 0 containers · 1 bounded contexts · 0 dependency edges (baseline)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

hkust-taco/mlscript was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.