com-lihaoyi/utest

This system is the uTest testing framework for Scala, providing a structured way to define, execute, and report test results across JVM, Scala.js, and Scala Native platforms. It supports Scala 2.12, 2.13, and 3.x, utilizing version-specific macro implementations to enable features like golden file testing, detailed assertion diffs, and compile-time error verification. The framework includes a refactored runner for selector-based execution, retry logic for flaky tests, and configurable output formatting to aid in debugging and test maintenance.

13 changes.

The project migrated its build system from SBT to Mill, enabling support for Scala 2.12, 2.13, and 3.x across multiple platforms. This period involved a comprehensive restructuring of the uTest framework internals, introducing new assertion APIs, golden file testing, and a refactored test runner architecture.

6 changes.

The project implemented comprehensive Scala 3 support by introducing version-specific macro implementations for test building, assertions, and golden file testing. It also expanded platform compatibility by adding Scala Native support and enhanced JVM capabilities with new golden file assertion methods. Additionally, performance issues in string comparison were resolved by integrating a shaded diff library.

The test framework now supports Scala Native by introducing platform-specific implementations for JS and Native targets. This includes \PlatformShims\ to handle asynchronous test execution (using \Await\ on Native) and reflective module loading, as well as \TestSuitePlatformSpecific\ stubs that currently provide no-op golden file processing. This enables running the utest suite on the Native platform, although golden literal printing is not yet implemented for this target.

utest/src-js, utest/src-native · high confidence

This change introduces the necessary framework components to support Scala 3 in the utest library. It adds Scala 3-specific macro implementations (\GoldenSpanMacros\) for golden file assertions, replacing the Scala 2 macros. It also includes a new \EnableReflectiveInstantiation\ annotation and a \PortableScalaReflectExcerpts\ object to handle reflective class loading and instantiation on the JVM, ensuring that test classes marked with the annotation can be properly discovered and instantiated during test execution.

utest/src-2-jvm, utest/src-3-jvm · high confidence

The test framework now includes \assertThrows\ for verifying that code blocks raise specific exceptions, \assertMatch\ for pattern-matching assertions on values, and a \retry\ helper to automatically re-run assertions that may fail intermittently. These additions provide more robust and expressive ways to validate behavior in tests.

utest/src/utest/asserts · high confidence

uTest now supports golden file testing via \assertGoldenFile\, \assertGoldenLiteral\, and \assertGoldenFolder\, allowing users to automatically update and compare expected outputs against source files. Additionally, assertion failures now display detailed line and character-level diffs with color highlighting, making it easier to spot differences in test output.

repository · high confidence

The JVM test framework now supports three new assertion methods for verifying outputs against stored reference data: \assertGoldenFile\ compares a string value against a file on disk, \assertGoldenLiteral\ compares a value against a literal in the source code (updating the source via PPrint when the environment variable \UTEST\_UPDATE\_GOLDEN\_TESTS=1\ is set), and \assertGoldenFolder\ recursively compares directory contents, handling binary files and reporting differences. These additions allow users to maintain and update golden test data directly from their test runs.

utest/src-jvm · high confidence

The core implementation files for the utest testing framework have been deleted, including the test model and execution logic (Model.scala, TestSuite.scala, package.scala), assertion and concurrency helpers (Asserts.scala, Concurrent.scala, package.scala), output formatting (Printers.scala), and the SBT integration layer (Framework.scala, Runner.scala). This change removes the ability to define, run, and report test results using the utest framework within this codebase.

src/main · high confidence

The assertion framework now provides richer debugging information when tests fail. Boolean assertions (\assert\) automatically trace and log the values of identifiers and equality comparisons, displaying a line-diff of pretty-printed output when \a == b\ fails. Additionally, \assertCompileError\ has been implemented to capture and report specific compilation errors (parse, type, or compile-time-only) with precise source positions, allowing users to verify that code correctly fails to compile.

utest/src-2/utest/asserts · high confidence

The project's build configuration has been migrated from SBT to Mill, introducing a new \build.mill\ script and the Mill launcher wrapper. This change updates the build to support Scala 2.12, 2.13, and 3.x across JVM, Scala.js, and Scala Native platforms, while also vendoring internal dependencies like \pprint\ and \fansi\ to avoid classpath conflicts. Additionally, an MIT License file has been added to the repository to clarify usage rights.

(repo-wide) · high confidence

The test framework has been restructured with new core components: DefaultFormatters handles summary and banner rendering, ExecutionContext provides a synchronous 'RunNow' mode for simpler async handling, Executor defines the lifecycle hooks (beforeEach, afterEach, afterAll), and StackMarker filters internal call-stack noise. Additionally, the Formatter trait now exposes configurable options for truncation height, wrap width, and color, along with customizable printers for golden literals and assertion errors, allowing users to tailor the appearance of test output.

utest/src/utest/framework · high confidence

The test runner implementation has been restructured to introduce \BaseRunner\ as the core execution engine, which now supports running tests based on specific selectors (paths) rather than just full suite names. This change enables more granular test execution and allows \class\es extending \TestSuite\ to be used as test suites, in addition to existing module-based suites. The refactoring also introduces \MasterRunner\ and \ScalaJsSlaveRunner\ to handle result aggregation and remote execution, improving how test results are reported and processed across different environments.

utest/src/utest/runner · high confidence

The Scala 2.12 build now includes the MacroCompat module, which was previously commented out. This restores the ability to use macro-based assertions in Scala 2.12, aligning its capabilities with the Scala 2.13 version which already supports these features.

utest/src-2.12 · high confidence

The test runner logic has been moved into a dedicated TestRunner object, centralizing how tests are executed asynchronously and how results are rendered for the user. Concurrent Future-based tests are now run sequentially to ensure deterministic ordering. Additionally, the TestSuite base class has been refactored to include retry capabilities via a Retries trait, allowing users to configure automatic retries for flaky tests.

utest/src/utest · high confidence

The utest library now includes a dedicated Scala 3 implementation located in the \src-3\ directory. This change introduces version-specific traits (\TestsVersionSpecific\, \TestSuiteVersionSpecific\, \TestSuiteCompanionVersionSpecific\, and \CompileErrorVersionSpecific\) that delegate to a new \TestBuilder\ macro. The \TestBuilder\ processes test definitions using Scala 3's \scala.quoted\ API, handling test tree construction, path resolution, and assertion inlining to ensure correct line numbers in failure traces. This replaces or supplements the previous Scala 2 implementation, enabling utest to function correctly on Scala 3 while maintaining backward compatibility through deprecated wrappers for older APIs.

utest/src-3/utest · high confidence

The assertion framework now uses Scala 3 macros (via \scala.quoted\) to provide richer debugging information when assertions fail. This implementation adds line-number preservation in failed assertion traces, prints a line-diff of pretty-printed output when equality checks fail, and traces variable values within expressions to help identify the source of failures. The changes are localized to the assertion macro layer (\AssertsVersionSpecific\, \ParallelVersionSpecific\, and \Tracer\), ensuring that test failures now include more contextual debug data without altering the core assertion logic.

utest/src-3/utest/asserts · high confidence

The \utest\ library now includes a dedicated \src-2\ source directory containing Scala 2-specific implementations for core testing components. This change introduces \Errors.scala\ with version-specific compile error handling, \TestSuite.scala\ providing deprecated macro-based entry points for test suites, and \Tests.scala\ which implements the macro logic for parsing test structures and building call trees. These files replace previous version-specific file naming conventions, consolidating the Scala 2 codebase into a single shared location while maintaining backward compatibility through deprecated APIs.

utest/src-2/utest · high confidence

A shaded copy of the stringdiff library has been added to the test suite to resolve pathological performance problems during string comparison. This change introduces a complete implementation of the Myers diff algorithm, including sequence and string diffing, tokenization, and ANSI/text formatting, ensuring that test output generation remains efficient even with large inputs.

utest/src/utest/shaded · high confidence

Added comprehensive test suite for uTest framework internals; Removed legacy utest test suite files.

The \build.sbt\ file has been deleted, removing the SBT-based build definition for the project. This change eliminates the legacy dependency on Scala 2.10.3 libraries (including \scala-reflect\, \quasiquotes\, and \test-interface\) and the associated compiler plugin setup, consistent with the project's migration away from SBT as indicated by the commit history.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 58.

• Code Health 96
• Architecture 100
• Maturity 46
• Readiness 56
• Security 67

• 300 commits — 275 feature/other, 25 fixes

• (root) — 108 commits
• utest/src-0 — 58 commits
• (repo) — 36 commits
• utest/src-3 — 28 commits
• utest/test — 26 commits
• utest/src — 18 commits
• .github/workflows — 8 commits
• utest/jvm — 6 commits
• utest/src-jvm — 4 commits
• utest/src-2 — 3 commits
• utest/js — 2 commits
• .github/FUNDING.yml — 1 commit
• utest/native — 1 commit
• utest/src-2.12 — 1 commit

• fix: 0.9.3 try to fix crash in named pattern match transformation
• fix: Fix Scala.js Scala 3
• fix: Fix Sonatype publishing
• fix: Fix assignment traversal
• fix: Fix failed assertion linenumbers (#381)
• fix: Fix given syntax
• fix: Fix identation
• fix: Fix identification of anonymous test{} blocks in Scala 3.x (#390)
• fix: Fix missing last test when return type is string (#391)
• fix: Fix pathological performance in def splitLines (#409)
• fix: Fixes to TestableString and TestableSymbol
• fix: Implicits wiring fixed
• fix: Merge pull request #196 from LGLO/async-tests-readme-fix
• fix: Merge pull request #238 from lolgab/fix-scala-js-scala3
• fix: Merge pull request #241 from com-lihaoyi/fix-sonatype-publishing
• fix: Merge remote-tracking branch 'origin/fix-ext' into dotty-community-build
• fix: Revert to def in Model.scala
• fix: Vendor stringdiff to fix pathological poor performance (#412)
• fix: fix linenumbers in failed assertion traces (#380)
• fix: fix linenumbers in failed assertion traces (#382)
• …and 280 more

• 0 containers · 1 bounded contexts · 0 dependency edges (baseline)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

com-lihaoyi/utest was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.