databricks/sjsonnet

Sjsonnet is a high-performance, multi-platform implementation of the Jsonnet data templating language, supporting JVM, JavaScript, Native, and GraalVM targets. It provides a robust interpreter with extensive standard library functions for compression, hashing, and YAML parsing, alongside a persistent server mode for efficient client-server communication. The system includes a browser-based playground for interactive evaluation and a comprehensive benchmarking suite to profile and optimize parsing, rendering, and execution performance.

12 changes.

The project migrated its build infrastructure to Mill and removed legacy shell-based test suites while expanding cross-platform support for Scala.js and JVM. Significant architectural work introduced a persistent background server mode and a high-performance byte-oriented JSON renderer to improve execution speed and resource reuse. Comprehensive test coverage was added to validate these new features, including concurrency safety, compression utilities, and error handling across all supported targets.

23 changes.

This period focused on introducing Scala Native support to enable low-latency standalone execution, alongside a major architectural reorganization of the standard library into dedicated submodules. Significant effort was also dedicated to expanding test coverage across JVM, JavaScript, and Native platforms, while optimizing performance through memory management improvements and comprehensive benchmarking.

8 changes.

This period focused on enhancing Sjsonnet's runtime performance through SIMD-accelerated base64 operations and fixes for recursive standard library hangs, while introducing a browser-based playground for interactive evaluation. Comprehensive testing efforts included benchmark suites for array views and JDK 17 string operations, alongside end-to-end integration tests for the client/server IPC flow.

A Java-compatible BitSet implementation has been added for both the JavaScript and Native targets, enabling code that relies on java.util.BitSet to function correctly on these platforms. The new classes provide core functionality including initialization, element retrieval and modification (get, set, clear), cardinality checks, stream conversion, and set operations (andNot), ensuring parity with the standard Java API for these specific use cases.

sjsonnet/src-js/java, sjsonnet/src-native/java · high confidence

Introduces the Scala.js-specific platform layer (Platform.scala) and character scanning utilities (CharSWAR.scala) to enable std.parseYaml() and optimized string rendering in JavaScript environments. This implementation provides the necessary cache structures and YAML decoding logic for the JS runtime, allowing users to parse YAML content and render strings efficiently in browser-based or Node.js contexts.

sjsonnet/src-js/sjsonnet · high confidence

The stdlib now includes a new \xz\ function module that allows users to compress strings or byte arrays using the XZ algorithm with an optional compression level. Additionally, Base64 encoding performance is improved on the JVM by introducing an \asciiSafe\ optimization path that uses ISO-8859-1 charset for ASCII-only inputs, avoiding the overhead of UTF-8 encoding when the input is known to be ASCII-safe.

sjsonnet/src-jvm/sjsonnet/stdlib · high confidence

Added a new set of benchmark resource files (bench.01 through bench.04) to the C++ benchmark suite, including both the Jsonnet test definitions and their corresponding golden output files. These resources provide standard computational benchmarks, such as recursive summation, Fibonacci sequence calculation, and string folding, to validate the performance and correctness of the C++ implementation.

_bench/resources/cpp\suite · high confidence

The benchmark suite now includes a README explaining the four benchmark categories (bug\_suite, cpp\_suite, go\_suite, sjsonnet\_suite) and a shell script to regenerate golden output files. The script dynamically determines the Scala version used by the build tool to ensure the generated goldens match the runtime environment, preventing drift issues that occurred when versions were pinned statically.

bench/resources · high confidence

sjsonnet now runs on Scala Native, enabling users to execute Jsonnet code as a standalone, low-latency binary without a JVM. This release introduces native-specific performance optimizations, including a SWAR-based escape character scanner for faster string processing and a buffered native output stream to reduce system call overhead. It also brings parity with the JVM version by implementing std.parseYaml() (including YAML 1.2 compliance for non-finite floats and octals), std.sha\* hashing, and std.native functions like gzip and regex support.

sjsonnet/src-native/sjsonnet · high confidence

This change adds the \SjsonnetMain\ Scala.js entrypoint, enabling sjsonnet to be executed directly in Node.js and browsers via \@JSExportTopLevel\. It also introduces the \sjsonnet.client\ Java package, providing \Locks\ and \FileLock\ for file-based process synchronization and a \ProxyOutputStream\ for multiplexed output streams, which supports the new async import loader and client-side integration patterns.

repository · high confidence

Added a new client-side playground (\playground/index.html\) that provides a single-file, browser-based IDE for writing, evaluating, and debugging Jsonnet code. The tool features a multi-file editor with CodeMirror, support for external and top-level arguments, dark/light theming, and local state persistence. It runs entirely in the browser using an inlined Sjsonnet engine, allowing users to experiment with Jsonnet syntax and imports without a backend.

playground · high confidence

The Sjsonnet tool now supports a persistent background server architecture, allowing clients to connect via Unix domain sockets to reuse state and avoid startup overhead. This change introduces a new \SjsonnetServerMain\ and \Server\ class that manage a long-running process, handling client connections, locking, and version compatibility checks. Users benefit from faster subsequent executions as the server maintains a parse cache across invocations, while the client handles communication through a dedicated protocol.

sjsonnet/server · high confidence

The JSON rendering pipeline now includes a new \BaseByteRenderer\ and \ByteRenderer\ that write directly to an \OutputStream\ using byte buffers, bypassing the previous character-based \BaseCharRenderer\ and \BaseRenderer\ to eliminate UTF-8 encoding overhead and reduce memory allocations. This change is accompanied by the introduction of a \DebugStats\ class and \--debug-stats\ CLI flag, which allows users to capture runtime performance metrics such as parse time, evaluation time, function call counts, and stack depth for performance analysis.

sjsonnet/src/sjsonnet · high confidence

The sjsonnet library introduces new internal helper classes (AbstractFunctionModule, FunctionBuilder, FunctionModule) in the functions package to streamline the creation of built-in functions. These utilities allow built-ins to be defined with type-safe evaluation logic and support qualified names for clearer error reporting, while also providing a mechanism to group functions into reusable module objects.

sjsonnet/src/sjsonnet/functions · high confidence

The sjsonnet JVM entry point now exposes new standard library functions: std.gzip and std.xz for compressing strings and bytes, and std.regex for regular expression matching using the RE2 library. These capabilities are wired into the main execution context via the new stdlib module, enabling users to perform compression and pattern matching directly within their Jsonnet code.

sjsonnet/src-jvm/sjsonnet · high confidence

The repository has been set up with a new Mill build system (build.mill) replacing previous build tools, configured to cross-compile for Scala 2.12, 2.13, and 3.3 on the JVM, plus Scala.js, Scala Native, and GraalVM targets. The project now uses scalafmt (version 3.11.5) for code formatting with a dedicated configuration file (.scalafmt.conf) and enforces formatting checks in CI. Documentation has been added including a comprehensive development guide (CLAUDE.md), agent instructions (AGENTS.md), a detailed changelog (CHANGELOG.md), and an updated README with usage examples. The build is configured to use Mill version 1.1.6 and includes JVM options for native access and memory settings.

(repo-wide) · high confidence

The \test\_suite\ directory has been removed, deleting the CMake build configuration (\CMakeLists.txt\), the shell-based test runner (\run\_tests.sh\, \count\_tests.sh\), and all associated \.jsonnet\ test scripts and \.golden\ expected-output files. This eliminates the legacy bash-based regression testing infrastructure for the JSONNET parser and runtime.

_test\suite · high confidence

The \test\_cmd\ directory's legacy shell-based test infrastructure has been removed. This includes the \run\_cmd\_tests.sh\ runner, the \cmd\tests.source\ helper library, and all associated golden output files (\.golden.\\) and test input files (\.jsonnet\, \.txt\). These files are no longer part of the project's test execution.

_test\cmd · high confidence

The Jsonnet standard library has been reorganized from a single monolithic module into separate submodules (Array, Encoding, Manifest, Math, NativeRegex, Object, Set, and StdLibModule). This structural change improves code maintainability and allows for more targeted performance optimizations and bug fixes within specific function groups without affecting unrelated parts of the library.

sjsonnet/src/sjsonnet/stdlib · high confidence

New benchmark resources have been added to the \jdk17\_suite\ directory to validate string manipulation and hashing performance. The suite includes \hash.jsonnet\, which tests MD5, SHA-1, SHA-256, SHA-512, and SHA-3 outputs on repeated seed payloads; \repeat\_format.jsonnet\, which verifies string repetition, decimal/hex formatting, and padding alignment; and \split\_resolve.jsonnet\, which tests string splitting, joining, and path resolution logic. Golden files are provided for each test to ensure output consistency.

_bench/resources/jdk17\suite · high confidence

The base64 encode and decode operations in the standard library now use a native implementation backed by the aklomp/base64 C library, enabling SIMD acceleration (SSSE3, AVX, NEON, etc.) for significantly faster performance. This change introduces a new \PlatformBase64.scala\ file that handles the foreign function interface and includes optimizations such as skipping UTF-8 conversion for ASCII-safe inputs. Additionally, the implementation enforces strict RFC 4648 compliance, meaning unpadded base64 input is now rejected during decoding, which differs from the previous lenient behavior on the JVM.

sjsonnet/src-native/sjsonnet/stdlib · high confidence

The sjsonnet library now avoids loading large imported files entirely into memory. A new caching mechanism checks file sizes against a configurable threshold; files exceeding this limit are read directly from disk on demand rather than being cached in RAM, which prevents out-of-memory errors when processing very large imports.

sjsonnet/src-jvm-native/sjsonnet · high confidence

The sjsonnet standard library now includes a specialized \DeepArrayActiveSet\ to track visited arrays during deep traversal operations. This change prevents infinite loops and hangs in \deepJoin\ and \flattenDeepArray\ when processing recursive or cyclic data structures, ensuring these functions terminate correctly.

sjsonnet/src-js/sjsonnet/stdlib · high confidence

Resolves an issue where the \deepJoin\ and \flattenDeepArray\ standard library functions could hang indefinitely during recursive traversal. This fix introduces a \DeepArrayActiveSet\ to track visited arrays and prevent infinite loops, alongside refactoring native functions like \gzip\ into dedicated module objects for better structure.

sjsonnet/src-jvm-native/sjsonnet/stdlib · high confidence

The sjsonnet client now waits for the background output pump thread to finish before exiting, ensuring that all stdout and stderr data is fully flushed to the terminal. Previously, the client could exit prematurely while the daemon thread was still writing, leading to truncated or empty output in certain scenarios. This change introduces dedicated input/output pumper classes and a locking mechanism to manage the client-server communication lifecycle more robustly.

sjsonnet/client · high confidence

Added CLI and error-handling test cases; Added GraalVM native image test suite; Added JVM-specific test coverage for core sjsonnet components; Added JVM/Native test infrastructure and coverage for shared functionality; Added JavaScript-specific test infrastructure and async import validation; Added base64 encoding benchmark test case; Added benchmark tests for array views and JSON import performance; Added end-to-end integration tests for the sjsonnet client/server IPC flow; Added go-jsonnet test suite for sjsonnet; Added golden output refresh script for test suites; Added regression benchmarks for deep array traversal and object assertions; Added test cases for import argument validation errors; Added test resources for mixins and string quoting; Added tests for external access to Val.Str extractor; Added tests for the native xz compression function; Added unit tests for client I/O utilities and lock implementations; Expanded test coverage for sjsonnet core behaviors; Expanded test coverage for sjsonnet core evaluation and optimization; New JMH benchmark suite for sjsonnet performance profiling; Updated test suite with new assertions and error cases.

This release updates several core dependencies, including ujson to 4.4.3, lz4-java to 1.10.3, snakeyaml to 2.6, and re2j to 1.8. It also upgrades the Scala version to 3.3.8 and enables the -Yfuture-lazy-vals compiler flag, which may affect code relying on lazy val initialization order. The build now targets JDK 11 for both compilation and runtime.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 66.

• Code Health 77
• Architecture 98
• Maturity 61
• Readiness 80
• Security 89
• Accessibility 59

• 300 commits — 146 feature/other, 154 fixes

• sjsonnet/src — 135 commits
• sjsonnet/test — 109 commits
• (root) — 23 commits
• sjsonnet/src-jvm-native — 8 commits
• .github/workflows — 6 commits
• bench/resources — 6 commits
• bench/src — 3 commits
• sjsonnet/client — 2 commits
• sjsonnet/server — 2 commits
• sjsonnet/src-native — 2 commits
• (repo) — 1 commit
• .github/scripts — 1 commit
• docs/perf-gap-vs-jrsonnet.md — 1 commit
• sjsonnet/version — 1 commit

• fix: Fix client/server E2E lock root isolation (#1048)
• fix: Fix concurrency races on shared std module and MemberList caches (#1047) (#1052)
• fix: Fix performance regression with std.range and regenerate bench goldens (#1059)
• fix: Fix profiler to handle json imports properly (#1103)
• fix: Fix std.thisFile to preserve absolute paths (#1025)
• fix: More race condition fixes in StaticOptimizer (#1057)
• fix: Revert "Upgrade Scala 3 to 3.3.8 with -Yfuture-lazy-vals (#920)" (#936)
• fix: Revert "chore: bump scalafmt to 3.11.5 (#1093)" (#1098)
• fix: Revert "fix: --yaml-stream produces correct output matching C++ jsonnet/go-jsonnet (#1026)" (#1051)
• fix: Revert "fix: std.format rejects boolean for numeric conversion codes (#1016)" (#1053)
• fix: Revert "perf: SIMD-accelerated FastBase64 for Scala Native via C FFI" (#777)
• fix: Revert Scala.js to 1.20.2 (#1107)
• fix: also revert scala-yaml, which requires scala.js 1.22 (#1109)
• fix: fix recursive deepJoin and flattenDeepArray traversal hangs (#1041)
• fix: fix scaladoc warnings (#1066)
• fix: fix(ci): prevent indefinite hang in native dependency resolution (#995)
• fix: fix(client): wait for output pump before exit to prevent truncated stdout (#1009)
• fix: fix(escapeString): output non-ASCII characters literally in escapeStringJson/Python (#993)
• fix: fix(format): honor sign flags for zero float conversions (#987)
• fix: fix(format): normalize scientific rounding for %e and %g (#988)
• …and 280 more

• 0 containers · 1 bounded contexts · 0 dependency edges (baseline)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

databricks/sjsonnet was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.