combust/mleap
58.7
Adequate · 14 November 2026
28.3k
lines of production code
Scala
with Python
7
measurements over time
What this system is
Features
Add ALS recommendation transformer
Introduces a new ALS (Alternating Least Squares) transformer in the recommendation module, enabling users to apply ALS-based recommendation models within their data pipelines. The transformer takes user and item identifiers as input and outputs the predicted rating or score using the underlying ALSModel.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/recommendation · high confidence
Add Avro serialization support for LeapFrames and Rows
This change introduces a new Avro serialization format for MLeap data structures. It adds \DefaultFrameReader\ and \DefaultFrameWriter\ to serialize entire LeapFrames to and from Avro binary files, as well as \DefaultRowReader\ and \DefaultRowWriter\ for individual row serialization. The implementation includes \SchemaConverter\ and \ValueConverter\ to map MLeap data types (including scalars, lists, maps, and tensors) to Avro schemas and handle value conversion, enabling users to persist and load model data using the Avro format.
mleap-avro/src/main · high confidence
Add CategoricalDrilldown transformer for label-specific model execution
A new CategoricalDrilldown transformer has been added to the runtime, enabling the application of specific sub-models based on a categorical label column. This component takes a label and a row as input, selects the corresponding transformer from an internal map, and returns the transformed output columns prefixed with 'drilldown.', allowing for conditional processing paths within ensemble models.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/ensemble · high confidence
Add Databricks runtime testkit with Spark ML, TensorFlow, and XGBoost validation
The Databricks runtime testkit now includes a new entry point (Boot.scala) and dedicated test suites for validating model serialization and deserialization. Users can verify that Spark ML pipelines (using Logistic Regression), TensorFlow 2.0 models (via the new tensorflow/java binding), and XGBoost classifiers are correctly bundled and loaded within the Databricks runtime environment.
mleap-databricks-runtime-testkit/src · high confidence
Add HDFS support for MLeap bundle storage
The new bundle-hdfs module introduces a HadoopBundleFileSystem implementation, enabling users to save and load MLeap model bundles directly to and from HDFS (and optionally S3) using standard URIs. This includes automatic registration within Spark contexts, configurable Hadoop settings via application.conf, and programmatic customization for runtime usage.
bundle-hdfs · high confidence
Add JSON serialization support for LeapFrames, Rows, and data types
This change introduces a new JSON serialization format for MLeap data structures. It adds \DefaultFrameReader\ and \DefaultFrameWriter\ to serialize entire LeapFrames, as well as \DefaultRowReader\ and \DefaultRowWriter\ for individual rows. The \JsonSupport\ trait and \RowFormat\ object provide the underlying logic, enabling the serialization of complex types including scalars, lists, maps, tensors, and byte strings, while correctly handling nullable fields according to their schema definitions.
mleap-runtime/src/main/scala/ml/combust/mleap/json · high confidence
Add MLeap PySpark integration and serialization support
This change introduces the initial PySpark integration for MLeap by adding the \\_\init\\_.py\, \py2scala.py\, and \spark\_support.py\ modules. It registers MLeap features (StringMap, MathBinary, MathUnary) under the \pyspark.ml.mleap\ namespace and patches the PySpark \Transformer\ class with \serializeToBundle\ and \deserializeFromBundle\ methods to enable model bundling. It also provides helper utilities for accessing Scala objects and handling Scala-specific types like \Some\ and \None\ from the Python side.
python/mleap/pyspark · high confidence
Add MLeap serialization support for Gensim Word2Vec models
Users can now serialize Gensim Word2Vec models to MLeap bundles. This change adds a new \word2vec.py\ module in the \python/mleap/gensim\ package that patches the Gensim \Word2Vec\ class with MLeap-specific methods (\serialize\_to\_bundle\, \sent2vec\, and \mlinit\). The implementation extracts word vectors and vocabulary keys to create a serializable bundle, enabling these models to be exported for use in MLeap-compatible environments.
python/mleap/gensim · high confidence
Add MLeap serialization support for XGBoost classification and regression models
This change introduces new MLeap model operators (\XGBoostClassificationModelOp\ and \XGBoostRegressionModelOp\) that enable serializing and deserializing XGBoost models for use outside of Spark. The classification operator now persists the model's objective function, thresholds, and other configuration parameters alongside the native booster, ensuring that deserialized models retain their original inference settings. The regression operator similarly persists configuration parameters like missing value handling and batch size. This allows users to export trained XGBoost models to MLeap bundles for deployment in non-Spark environments.
mleap-xgboost-spark/src/main/scala · high confidence
Add MLeap serialization support for scikit-learn Decision Trees
Users can now serialize scikit-learn DecisionTreeClassifier and DecisionTreeRegressor models to MLeap bundles. This change introduces the \python/mleap/sklearn/tree\ module, which patches the scikit-learn classes to include MLeap-specific initialization and serialization logic, allowing these models to be exported for use in MLeap-based serving environments.
python/mleap/sklearn/tree · high confidence
Add MLeap-compatible Imputer and DefineEstimator extensions for sklearn pipelines
Users can now use the new \Imputer\ and \DefineEstimator\ classes in \python/mleap/sklearn/extensions/data.py\ to integrate sklearn imputation and estimator logic into MLeap pipelines. The \Imputer\ wraps \sklearn.impute.SimpleImputer\ to handle scalar data correctly during serialization, ensuring compatibility with MLeap's scoring engine, while \DefineEstimator\ allows splitting input matrices to pass specific columns to underlying transformers or estimators.
python/mleap/sklearn/extensions · high confidence
Add S3-backed repository support for MLeap bundles
Users can now download MLeap bundles directly from Amazon S3 using s3:// URIs. This change introduces the S3Repository implementation and its provider, allowing the executor to fetch model artifacts from S3 buckets by configuring the S3RepositoryProvider alongside existing file and HTTP repository providers. Credentials and region are resolved via the standard AWS default credential provider chain.
mleap-repository-s3 · high confidence
Add TensorFlow transformer with support for SavedModel and frozen graph formats
Introduces the MLeap TensorFlow integration, allowing users to load and execute TensorFlow models within MLeap pipelines. The new \TensorflowTransformer\ and \TensorflowModel\ components support both the legacy frozen graph format (.pb) and the newer SavedModel format (.zip), handling serialization and deserialization of model bytes, input/output schemas, and node configurations via the \TensorflowTransformerOp\ bundle operation.
mleap-tensorflow/src/main/scala/ml/combust/mleap/tensorflow · high confidence
Add bundle serialization support for OneVsRest and SVM classifiers
Users can now serialize and deserialize OneVsRest and Support Vector Machine (SVM) classification models in MLeap bundles. This change introduces new bundle operations (OneVsRestOp and SupportVectorMachineOp) that handle the storage and loading of these specific Spark ML transformers, enabling these models to be exported and used in non-Java/Scala environments.
mleap-spark-extension/src/main/scala/org/apache/spark/ml/bundle/extension/ops/classification · high confidence
Add clustering transformers for K-Means, Gaussian Mixture, Bisecting K-Means, and LDA
New transformer implementations have been added to the runtime to support inference with four clustering models: K-Means, Gaussian Mixture Model (GMM), Bisecting K-Means, and Latent Dirichlet Allocation (LDA). Each transformer wraps its respective core model (KMeansModel, GaussianMixtureModel, BisectingKMeansModel, LocalLDAModel) and exposes a UserDefinedFunction to process feature tensors. The GMM transformer specifically handles both prediction and probability outputs, while the others provide direct topic or cluster assignments.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/clustering · high confidence
Added BuildInfo and ClassLoaderUtil components to mleap-base
The mleap-base module now includes a new BuildInfo case class to expose build metadata (name, version, git SHA) and a ClassLoaderUtil object to assist in locating the appropriate class loader by inspecting the call stack. These additions provide foundational utilities for build identification and class loading within the base library.
mleap-base · high confidence
Added Spark demo data file
A new CSV file (examples/spark-demo.csv) has been added to the examples directory, containing sample test data with string and double columns for Spark-related demonstrations.
examples · high confidence
Added utility for direct PipelineModel creation
A new SparkUtil object has been added to the Spark MLEAP base module, providing helper methods to instantiate PipelineModel objects directly from an array of Transformers. This allows users to construct pipeline models without relying on the standard Pipeline builder, offering a more direct programmatic approach to model assembly.
mleap-spark-base/src/main/scala/org/apache/spark/ml/mleap · high confidence
Automated Scala classpath setup for Python tests
A new shell script, scripts/scala\_classpath\_for\_python.sh, has been added to streamline the environment setup for running Python tests that depend on Scala components. Sourcing this script automatically compiles the mleap-spark-extension and mleap-spark modules using sbt and exports the resulting dependency classpath into the SCALA\_CLASS\_PATH environment variable, removing the need for manual compilation or classpath configuration before executing Python-based tests.
scripts · high confidence
Automated release and Docker publishing pipeline
Added a set of Travis CI deployment scripts (docker.sh, extract.sh, publish.sh, release.sh, travis\_publish.sh) that enable automated signed publishing and Docker image generation for the mleap-serving and mleap-spring-boot modules. The pipeline decrypts release credentials, configures Git identity, switches to SSH for GitHub access, and triggers sbt release and Docker publish tasks only on the master branch when not triggered by a pull request.
travis · high confidence
Initial CI infrastructure and documentation overhaul
This change introduces the initial continuous integration setup for the project, adding a \.travis.yml\ configuration to run Scala and Python tests on Ubuntu Jammy with OpenJDK 11, and a \Makefile\ to standardize local and CI test execution. It also includes a comprehensive \RELEASE\_NOTES.md\ documenting the project's history from version 0.16.0 through 0.21.0, and updates the \README.md\ with modern badges, installation instructions for Maven/SBT/PyPI, and usage examples. Additionally, the repository metadata is cleaned up by fixing the \bundle-protobuf\ submodule URL, updating copyright notices in \LICENSE\ and \NOTICE\, and refining \.gitignore\.
(repo-wide) · high confidence
Initial Spark integration layer for MLeap serialization and data conversion
This change introduces the core Spark integration components for MLeap, enabling users to serialize and deserialize Spark ML transformers into MLeap bundles and convert data between Spark DataFrames and MLeap LeapFrames. The new \SimpleSparkSerializer\ class allows saving transformer bundles with a configurable serialization format (defaulting to JSON) and loading them back. \SparkLeapFrame\ provides the internal representation for MLeap data within the Spark ecosystem, supporting column operations like adding, selecting, dropping, and filtering. \SparkSupport\ supplies implicit conversions and extension methods, such as \toSparkLeapFrame\ and \toSpark\, facilitating seamless interoperability between Spark and MLeap data structures.
mleap-spark-base/src/main/scala/ml · high confidence
Initial import of MLeap core ML model implementations
This change introduces the foundational core models for MLeap, porting key Spark MLlib algorithms to run independently of a Spark context. The diff adds implementations for classification models (GBTClassifier, LinearSVC, MultiLayerPerceptron, NaiveBayes), clustering models (KMeans, BisectingKMeans, GaussianMixture, LDA), and feature transformers (Binarizer, Bucketizer, ChiSqSelector, CountVectorizer). It also includes the underlying neural network layer definitions (AffineLayer, LossFunctions) and utility objects (BreezeUtil) required for these models, establishing the core inference engine.
mleap-core/src/main · high confidence
Initial registration of Spark ML bundle operations
The \reference.conf\ file now defines the \ml.combust.mleap.spark.registry.builtin-ops\ list, registering a comprehensive set of Spark ML operators for serialization. This includes operations for classification (e.g., DecisionTree, NaiveBayes, GBT), clustering (e.g., KMeans, GaussianMixture), feature transformation (e.g., Tokenizer, VectorAssembler, StringIndexer), regression (e.g., LinearRegression, RandomForest), and tuning (CrossValidator). This configuration enables the runtime to recognize and bundle these specific Spark ML models and transformers.
mleap-spark/src/main/resources · high confidence
Initial release of MLeap benchmarking tool
Adds a new benchmarking utility in the mleap-benchmark module that allows users to measure the performance of MLeap and Spark model transformations. The tool provides command-line entry points for running MLeap row transforms, MLeap frame transforms, and Spark frame transforms against specified model bundles and data frames, with configurable iteration ranges and run counts defined in resource configuration files.
mleap-benchmark/src · high confidence
Initial release of MLeap runtime bundle serialization ops
This change introduces the core serialization and deserialization infrastructure for MLeap models. It adds the base \MleapOp\ and \MultiInOutMleapOp\ classes to handle model loading and node shape management, including legacy port name translation for multi-input/output transformers. It also implements the \PipelineOp\ for serializing transformer pipelines, and provides specific bundle operations for a wide range of ML components, including classification models (Decision Tree, GBT, Logistic Regression, Naive Bayes, SVM, Random Forest, One-vs-Rest, Multi-Layer Perceptron), clustering models (K-Means, Bisecting K-Means, Gaussian Mixture, LDA), and feature transformers (Binarizer, Bucketizer, Chi-Sq Selector, Coalesce, Count Vectorizer, DCT, Elementwise Product, Feature Hasher).
mleap-runtime/src/main/scala/ml/combust/mleap/bundle · high confidence
Initial release of the MLeap Python integration package
This change introduces the MLeap Python package, enabling users to serialize and deserialize machine learning models to MLeap Bundles. The package provides integration with PySpark and Scikit-Learn, including support for transformers like PCA, StringMap, and MathUnary. It also includes optional integration for gensim Word2Vec and a preliminary TensorFlow module. The package requires Python 3.10+ and manages dependencies via setup.py, with testing handled by tox.
python · high confidence
Initial scikit-learn preprocessing integration for MLeap serialization
This change introduces the core implementation for serializing scikit-learn preprocessing transformers (StandardScaler, MinMaxScaler, SimpleImputer, OneHotEncoder, Binarizer, and PolynomialFeatures) to MLeap bundles. The code in \python/mleap/sklearn/preprocessing/data.py\ extends these scikit-learn classes with MLeap-specific attributes and methods, enabling them to be serialized and deserialized within the MLeap runtime environment.
python/mleap/sklearn/preprocessing · high confidence
Initial serialization and deserialization infrastructure for MLeap bundles
The \python/mleap/bundle\ package now provides the core framework for serializing and deserializing machine learning models into MLeap bundle format. This includes the \MLeapSerializer\ class, which converts transformer attributes (such as scalars, vectors, tensors, and strings) into JSON structures for \model.json\ and \node.json\ files, and the \MLeapDeserializer\ class, which begins the process of reconstructing transformers from these bundles. This change establishes the foundational mechanism for exporting and importing models in a standardized way.
python/mleap/bundle · high confidence
Initial support for Spark ML serialization operations
This change introduces the initial set of Spark ML bundle operations (ops) for serializing and deserializing various Spark ML models and transformers into MLeap bundles. The diff adds new Scala files for classification models (GBTClassifier, MultiLayerPerceptronClassifier, NaiveBayes), clustering models (BisectingKMeans, GaussianMixture, KMeans, LDA), and feature transformers (Binarizer, BucketedRandomProjectionLSH, Bucketizer, ChiSqSelector, CountVectorizer, DCT, ElementwiseProduct, FeatureHasher, IDF, Interaction, MaxAbsScaler, MinHashLSH, MinMaxScaler, NGram, Normalizer). It also includes a utility object OpsUtils to copy Spark stage parameters. This enables users to save and load these specific Spark ML components using MLeap's bundle format.
mleap-spark/src/main/scala · high confidence
Initial support for serializing Scikit-learn models to MLeap bundles
This change introduces the \python/mleap/sklearn\ module, enabling users to serialize and deserialize Scikit-learn models into MLeap bundle formats. The implementation adds serialization support for \LinearRegression\, \LogisticRegression\ (including multi-class handling), \SVC\, \LinearSVC\, \FeatureUnion\, and \Pipeline\. By attaching specific \serialize\_to\_bundle\ and \deserialize\_from\_bundle\ methods to these Scikit-learn classes, the library allows these models to be exported as portable bundles containing model attributes, node definitions, and metadata.
python/mleap/sklearn · high confidence
Introduce JSON serialization support for Bundle.ML core types
Added a new \JsonSupport\ trait in the \bundle-ml\ module that provides \spray.json\ formats for Bundle.ML protobuf objects. This enables users to serialize and deserialize core data structures—including \BasicType\, \ByteString\, \Tensor\, \Scalar\, and \NodeShape\—to and from JSON, facilitating easier bundle inspection, debugging, and integration with JSON-based workflows.
bundle-ml/src/main/scala/ml/combust/bundle/json · high confidence
Introduce Java DSL for MLeap runtime operations
Added a new Java DSL in the \javadsl\ package to simplify MLeap usage from Java. This includes \BundleBuilder\ and \ContextBuilder\ for loading/saving bundles and creating contexts, \LeapFrameBuilder\ for constructing frames, schemas, rows, and data types (including tensor and map support), and support classes (\LeapFrameSupport\, \RowTransformerSupport\, \TensorSupport\) that expose core runtime capabilities like collecting frames, selecting/dropping columns, and converting tensors to arrays in a Java-friendly manner.
mleap-runtime/src/main/java · high confidence
Introduce MLeap Executor as a new backend engine for model serving
The mleap-executor module introduces a new backend engine built on Akka actors and streams to handle model loading, unloading, and data transformation. This component supports loading MLeap bundles from local file systems and HTTP sources via a configurable repository system, and provides both single-frame transforms and high-performance streaming row transforms with configurable parallelism, buffering, throttling, and idle timeouts. It serves as the underlying execution layer for higher-level integrations like mleap-spring-boot and mleap-grpc-server.
mleap-executor · high confidence
Introduce base Spark bundle serialization infrastructure
Adds the foundational classes and utilities for serializing Spark ML transformers into MLeap bundles. This includes \SimpleSparkOp\ and \MultiInOutSparkOp\ to handle single and multi-column input/output transformations, \SparkShapeSaver\ and \SparkShapeLoader\ to map Spark DataFrame schemas to bundle node shapes, \BundleTypeConverters\ for Spark-to-MLeap type mapping, and \SparkBundleContext\ to manage the serialization context and dataset. These components enable the framework to correctly capture and restore the structure and parameters of Spark ML pipelines.
mleap-spark-base/src/main/scala/org/apache/spark/ml/bundle · high confidence
Introduction of Bundle.ML serialization infrastructure
This change introduces the core Bundle.ML serialization framework, adding new components for managing model bundles. Users can now load and save ML models using the new BundleFile and BundleWriter classes, which support multiple storage backends including local files, JAR resources, and custom URI schemes via the BundleFileSystem interface. The BundleRegistry has been refactored to support context-aware serialization and dynamic loading of operations and file systems from configuration, enabling a more flexible and extensible model persistence system.
bundle-ml/src/main/scala/ml/combust/bundle · high confidence
Introduction of binary serialization format for LeapFrames and Rows
This change introduces a new binary serialization format for MLeap, adding \DefaultFrameReader\, \DefaultFrameWriter\, \DefaultRowReader\, and \DefaultRowWriter\ components to the \mleap-runtime\ module. These classes enable efficient binary encoding and decoding of LeapFrames and individual Rows, supporting nullable data types, basic primitives, strings, ByteStrings, lists, and tensors (both dense and sparse) via the new \ValueSerializer\ infrastructure. Users can now serialize and deserialize model execution contexts and data using this compact binary format alongside existing JSON/Avro methods.
mleap-runtime/src/main/scala/ml/combust/mleap/binary · high confidence
Introduction of core tensor data structures and JSON serialization
The mleap-tensor module now provides the foundational data structures for handling multi-dimensional arrays, introducing the \Tensor\ trait along with \DenseTensor\ and \SparseTensor\ implementations. These classes support various data types (Boolean, Byte, Short, Int, Long, Float, Double, String, and ByteString) and include logic for dimension normalization and value mapping. Additionally, a new \JsonSupport\ trait enables the serialization and deserialization of these tensors to and from JSON formats, allowing users to easily exchange tensor data in JSON-compatible environments.
mleap-tensor/src/main · high confidence
JSON serialization support for decision and clustering trees
Decision trees and clustering models can now be serialized to and deserialized from JSON format in addition to the existing Protobuf support. This change introduces new serializers and JSON format definitions for tree node structures (internal, leaf, categorical/continuous splits), enabling users to save and load tree-based machine learning models using human-readable JSON files alongside the binary Protobuf format.
bundle-ml/src/main/scala/ml/combust/bundle/tree · high confidence
New MLeap Executor Testkit for validating TransformService implementations
A new testkit module has been added to provide shared utilities for verifying implementations of the MLeap \TransformService\ trait. It introduces a \TransformServiceSpec\ trait that developers can mix into their own specs to run a comprehensive suite of tests covering model loading, unloading, transformation, and streaming. The module also includes \TestUtil\ with sample models (Airbnb regression and logistic regression bundles) and leap frames to facilitate these validations, ensuring consistent behavior across executor implementations like \mleap-executor-tests\ and \mleap-grpc-server\.
mleap-executor-testkit · high confidence
New MLeap gRPC client module for remote model serving
The mleap-grpc module now provides a client-side implementation of the MLeap gRPC protocol, allowing users to load, unload, transform, and stream data against models hosted by a separate mleap-grpc-server process. This change introduces the GrpcClient, which implements the TransformService trait, along with TypeConverters for translating between MLeap types and Protobuf representations, and GrpcAkkaStreams helpers for managing Akka streams over gRPC connections.
mleap-grpc · high confidence
New MLeap gRPC server module for remote model serving
The mleap-grpc-server module introduces a standalone gRPC server that exposes the MLeapExecutor to remote clients, enabling them to load, unload, transform, and stream data against MLeap models. The server is bootstrapped via the \ml.combust.mleap.grpc.server.Boot\ entry point, which reads configuration (such as the listen port, defaulting to 65328) from Typesafe config and command-line arguments. It implements the \Mleap\ gRPC service by delegating to the executor, handling frame and row transformations via streaming RPCs, and includes an \ErrorInterceptor\ to map internal exceptions (like \NotFoundException\ or \TimeoutException\) to appropriate gRPC status codes.
mleap-grpc-server · high confidence
New MLeap serving module with unified gRPC and HTTP servers
A new mleap-serving module has been added, providing a single entry point (RunServer) that starts both the gRPC server (from mleap-grpc-server) and the HTTP server (from mleap-spring-boot) backed by a shared MLeapExecutor. This allows models to be served over both protocols from one process, with configuration accessible via command-line arguments such as --grpc-port.
mleap-serving · high confidence
New PySpark feature transformers for math operations and string mapping
This change introduces three new PySpark transformer components in the \python/mleap/pyspark/feature\ package: \MathBinary\, \MathUnary\, and \StringMap\. \MathBinary\ enables binary mathematical operations (such as Add, Subtract, Multiply, Divide, Min, Max, and LogN) on two input columns, supporting default values for missing inputs. \MathUnary\ applies single-column mathematical transformations (including Sin, Cos, Tan, Log, Exp, Abs, Sqrt, and Logit). \StringMap\ provides label encoding functionality, allowing users to map string labels to numeric values via a dictionary or a Spark DataFrame, with configurable handling for invalid labels (\handleInvalid\) and default values. These components bridge MLeap's core feature models to PySpark pipelines.
python/mleap/pyspark/feature · high confidence
New TypeConverters for Spark-MLeap schema and value mapping
A new TypeConverters trait has been introduced in the mleap-spark-base module to handle the conversion of data types and schemas between Apache Spark DataFrames and MLeap LeapFrames. This change adds support for mapping Spark types (such as Vector, Matrix, Decimal, and various array types) to their MLeap equivalents (Tensor, List, Scalar) and vice versa, including logic to infer vector sizes from Spark metadata and handling nullable fields during conversion.
mleap-spark-base/src/main/scala/org/apache/spark/sql · high confidence
New feature transformer bundle operations
Added bundle serialization and deserialization support for several new feature transformers: Imputer, MapEntrySelector, MathBinary, MathUnary, MultinomialLabeler, StringMap, and WordLengthFilter. These changes allow models using these specific Spark ML/MLeap feature operations to be correctly saved to and loaded from MLeap bundles, enabling deployment of pipelines that utilize missing value imputation, map entry selection, mathematical transformations, multinomial labeling, string mapping with invalid value handling, and word length filtering.
mleap-spark-extension/src/main/scala/org/apache/spark/ml/bundle/extension/ops/feature · high confidence
New feature transformers for data imputation, mapping, and math operations
This update introduces several new Spark ML transformers to the MLeap feature library, enabling users to handle missing data, extract map entries, perform arithmetic, and filter text. The new Imputer replaces missing values in numeric columns using either the mean or median. MapEntrySelector allows extracting specific keys from map-type columns with a configurable default value. MathUnary and MathBinary apply unary and binary mathematical operations to numeric columns, supporting both column-based and constant-value inputs. StringMap maps string labels to numeric values, handling invalid inputs via a configurable strategy. Finally, WordLengthFilter removes words from string arrays that are shorter than a specified minimum length.
mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/feature · high confidence
New parameter traits for labels and probabilities columns
This change introduces two new parameter traits, HasLabelsCol and HasProbabilitiesCol, which allow Spark transformers to configure specific output columns for storing label values and probability scores, respectively. Additionally, the existing HasDropLast trait has been moved from the mleap-spark module to this extension module to consolidate shared parameter definitions.
mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/param · high confidence
Register XGBoost model operations in MLeap Spark
The MLeap Spark integration now automatically registers the XGBoost classification and regression model operations. Users can utilize XGBoost models within the MLeap Spark runtime without needing to manually configure the operation registry, as the default operations list now includes the specific XGBoost model operators.
mleap-xgboost-spark/src/main/resources · high confidence
Register built-in MLeap runtime operations
A new \reference.conf\ file has been added to the MLeap runtime resources, defining the \ml.combust.mleap.registry.builtin-ops\ configuration list. This configuration explicitly registers a comprehensive set of built-in operations for classification, clustering, feature transformation, regression, ensemble methods, and recommendation models, ensuring these components are automatically available in the runtime registry without manual configuration.
mleap-runtime/src/main/resources · high confidence
Support for String and ByteString tensor types in TensorFlow conversion
The TensorFlow converter now supports converting String and ByteString data types between MLeap and TensorFlow tensors. Previously, only numeric types (int, float, double, byte) were handled; the new implementation in MleapConverter and TensorflowConverter allows scalar and array/string-tensor conversions for text-based data, enabling models that output or require string inputs/outputs to be used within MLeap pipelines.
mleap-tensorflow/src/main/scala/ml/combust/mleap/tensorflow/converter · high confidence
Support for serializing Scikit-learn CountVectorizer and TfidfVectorizer
Users can now serialize Scikit-learn text feature extraction transformers (CountVectorizer and TfidfVectorizer) to MLeap bundles. This change adds the necessary serialization logic in the feature\_extraction module, allowing these models to be exported and used in MLeap-compatible environments.
_python/mleap/sklearn/feature\extraction · high confidence
Support for serializing scikit-learn Random Forest models
Users can now serialize scikit-learn Random Forest Regressor and Classifier models to MLeap bundles. This change adds the necessary integration code in the sklearn ensemble module to handle the serialization of individual trees within the forest, including their weights and structure, allowing these models to be exported for use in MLeap-based pipelines.
python/mleap/sklearn/ensemble · high confidence
Removals
Removal of Bundle.ML DSL and serialization support
The \bundle-ml\ module's domain-specific language (DSL) and associated serialization infrastructure have been removed. This includes the deletion of core DSL classes such as \Bundle\, \Model\, \Node\, \Shape\, \Attribute\, and \Value\, which previously allowed users to construct and manipulate ML pipeline graphs programmatically. Additionally, the JSON serialization support provided by \JsonSupport\ and the tree serialization logic in \TreeSerializer\ are no longer available, meaning users can no longer serialize or deserialize Bundle.ML models using these specific Scala APIs.
bundle-ml/src/main/scala/ml/bundle/dsl · high confidence
Removal of legacy MLeap serialization and registry components
The \MleapBundle\ and \MleapRegistry\ objects in the \mleap-runtime\ serialization bundle package have been removed. This eliminates the previous implementation for reading and writing transformer graphs and individual transformers, as well as the static registry instance that registered specific classification, feature, and regression operations (such as OneVsRest, StandardScaler, and LinearRegression). Users relying on these specific entry points for bundle serialization will need to adopt the updated serialization mechanisms provided by the current MLeap and Spark registry integration.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/serialization/bundle · high confidence
Removal of serialization ops for classification, regression, and feature transformers
The serialization operations for several machine learning models and feature transformers have been removed from the \mleap-runtime\ bundle. Specifically, the code handling serialization for \Pipeline\, \DecisionTreeClassifier\, \LogisticRegression\, \OneVsRest\, \RandomForestClassifier\, \SupportVectorMachine\, \ReverseStringIndexer\, \StandardScaler\, \StringIndexer\, \VectorAssembler\, \DecisionTreeRegression\, \LinearRegression\, and \RandomForestRegression\ has been deleted. This change eliminates the ability to serialize and deserialize these specific model types using the MLeap bundle format.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/serialization/bundle/ops · high confidence
Removed legacy Bundle.ML serializer implementation
The serialization logic in \bundle-ml/src/main/scala/ml/bundle/serializer\ has been removed. This includes the \BundleContext\, \BundleSerializer\, \ModelSerializer\, \NodeSerializer\, and associated format definitions (\SerializationFormat\, \SerializationContext\) and custom type serializers. This cleanup removes the previous JSON/Protobuf/Mixed serialization infrastructure to prepare for the updated MLeap/Spark integration.
bundle-ml/src/main/scala/ml/bundle/serializer · high confidence
Security
Zip extraction now validates against path traversal attacks
The FileUtil utility now includes Zip Slip validation when extracting archives. Before writing files to disk, the code verifies that each entry's canonical path remains within the intended destination directory, throwing an exception if a malicious archive attempts to escape the target folder. This prevents directory traversal vulnerabilities during bundle extraction.
bundle-ml/src/main/scala/ml/combust/bundle/util · high confidence
Behavioural changes
1 commit (0 fixes) modifying mleap-xgboost-benchmark, mleap-xgboost-runtime, python
A change to existing behaviour in mleap-xgboost-benchmark, mleap-xgboost-runtime, python — 1 commit, 7 files: “Read a sparse gap as missing for XGBoost prediction (\#909)” (refs \#909). Also in this area: Added benchmark for pure-JVM XGBoost predictor performance.
mleap-xgboost-benchmark · low confidence · unverified
Bundle.ML serializer refactored to support JSON and Protobuf formats with improved error handling
The serialization layer in bundle-ml has been restructured to introduce explicit support for both JSON and Protobuf serialization formats via new \SerializationFormat\, \ModelSerializer\, and \NodeSerializer\ components. This change replaces the previous \GraphSerializer\ with a more robust implementation that wraps write and read operations in \scala.util.Try\, allowing callers to handle serialization and deserialization errors gracefully rather than encountering unhandled exceptions.
bundle-ml/src/main/scala/ml/combust/bundle/serializer · high confidence
Classification transformers now support raw predictions and probabilities
The classification transformers in the runtime (DecisionTree, LogisticRegression, RandomForest, SupportVectorMachine, OneVsRest, NaiveBayes, GBT, MultiLayerPerceptron, and the new LinearSVC) have been refactored to extend MultiTransformer instead of the legacy Transformer interface. This change allows these models to optionally output raw predictions and class probabilities alongside the final prediction, determined by the node's output schema configuration, rather than only returning a single prediction column.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/classification · high confidence
Default Spark transformer operations registered in reference configuration
The MLeap Spark extension now automatically registers a set of default Spark transformer operations via the new reference.conf file. This configuration includes operators for classification (OneVsRest, SVM), feature engineering (Imputer, MapEntrySelector, MathBinary, MathUnary, MultinomialLabeler, WordLengthFilter, StringMap), ensuring these transformers are available out-of-the-box without manual registration.
mleap-spark-extension/src/main/resources · high confidence
Feature transformers migrated to the new SimpleTransformer interface
The feature transformers in \mleap-runtime\ (including Binarizer, Bucketizer, CountVectorizer, IDF, Imputer, Normalizer, StandardScaler, StringIndexer, and others) have been refactored from the legacy \Transformer\ interface with manual \build\ logic to the new \SimpleTransformer\ interface. This change replaces explicit column registration and row-based execution with a declarative \NodeShape\ and \model\-driven \UserDefinedFunction\ execution, simplifying the transformer implementation and aligning with the updated runtime execution model.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/feature · high confidence
MLeap runtime refactored to use new frame and serialization APIs
The MLeap runtime has been restructured to align with the new Bundle.ML architecture. The legacy \LeapFrame\, \DefaultLeapFrame\, \Row\, and \Dataset\ types have been removed from the root \ml.combust.mleap.runtime\ package and are now accessed via the \frame\ subpackage, with backward-compatibility aliases marked for removal in version 1.0. The \MleapSupport\ trait has been updated to provide new implicit operations for writing bundles, loading bundles from URIs, and converting case classes to LeapFrames using the new \LeapFrameConverter\. Additionally, new utility classes for selectors (\Selector\, \FieldSelector\, \StructSelector\) and user-defined functions (\UserDefinedFunction\) have been introduced to support the updated function execution model.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime · high confidence
MLeap serving overhaul with new REST API and error handling
The MLeap Spring Boot module has been overhauled to provide a modernized serving interface. A new GlobalExceptionHandler has been added to standardize error responses for various HTTP and application exceptions. The serving API is now split into dedicated controllers: JsonScoringController and ProtobufScoringController handle model lifecycle operations (load, unload, get, meta) and data transformation via JSON and Protobuf formats respectively, while LeapFrameScoringController specifically handles Leap Frame transformations in JSON and binary formats. Additionally, a new ModelLoader component enables automatic loading of models from a configured directory or file path at startup, and the underlying infrastructure (StarterConfiguration, TypeConverters) has been updated to support these new components and data formats.
mleap-spring-boot/src/main/scala · high confidence
Migrate to ml.combust.bundle package and introduce context-aware serialization
The Bundle.ML DSL and core serialization types have been moved from the \ml.bundle\ package to \ml.combust.bundle\, aligning with the library's new Maven coordinates. This change introduces a generic \Context\ type parameter to the \OpModel\ and \OpNode\ traits, allowing serialization and deserialization logic to access a \BundleContext\ for handling custom types and deferred operations. Additionally, the DSL now includes dedicated \Attributes\, \Model\, \Node\, and \NodeShape\ classes to represent the bundle structure, replacing the previous implicit or simpler representations.
bundle-ml/src/main/scala/ml/combust/bundle/dsl · high confidence
New binary serialization for tensors and array types
The bundle-ml module now uses a new binary serialization format for tensors and their underlying array data. This change introduces dedicated serializers for Boolean, Byte, Short, Int, Long, Float, Double, String, and ByteString arrays, replacing previous methods. Users relying on the internal representation of serialized tensor bundles may see changes in the binary format, although the logical tensor data remains the same.
bundle-ml/src/main/scala/ml/combust/bundle/tensor · high confidence
OneVsRest and SVM classification models updated for Spark compatibility and behavior
The OneVsRest and SVM classification components have been moved to the mleap-spark-extension module and updated to align with Spark's API changes. In OneVsRestModel, prediction logic now correctly handles ProbabilisticClassificationModel instances by using the probability column instead of raw predictions, and the model loading process has been adjusted to use the metadata object for parameter setting. For SVM, the default threshold is no longer hardcoded to 0.5; instead, it is explicitly set from the underlying model if available, and a clearThreshold method has been added. Additionally, SVMModel's copy and raw2probabilityInPlace methods have been simplified to match Spark's expected behavior.
mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/classification · high confidence
Overhaul of the MLeap type system and introduction of Bundle serialization converters
The internal type system has been refactored, replacing the previous sealed trait hierarchy (BasicType, TensorType, ListType) with a new structure that includes a ByteString type. To support serialization, a new BundleTypeConverters trait has been added, providing implicit conversions between MLeap's internal types and the external Bundle protocol format, covering basic types, data shapes (Scalar, List, Tensor), schemas, and node shapes.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/types · high confidence
Refactored LeapFrame runtime with consistent interface and asynchronous support
The \mleap-runtime\ frame package has been restructured to provide a consistent, trait-based interface for data frames. A new \FrameBuilder\ trait and \LeapFrame\ trait define a unified API for schema operations (select, withColumn, drop, filter), replacing previous ad-hoc implementations. The runtime now includes \ArrayRow\ as the default row implementation and \DefaultLeapFrame\ for in-memory storage, along with \RowTransformer\ for efficient, lazy row transformations. Additionally, the \Transformer\ trait now supports asynchronous execution via \transformAsync\, enabling non-blocking model inference.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/frame · high confidence
Register TensorFlow transformer operations by default
A new reference.conf file registers the TensorflowTransformerOp and the general TensorFlow operations package as default operations in the MLeap registry. This ensures that TensorFlow transformers are automatically available without requiring manual configuration.
mleap-tensorflow/src/main/resources · high confidence
Regression transformers adopt new transformer interface and add new model support
The regression transformer components in the runtime library have been refactored to use a new \SimpleTransformer\ and \MultiTransformer\ base classes with a \NodeShape\-based schema, replacing the previous \TransformBuilder\-based approach that relied on explicit column names. This change simplifies the transformer implementation and aligns with the updated typing system. Additionally, support for new regression models has been added, including AFT Survival Regression, Gradient Boosted Tree (GBT) Regression, Generalized Linear Regression, and Isotonic Regression, allowing users to deploy these specific model types.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/regression · high confidence
Serialization architecture refactored to separate readers and writers
The serialization module has been restructured to decouple reading and writing operations. The previous \FrameSerializer\ trait, which combined both concerns, has been removed and replaced with distinct \FrameReader\ and \FrameWriter\ traits. This change introduces a factory-based approach where readers and writers are instantiated dynamically by format (e.g., JSON, binary, Avro) via \BuiltinFormats\, allowing users to explicitly choose the appropriate reader or writer for their data operations rather than using a monolithic serializer.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/serialization · high confidence
Transformer interface overhaul and Pipeline schema support
The transformer API has been migrated from the legacy \ml.combust.mleap.runtime.transformer\ traits to the new \ml.combust.mleap.runtime.frame\ types (\FrameTransformer\, \FrameBuilder\), with the old types marked as deprecated for removal in MLeap 1.0. This change introduces an asynchronous transformation interface (\transformAsync\) and enforces explicit schema management on \Pipeline\, which now exposes distinct input, output, intermediate, and strict output schemas derived from its constituent transformers.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer · high confidence
XGBoost inference now uses a pure-JVM predictor for better performance
MLeap's XGBoost runtime has migrated from the JNI-based xgboost4j Booster to the pure-JVM com.yelp:xgboost-predictor engine. This change removes the native library dependency on the hot path, improving inference performance and deployment simplicity. The runtime now supports both classification and regression models via new ops (XGBoostPredictorClassificationOp, XGBoostPredictorRegressionOp) and models that wrap the Predictor. It handles dual-format bundle loading (legacy binary and UBJSON) so existing deployed bundles continue to work without migration. Missing value handling is preserved, with support for models trained with missing=0.0f (treated as absent) via the treatsZeroAsNA flag.
mleap-xgboost-runtime/src/main · high confidence
Fixes
Fix polynomial feature ordering mismatch between MLeap and scikit-learn
The PolynomialFeatures transformer in MLeap now generates polynomial features in the same order as scikit-learn, ensuring consistent output when migrating models. This change resolves discrepancies where the sequence of generated polynomial terms differed between the two libraries, which could lead to incorrect predictions if model expectations were not aligned.
mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/sklearn · high confidence
Test coverage
Added PySpark feature tests for MathBinary, MathUnary, and StringMap; Added PySpark test infrastructure and utilities; Added Python test suite for scikit-learn model serialization and deserialization; Added XGBoost training dataset for tests; Added and standardized regression model tests; Added and standardized unit tests for MLeap feature models; Added comprehensive tests for Tensor equality and legacy indexing behavior; Added integration tests for JSON and Protobuf scoring endpoints; Added integration tests for MLeap Executor components; Added parity test for LinearSVC transformer; Added parity test for Support Vector Machine transformer; Added parity tests for ALS recommendation model; Added parity tests for CrossValidator and TrainValidationSplit; Added parity tests for Spark ML feature transformers; Added parity tests for XGBoost classification and regression models; Added parity tests for clustering algorithms; Added parity tests for new and updated feature transformers; Added parity tests for regression models; Added test coverage for MLeap runtime core components; Added test coverage for TensorFlow integration; Added test coverage for classification model schemas and predictions; Added test for null vocabulary serialization error; Added tests for ALSModel prediction and schema; Added tests for Avro serialization of LeapFrames and rows; Added tests for Bundle.ML serialization and file system operations; Added tests for Gensim Word2Vec integration; Added tests for MLeap vector conversion utilities; Added tests for MleapReflection utility methods; Added tests for PolynomialFeaturesModel; Added tests for multi-output transformer support; Added tests for type casting and structural type operations; Added unit tests for core clustering model schemas and behavior; Expanded test coverage for XGBoost runtime models and predictors; New Spark/MLeap parity testing infrastructure and sample datasets.
Dependencies
MLeap build infrastructure upgraded to Scala 2.13, Java 17, and Spark 4.1.2
The project's build system has been significantly modernized to support newer runtime environments. The default Scala version is now 2.13.18 (up from 2.11.7) and Java compilation targets Java 17, requiring the use of specific JVM flags for module access. The primary Spark dependency has been upgraded to version 4.1.2, and Spring Boot has been updated to 3.2.0 to align with these Java 17 requirements. Additionally, the build tool itself has been upgraded from sbt 0.13.12 to 1.12.5, and the publishing infrastructure has migrated from the legacy Sonatype API to the new Sonatype Central Portal using the sbt-ci-release plugin.
project · high confidence
Reorganized build structure and updated Python dependencies
The build system has been reorganized to use a centralized project definition in \MleapProject\, splitting the monolithic \build.sbt\ into individual module files (e.g., \mleap-core/build.sbt\, \mleap-spark/build.sbt\) that import shared settings and dependencies. In the Python environment, \pyspark\ is pinned to version 4.1.2, \scikit-learn\ is restricted to the 1.0.x series, \gensim\ is set to \>4.1.0, and \urllib3\ is updated to 1.26.17.
(dependencies) · high confidence
Housekeeping
MLeap Python package version updated to 0.25.3
The MLeap Python package version has been updated to 0.25.3. This change ensures that the \mleap\ module exposes the correct version string via \\_\version\\_\, reflecting the current release state.
python/mleap · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 58 → 59 (+0.4)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 53 → 53 (-0.6)
- Architecture 98 → 98 (+0.0)
- Maturity 57 → 56 (-0.2)
- Readiness 57 → 61 (+4.1)
- Security 74 → 70 (-4.0)
- Performance 80 → 91 (+11.4)
Resolved (4)
- Duplicated block (7 lines × 2) (python/mleap/sklearn/preprocessing/data.py)
- Further sole-owners (lower concentration)
- No ADRs found
- Off-boarding risk: anonymized user #1
New (55)
- Edited copies of one member (3 members, 15+ corresponding lines) (mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/feature/Imputer.scala)
- Edited copies of one member (3 members, 15+ corresponding lines) (mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/feature/MathBinary.scala)
- Edited copies of one member (3 members, 15+ corresponding lines) (python/mleap/sklearn/preprocessing/data.py)
- Edited copies of one member (3 members, 5+ corresponding lines) (mleap-spring-boot/src/main/scala/ml/combust/mleap/springboot/JsonScoringController.scala)
- Edited copies of one member (3 members, 9+ corresponding lines) (mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/feature/MathUnary.scala)
- Edited copy of a member (1 corresponding lines) (mleap-grpc/src/main/scala/ml/combust/mleap/grpc/TypeConverters.scala)
- Edited copy of a member (10 corresponding lines) (mleap-spark-base/src/main/scala/org/apache/spark/sql/mleap/TypeConverters.scala)
- Edited copy of a member (11 corresponding lines) (mleap-executor/src/main/scala/ml/combust/mleap/executor/service/FrameStreamActor.scala)
- Edited copy of a member (11 corresponding lines) (mleap-runtime/src/main/scala/ml/combust/mleap/runtime/frame/IndexedRowUtil.scala)
- Edited copy of a member (12 corresponding lines) (mleap-runtime/src/main/scala/ml/combust/mleap/runtime/frame/DefaultLeapFrame.scala)
- Edited copy of a member (14 corresponding lines) (mleap-runtime/src/main/scala/ml/combust/mleap/runtime/frame/DefaultLeapFrame.scala)
- Edited copy of a member (15 corresponding lines) (mleap-executor/src/main/scala/ml/combust/mleap/executor/service/BundleActor.scala)
- Edited copy of a member (16 corresponding lines) (mleap-runtime/src/main/scala/ml/combust/mleap/runtime/frame/RowTransformer.scala)
- Edited copy of a member (16 corresponding lines) (mleap-spark-extension/src/main/scala/org/apache/spark/ml/mleap/feature/MapEntrySelector.scala)
- Edited copy of a member (16 corresponding lines) (mleap-spring-boot/src/main/scala/ml/combust/mleap/springboot/LeapFrameScoringController.scala)
- Edited copy of a member (16 corresponding lines) (python/mleap/sklearn/base.py)
- Edited copy of a member (17 corresponding lines) (mleap-core/src/main/scala/ml/combust/mleap/core/feature/FeatureHasherModel.scala)
- Edited copy of a member (17 corresponding lines) (mleap-executor/src/main/scala/ml/combust/mleap/executor/Transform.scala)
- Edited copy of a member (17 corresponding lines) (mleap-executor/src/main/scala/ml/combust/mleap/executor/service/FrameStreamActor.scala)
- Edited copy of a member (18 corresponding lines) (mleap-spark/src/main/scala/org/apache/spark/ml/bundle/ops/feature/OneHotEncoderOp.scala)
- …and 35 more
Changes since last survey
- 1 commits — 1 feature/other, 0 fixes
By area
- mleap-xgboost-runtime/src — 1 commit
Notable commits
- change: Read a sparse gap as missing for XGBoost prediction (#909)
Architecture
- Containers 0 added · 0 removed · contexts 0 added · 0 removed · edges 5 added · 13 removed
Added dependency edges (5)
- bundle-ml → mleap-tensor (coupling)
- mleap-runtime → mleap-tensor (coupling)
- mleap-spark-testkit → mleap-core (coupling)
- mleap-tensorflow → bundle-ml
- mleap-xgboost-runtime → bundle-ml
Removed dependency edges (was a dependency, now gone) (13)
- bundle-ml → mleap-core (coupling)
- bundle-ml → mleap-runtime
- bundle-ml → mleap-tensor
- mleap-base → mleap-core
- mleap-core → bundle-ml (coupling)
- mleap-runtime → mleap-tensor
- mleap-spark-extension → mleap-runtime (coupling)
- mleap-spark-testkit → bundle-ml (coupling)
- mleap-spark-testkit → mleap-core
- mleap-tensor → bundle-ml
- mleap-tensor → mleap-core
- mleap-tensorflow → bundle-ml (coupling)
- mleap-xgboost-runtime → bundle-ml (coupling)
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
combust/mleap was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 9ffe17064a42459556617c1de9ae31f51cf6f90e — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-36d4cd83012f.