feathr-ai/feathr

This release introduces a comprehensive feature registry with SQL and Microsoft Purview backends, enabling robust feature management and lineage tracking. The Feathr Python SDK is updated to version 1.0.0, featuring a redesigned API, YAML-based configuration, and support for local, Databricks, and Azure Synapse runtimes. Significant architectural changes include a migration from SBT to Gradle, the introduction of a new compute graph model, and the addition of an interactive Web UI for feature visualization. The release also adds a local sandbox environment, sample datasets, and extensive test coverage for the new components.

Users can now interact with an experimental chatbot inside Jupyter notebooks via the \feathr\ magic command. The chatbot, powered by ChatGPT, allows users to ask questions about Feathr DSL, receive code snippets for feature engineering, and execute generated code directly in the notebook. The feature requires setting the \CHATGPT\_API\_KEY\ environment variable and creating a \FeathrClient\ instance named \client\ before use.

_feathr\project/feathr/chat · high confidence

Users can now easily access pre-configured sample datasets for common machine learning scenarios, including NYC Taxi fare prediction, fraud detection, product recommendation, and hotel reviews. The new \feathr.datasets\ module provides utility functions to download and retrieve these datasets as Pandas or Spark DataFrames, with specific path handling for Databricks environments.

_feathr\project/feathr/datasets · high confidence

The Feathr CLI now includes a new 'hocon' command that scans Python-based feature definitions and converts them to HOCON config files, allowing users to generate configuration files for their features. Additionally, the 'join' and 'deploy' commands have been updated to use new internal client methods (\_get\_offline\_features\_with\_config and \_materialize\_features\_with\_config), which likely reflects changes in how feature joins and deployments are executed, potentially improving configuration handling or integration with the registry system.

_feathr\project/feathrcli · medium confidence

A new Python script, migrate\_legacy\_project.py, has been added to the registry/purview-registry/script directory. This script automates the migration of legacy workspace data into the Purview registry by querying for specific entity types (feathr\_workspace\_v1, feathr\_anchor\_v1, etc.) and generating the necessary process entities and relations (CONTAINS, BELONGS\_TO, CONSumES, PRODUCES) to update the registry.

registry/purview-registry/script · medium confidence

Added three new Python files defining feature sets for the NYC taxi dataset: \agg\_features.py\ introduces window-based aggregations (average and maximum fare per location), \non\_agg\_features.py\ defines non-aggregated features like trip distance and day of week, and \request\_features.py\ specifies request-time features including derived features for trip time and distance calculations.

_feathr\_project/feathrcli/data/feathr\_user\workspace/features · high confidence

Users can now generate DSL (Domain Specific Language) representations for their feature definitions. This includes a new \dsl\_generator.py\ module that tokenizes and parses feature transformations into an Abstract Syntax Tree (AST), and a \functions.py\ file that lists all supported SQL-like functions (e.g., \abs\, \concat\, \date\_add\) used in feature transformations. A shell script \gen\_function\_list.sh\ is also added to automatically update the supported function list from the Feathr online pipeline source code. Tests are included to verify the DSL generation for supported features and to ensure unsupported features (like \WindowAggTransformation\) raise errors.

_feathr\project/feathr/utils/dsl · high confidence

Users can now define Python UDFs for feature preprocessing, which are executed via PySpark. The system generates a PySpark driver script and manages the mapping of features to their corresponding UDFs, allowing Python-based preprocessing logic to be integrated into the feature engineering pipeline.

_feathr\project/feathr/udf · high confidence

The UI public directory now includes essential static files to support the Feathr Feature Store Web UI. This includes an environment configuration script (env-config.js) that allows runtime injection of variables, the main HTML entry point (index.html) referencing the React root, a PWA manifest (manifest.json) defining the app's identity and icons, a robots.txt file, and a static web app configuration (staticwebapp.config.json) for routing and MIME types.

ui/public · high confidence

The Feathr Python SDK has been updated to version 1.0.0, introducing a redesigned, Pythonic API for feature definition and client initialization. Configuration is now managed via a YAML file (feathr\_config.yaml) and environment variables, replacing the previous environment-variable-only approach. The \FeathrClient\ has been refactored to support local, Databricks, and Azure Synapse runtimes, with improved error handling and logging. The feature registry integration with Azure Purview has been updated to use a new client, and the SDK now exposes a comprehensive set of classes for defining anchors, features, and transformations, along with utilities for feature printing and job parameter management.

_feathr\project/feathr · high confidence

The Feathr Web UI is introduced as a new component, accompanied by comprehensive development tooling and configuration files. This includes ESLint and Prettier for code linting and formatting, a TypeScript configuration, and a CRACO build configuration that enables Less module support and exposes the application version. Additionally, environment variable files (.env) are added to manage Azure AD authentication and API endpoints, allowing developers to easily override configurations for local development.

ui · high confidence

Adds a new Access Control Gateway plugin that sits in front of the feature registry API to enforce read, write, and manage permissions based on user roles stored in a SQL database. The change introduces environment variables (RBAC\_API\_BASE, RBAC\_API\_CLIENT\_ID, etc.) to configure the gateway, defines a \userroles\ table schema for storing project and global admin roles, and exposes management APIs (GET/POST/DELETE /userroles) for project administrators to assign or revoke roles. Users must enable the \ENABLE\_RBAC\ flag and configure the gateway to activate this security layer.

_registry/access\control · high confidence

The Feathr library now includes a new registry module that enables users to register, list, and delete features and anchors. This change introduces a \FeathrRegistry\ abstract base class and concrete implementations for interacting with a remote registry, including a new \\_FeatureRegistry\ client and a \\_PurviewRegistry\ client for Azure Purview. Users can now synchronize local feature definitions with a central registry, allowing for better feature discovery and lineage tracking.

_feathr\project/feathr/registry · high confidence

The UI now includes a complete frontend application, starting with an API client layer that handles authentication via Azure MSAL and manages requests for projects, features, and data sources. A new interactive FlowGraph component renders feature lineage as a visual graph, allowing users to explore dependencies between features. The interface also provides a sidebar for navigation, a header with project switching, and reusable components like editable tables and resizable columns, enabling users to manage and view their Feathr assets directly through the web interface.

ui/src · high confidence

The Feathr compute module now includes a new internal API for representing and manipulating a compute graph. This adds classes for building the graph (ComputeGraphBuilder), merging and validating graph structures (ComputeGraphs), managing node dependencies (Dependencies), and resolving feature requests into optimized subgraphs (Resolver). It also introduces a set of operator identifiers for MVEL, Java UDFs, and Spark SQL, alongside utility classes for SQL parsing and default value handling. This enables the system to define, optimize, and execute feature computation pipelines.

(repo-wide) · high confidence

Added new Pydantic-based data models in the registry/data-models directory to define the abstract backend data structures for the Feathr feature registry. This includes core entity definitions such as Project, FeatureName, Feature, and Anchor, along with supporting types for transformations, sliding window aggregations, and tensor feature formats. These models decouple the backend's internal representation from API-specific data models, enabling consistent serialization and validation for feature metadata and transformations.

registry/data-models · high confidence

Added a new Feathr sandbox environment that provides a complete local setup for development and testing. This includes a Python initialization script that configures a local Spark and Redis environment to run Feathr feature definitions, alongside a shell script that starts the necessary services (Nginx, API, Redis, and Jupyter Notebook) to support the local sandbox experience.

feathr-sandbox · high confidence

The codebase now includes a new \purview-registry\ module that provides an implementation of the \Registry\ interface for interacting with Microsoft Purview. This adds support for managing and querying entities such as projects, sources, anchors, and features within the Purview data catalog, including operations for creation, retrieval, and lineage tracking.

registry/purview-registry/registry · high confidence

The registry/purview-registry directory now contains a complete, reference implementation of the Feathr API spec backed by Microsoft Purview. This includes the FastAPI application (main.py) exposing endpoints for managing projects, data sources, anchors, and features, along with the corresponding API specification (api-spec.md) and supporting configuration files (Dockerfile, .dockerignore, .gitignore).

registry/purview-registry, registry/sql-registry/registry · high confidence

The registry's access control logic is now implemented in the new \registry/access\_control/rbac\ package. This adds a FastAPI-based RBAC system that validates user and application tokens via Azure AD, enforces project-level read/write/manage permissions against a SQL database, and automatically initializes project administrators upon project creation.

_registry/access\control/rbac · high confidence

Adds a new SQL-based registry implementation for Feathr, providing a reference backend for the Feathr API spec. This includes the core application logic (main.py), database schema (schema.sql), and test data (test\_data.sql). The implementation supports creating and managing projects, datasources, anchors, and features, along with deletion APIs and entity lineage tracking.

registry/sql-registry · high confidence

Added a new client entry point, FeathrClient2, which accepts a ComputeGraph and join configuration to join observation data with features, replacing or supplementing the previous client interface. The change also introduces a new DataLocation trait and its implementations (SimplePath, Jdbc, GenericLocation, etc.) to abstract data source loading and writing, along with supporting converters for Pegasus records and UDF plugin context for external UDFs.

feathr-impl/src/main/scala · high confidence

The \feathr/utils\ package now includes several new modules that enhance configuration handling, job execution, and environment detection. The \config\ module provides a \generate\_config\ function that automatically creates Feathr configuration files for local, Databricks, and Azure Synapse environments, supporting environment variable overrides and Azure Key Vault integration. The \job\_utils\ module introduces \get\_result\_df\ and \get\_result\_pandas\_df\ functions to download and load job results as DataFrames, with support for multiple data formats (Parquet, Delta, Avro, CSV) and local caching. Additionally, \platform.py\ adds helper functions (\is\_databricks\, \is\_synapse\, \is\_jupyter\) to detect the current execution environment, while \\_env\_config\_reader.py\ provides a centralized way to read configuration from environment variables, YAML files, or Azure Key Vault. These changes streamline setup, improve environment awareness, and simplify result retrieval for users.

_feathr\project/feathr/utils · high confidence

The Spark provider module has been restructured to include dedicated launchers for Databricks (\_databricks\_submission.py), Azure Synapse (\_synapse\_submission.py), and local Spark (\_localspark\_submission.py). This introduces support for submitting feature jobs to these specific environments, each handling file uploads, job submission, and status tracking according to their respective APIs. The abstract base class (now at spark\_provider/\_abc.py) was updated to include a 'properties' argument for passing system properties to the Spark jobs.

_feathr\_project/feathr/spark\provider · high confidence

The \feathr-config\ module introduces a new set of configuration classes to support a new compute model. This includes \FeatureDefinitionLoader\ and its factory for loading feature definitions, along with a suite of config objects for feature generation (e.g., \FeatureGenConfig\, \OperationalConfig\, \OfflineOperationalConfig\, \NearlineOperationalConfig\) and feature consumption (e.g., \JoinConfig\, \ObservationDataTimeSettingsConfig\, \KeyedFeatures\). These classes define the structure for specifying feature sources, anchors, derivations, and time-based settings for both offline and nearline processing.

feathr-config · high confidence

Added configuration files for Read the Docs, Azure Pipelines component governance, and Git blame ignore rules. Introduced Dockerfiles for the Feathr Registry and Sandbox environments, including a new Gradle wrapper for Java-based builds. Updated the .gitignore to exclude Gradle and Metals build artifacts, and revised the CONTRIBUTING.md to include new pull request and new contributor guidelines.

(repo-wide) · high confidence

The sliding window join implementation and all associated aggregation classes have been removed from the codebase. This includes the core \SlidingWindowJoin\ object, data definition classes like \LabelData\ and \FactData\, and all aggregation implementations such as \AvgAggregate\, \SumAggregate\, \MaxAggregate\, and their corresponding pooling variants. This change eliminates the library's ability to perform sliding window aggregations on Spark DataFrames.

src/main/scala/com/linkedin/feathr/swj · high confidence

The following abstract classes have been removed from the Spark common module: ComplexAggregation, FeatureDerivationFunctionSpark, OutputProcessor, SeqJoinCustomAggregation, SimpleAnchorExtractorSpark, and SourceKeyExtractor. These were the primary extension points for custom feature derivation, aggregation, and anchor extraction logic in Spark. Users relying on these base classes to implement custom transformations or data processing steps will need to migrate to the current supported APIs.

src/main/scala/com/linkedin/feathr/sparkcommon · high confidence

Removed several legacy classes and objects from the offline feature generation and join pipelines, including FeatureGroupsUpdater, FeatureGenerationPathName, SparkIOUUtil, PushToRedisOutputProcessor, DataSourceUtils, FeathrUdfRegistry, FeatureGenConfigOverrider, FeatureGenContext, JoinJobContext, LocalFeatureJoinJob, OutputUtils, DataFrameKeyCombiner, ExecutionContext, and various join algorithm implementations. This cleanup eliminates unused or deprecated code paths for local debugging, output processing, and join execution.

src/main/scala/com/linkedin/feathr/offline · high confidence

A new pre-commit hook has been added to the project, which automatically runs lint-staged to enforce code formatting and linting standards before each commit.

.husky · high confidence

The deployment is now containerized, using a new nginx configuration to serve the UI and proxy API requests. A startup script dynamically generates a JavaScript configuration file from environment variables (REACT\_APP\_AZURE\_CLIENT\_ID, REACT\_APP\_AZURE\_TENANT\_ID, REACT\_APP\_ENABLE\_RBAC). The startup script also conditionally launches the RBAC application alongside the registry if RBAC is enabled, or runs the registry alone otherwise, supporting both SQL and Purview backends.

deploy · high confidence

The \FeatureVariableResolver\ was updated to accept an optional \FeathrExpressionExecutionContext\, allowing MVEL-based derivations to handle non-string default values. This change enables the use of alien values in MVEL-based feature derivations, improving flexibility in how default values are processed during expression evaluation.

feathr-impl/src/main/java · high confidence

The \feathr/definition\ module has been refactored into a new modular structure, introducing dedicated classes for feature types (\dtype.py\), feature anchors (\anchor.py\), and configuration helpers (\config\_helper.py\). This change introduces a new \Aggregation\ enum with support for \FIRST\ and element-wise aggregation functions, and enables \override\_time\_delay\ for simulating time delays in feature queries. Additionally, the system now supports saving feature definitions to HOCON config files and allows for automatic conflict resolution when feature names clash with observation dataset columns.

_feathr\project/feathr/definition · high confidence

The Python client's build configuration has been updated to use the \feathr/version.py\ file for versioning and to read the README from the \docs/\ directory. The \setup.py\ file now pins several dependency versions to resolve conflicts (such as \azure-core\ and \msrest\ incompatibilities) and adds optional dependencies for development (\black\, \pytest\), notebooks (\jupyter\, \scikit-learn\), and all extras. Additionally, the project now includes a \.readthedocs.yml\ configuration for documentation builds, a \MANIFEST.in\ to include specific data files and JARs in the distribution, and generated Protocol Buffer Python code for feature values.

_feathr\project · high confidence

The MANIFEST.MF file in the src/META-INF directory has been removed. This file previously defined the Main-Class as com.linkedin.feathr.cli.FeatureExperimentEntryPoint, which typically specifies the entry point for executable JARs. Its removal suggests a change in how the application is packaged or launched, potentially shifting away from a standalone executable JAR approach or moving the main class definition elsewhere.

src/META-INF · medium confidence

The project's SBT build configuration has been removed. Specifically, the files Dependencies.scala, assembly.sbt, build.properties, and plugins.sbt have all been deleted from the project directory. This eliminates the previous SBT-based build system, including the sbt-assembly plugin and ScalaTest dependency definitions.

project · high confidence

The default Feathr configuration has been updated to support additional storage backends, including Snowflake and Azure Blob Storage (BLOB), and introduces a new 'secrets' section for Azure Key Vault integration. The 'feathr\_runtime\_location' for both Azure Synapse and Databricks has been updated to point to the latest JAR from Maven, and the Databricks config template now supports Maven libraries. Additionally, the 'nyc\_driver\_demo.ipynb' notebook has been removed from the workspace.

_feathr\_project/feathrcli/data/feathr\_user\workspace · medium confidence

Add test coverage for sql-registry and purview-registry; Add unit tests for NYC Taxi dataset utilities; Added comprehensive end-to-end and unit tests for Feathr features; Added notebook execution tests for sample workflows; Added test coverage for Feathr core components; Added test utility modules for data generation, SQL querying, and Spark UDFs; Added test workspace configuration files and sample data for integration tests; Added tests for Purview Registry entity creation and retrieval; Added tests for SQL registry entity and project operations; Added unit tests for TypedKey and Feature type validation; Added unit tests for UDF preprocessing manager; Added unit tests for local Spark job submission; Added unit tests for utils module; Removed obsolete test files and mock data.

The project's build system has been migrated from SBT to Gradle, introducing a new multi-module structure (feathr-impl, feathr-compute, feathr-config, feathr-data-models) with explicit dependency declarations for libraries such as Spark, Jackson, and Pegasus. Additionally, Python dependencies are now explicitly pinned in \requirements.txt\ files across various modules (e.g., \registry/access\_control\, \registry/purview-registry\), and \pyproject.toml\ has been updated with configuration for code formatting and testing.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 45 → 62 (+16.8)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 83 → 90 (+6.7)
• Architecture 89 → 67 (-22.5)
• Maturity 62 → 68 (+5.4)
• Readiness 22 → 58 (+35.7)
• Security 57 → 67 (+9.7)
• Accessibility 60 (new)

• Coverage not measured — test suite did not build
• Critical CVE: [GHSA redacted] (ui/package-lock.json)
• Critical CVE: [GHSA redacted] (ui/package-lock.json)
• Critical CVE: [GHSA redacted] (ui/package-lock.json)
• Critical CVE: [GHSA redacted] (ui/package-lock.json)
• Critical IaC: AZU-0011 (docs/how-to-guides/azure_resource_provision.json)
• Critical IaC: AZU-0012 (docs/how-to-guides/azure_resource_provision.json)
• Critical IaC: AZU-0013 (docs/how-to-guides/azure_resource_provision.json)
• Dimension evaluation failed
• Duplicated block (106 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (106 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (106 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (106 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (106 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (11 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (11 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (11 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (11 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (11 lines × 2) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• Duplicated block (111 lines × 6) (feathr-impl/src/main/java/com/linkedin/feathr/common/types/protobuf/FeatureValueOuterClass.java)
• …and 101 more

• AnchorConfigConverter.buildDataSource (cognitive 26) (feathr-compute/src/main/java/com/linkedin/feathr/compute/converter/AnchorConfigConverter.java)
• AnchorConfigConverter.getOperator (cognitive 19) (feathr-compute/src/main/java/com/linkedin/feathr/compute/converter/AnchorConfigConverter.java)
• AnchorLoader.deserialize (cognitive 39) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/config/FeathrConfigLoader.scala)
• AnchorLoader.deserialize (cyclomatic 34) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/config/FeathrConfigLoader.scala)
• AnchorLoader.loadSourceKeyExtractor (cognitive 32) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/config/FeathrConfigLoader.scala)
• AnchorToDataSourceMapper.getBasicAnchorDFMapForJoin (cognitive 17) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/transformation/AnchorToDataSourceMapper.scala)
• AnchorUDFOperator.computeUDFResult (cognitive 34) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/evaluator/transformation/AnchorUDFOperator.scala)
• AnchorUDFOperator.computeUDFResult (cyclomatic 20) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/evaluator/transformation/AnchorUDFOperator.scala)
• AnchoredFeatureJoinStep.joinFeaturesOnSingleDF (cognitive 28) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/join/workflow/AnchoredFeatureJoinStep.scala)
• AvgAggregate.genRow (cognitive 22) (feathr-impl/src/main/scala/com/linkedin/feathr/swj/aggregate/AvgAggregate.scala)
• BaseDerivedFeatureOperator.applyDerivationFunction (cognitive 17) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/evaluator/transformation/BaseDerivedFeatureOperator.scala)
• BatchDataLoader.loadDataFrameWithRetry (cognitive 16) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/source/dataloader/BatchDataLoader.scala)
• CoercionUtils.coerceToVector (cognitive 27) (feathr-impl/src/main/java/com/linkedin/feathr/common/util/CoercionUtils.java)
• CoercionUtils.coerceToVector (cyclomatic 17) (feathr-impl/src/main/java/com/linkedin/feathr/common/util/CoercionUtils.java)
• CoercionUtils.getCoercedFeatureType (cognitive 20) (feathr-impl/src/main/java/com/linkedin/feathr/common/util/CoercionUtils.java)
• CoercionUtilsScala.coerceFeatureValueToStringKey (cognitive 17) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/util/CoercionUtilsScala.scala)
• CoercionUtilsScala.coerceFieldToFeatureValue (cognitive 25) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/util/CoercionUtilsScala.scala)
• CoercionUtilsScala.coerceFieldToFeatureValue (cyclomatic 22) (feathr-impl/src/main/scala/com/linkedin/feathr/offline/util/CoercionUtilsScala.scala)
• CompatibilityUtils.getCategoricalString (cognitive 23) (feathr-impl/src/main/java/com/linkedin/feathr/common/CompatibilityUtils.java)
• Critical CVE: [GHSA redacted] (ui/package-lock.json)
• …and 448 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

feathr-ai/feathr was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.