PrefectHQ/prefect
53.1
Adequate · 17 November 2026
368.5k
lines of production code
Python
with TypeScript
5
measurements over time
What this system is
Features
Add AWS S3 and GCP GCS bundle upload and execution steps
New \upload\ and \execute\ modules are added to the \prefect\_aws.bundles\ and \prefect\_gcp.bundles\ packages, enabling users to upload flow bundles to S3 and GCS buckets and execute them remotely. The AWS implementation provides \upload\_bundle\_to\_s3\ and \execute\_bundle\_from\_s3\ functions (with CLI wrappers) that handle downloading, extracting, and running bundles using \AwsCredentials\. The GCP implementation provides equivalent \upload\_bundle\_to\_gcs\ and \execute\_bundle\_from\_gcs\ functions using \GcpCredentials\. Both implementations include fallback logic to support older Prefect versions by importing from experimental bundle paths if the GA paths are unavailable.
_src/integrations/prefect-aws/prefect\_aws/bundles, src/integrations/prefect-gcp/prefect\gcp/bundles · high confidence
Add Azure Blob Storage deployment steps with service principal support
The \prefect\_azure.deployments.steps\ module now provides \push\_to\_azure\_blob\_storage\ and \pull\_from\_azure\_blob\_storage\ functions, enabling users to define default or specific push/pull steps in their \prefect.yaml\ files for code storage in Azure Blob Storage. These steps support authentication via connection strings or account URLs, and specifically introduce support for Service Principal Name (SPN) authentication using \ClientSecretCredential\ when tenant, client, and secret IDs are provided, falling back to \DefaultAzureCredential\ otherwise.
_src/integrations/prefect-azure/prefect\azure/deployments · high confidence
Add ECS worker infrastructure CloudFormation templates
Added CloudFormation template files for deploying Prefect ECS worker infrastructure, including an events-only stack for capturing ECS task state changes via SQS and a full service stack for configuring the worker tasks, networking, secrets, and auto-scaling parameters.
_src/integrations/prefect-aws/prefect\aws/templates · high confidence
Added AI agent guidance for writing documentation
Introduced a new \write-docs\ skill and associated templates (\template-concept.mdx\, \template-howto.mdx\) within the \.claude/skills\ directory. These files provide structured guidance, page templates, and Mintlify component examples to help AI agents generate and update Prefect documentation consistently.
.claude/skills/write-docs · high confidence
Added GitHub GraphQL schema definitions for the Prefect integration
The \prefect-github\ integration now includes generated Python types and the raw GraphQL schema file for the GitHub API. This adds the necessary type definitions to support structured queries against GitHub's GraphQL endpoint, enabling the integration to interact with GitHub features using strongly-typed schema objects.
_src/integrations/prefect-github/prefect\github/schemas · high confidence
Added TLA+ model for deployment concurrency leases
Added a formal TLA+ model in \formal/tla/deployment-concurrency/\ to verify the safety of the deployment concurrency lease protocol. The model defines a target protocol where a durable SQL claim acts as the sole decrement authority and includes counterexample configurations to isolate known failure modes, such as stale expiry scans, duplicate reapers, and acquisition aborts. This formal verification ensures that cleanup operations do not incorrectly revoke renewed leases or decrement other runs' capacity, providing a safety guarantee that unit tests alone cannot establish.
formal · high confidence
Added repository-hosted block and worker logos
The assets directory now includes a new \block-logos\ folder containing a \mapping.json\ file that maps external Sanity CDN URLs to local logo files (such as AWS, Docker, Kubernetes, Slack, and others) and a \validate.py\ script to ensure these assets are valid 256x256 PNG images. This change provides the necessary static assets and validation tooling for displaying block and worker logos within the application.
assets · high confidence
Adds GCS bundle upload and download steps for Prefect deployments
Users can now configure deployment build and pull steps to store and retrieve code bundles in Google Cloud Storage. The new \push\_to\_gcs\ step uploads the current working directory to a specified GCS bucket (respecting \.prefectignore\ patterns), and \pull\_from\_gcs\ downloads the bundle back to the local environment. Both steps support authentication via application default credentials, service account info dictionaries, or service account file paths.
_src/integrations/prefect-gcp/prefect\gcp/deployments · high confidence
Automatic symlink creation for AGENTS.md to CLAUDE.md
A new hook script has been added to automatically create symlinks from CLAUDE.md to AGENTS.md files within the project directory. This ensures that Claude Code can access agent configuration files regardless of whether they are named AGENTS.md or CLAUDE.md, improving compatibility with existing project setups.
.claude/hooks · high confidence
Azure Container Instances worker now available
The Azure Container Instances (ACI) worker has been introduced, allowing users to execute Prefect flow runs within Azure ACI containers. This new component includes the worker implementation and configuration, supporting custom ARM templates for flexible infrastructure definition, secure environment variable handling (including \PREFECT\_API\_AUTH\_STRING\), and integration with \prefect-docker\'s \DockerRegistryCredentials\ for image registry authentication.
_src/integrations/prefect-azure/prefect\azure/workers · high confidence
Initial SQLite database schema and migration history
This change introduces the foundational Alembic migration files for the SQLite database backend, establishing the initial schema and subsequent structural evolutions. The initial migration creates core tables for flows, logs, concurrency limits, and saved searches, while later migrations progressively introduce support for agents and work queues, block storage (renamed from block\_data to block, then block\_document), block schemas, and flow run alert policies. The diff also includes data migrations that backfill state names for flow and task runs, and refactors the block model to replace version strings with checksums and introduce block types, ensuring the database structure aligns with the server's object model.
_src/prefect/server/database/\migrations/versions/sqlite · high confidence
Initial UI scaffolding and configuration
The UI application has been scaffolded with a new project structure, including configuration for ESLint, PostCSS, and Tailwind CSS, as well as VS Code workspace settings and Vue component snippets. The entry point is now \index.html\, which loads the Vue app from \src/main.ts\ and includes favicons that adapt to the user's system color scheme preference.
ui · high confidence
Initial packaging and documentation infrastructure for prefect-sqlalchemy
The prefect-sqlalchemy integration package is now properly structured for distribution and documentation. This includes adding the Apache 2.0 LICENSE file, a MANIFEST.in to ensure correct packaging of license and manifest files, a README.md with PyPI badges and documentation links, and a justfile to automate testing and API reference generation using mdxify.
src/integrations/prefect-sqlalchemy · high confidence
Initial packaging and documentation scaffolding for prefect-kubernetes
The prefect-kubernetes integration package is now structured with standard distribution files, including an Apache 2.0 LICENSE, a MANIFEST.in to ensure these files are included in source distributions, and a README.md pointing to the official documentation. A justfile has been added to streamline local development tasks, specifically providing commands to run tests and generate API reference documentation using mdxify.
src/integrations/prefect-kubernetes · high confidence
Initial release of the prefect-redis integration package
This change introduces the \prefect-redis\ integration package into the repository, providing the foundational structure for Redis storage support in Prefect. The addition includes the Apache 2.0 license, package manifest configuration, and a \justfile\ that enables automatic generation of API reference documentation using \mdxify\. This establishes the package as a standalone, documented component for users to integrate Redis with their Prefect workflows.
src/integrations/prefect-redis · high confidence
Initial repository scaffolding and build configuration
This change establishes the foundational structure for the Prefect project, introducing the core build and development tooling. It adds a multi-stage Dockerfile that builds both the legacy V1 and new React V2 UIs, packages them into the Python distribution, and installs dependencies using \uv\ with locked versions for reproducible images. A \justfile\ is introduced as the primary task runner for documentation generation, testing, and local development. The repository also sets up a comprehensive pre-commit configuration using \ruff\, \codespell\, and \mypy\, alongside a \pyright\ configuration for type checking. Additionally, it includes a \uv.lock\ file for dependency management, a \renovate.json\ for automated dependency updates, and standard documentation files like \AGENTS.md\, \CODE\_OF\_CONDUCT.md\, and \SECURITY.md\.
(repo-wide) · high confidence
Initializes telemetry module
The telemetry module is initialized with an empty \_\init\\_.py file, establishing the package structure for future telemetry features.
src/prefect/telemetry · high confidence
Internal SDK generation utilities for naming, schema conversion, and templates
Added internal modules in \src/prefect/\_sdk\ to support auto-generated typed client SDKs for Prefect deployments. This includes \naming.py\ for converting arbitrary names into valid Python identifiers and PascalCase class names, \schema\_converter.py\ for translating JSON Schema definitions into Python type annotations, \unions.py\ for handling union type strings, and Jinja2 templates in \templates/sdk.py.jinja\ that generate the final deployment client code. These components are private implementation details for SDK generation and are not part of the public API.
_src/prefect/\sdk · high confidence
Internal refactoring and new utility modules
This change consolidates various internal utilities into the new \prefect.\_internal\ package, introducing modules for safe logging (\\_logging.py\), AST-based function discovery (\ast\_utils.py\), Docker buildx integration (\buildx.py\), and infrastructure exit code hints (\infrastructure\_exit\_codes.py\). It also adds support for package installation via \uv\ (\installation.py\), lazy-loading dictionaries (\lazy.py\), and OpenTelemetry resource metrics collection (\metrics.py\).
_src/prefect/\internal · high confidence
Introduce Asset model and @materialize decorator
Users can now define data assets using the new \Asset\ class and \AssetProperties\ (which enforces a 2500-character limit on descriptions) and mark tasks that produce them with the \@materialize\ decorator. This allows tasks to explicitly declare which assets they materialize and optionally specify the tool responsible (e.g., dbt, spark), enabling better lineage tracking and dependency management within flows.
src/prefect/assets · high confidence
Introduce Azure Blob Storage bundle upload and execution capabilities
This change adds new modules for managing Prefect bundles with Azure Blob Storage. Users can now upload local bundle files to Azure containers via the new \upload\_bundle\_to\_azure\_blob\_storage\ function and CLI, which supports optional credential blocks and ensures idempotent uploads by overwriting existing blobs (essential for reused sidecar keys). Additionally, a new \execute\_bundle\_from\_azure\_blob\_storage\ function and CLI allow downloading and executing bundles directly from Azure, including support for extracting included files from content-addressed sidecars. The implementation includes fallback imports to support older Prefect versions within the declared dependency range.
_src/integrations/prefect-azure/prefect\azure/bundles · high confidence
Introduce CDK infrastructure for Prefect ECS workers
Adds AWS CDK stacks to define and deploy the infrastructure for Prefect ECS workers, separating concerns into an events stack (EventBridge and SQS) and a service stack (ECS service, IAM, and secrets). The service stack targets existing ECS clusters and VPCs, supports API key or auth-string authentication via AWS Secrets Manager, and includes a deployment circuit breaker with automatic rollback. CDK synthesis is configured to skip bootstrap version checks and disable version reporting to ensure deterministic CloudFormation template generation.
src/integrations/prefect-aws/infra · high confidence
Introduce CLI for managing ECS worker infrastructure
Users can now deploy and manage AWS ECS worker infrastructure directly from the command line using the new \prefect-aws\ CLI. This tool provides commands to deploy ECS service stacks, configure work pools, and manage CloudFormation resources, including validation of AWS credentials, VPC, and subnets, as well as support for both Prefect Cloud and self-hosted servers.
_src/integrations/prefect-aws/prefect\_aws/\cli · high confidence
Introduce Redis integration with Sentinel, Cluster, and concurrency support
The \prefect-redis\ package is now available, providing a \RedisDatabase\ block for use as result storage (with optional per-key TTL), a \RedisLockManager\ for distributed locking, and a \ConcurrencyLeaseStorage\ implementation for managing concurrency limits. It adds Redis-backed worker cleanup queues, causal event ordering, and messaging (publisher/consumer) with configurable settings. Connection handling is enhanced with support for Redis Sentinel (via \redis+sentinel://\ URLs) and Redis Cluster (via \redis+cluster://\ URLs), including cluster-aware key prefixing. The integration also includes prebuilt tasks (\redis\_set\, \redis\_get\, etc.) and ensures compatibility with \redis-py\ 8 defaults.
_src/integrations/prefect-redis/prefect\redis · high confidence
Introduce UI v2 as the default interface for self-hosted deployments
The React-based UI v2 is now the default user interface for self-hosted Prefect servers, replacing the legacy Vue application. This change brings a modernized interface built with React 19, Tailwind CSS, and TanStack Router, featuring improved dark mode support, a new sidebar navigation, and enhanced data visualization components. Users will automatically see the new UI upon upgrading, with the legacy UI remaining available for those who need to opt out.
ui-v2 · high confidence
Introduce client-side events subsystem with automations and interleaved flow run streaming
The \src/prefect/events\ module now provides the complete client-side implementation for Prefect's event system, including Pydantic schemas for events, automations, and deployment triggers, as well as clients for emitting and subscribing to events. Users can now define and manage automations (triggers and actions like \RunDeployment\, \SendNotification\, \CallWebhook\) directly from the client side, with metric triggers restricted to Cloud. The subsystem introduces \FlowRunSubscriber\, which interleaves real-time events and logs for a specific flow run into a single stream, and \EventsWorker\, which manages the background emission of events with robust checkpointing (count-based and time-based) and retry logic. Additionally, the \emit\_event\ utility allows programmatic event emission, automatically attaching related resources (flow, task, deployment, work pool) from the current run context.
src/prefect/events · high confidence
Introduce core execution engine with PrefectDbtRunner and lifecycle hooks
The prefect-dbt integration now includes a new core execution engine (\prefect\_dbt.core\) featuring \PrefectDbtRunner\ and \PrefectDbtOrchestrator\. \PrefectDbtRunner\ wraps the dbt Python API to invoke CLI-style commands, while \PrefectDbtOrchestrator\ provides wave-based execution with per-node or per-wave Prefect task scheduling. Both components support lifecycle hooks (\on\_run\_start\, \post\_model\, \on\_run\_end\) via \DbtHookMixin\, allowing users to register callbacks for monitoring and reacting to dbt runs. The engine also introduces \PrefectDbtSettings\ for configuration, automatic asset creation for dbt nodes, and robust cancellation handling to ensure dbt worker threads are properly shut down during Prefect task timeouts or cancellations.
_src/integrations/prefect-dbt/prefect\dbt/core · high confidence
Introduce database-backed event storage and querying
Added new modules in \src/prefect/server/events/storage\ that provide the infrastructure for persisting and retrieving events from a database. This includes \database.py\, which implements asynchronous functions to query, count, and paginate events using SQLAlchemy, and \\_\init\\_.py\, which defines helper utilities for page token serialization and time-based count processing. This change enables the server to store event data and supports features like event filtering, distinct resource counting, and paginated event lists.
src/prefect/server/events/storage · high confidence
Introduce declarative Service Level Agreement (SLA) types for deployments
Users can now define and apply Service Level Agreements to Prefect deployments using three new declarative SLA types: TimeToCompletion (triggered when a flow run exceeds a specified duration), Frequency (triggered when a flow run is not detected within a stale period), and Lateness (triggered when a flow run does not start within a specified window). These SLAs are managed via the new \SlaClient\ and \SlaAsyncClient\, which expose an \apply\_slas\_for\_deployment\ method that allows users to create, update, or delete SLAs by name for a given deployment, with the backend returning a summary of created, updated, and deleted SLA names.
_src/prefect/\experimental/sla · high confidence
Introduce deployment steps framework for pull, build, and utility actions
Users can now define modular, reusable steps in their prefect.yaml files to manage deployment workflows. The new \prefect.deployments.steps\ module provides core primitives for executing these steps, including \git\_clone\ for pulling code (with support for credentials, sparse checkout, and retries), \set\_working\_directory\ for path management, \run\_shell\_script\ for executing shell commands (with environment variable expansion and shell mode support), and \pip\_install\_requirements\ for installing dependencies. This framework allows users to compose complex build and pull logic using standard YAML configurations, replacing previous ad-hoc or monolithic deployment scripts.
src/prefect/deployments/steps · high confidence
Introduce dynamic runtime context for flows, tasks, and deployments
Users can now access dynamic attributes of the current execution context via the new \prefect.runtime\ module. This includes \prefect.runtime.flow\_run\ (providing flow run ID, name, version, parameters, tags, run count, parent IDs, and API/UI URLs), \prefect.runtime.task\run\ (providing task run ID, name, parameters, tags, run count, task name, and API/UI URLs), and \prefect.runtime.deployment\ (providing deployment ID, name, version, parameters, and flow run ID). These attributes are populated from the current execution context or fetched from the API if not immediately available, and can be mocked for testing via environment variables prefixed with \PREFECT\\RUNTIME\\_\. This allows scripts and flows to dynamically adapt their behavior based on how they are being run.
src/prefect/runtime · high confidence
Introduce experimental Snowpark Container Services (SPCS) worker
Users can now run Prefect flows on Snowflake's Snowpark Container Services using a new experimental worker. This addition allows flows to execute within managed container services, supporting configuration for compute pools, resource requests (CPU, memory, GPU), custom images, and external access integrations.
_src/integrations/prefect-snowflake/prefect\snowflake/experimental · high confidence
Introduce new YAML-driven deployment configuration and lifecycle system
The \src/prefect/deployments\ module has been restructured to support a new YAML-driven configuration model for packaging, publishing, and triggering flow runs. This change introduces \prefect.yaml\ as the central configuration file, managed by the new \initialize\_project\ function in \base.py\ and a new 'Steps' system (\steps/core.py\, \steps/pull.py\, \steps/utility.py\) that executes lifecycle actions like \git\_clone\ and \pip\_install\_requirements\. The module now exposes \deploy\ (via \runner.py\) for creating deployments from \Flow\ objects and \run\_deployment\/\arun\_deployment\ (via \flow\runs.py\) for triggering them, including support for subflows and trace context propagation. Entrypoints now support both file paths and module paths, with the latter handled by temporarily prepending the storage destination to \sys.path\. The public API is exposed through a lazy-loading \\\init\\_.py\ that defers imports for performance.
src/prefect/deployments · high confidence
Introduce new bundle execution infrastructure with sidecar file support
The \src/prefect/bundles\ module has been restructured to support a new bundle execution model. This includes a new \\_\init\\_.py\ that defines serialization logic for bundles (including optional sidecar zip files for included assets), a new \execute.py\ module that handles loading and running these bundles (automatically extracting sidecar files if present), and a new \\_zip\_extractor.py\ module that safely extracts zip archives while validating against path traversal and preserving file permissions.
src/prefect/bundles · high confidence
Introduces WebSocket-based Worker Channel with REST fallback
Adds a new \src/prefect/workers/\_worker\_channel\ module that enables workers to maintain a persistent WebSocket connection to the Prefect API for real-time communication, including a handshake protocol, heartbeat monitoring, and work-pool snapshot handling. The implementation includes a robust fallback mechanism: if the WebSocket connection cannot be established or is lost, the worker automatically reverts to using standard REST API calls for heartbeats and work-pool synchronization, ensuring continuous operation. This change also introduces specific error handling for authentication (401) and authorization (403) failures during the channel upgrade, as well as support for cleanup delivery capabilities.
_src/prefect/workers/\_worker\channel · high confidence
Introduces configurable causal ordering implementations for event processing
The \src/prefect/server/events/ordering\ module now provides the infrastructure for managing partial causal ordering of events, allowing the system to buffer and process events in their correct causal sequence. This change introduces two concrete implementations: an in-memory \CausalOrdering\ class in \memory.py\ that uses \TTLCache\ to prevent memory leaks by automatically expiring seen events and followers, and a database-backed \CausalOrdering\ class in \db.py\ that persists follower relationships to the database. A new \\_\init\\_.py\ file exposes a \get\_triggers\_causal\_ordering()\ function that dynamically loads the active implementation based on the \server.events.causal\_ordering\ setting, enabling users to switch between memory-based and database-based ordering strategies.
src/prefect/server/events/ordering · high confidence
Introduces in-memory cleanup queue storage for worker channel delivery
Adds a new \WorkerCleanupQueue\ interface and its in-memory singleton implementation within the \src/prefect/server/worker\_communication/cleanup\_queue\ module. This component provides the backend storage layer for worker cleanup messages, supporting operations such as enqueueing, reserving, acknowledging, releasing, and renewing leases, along with dead-letter queue handling and idempotency tracking. It also exposes Prometheus metrics for monitoring cleanup queue operations, lease expirations, and dead-letter events.
_src/prefect/server/worker\communication · high confidence
Introduces infrastructure provisioners for push work pools
Users can now automatically provision required cloud resources and credentials when creating push work pools for Cloud Run, Azure Container Instances, ECS, Modal, and Coiled. The new \prefect.infrastructure.provisioners\ module registers these provisioners and handles setup tasks such as creating service accounts, IAM policies, and credential blocks, while deferring heavy SDK imports to avoid slowing down module load times.
src/prefect/infrastructure/provisioners · high confidence
Introduces new abstract block interfaces and dedicated system/notification blocks
This change establishes a new structural foundation for the blocks module by introducing abstract base classes in \abstract.py\—including \CredentialsBlock\, \NotificationBlock\, \JobBlock\, and \SecretBlock\—to standardize capabilities and API shapes. It adds dedicated implementation modules for core functionality: \system.py\ provides the new \Secret\ block for storing and obfuscating sensitive values (supporting both strings and JSON), \notifications.py\ consolidates notification logic using Apprise with SSRF protection and async dispatch, \webhook.py\ introduces a generic \Webhook\ block with configurable HTTP methods and secure transport options, and \redis.py\ provides \RedisStorageContainer\ for file-system-like interactions with Redis. The \\_\init\\_.py\ module is updated to ensure these core blocks are automatically registered upon import.
src/prefect/blocks · high confidence
Introduces server-side automations and deployment trigger schemas
This change adds the core Pydantic schema models for Prefect's new automations system within \src/prefect/events/schemas\. It defines the \AutomationCore\ and \Trigger\ hierarchies (including \EventTrigger\, \MetricTrigger\, \CompoundTrigger\, and \SequenceTrigger\) which allow users to define automated workflows based on resource events or metrics. Additionally, it introduces a parallel set of \Deployment\*Trigger\ schemas (e.g., \DeploymentEventTrigger\) designed for use in deployment YAML files, providing a simplified interface that automatically generates the corresponding automation and \RunDeployment\ action. The update also includes supporting schemas for \Resource\ and \RelatedResource\ labelling, enabling precise filtering of triggers by resource labels and relationships.
src/prefect/events/schemas · high confidence
Introduces server-side automations and event streaming capabilities
This change adds the core server-side infrastructure for Prefect Automations, including the event pipeline, trigger evaluation logic, and action execution framework. Users can now define automations that react to specific events or time intervals, with support for complex filtering, causal event ordering, and various action types such as running deployments, sending notifications, or changing flow run states. Additionally, the system now supports streaming events to clients via websockets through a new Distributor service, enabling real-time event monitoring and integration.
src/prefect/server/events · high confidence
Introduces server-side schema models for Automations and Events
This change adds the foundational Pydantic schema models for the new server-side Automations and Events system. It defines the data structures for Events (including Resource and RelatedResource validation), Automations, and Triggers (such as Compound and Sequence triggers), and introduces lifecycle event builders for domain objects like Variables, Flows, and Blocks. These schemas enable the server to process, validate, and store automation-related events and trigger states.
src/prefect/server/events/schemas · high confidence
Introduction of the \`prefect-client\` package for lightweight environments
A new \prefect-client\ package is introduced, providing a minimal installation of Prefect designed for lightweight or resource-constrained environments like serverless functions. This package is built by stripping server-side components, CLI tools, and heavy dependencies from the main \prefect\ library, resulting in a smaller footprint. The \client/\ directory now contains the build scripts (\build\_client.sh\), configuration (\pyproject.toml\), and a stub CLI (\prefect-cli-stub\) that only supports \flow-run execute\, enabling users to interact with remote Prefect servers without installing the full suite.
client · high confidence
New DockerImage class with opt-in BuildKit/buildx support
The \src/prefect/docker\ module now exposes a \DockerImage\ class that provides a structured interface for building and pushing Docker images. Users can choose between the standard \docker-py\ backend or an opt-in \buildx\ backend (powered by \python-on-whales\) to enable advanced features like build secrets, SSH forwarding, and multi-platform builds. The class also supports interactive progress streaming via a new \stream\_progress\_to\ parameter and automatically generates a default Dockerfile when none is specified. A compatibility shim in \\_buildx.py\ ensures that older integration packages relying on the previous internal paths continue to function.
src/prefect/docker · high confidence
New UI-specific API endpoints for flow, task, and schema operations
This change introduces a new set of API routes under the \/ui\ prefix, specifically designed to support the Prefect UI. The new endpoints include \/ui/flow\_runs/history\ for retrieving flow run history with work pool filtering, \/ui/flow\_runs/count-task-runs\ for bulk task run counts, \/ui/flows/next-runs\ to fetch the next scheduled run for specific flows, and \/ui/flows/count-deployments\ for deployment counts. Additionally, \/ui/task\_runs/dashboard/counts\ provides time-bucketed task run success/failure metrics for the dashboard, \/ui/task\_runs/count\ returns task run counts by state, and \/ui/task\_runs/{id}\ retrieves detailed task run information including the parent flow run name. A new \/ui/schemas/validate\ endpoint allows the UI to validate JSON schemas and their values against the database context. These endpoints consolidate UI-specific data access patterns into a dedicated module.
src/prefect/server/api/ui · high confidence
New \`prefect transfer\` CLI command for migrating resources between profiles
Users can now use the \prefect transfer\ command to move resources (flows, deployments, work pools, automations, variables, and blocks) from one Prefect profile to another. The command automatically resolves dependencies and transfers resources in the correct order using a dependency graph, handles missing or already-existing resources gracefully, and emits events to track the transfer progress and outcome.
src/prefect/cli/transfer · high confidence
New background services for event persistence, trigger evaluation, and debugging
The Prefect server now includes dedicated background services to handle core event automation workflows. The EventPersister service consumes events from the bus and persists them to the database using configurable batching, flushing intervals, and memory safeguards to prevent queue overflow. The ReactiveTriggers service evaluates automation triggers as events arrive, while a new periodic service handles proactive trigger evaluation on a scheduled basis. Additionally, an EventLogger service is available for debugging, which prints incoming events to the console in real-time.
src/prefect/server/events/services · high confidence
New benchmarking suite for CLI, flows, tasks, and performance hotspots
A new \benches/\ directory has been added to provide a structured way to measure and track performance regressions. This includes a runner (\\_\main\\_.py\) and configuration (\cli-bench.toml\) for CLI startup benchmarks, specific benchmarks for flow and task creation/calling, cold-import timing measurements, and targeted performance tests for database operations like \count\_flow\_runs\ and \read\_block\_schemas\. These benchmarks use \pytest-benchmark\ and CodSpeed to ensure that changes to the CLI, engine, or database queries do not introduce unexpected latency.
benches · high confidence
New concurrency lease storage system with filesystem and memory backends
Prefect introduces a new concurrency lease storage system to manage concurrency limits more robustly. This change adds a \ConcurrencyLeaseStorage\ protocol and two concrete implementations: a file-based backend (\filesystem.py\) that persists leases to disk using atomic writes and an expiration index, and an in-memory backend (\memory.py\) for simpler or testing scenarios. The system supports creating, renewing, and revoking leases, tracking lease holders, and listing active/expired leases, providing the foundation for improved concurrency control in task orchestration.
src/prefect/server/concurrency · high confidence
New dbt Cloud integration with per-node orchestration and asset materialization
The \prefect\_dbt.cloud\ package introduces a new integration for dbt Cloud, providing \DbtCloudCredentials\ for API authentication and dedicated clients for the administrative and metadata APIs. A key addition is \DbtCloudExecutor\, which enables per-node orchestration by creating ephemeral dbt Cloud jobs for individual DAG nodes, handling manifest resolution, job triggering, polling, and cleanup. The integration also adds tasks and flows to create and delete dbt Cloud jobs, manage job runs, and automatically materialize Prefect assets from successful dbt run results.
_src/integrations/prefect-dbt/prefect\dbt/cloud · high confidence
New dedicated automation client with configurable filtering
A new \AutomationClient\ and \AutomationAsyncClient\ have been introduced in the orchestration module to manage automations via dedicated API endpoints. This change adds support for configurable filter parameters (such as \automations\, \sort\, \limit\, and \offset\) when reading automations, allowing users to query and filter automation resources more precisely than before.
_src/prefect/client/orchestration/\automations · high confidence
New dedicated log client for creating and reading logs
A new \LogClient\ and \LogAsyncClient\ have been introduced in the orchestration client module, providing dedicated methods to create logs (\create\_logs\) and read logs (\read\_logs\) for flow and task runs. This change separates log-specific API interactions into their own client classes, allowing users to interact with the \/logs/\ and \/logs/filter\ endpoints directly through these specialized clients rather than generic orchestration methods.
_src/prefect/client/orchestration/\logs · high confidence
New deployment steps for building and pushing Docker images
The \prefect\_docker.deployments.steps\ module has been introduced, providing \build\_docker\_image\ and \push\_docker\_image\ functions that can be used within \prefect.yaml\ files to define default or specific build steps for deployments. The build step supports auto-generated Dockerfiles (with an option to persist them), configurable tags, and a choice of build backends (\docker-py\ or \buildx\ for BuildKit features). It also includes input validation for image names and a caching mechanism to avoid redundant builds.
_src/integrations/prefect-docker/prefect\docker/deployments · high confidence
New deployment steps for pushing and pulling code to/from S3
The \prefect\_aws.deployments.steps\ module now provides \push\_to\_s3\ and \pull\_from\_s3\ functions, allowing users to configure S3 as a code storage backend in their deployment flows. The \push\_to\_s3\ step uploads the current working directory to a specified S3 bucket folder, respecting patterns from an ignore file, while \pull\_from\_s3\ downloads files from a specific S3 folder back to the local environment. Both steps utilize the \AwsCredentials.get\_s3\_client()\ method for authentication and support optional client parameters.
_src/integrations/prefect-aws/prefect\aws/deployments · high confidence
New example library for Prefect v3 workflows
The examples directory has been populated with a new set of Python scripts demonstrating core Prefect v3 capabilities. These include a 'Hello, world!' flow for basic orchestration, an API-sourced ETL pipeline using pandas, a simple web scraper with retry logic, and a dbt model orchestration example using the prefect-dbt integration. Advanced patterns are also covered, such as asset-based data pipelines with automatic dependency tracking, per-worker task concurrency using Global Concurrency Limits for GPU resources, and automating flow run resumption via GitHub webhooks and Prefect Cloud automations.
examples · high confidence
New filesystem and in-memory lock managers for transaction safety
Prefect introduces a new locking subsystem in \src/prefect/locking\ to prevent race conditions during concurrent transaction updates. This change adds a \LockManager\ protocol and two concrete implementations: \FileSystemLockManager\, which uses atomic file creation and PID-based stale-lock detection for cross-process safety, and \MemoryLockManager\, which uses in-memory threading locks for single-process scenarios. These managers support both synchronous and asynchronous acquisition with configurable timeouts, ensuring that transaction records are protected from concurrent modification.
src/prefect/locking · high confidence
New flow run input management API
The \prefect.input\ module now exposes a comprehensive API for managing flow run inputs, allowing users to create, read, filter, and delete input values associated with specific flow runs. This includes both synchronous and asynchronous functions (e.g., \create\_flow\_run\_input\, \aread\_flow\_run\_input\) and specialized helpers like \create\_flow\_run\_input\_from\_model\ for storing Pydantic model instances. The module also introduces \receive\_input\ and \send\_input\ for handling typed input from external sources, along with supporting classes like \RunInput\ and \Keyset\ to facilitate structured data handling within flow runs.
src/prefect/input · high confidence
New internal concurrency management module
Prefect introduces a new internal concurrency module at \src/prefect/\_internal/concurrency\ to manage the execution of synchronous and asynchronous calls without blocking threads. This module provides a developer-facing API (\prefect.\_internal.concurrency.api\) with \from\_sync\ and \from\_async\ namespaces for scheduling work in worker threads or the global event loop. It implements core data structures like \Call\ for capturing function context, \Waiter\ for non-blocking result retrieval, and \Future\ with cancellation support. The system also includes thread-safe primitives (\Event\), dedicated thread management (\WorkerThread\, \EventLoopThread\), and queue-based services (\QueueService\) that handle background tasks like logging and events. Additionally, it features robust fork handling to prevent multiprocessing deadlocks on Linux by resetting service and thread states in child processes.
_src/prefect/\internal/concurrency · high confidence
New process utilities module for subprocess management
A new \processutils\ module has been introduced to centralize cross-platform subprocess primitives. It provides \sanitize\_subprocess\env\ to safely handle environment variables (including removing \None\ values and cleaning inherited vars like \PREFECT\\_DEPLOYMENT\_NAME\), \run\_process\ for async subprocess execution with output streaming and signal forwarding, and helpers like \command\_to\_string\/\command\_from\_string\ for platform-neutral command serialization. The module also includes \consume\_process\_output\ and \stream\_text\ for handling process streams, with specific behavior for non-UTF-8 output (silently replacing invalid bytes) and Windows-specific process group handling.
src/prefect/utilities/processutils · high confidence
New scripts for documentation generation, link auditing, and CI tooling
This change introduces a suite of new scripts in the \scripts/\ directory to support documentation, testing, and release workflows. Key additions include \all\_links\_should\_be\_ok.py\ to audit HTTP links in documentation, \generate\_api\_ref.py\ and \generate\_mintlify\_openapi\_docs.py\ for building API reference pages, \generate\_settings\_ref.py\ and \generate\_settings\_schema.py\ for settings documentation, and \generate\_example\_pages.py\ for example documentation. It also adds \backfill\_release\_notes.py\ and \prepare\_integration\_release\_notes.py\ for managing release notes, \collections-manager\ for bulk integration package operations, \check\_pyright\_report.py\ to validate type-checking runs, and \generate-lower-bounds.py\ for dependency testing. The \entrypoint.sh\ script is updated to use \uv\ for installing extra pip packages. Also in this area: 1 commit (0 fixes) modifying scripts.
scripts, scripts/templates · low confidence
New server utilities module for database, encryption, and worker channel management
The \src/prefect/server/utilities\ package has been introduced, providing core infrastructure for the Prefect REST API. This includes \database.py\ for platform-independent ORM utilities (UUID generation, timestamp handling) across SQLite and PostgreSQL, \encryption.py\ for Fernet-based data encryption using configuration or environment keys, and \postgres\_listener.py\ which establishes raw asyncpg connections for LISTEN/NOTIFY, normalizing DSNs for multihost support and preserving TLS/Kerberos settings. The module also adds \worker\_channel.py\ and \worker\_channel\_cleanup.py\ to manage WebSocket-based worker communication, heartbeat tracking, work-pool snapshot delivery, and cleanup queue dispatching with Prometheus metrics. Additional utilities include \subscriptions.py\ for authenticated WebSocket handshakes, \leasing.py\ for resource lease protocols, \text\_search\_parser.py\ for query parsing, \user\_templates.py\ for secure Jinja2 sandboxing, and \http.py\ for sensitive header redaction.
src/prefect/server/utilities · high confidence
New server-side schema utilities and base models
The server-side schema utilities have been reorganized with the introduction of a new \src/prefect/server/utilities/schemas\ package. This change adds a set of base models (\ActionBaseModel\, \IDBaseModel\, \ORMBaseModel\, \PrefectBaseModel\, \PrefectDescriptorBase\, \TimeSeriesBaseModel\) and a helper function (\get\_class\_fields\_only\) to standardize data structures on the server. Additionally, new serialization utilities (\orjson\_dumps\, \orjson\_dumps\_extra\_compatible\) have been added to support efficient JSON encoding using \orjson\, including support for non-string keys and NumPy types.
src/prefect/server/utilities/schemas · high confidence
New utilities package with annotations, collection helpers, and service loops
The \prefect.utilities\ package has been introduced to centralize general-purpose helpers and cross-cutting tools used throughout the SDK. This location now provides \prefect.utilities.annotations\ (including \unmapped\, \allow\_failure\, \quote\, \opaque\, and \freeze\ for flow/task signatures), \prefect.utilities.collections\ (with \visit\_collection\, \flatten\, and \dict\_to\_flatdict\ for data manipulation), and \prefect.utilities.services\ (featuring \critical\_service\_loop\ with backoff and jitter for resilient client operations). It also includes \prefect.utilities.dispatch\ for dynamic type dispatch, \prefect.utilities.hashing\ for stable hashing, and \prefect.utilities.dockerutils\ for image building, alongside documentation in \AGENTS.md\.
src/prefect/utilities · high confidence
Prefect 3.0 SDK core module initialization and plugin system
The \src/prefect\ package is initialized with a new entrypoint structure that enables the plugin system and SDK analytics on import. The \\_\init\\_.py\ module now explicitly defines the public API surface (including \flow\, \task\, \State\, \serve\, \deploy\, and context managers) and runs \\_initialize\_plugins()\ and \\_initialize\_sdk\_analytics()\ during import. A new \\_flow\_run\_suspension.py\ module introduces in-process flow run suspension requests, allowing flows and tasks to coordinate suspension states across contexts. The package also includes a \.prefectignore\ file to exclude common development artifacts and a migration helper in \agent.py\ to guide users upgrading from Prefect 2.x.
src/prefect · high confidence
prefect-aws integration package structure and documentation
The prefect-aws integration package now includes a dedicated Dockerfile for building images with pinned prefect and prefect-aws versions, a MANIFEST.in file to ensure the Apache 2.0 LICENSE is included in distributions, and a justfile to automate API reference documentation generation. Additionally, an AGENTS.md file documents the workflow for managing ECS worker infrastructure via AWS CDK, and a .gitignore file excludes generated CloudFormation templates and CDK build artifacts.
src/integrations/prefect-aws · high confidence
prefect-aws integration promoted to GA with new AWS capabilities
The prefect-aws integration is now generally available, bringing several new capabilities and improvements. Users can now assume IAM roles via the new AwsCredentials.assume\_role\_arn and AssumeRoleParameters fields, enabling temporary credential management. AWS Batch job submission is now supported through the new batch\_submit task. The ECSWorker decorator now supports bundling local files with flows via the include\_files and include\_files\_base\_dir parameters. SSL certificate verification behavior has been refined in AwsClientParameters with a new verify\_cert\_path field and deprecation of passing file paths directly to verify. Additionally, the integration now supports IAM-based authentication for RDS PostgreSQL connections, and async tasks across modules (S3, Lambda, Batch, Secrets Manager, Client Waiter) have been migrated to use the new async\_dispatch system for consistent asynchronous execution.
_src/integrations/prefect-aws/prefect\aws · high confidence
prefect-azure now supports managed identity authentication for Azure Database for PostgreSQL
Users can now connect the Prefect server to Azure Database for PostgreSQL using Microsoft Entra ID (managed identity) instead of a database password. This is enabled by setting the \PREFECT\_INTEGRATIONS\_AZURE\_POSTGRES\_MANAGED\_IDENTITY\_ENABLED\ environment variable, which allows the server to acquire short-lived tokens via \DefaultAzureCredential\ and supply them to \asyncpg\ on every new connection, automatically refreshing tokens without storing credentials. The integration also includes a new Dockerfile for building images with pinned versions of \prefect\ and \prefect-azure\, and adds a \justfile\ to generate API reference documentation using \mdxify\.
_src/integrations/prefect-azure, src/integrations/prefect-azure/prefect\azure · high confidence
prefect-bitbucket package structure and documentation tooling added
The prefect-bitbucket integration package now includes its own LICENSE (Apache 2.0), MANIFEST.in, README, and a justfile. The justfile provides a local command to generate API reference documentation using mdxify, ensuring the package is self-contained for distribution and documentation generation.
src/integrations/prefect-bitbucket · high confidence
prefect-databricks integration package scaffolding and documentation tooling
The prefect-databricks integration package is now structured with standard distribution files, including an Apache 2.0 LICENSE, a MANIFEST.in for packaging, and a README.md pointing to the official documentation. Additionally, a justfile has been added to streamline local development by providing commands for running tests and generating API reference documentation using mdxify.
src/integrations/prefect-databricks · high confidence
prefect-dbt integration package structure and documentation setup
The prefect-dbt integration package now includes its own LICENSE (Apache 2.0), MANIFEST.in, README, and a justfile to streamline local development and documentation generation. The justfile configures mdxify to automatically generate API reference documentation for the prefect\_dbt module, ensuring that users have up-to-date SDK references available in the docs. This change standardizes the package structure and improves the developer experience for maintaining and consuming the prefect-dbt integration.
(repo-wide) · high confidence
prefect-github integration added to repository
The prefect-github integration package has been added to the repository, including its source code structure, Apache 2.0 license, and build configuration files (MANIFEST.in, justfile). This change introduces the integration as a distinct component within the core project, enabling users to install and use GitHub-specific Prefect flows and tasks.
src/integrations/prefect-github · high confidence
prefect-ray integration package structure and documentation tooling added
The prefect-ray integration package now includes its own standalone directory structure with a LICENSE (Apache 2.0), MANIFEST.in, README, and a justfile for generating API reference documentation via mdxify. This establishes the packaging and documentation infrastructure for the prefect-ray integration, making it a first-class citizen in the core repository with proper licensing and automated API ref generation capabilities.
src/integrations/prefect-ray · high confidence
prefect-shell integration package initialization and documentation setup
The prefect-shell integration package has been added to the repository, including its source code structure, Apache 2.0 license, and build manifest (MANIFEST.in). This change also introduces automated API reference documentation generation for the package using mdxify, ensuring that the shell command integration tools are properly documented and distributed.
src/integrations/prefect-shell · high confidence
Security
SSRF protection and hydration validation in schema tools
The schema\_tools module now enforces security and behavioral improvements for deployment parameters and automation payloads. Validation functions (validate, is\_valid\schema) use a non-fetching registry to prevent Server-Side Request Forgery (SSRF) via remote $ref URLs in JSON schemas. Additionally, the hydration process for \\_prefect\_kind structures (jinja, json, workspace\_variable) is documented with specific type-preservation patterns, and validation errors are prioritized to surface placeholder errors (e.g., missing variables) over standard schema violations.
_src/prefect/utilities/schema\tools · high confidence
Security fix for SSRF via remote JSON schema references
The JSON schema validation logic in \src/prefect/\_internal/schemas\ has been hardened to prevent Server-Side Request Forgery (SSRF) attacks. A new \non\_fetching\_registry\ is used to disable the default behavior of the \jsonschema\ library, which previously allowed fetching remote \$ref\ URLs over the network during validation. External references now raise an error without making any network requests, while in-document references continue to resolve normally.
_src/prefect/\internal/schemas · high confidence
Architecture
Prefect REST API restructured into modular route files
The server API has been reorganized from a monolithic structure into a modular package under \src/prefect/server/api\. This change introduces lazy-loading for all route modules (such as \admin\, \artifacts\, \automations\, \block\_documents\, \concurrency\limits\, and \ui\) via a new \\\init\\.py\ and \\\getattr\\_\ mechanism, which improves startup performance and reduces circular dependencies. The diff also includes new internal components for managing UI static file bundles (\\_ui\_static.py\) and a dedicated background worker service (\background\_workers.py\) that registers perpetual services and task functions using the \docket\ library.
src/prefect/server/api · high confidence
Prefect client schemas are restructured into a modular package with lazy loading
The \src/prefect/client/schemas\ directory has been reorganized from a single monolithic module into a structured package containing separate files for actions, objects, filters, responses, schedules, sorting, events, and the worker channel protocol. To maintain backward compatibility and improve import performance, the package now uses a \\_\getattr\\_\ mechanism to lazily load these submodules only when their specific classes (such as \FlowRun\, \StateCreate\, or \DeploymentScheduleCreate\) are accessed. This change also introduces new schema definitions for event pagination, worker channel protocol contracts, and deployment schedule actions, while moving validation logic to internal modules.
src/prefect/client/schemas · high confidence
Prefect server schemas are reorganized into a modular package
The server-side Pydantic schemas have been restructured from a single monolithic module into a dedicated \prefect.server.schemas\ package. This change introduces lazy loading via \\_\getattr\\\ in the package \\\init\\_.py\ to improve import performance, and splits the schema definitions into distinct modules for actions, core models, filters, responses, schedules, sorting, states, statuses, and UI settings. This modularization isolates server-specific schema logic from client-side definitions and supports new capabilities like deployment schedule slugs, UI version switching, and granular filtering.
src/prefect/server/schemas · high confidence
Refactored orchestration client into modular components
The orchestration client has been restructured from a single monolithic file into a modular package with dedicated submodules for each resource type (e.g., blocks, events, deployments, automations). This change improves code organization and maintainability by separating concerns, while preserving the existing public API surface for users.
src/prefect/client/orchestration · high confidence
Runner internals refactored into single-responsibility services
The \Runner\ class has been refactored from a monolithic implementation into a thin facade that delegates to extracted, single-responsibility services. This change introduces new internal modules for managing specific concerns: \FlowRunExecutor\ for per-run lifecycle, \ProcessManager\ for process tracking and termination, \StateProposer\ for API state transitions, \CancellationManager\ for the kill-hooks-state-event sequence, \HookRunner\ for executing cancellation and crash hooks, \EventEmitter\ for event emission with WebSocket degradation, \LimitManager\ for concurrency limiting, \DeploymentRegistry\ for deployment data, and \ScheduledRunPoller\ for scheduling. Legacy methods on \Runner\ are now deprecated and will be removed once internal callers are migrated. This structural change improves maintainability and isolates behavior to specific components.
src/prefect/runner · high confidence
Behavioural changes
4 commits (0 fixes) modifying src/integrations
A change to existing behaviour in src/integrations — 4 commits, 15 files: “Acknowledge \W1030\ validation warnings in prefect-aws ECS worker CDK synthesis (\#23899)”; “docs: note integrations' reliance on private core engine APIs in AGENTS.md (\#23884)”; “Add a Lambda MicroVM base image and entrypoint to prefect-aws (\#23844)”; and 1 more (refs \#23899, \#23884, \#23844, \#23900).
_load\_testing/local-telemetry, src/integrations/prefect-aws/prefect\aws/microvm, src/integrations/prefect-aws/tests/microvm · medium confidence · unverified
4 commits (1 fix) modifying src/prefect/utilities/filesystem
A change to existing behaviour in src/prefect/utilities/filesystem — 4 commits (1 fix), 2 files: “Split utilities AGENTS.md into per-subpackage intent nodes (\#21579)”; “Add opt-in POSIX paths to filter\_files while preserving the native default (\#23292)”; “Fix \filter\_files\ aborting on a directory symlink cycle (\#23307)”; and 1 more (refs \#21579, \#23292, \#23307, \#22558).
_load\testing, src/prefect/utilities/filesystem · medium confidence · unverified
Artifact creation and updates now emit system events
The artifact client now automatically emits \prefect.artifact.created\ and \prefect.artifact.updated\ events whenever an artifact is created or modified. This enables external systems and workflows to react to artifact lifecycle changes via the Prefect events mechanism.
_src/prefect/client/orchestration/\artifacts · high confidence
Background services now run as distributed perpetual functions via Docket
The Prefect server's background services (such as the scheduler, foreman, late runs monitor, and cancellation cleanup) have been refactored to run as distributed perpetual functions using the Docket library. This change replaces the previous loop-based service model with a centralized, HA-aware scheduling system that manages service lifecycle, automatic retries, and error isolation. Operators can now enable or disable these services independently via settings (e.g., \PREFECT\_SERVER\_SERVICES\_SCHEDULER\_ENABLED\), and the system ensures that services run consistently across multi-server deployments without duplicate execution.
src/prefect/server/services · high confidence
Build process now generates analytics configuration
The build tooling has been updated to automatically generate an analytics configuration file containing an Amplitude API key during the build process. This change ensures that analytics settings are embedded into the project at build time, allowing the application to track usage metrics based on the environment-provided API key.
tools · high confidence
Centralized type definitions and validation logic in src/prefect/types
The \src/prefect/types\ package has been introduced to centralize type aliases, validation rules, and helper functions previously scattered across the codebase. This change consolidates datetime handling (including timezone-aware coercion and Python 3.13 compatibility via \whenever\ or \pendulum\), standardizes name validation (enforcing banned characters and length limits for variables, assets, and keys), and introduces self-validating types for concurrency leases and entrypoints. Users benefit from stricter input validation, consistent behavior for time-related fields across different Python versions, and cleaner error messages when invalid names or values are provided.
src/prefect/types · high confidence
Clear error when passing FlowRun or TaskRun objects as task arguments
Prefect now raises a clear error when a \FlowRun\ or \TaskRun\ object is passed as a task argument without the \opaque\ annotation. Previously, these Pydantic models would be recursively inspected by \visit\_collection\, causing the embedded \.state\ attribute to be mistakenly treated as an upstream task dependency. Users must now pass individual fields (e.g., \flow\_run.id\) or wrap the object with \opaque()\ to skip dependency traversal.
src/prefect/utilities/engine · high confidence
Concurrency module refactored to use lease-based slot acquisition with renewal and strict mode
The \prefect.concurrency\ module has been rewritten to use a lease-based model for managing concurrency slots, replacing the legacy slot API. The \concurrency()\ context manager now acquires a lease that is actively renewed by a background daemon thread (renewing at 75% of the lease duration with exponential backoff retries) to prevent slot starvation during long-running tasks. A new \strict\ parameter allows users to raise an error if a named concurrency limit does not exist, instead of the previous default behavior of logging a warning. Additionally, a \raise\_on\_lease\_renewal\_failure\ parameter lets users control whether a renewal failure terminates the task or merely logs a warning. The module now emits structured events for slot acquisition and release, and includes client-side caching to avoid redundant API calls for limits known to not exist.
src/prefect/concurrency · high confidence
Database migration infrastructure reorganization and documentation
The database migration system has been reorganized: the migration module is now located at \prefect.server.database.\_migrations\ (previously \prefect.server.database.migrations\), and a new \MIGRATION-NOTES.md\ file documents the history, purpose, and revision IDs for all schema changes. The Alembic environment (\env.py\) has been updated to handle SQLite-specific constraints (disabling foreign keys during migrations) and to use a dedicated timeout for PostgreSQL migrations to prevent timeouts during long-running index builds.
_src/prefect/server/database/\migrations · high confidence
Databricks integration now supports Service Principal authentication
The Databricks credentials block now supports authenticating via Service Principal (OAuth 2.0) in addition to the existing Personal Access Token method. Users can configure \client\_id\ and \client\_secret\ (with an optional \tenant\_id\ for Azure Databricks) to enable OAuth-based access, with automatic token caching and refresh handled internally. The integration also includes a restored \jobs\_runs\_submit\_and\_wait\_for\_completion\ flow and updated telemetry attribution headers to comply with Databricks partner requirements.
_src/integrations/prefect-databricks/prefect\databricks · high confidence
Deploy command refactored to cyclopts with improved YAML validation and configuration handling
The \prefect deploy\ and \prefect init\ CLI commands have been migrated from Typer to the cyclopts framework, reorganizing the codebase into a modular structure under \src/prefect/cli/deploy/\. This change introduces Pydantic-based models (\PrefectYamlModel\) for validating \prefect.yaml\ files, resulting in more detailed and user-friendly error messages for invalid configurations. The deployment configuration now supports single-step mappings in action sections (build, push, pull) and preserves explicit empty \pull: \[\]\ definitions instead of forcing a default working directory. Additionally, the CLI now supports ISO 8601 duration strings for interval schedules, a \replaces\ field for renaming schedule slugs, and the \day\_or\ parameter for cron schedules.
src/prefect/cli/deploy · high confidence
Deprecate experimental plugin imports in favor of stable API
Imports from the \prefect.\_experimental.plugins\ path now emit a \DeprecationWarning\ and redirect users to the stable \prefect.plugins\ module. This change reflects the graduation of the plugin system from experimental to general availability, ensuring that existing code using the experimental path continues to function while encouraging migration to the public API.
_src/prefect/\experimental/plugins · high confidence
Deprecation of experimental Docker decorator import paths
The \prefect\_docker.experimental\ and \prefect\_docker.experimental.decorators\ modules now emit deprecation warnings when imported, directing users to migrate to \prefect\_docker.decorators\. The experimental location no longer contains implementation code but serves as a shim to guide users to the stable decorator location, with the old import paths scheduled for removal in a future release.
_src/integrations/prefect-docker/prefect\docker/experimental · high confidence
Deprecation of experimental GCP bundle modules
The GCP bundle modules previously located under \prefect\_gcp.experimental.bundles\ (including \upload\, \execute\, and their submodules) have been deprecated and now emit warnings directing users to migrate to the stable \prefect\_gcp.bundles\ paths. The code in this location now acts as a shim, re-exporting the functionality from the new stable location while alerting users that the old import paths will be removed in a future release.
_src/integrations/prefect-gcp/prefect\gcp/experimental/bundles · high confidence
Deprecation of experimental GCP decorators
The \prefect\_gcp.experimental\ module and its \decorators\ submodule have been deprecated and now redirect to \prefect\_gcp.decorators\. Importing from the old experimental paths will trigger a deprecation warning, indicating that these imports will be removed in a future release. Users should update their code to import \cloud\_run\ and \vertex\_ai\ directly from \prefect\_gcp.decorators\.
_src/integrations/prefect-gcp/prefect\gcp/experimental · high confidence
Deprecation of experimental import paths in Prefect Azure integration
The \prefect\_azure.experimental\ module and its submodules (\decorators\, \bundles\, \bundles.execute\, \bundles.upload\) now emit deprecation warnings when imported, indicating that these components have moved to their stable counterparts in \prefect\_azure.decorators\ and \prefect\_azure.bundles\. Users relying on the old experimental import paths will see warnings advising them to update their imports to the new locations before the old paths are removed in a future release.
_src/integrations/prefect-azure/prefect\azure/experimental · high confidence
Deprecation of experimental import paths in prefect-aws
The \prefect\_aws.experimental\ module and its submodules (\decorators\, \bundles.execute\, \bundles.upload\) now emit deprecation warnings when imported, indicating that these components have moved to stable locations (\prefect\_aws.decorators\ and \prefect\_aws.bundles\). Users should update their imports to use the new paths, as the old experimental paths will be removed in a future release.
_src/integrations/prefect-aws/prefect\aws/experimental · high confidence
Deprecation of experimental import paths in prefect-aws
The \prefect\_aws.experimental\ module and its submodules (\decorators\, \bundles.execute\, \bundles.upload\) now emit deprecation warnings when imported, indicating that these components have moved to stable locations (\prefect\_aws.decorators\ and \prefect\_aws.bundles\). Users should update their imports to use the new paths, as the old experimental paths will be removed in a future release.
_src/integrations/prefect-kubernetes/prefect\kubernetes/experimental · high confidence
Deprecation of experimental launcher utilities
The launcher utilities previously available at \prefect.\_experimental.\_launchers\ have been moved to \prefect.\_internal.launchers\ as the infrastructure decorators feature has graduated to general availability. The old import path now re-exports the public symbols but emits a \DeprecationWarning\, indicating that users should migrate their imports to the new location as the old path will be removed in a future release.
_src/prefect/\experimental · high confidence
Deprecation of legacy bundle import paths
The bundle implementation has graduated to general availability at \prefect.bundles\, and the legacy \prefect.\_experimental.bundles\ path is now deprecated. Importing from the old location or running \python -m prefect.\_experimental.bundles.execute\ will now emit a \DeprecationWarning\ and forward to the new GA modules, ensuring backward compatibility for existing work pool storage configurations while encouraging migration to the stable API.
_src/prefect/\experimental/bundles · high confidence
Deprecation of the prefect\_dbt.cli module in favor of prefect\_dbt.core
The \prefect\_dbt.cli\ package is now deprecated and will be removed in a future release; users should migrate to \prefect\_dbt.core\. The \cli\ module now serves only as a compatibility shim that re-exports legacy classes like \DbtCliProfile\ and \DbtCoreOperation\ while emitting a \UserWarning\ to alert users of the change. Existing code using these imports will continue to function temporarily but must be updated to use the new core module to avoid future breakage.
_src/integrations/prefect-dbt/prefect\dbt/cli · high confidence
Introduce FlexibleScheduleList type alias for schedule definitions
A new \FlexibleScheduleList\ type alias has been added to the client types module, allowing schedules to be defined using a sequence of deployment schedule creation objects, dictionaries, or existing schedule types. This change supports the ability to define per-schedule parameters via \.serve\ and \.deploy\ commands by providing a unified type for flexible schedule inputs.
src/prefect/client/types · medium confidence
Introduce automation change notifications and composite trigger race-condition handling
The \src/prefect/server/events/models\ module now includes logic to emit lifecycle events (created, updated, deleted) for automations and manages composite trigger state. For PostgreSQL, automation changes trigger a \NOTIFY\ on a dedicated channel to update in-memory caches, while SQLite updates the cache directly after commit. Additionally, a new \composite\_trigger\_child\_firing\ module serializes concurrent evaluations of compound triggers using PostgreSQL advisory locks and tracks child firing states to prevent race conditions where multiple transactions might otherwise miss firing the parent trigger.
src/prefect/server/events/models · high confidence
Introduce internal Pydantic v2 compatibility layer for function validation
Prefect adds a new internal module at src/prefect/\_internal/pydantic to support Pydantic v2, including a ValidatedFunction class that validates function arguments using Pydantic v2 models, schema generation utilities for both v1 and v2, and helpers to detect v1/v2 types in function signatures. This enables flow parameters and function signatures to work correctly with Pydantic v2 constructs and resolves forward reference and container-type annotation issues that previously caused errors or deprecation warnings.
_src/prefect/\internal/pydantic · high confidence
Introduces structured server settings models
The server configuration is now organized into a hierarchical set of Pydantic models under \src/prefect/settings/models/server\, replacing the previous flat structure. This change introduces dedicated settings groups for the API (including CSRF protection, CORS, and WebSocket ping intervals), database (PostgreSQL connection tuning, mTLS, and search path), concurrency (lease storage and duration), events (retention, caching, and messaging), services (docket, event persister, and database vacuum), tasks, and the UI (V2 default and promotional content). Users can now configure these server behaviors through a more granular and type-safe settings interface.
src/prefect/settings/models/server · high confidence
Migration helpers for Prefect 3.0 infrastructure module changes
The \src/prefect/infrastructure\ module now includes \\_\init\\_.py\ and \base.py\ files that provide actionable error messages for users upgrading to Prefect 3.0. These files utilize a migration helper to intercept imports of moved or removed objects, guiding users through the transition rather than failing with opaque errors.
src/prefect/infrastructure · high confidence
New deployment client module with branching, versioning, and pause/resume capabilities
The deployment client logic has been reorganized into a new module (\src/prefect/client/orchestration/\_deployments\), introducing support for deployment versioning and branching via new \version\_info\, \branch\, \base\, and \root\ parameters in \create\_deployment\. The client now exposes explicit \pause\_deployment\ and \resume\_deployment\ methods (with the previous \\_set\_deployment\_paused\_state\ marked as deprecated) and normalizes empty \parameter\_openapi\_schema\ values to ensure valid OpenAPI format. Additionally, concurrency options are serialized using \exclude\_unset\ to avoid sending unset \grace\_period\_seconds\ fields.
_src/prefect/client/orchestration/\deployments · high confidence
New internal compatibility utilities for Prefect 3.0 migration and async handling
This change introduces a new \prefect.\_internal.compatibility\ package containing utilities to support the Prefect 3.0 upgrade and modernize internal async patterns. It adds \async\_dispatch\, a decorator that allows methods to automatically run synchronously or asynchronously based on the execution context, replacing the older \sync\_compatible\ approach. It also provides \migration.py\ with \MOVED\_IN\_V3\ and \REMOVED\_IN\_V3\ mappings to guide users through import path changes and removed classes (like \Deployment\ and \PrefectAgent\) with helpful error messages. Additional helpers include \deprecated\_paths.py\ for managing deprecated module re-exports, \backports.py\ for Python version-specific features (like TOML parsing), and \starlette.py\ to maintain compatibility with renamed HTTP status codes in newer Starlette versions.
_src/prefect/\internal/compatibility · high confidence
New structured settings system with TOML profiles and pydantic models
Prefect replaces its legacy flat settings with a new hierarchical system built on Pydantic models, allowing settings to be organized into logical groups (such as API, server, and client) and accessed via nested object paths (e.g., \settings.server.api.host\). Users can now configure Prefect using \prefect.toml\ or \pyproject.toml\ files, and manage environment-specific configurations through named profiles stored in \profiles.toml\. The system maintains backward compatibility by supporting legacy environment variable names and providing a \temporary\_settings\ context manager that accepts both old-style setting objects and new dotted-path string keys.
src/prefect/settings · high confidence
PostgreSQL database schema migrations for block and flow run structures
This update applies a series of PostgreSQL Alembic migrations to the Prefect server database. It introduces new tables for block schemas, block documents, block types, and block references to support a more robust block system, while also adding tables for flow run notification policies and queues. Additionally, it adds state\_name columns to flow\_run and task\_run tables, creates various indexes for performance, and renames legacy tables such as block\_data to block\_document.
_src/prefect/server/database/\migrations/versions/postgresql · high confidence
Prefect CLI is now powered by cyclopts
The Prefect command-line interface has migrated from Typer to cyclopts, introducing native lazy loading of command modules to significantly improve CLI startup performance. This change restructures the CLI codebase (e.g., \\_app.py\, \\_utilities.py\) to use cyclopts for command registration and argument parsing while retaining Rich for console output. Users benefit from faster command execution and a more robust internal architecture, with no changes to the external command syntax or behavior.
src/prefect/cli · high confidence
Prefect Cloud CLI rewritten with Cyclopts and new management commands
The \prefect cloud\ command-line interface has been rebuilt using the Cyclopts framework, replacing the previous implementation. This change introduces new subcommands for managing Cloud resources: \prefect cloud asset\ (list and delete assets), \prefect cloud webhook\ (list, get, create, rotate, toggle, and update webhooks), and \prefect cloud ip-allowlist\ (enable, disable, list, add, and remove IP allowlist entries). The core \prefect cloud login\ workflow has also been updated to support interactive workspace selection, profile switching, and environment variable handling, while maintaining JSON output support across the new commands.
src/prefect/cli/cloud · high confidence
Prefect GCP worker integration migrated to core repository with Pydantic 2 and retry improvements
The GCP worker implementations (Cloud Run, Cloud Run v2, and Vertex AI) have been moved from the separate prefect-gcp package into the core Prefect repository, bringing them under the main codebase's maintenance and release cycle. This migration includes an upgrade to Pydantic 2, requiring updates to field definitions and validators across the worker classes. Additionally, the Cloud Run v2 worker now features configurable retry settings for transient HTTP errors during job creation, submission, and readiness polling, improving reliability in unstable network conditions. The workers also support injecting Prefect labels onto GCP resources and include fixes for service account configuration and null scheduling cases.
_src/integrations/prefect-gcp/prefect\gcp/workers · high confidence
Prefect GitHub integration migrated to core repository with Pydantic v2 and security hardening
The \prefect-github\ integration has been moved into the core \prefect\ repository, bringing the \GitHubCredentials\ and \GitHubRepository\ blocks into the main codebase. The credential model has been migrated to Pydantic v2, and the \token\ field is now explicitly typed as \Optional\[SecretStr\]\. To prevent token leakage, git error messages are now sanitized using a shared sanitizer that strips authentication details, and the \GitHubRepository\ block preserves symlinks when cloning directories. Additionally, async tasks now use \@async\_dispatch\ instead of the deprecated \@sync\_compatible\ decorator.
_src/integrations/prefect-github/prefect\github · high confidence
Prefect GitLab integration moved to core with Pydantic v2 and credential formatting fixes
The prefect-gitlab integration has been migrated into the core Prefect codebase, upgrading its data models to Pydantic v2 and replacing the previous async compatibility layer with async\_dispatch. This update introduces a dedicated GitLabCredentials block to manage authentication, which now correctly formats personal access tokens with the required 'oauth2:' prefix while preserving existing deploy token formats to prevent authentication failures. Additionally, the integration now uses a shared output sanitizer to ensure that sensitive credentials are stripped from git command error messages before they are surfaced to the user.
_src/integrations/prefect-gitlab/prefect\gitlab · high confidence
Prefect Kubernetes integration restructured for Prefect 3.x with async support and new worker decorators
The \prefect-kubernetes\ integration has been migrated to the core Prefect repository and rebuilt for Prefect 3.x, introducing several key changes for users. The integration now uses \kubernetes-asyncio\ for all API interactions, requiring async-aware usage patterns. A new \@kubernetes\ decorator is available to bind flows directly to a Kubernetes work pool, simplifying infrastructure configuration. The \KubernetesWorker\ has been refactored to reduce statefulness and now supports ad-hoc job submission via a new \.submit\ method. Credential handling is updated to use Pydantic 2-based blocks (\KubernetesCredentials\, \KubernetesClusterConfig\), and the integration exposes new settings for configuring the Kubernetes observer, worker retry behavior, and API key secret management. Existing code using the old synchronous client or pre-3.x worker patterns will need updates to align with these new async and configuration models.
_src/integrations/prefect-kubernetes/prefect\kubernetes · high confidence
Prefect Slack integration migrated to core with Pydantic 2 and async support
The \prefect-slack\ integration has been moved into the core repository, bringing the \SlackCredentials\ and \SlackWebhook\ blocks up to date with Pydantic 2 and modern async patterns. Users can now use \SlackWebhook.notify()\ with automatic async dispatch, and the \SlackWebhook\ block now supports an optional \subject\ parameter in notifications to include a subject line in the message body. The integration also includes updated logo URLs and documentation links.
_src/integrations/prefect-slack/prefect\slack · high confidence
Prefect Snowflake integration migrated to core repository with Pydantic 2 and WIF support
The \prefect-snowflake\ integration has been moved into the core Prefect repository, bringing the \SnowflakeCredentials\ and \SnowflakeConnector\ blocks up to date with Pydantic 2. This update introduces support for Workload Identity Federation (WIF) via the new \workload\_identity\_provider\ field, improves private key validation with a stricter regex, and fixes several bugs in fetch operations and connection closing. Users should ensure their environments are updated to use the core Prefect package for this integration.
_src/integrations/prefect-snowflake/prefect\snowflake · high confidence
Prefect client SDK restructured with new architecture and attribution headers
The \src/prefect/client\ module has been reorganized into a new structure featuring domain-specific orchestration submodules, a dedicated \CloudClient\, and WebSocket subscription support. A key behavioral addition is the automatic inclusion of attribution headers (e.g., \X-Prefect-Worker-Id\, \X-Prefect-Flow-Id\) in API requests to improve usage tracking and rate limit debugging. The HTTP client now includes robust retry logic with jitter, CSRF support, and informative error wrapping. Additionally, the client automatically starts a local server subprocess if no API URL is specified, and WebSocket subscriptions now provide clearer authentication error messages.
src/prefect/client · high confidence
Prefect server models are rewritten to use SQLAlchemy ORM and emit lifecycle events
The server's data access layer in \src/prefect/server/models\ has been completely rewritten to use SQLAlchemy ORM models (\orm\_models\) instead of the previous interface, and now emits lifecycle events for key domain objects. This change introduces event emission for the creation, update, and deletion of artifacts, block documents, block types, block schemas, concurrency limits (v1 and v2), and deployments, enabling external systems to react to these state changes. The models also now handle specific operational logic such as CSRF token management, configuration caching, and block registration with schema validation.
src/prefect/server/models · high confidence
Prefect settings are reorganized into a structured Pydantic model hierarchy
The configuration system has been refactored from a flat structure into a nested hierarchy of Pydantic models (e.g., \ClientSettings\, \FlowsSettings\, \TasksSettings\, \PluginsSettings\). This change introduces new, structured settings paths such as \PREFECT\_CLIENT\_CUSTOM\_HEADERS\ and \PREFECT\_TASKS\_DEFAULT\_NO\_CACHE\, while maintaining backward compatibility through validation aliases for legacy environment variable names (e.g., \PREFECT\_CLIENT\_ENABLE\_METRICS\). The new structure also standardizes default behaviors, such as setting the default local storage path to \$PREFECT\_HOME/storage\ and enabling OS-level resource metric collection by default.
src/prefect/settings/models · high confidence
Prefect v1 concurrency now supports both synchronous and asynchronous execution
The concurrency v1 module has been refactored to provide explicit synchronous and asynchronous entry points. Users can now import \concurrency\ from \prefect.concurrency.v1.sync\ for use in standard functions or \prefect.concurrency.v1.asyncio\ for async contexts. This change replaces the previous \@sync\_compatible\ decorator pattern with a dedicated \@async\_dispatch\ mechanism, ensuring that slot acquisition and release logic correctly handles both sync and async clients without relying on implicit runtime detection.
src/prefect/concurrency/v1 · high confidence
Prefect-Dask integration migrated to core repository with new client and task runner implementations
The \prefect-dask\ package has been moved into the core Prefect repository, introducing a new \PrefectDaskClient\ that wraps Dask futures to integrate with Prefect's task engine, allowing Prefect \Task\ objects to be submitted directly to Dask. The \DaskTaskRunner\ has been updated to support lazy client loading, custom cluster class instantiation, and performance report generation, while utility functions \get\_dask\_client\ and \get\_async\_dask\_client\ are now available to help users distribute work across Dask workers within tasks.
_src/integrations/prefect-dask/prefect\dask · high confidence
Prefect-Ray integration migrated to core with new context and future handling
The prefect-ray integration has been moved into the core codebase, introducing a new \RayTaskRunner\ and \PrefectRayFuture\ implementation. This update includes a new \remote\_options\ context manager that allows users to dynamically pass keyword arguments (such as CPU/GPU limits) to Ray \@remote\ calls within a flow scope. The task runner now uses lazy imports for the \ray\ library to prevent startup issues and ensures the Ray driver is only shut down in the task runner's exit if it was the one that started it, preventing conflicts when running on existing Ray workers.
_src/integrations/prefect-ray/prefect\ray · high confidence
Prefect-dbt integration restructured with lazy loading and new utility functions
The prefect-dbt package has been reorganized to support lazy loading of optional dependencies, allowing the core module to import without requiring database-specific config packages (like Snowflake or BigQuery) to be installed. The public API now explicitly exposes \PrefectDbtRunner\, \PrefectDbtSettings\, and \DbtCloudJob\ at the top level, while other components like \TargetConfigs\ are loaded on demand. Additionally, new utility functions have been added to handle dbt CLI argument conversion (\kwargs\_to\_args\), format asset keys by stripping restricted characters, and locate the dbt profiles directory.
_src/integrations/prefect-dbt/prefect\dbt · high confidence
Prefect-dbt target configs migrated to Pydantic v2 and core Prefect patterns
The dbt target configuration classes (TargetConfigs, BigQueryTargetConfigs, PostgresTargetConfigs, SnowflakeTargetConfigs) have been rewritten to use Pydantic v2 and inherit from the core Prefect Block class. This migration introduces a new \from\_profiles\_yml\ class method on TargetConfigs to instantiate configs directly from a dbt profiles.yml file, adds an \extras\ field to pass through unsupported dbt keywords, and implements \serialization\alias\ handling to ensure correct key mapping (e.g., \schema\\ to \schema\) when generating the final dbt configuration JSON. The specific adapter configs now also explicitly map Prefect connector fields to the exact keys expected by dbt profiles.
_src/integrations/prefect-dbt/prefect\dbt/cli/configs · high confidence
Promote plugin system from experimental to GA
The internal plugin system located in \src/prefect/\_internal/plugins\ is now promoted to general availability, providing a stable foundation for third-party extensions. This change introduces a robust plugin manager built on \pluggy\ that discovers and loads plugins via entry points, while adding support for both synchronous and asynchronous \setup\_environment\ hooks. Users benefit from improved reliability through configurable allow/deny lists, timeout handling for plugin setup, and strict-mode enforcement that can halt startup if required plugins fail. Additionally, the system now safely applies environment variable changes with automatic redaction of sensitive values (secrets, tokens, passwords) in logs and diagnostics, ensuring that plugin interactions are both powerful and secure.
_src/prefect/\internal/plugins · high confidence
Refactored Flow and FlowRun CRUD methods in client
The flow client implementation has been restructured into a dedicated module (\src/prefect/client/orchestration/\_flows\), introducing explicit \FlowClient\ and \FlowAsyncClient\ classes. This change consolidates flow-related operations such as creating, reading, filtering, and deleting flows, while also documenting the default server limit behavior for client queries to improve clarity for users interacting with the API.
_src/prefect/client/orchestration/\flows · medium confidence
Refactored Variable CRUD operations into dedicated client classes
The variable management logic has been reorganized into new \VariableClient\ and \VariableAsyncClient\ classes within the orchestration client module. This change introduces specific methods for creating, reading, updating, and deleting variables (including reading by name and filtering), replacing the previous implementation structure. Users interacting with variables via the client will now use these dedicated classes, which handle HTTP requests to the \/variables/\ endpoints and map responses to \Variable\ schema objects, while preserving existing error handling for conflicts and not-found scenarios.
_src/prefect/client/orchestration/\variables · high confidence
Refactored block schema and type client methods
The client methods for managing block schemas and block types have been reorganized into dedicated modules (\\_blocks\_schemas\ and \\_blocks\_types\). This change improves code structure and maintainability without altering the external API behavior for users interacting with block definitions.
_src/prefect/client/orchestration/\_blocks\_schemas, src/prefect/client/orchestration/\_blocks\types · high confidence
Refactored concurrency limit client into a dedicated module
The concurrency limit client code has been reorganized into a new \src/prefect/client/orchestration/\_concurrency\limits\ package, splitting the implementation into \\\init\\_.py\ and \client.py\. This change consolidates the CRUD operations for tag-based concurrency limits (create, read, reset, delete) and V1 slot management into a single, structured location, improving code maintainability without altering the external API surface for users.
_src/prefect/client/orchestration/\_concurrency\limits · high confidence
Refactored database layer with unified interface and dependency injection
The database module has been restructured to use a unified \PrefectDBInterface\ that integrates database configuration, query components, and ORM models. This change introduces a new dependency injection system via \db\_injector\ and \inject\_db\ decorators, replacing the previous \OrionDBInterface\ (renamed to \PrefectDBInterface\) to improve type safety and consistency. The refactoring also includes moving migration scripts to a private \\_migrations\ module and adding thread-safe locking for Alembic commands to prevent concurrent execution issues.
src/prefect/server/database · high confidence
Refactored flow run client into a dedicated module
The flow run client logic has been reorganized into a new \src/prefect/client/orchestration/\_flow\runs\ package, splitting the implementation into \\\init\\_.py\ and \client.py\. This change introduces a dedicated \FlowRunClient\ class that handles creating, updating, and deleting flow runs, including support for specifying work pools, work queues, and job variables during creation. This modularization separates flow run orchestration details from the broader client interface.
_src/prefect/client/orchestration/\_flow\runs · high confidence
Refactored orchestration engine into modular policy and rule components
The orchestration logic in \src/prefect/server/orchestration\ has been restructured to improve modularity and observability. The core state-transition rules are now organized into distinct policy classes (\CoreFlowPolicy\, \CoreTaskPolicy\, \GlobalFlowPolicy\, \GlobalTaskPolicy\) that define the priority order of execution. New universal transforms handle bookkeeping tasks such as ensuring unique state timestamps (\EnsureFlowRunStateTimestampIsUnique\) and managing concurrency lease releases (\\_release\_concurrency\_lease\). Additionally, a new instrumentation policy (\InstrumentFlowRunStateTransitions\) automatically emits Prefect Server Events for every flow run state change, enhancing observability without requiring user configuration.
src/prefect/server/orchestration · high confidence
Refactored work pool client into a dedicated module
The work pool client logic has been reorganized into a new \src/prefect/client/orchestration/\_work\pools\ package, splitting the implementation into \\\init\\_.py\ and \client.py\. This change consolidates methods for managing work pools and workers—such as reading work pools, creating pools, filtering workers, and sending heartbeats—into a single, structured client class, improving code maintainability without altering the external API surface.
_src/prefect/client/orchestration/\_work\pools · high confidence
Restructured logging module with new public API and configuration entry points
The logging infrastructure has been reorganized into the \src/prefect/logging\ package, introducing a cleaner public API and more robust configuration management. Users can now access the standard logger via \prefect.logging.get\_logger()\ and the run-context-aware logger via \prefect.logging.get\_run\_logger()\, along with testing utilities like \LogEavesdropper\ and \disable\_run\_logger()\. The module now includes a dedicated \ensure\_logging\_setup()\ function to safely initialize logging in remote execution environments (such as Dask or Ray workers) where the standard import path might not trigger configuration. Additionally, the logging configuration system has been enhanced to support incremental updates, allowing user-defined handlers to be preserved across multiple setup calls, and the \logging.yml\ template now supports environment variable overrides for granular control over log levels, formatters, and handlers.
src/prefect/logging · high confidence
Server package exports are lazily loaded to improve startup performance
The \prefect.server\ package now uses lazy loading for its submodules (\models\, \orchestration\, \schemas\, \services\). This change reduces the initial import overhead and memory footprint when the server package is imported, as submodules are only loaded when they are first accessed. This is a performance optimization for server startup and does not change the public API.
src/prefect/server · high confidence
Shell command execution now kills entire process trees and preserves sync output logs
The \prefect-shell\ integration has been updated to improve reliability and observability when running shell commands. On POSIX systems, spawned shell processes are now isolated in their own process groups, ensuring that the entire process tree (including detached child processes) is terminated during cleanup, which prevents orphaned processes from outliving cancelled flow runs. Additionally, the \shell\_run\_command\ task now correctly preserves and logs stream output in synchronous execution contexts, and incremental streaming is handled properly in \wait\_for\_completion\ scenarios.
_src/integrations/prefect-shell/prefect\shell · high confidence
Split SQLAlchemy integration into synchronous and asynchronous connectors
The prefect-sqlalchemy integration has been restructured to separate database connectivity into two distinct blocks: SqlAlchemyConnector for synchronous operations and AsyncSqlAlchemyConnector for asynchronous operations. This change ensures that users must explicitly choose the correct connector type for their driver, preventing errors when attempting to use async drivers with the synchronous connector. The new structure also includes updated Pydantic v2 typing and improved URL validation for connection strings.
_src/integrations/prefect-sqlalchemy/prefect\sqlalchemy · high confidence
Support for AnyIO 4.14 task group interface
The async utilities in \src/prefect/utilities/asyncutils\ have been updated to support the AnyIO 4.14 task group interface. This change ensures compatibility with the latest AnyIO version, allowing users to leverage new concurrency primitives and interface updates without breaking existing async workflows.
src/prefect/utilities/asyncutils · medium confidence
Support for self-hosted BitBucket instances and improved credential handling
The BitBucket integration now supports authentication for self-hosted BitBucket Server instances that do not include 'bitbucketserver' in their hostname, allowing users to provide a username and password/token for these environments. Additionally, credentials (usernames and tokens) are now properly URL-encoded when embedded in git clone URLs, preventing failures when special characters are present in tokens or usernames. The \BitBucketCredentials\ block also validates usernames to allow common special characters like '@' and '.', and enforces that credentials are only used with HTTPS repository URLs.
_src/integrations/prefect-bitbucket/prefect\bitbucket · high confidence
Worker module restructured with new healthcheck, cleanup, and configuration components
The \src/prefect/workers\ package has been reorganized to support a more robust worker lifecycle. A new slim healthcheck webserver (built on FastAPI and uvicorn) is now available to monitor worker status, including a startup phase before the worker is fully initialized. Cleanup logic for cancelling flow runs has been extracted into a dedicated \CancellingTimeoutTeardownHandler\ that safely tears down infrastructure. The \ProcessWorker\ now delegates execution to a \FlowRunExecutor\ with configurable working directories and command starters, and its configuration model explicitly supports \working\_dir\ and \stream\_output\ settings. Additionally, utility functions have been added to fetch available work pool types and default job templates from the collections registry, and attribution environment variables are now managed centrally in \BaseWorker\ to ensure consistent API request headers.
src/prefect/workers · high confidence
Workers now skip in-process retries when fetching scheduled flow runs
The SQL queries used to retrieve scheduled flow runs for workers have been updated to explicitly exclude runs with an in-process retry type. By filtering out these runs, workers will no longer attempt to execute flow runs that are already being retried within the same process, preventing duplicate execution attempts.
src/prefect/server/database/sql · high confidence
prefect-docker integration migrated to core repository with Pydantic 2 and new worker features
The prefect-docker integration has been moved into the core prefect repository, bringing the DockerWorker, DockerHost, and DockerRegistryCredentials blocks up to date with Pydantic 2. This update includes a new @docker decorator that allows flows to be bound to a Docker work pool, supporting configurable file inclusion and base directories. The DockerWorker now supports anonymous volumes and resolves relative bind-volume sources against the worker's current working directory. Additionally, Python 3.10 support has been dropped.
_src/integrations/prefect-docker/prefect\docker · high confidence
prefect-email integration migrated to Pydantic v2 with inline image support
The prefect-email integration has been updated to use Pydantic v2 for its data models, including the EmailServerCredentials block, which now utilizes Pydantic v2 field validators. Additionally, the email\_send\_message task now supports embedding images directly into the email body via the new inline\_images parameter, allowing users to reference local image files by content ID within HTML messages.
_src/integrations/prefect-email/prefect\email · high confidence
prefect-gcp integration migrated to core Prefect with Pydantic 2 and async improvements
The \prefect-gcp\ integration has been moved into the core Prefect repository, bringing GCP workers (Cloud Run V2, Vertex AI), storage blocks (GcsBucket), and data blocks (BigQuery, Secret Manager) under the \prefect\_gcp\ namespace. The codebase has been upgraded to Pydantic 2, and async support has been standardized using \async\_dispatch\ for key operations. Users also benefit from configurable retry settings for Cloud Run V2 workers and improved handling of BigQuery job configurations.
_src/integrations/prefect-gcp/prefect\gcp · high confidence
prefect-gcp integration moved to core repository with build and documentation tooling
The \prefect-gcp\ integration has been migrated into the core \prefect\ repository. This change introduces a Dockerfile for building container images with pinned versions of \prefect\ and \prefect-gcp\, along with a smoke test to verify installation. Additionally, standard package metadata files (LICENSE, MANIFEST.in) and a \justfile\ for running tests and generating API reference documentation via \mdxify\ have been added to support the integration's lifecycle within the main codebase.
src/integrations/prefect-gcp · high confidence
prefect-slack integration added to core with Python 3.10+ requirement
The prefect-slack integration has been migrated into the core repository, bringing with it official support for Prefect 3.x and a minimum Python version of 3.10. This change includes the addition of standard package metadata files (LICENSE, MANIFEST.in, README.md) and an automated API reference documentation generation setup via mdxify, ensuring the integration is now maintained as a first-party component with updated licensing and documentation infrastructure.
src/integrations/prefect-slack · high confidence
Fixes
Add unique consumer name generation for multi-server setups
A new utility function \generate\_unique\_consumer\_name\ has been added to the messaging utilities to support multi-server deployments. This function creates distinct consumer identifiers by combining the logical group name, the machine hostname, the process ID, and a short UUID, ensuring that consumers remain unique across different machines and processes.
src/prefect/server/utilities/messaging · high confidence
Test coverage
3 commits adding/updating tests in scripts/tests; 3 commits adding/updating tests in src/integrations/prefect-aws/tests/cli; Added comprehensive test suite for ECS Worker; Added comprehensive test suite for prefect-dbt core orchestration; Added comprehensive test suite for prefect-kubernetes integration; Added integration test harness for the Kubernetes worker; Added proxy-test script to validate WebSocket connections through HTTP proxies; Added standard test suites for Blocks and Worker Cleanup Queues; Added test project flow for prefect-docker integration; Added test suite for prefect-azure integration; Added test suite for prefect-dbt integration; Added tests for AWS ECS decorator and bundle execution; Added tests for Azure Blob Storage bundle execution and upload; Added tests for Azure Blob Storage deployment steps; Added tests for Azure Container and GCP Cloud Run/Vertex AI flow decorators; Added tests for Docker deployment steps; Added tests for ECS worker CLI, observer diagnostics, and crash detection; Added tests for GCS bundle upload and execution; Added tests for S3 deployment steps; Added tests for dbt CLI configuration classes; Added tests for dbt Cloud integration components; Added tests for prefect-dbt CLI commands and credentials; Added tests for the @docker decorator and its configuration options; Added tests for the Kubernetes flow decorator; Added unit tests for the experimental SPCS worker; Comprehensive test suite for prefect-redis integration; Expanded test coverage for Prefect AWS integration; Extensive test suite updates and fixes; Initial test suite for prefect-snowflake integration; Initial test suite for the prefect-github integration; New integration test suite for Prefect orchestration behaviors; New testing utilities and fixtures for local server testing.
Dependencies
Update Node.js to v24
The project's Node.js runtime requirement has been upgraded to version 24.
(dependencies) · high confidence
Vendor croniter library for cron expression handling
The croniter library has been vendored into the Prefect codebase at src/prefect/\_vendor/croniter. This adds the croniter module and its associated classes, exceptions, and utilities (such as croniter\_range and datetime\_to\_timestamp) directly to the project, enabling internal scheduling logic to parse and evaluate cron expressions without relying on an external package dependency.
_src/prefect/\vendor · high confidence
Housekeeping
Add link auditing and pyright diff utility scripts
Added \scripts/all\_links\_should\_be\_ok.py\ to scan project files for broken HTTP links and \scripts/pyright\_diff.py\ to compare pyright type-checking outputs and display new errors in a markdown table.
prefect · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 56.7 → 53.1 (-3.6)
- Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 57.7 → 44.9 (-12.8)
- Architecture 66.8 → 66.8 (0.0)
- Maturity 75.8 → 75.9 (+0.1)
- Readiness 56.6 → 61.4 (+4.8)
- Security 72.3 → 71.9 (-0.4)
- Domain Modelling 100.0 → 100.0 (0.0)
- Accessibility 50.7 → 50.7 (0.0)
- Performance 91.4 → 91.4 (0.0)
Resolved (75)
- BaseWorker._submit_run (cognitive 18) (src/prefect/workers/base.py)
- Duplicated block (106–109 lines × 2) (src/prefect/cli/work_pool.py)
- Duplicated block (10–20 lines × 2) (src/integrations/prefect-aws/prefect_aws/_cli/ecs_worker.py)
- Duplicated block (11 lines × 2) (src/prefect/cli/automation.py)
- Duplicated block (11 lines × 4) (src/prefect/server/database/_migrations/versions/postgresql/2024_04_03_112409_aeea5ee6f070_automations_models.py)
- Duplicated block (11–12 lines × 3) (src/prefect/server/database/_migrations/versions/sqlite/2023_01_12_000042_f92143d30c24_implement_artifact_table.py)
- Duplicated block (11–13 lines × 2) (src/prefect/server/database/_migrations/versions/sqlite/2023_01_12_000042_f92143d30c24_implement_artifact_table.py)
- Duplicated block (12 lines × 2) (src/integrations/prefect-gcp/prefect_gcp/workers/cloud_run.py)
- Duplicated block (12 lines × 2) (src/prefect/server/database/_migrations/versions/sqlite/2024_04_03_111618_07ed05dfd4ec_automations_models.py)
- Duplicated block (12–14 lines × 3) (src/prefect/server/database/_migrations/versions/sqlite/2023_01_12_000042_f92143d30c24_implement_artifact_table.py)
- Duplicated block (12–15 lines × 2) (src/prefect/server/database/_migrations/versions/sqlite/2023_01_12_000042_f92143d30c24_implement_artifact_table.py)
- Duplicated block (13 lines × 3) (src/integrations/prefect-dbt/prefect_dbt/cli/commands.py)
- Duplicated block (14 lines × 2) (src/integrations/prefect-aws/prefect_aws/_cli/ecs_worker.py)
- Duplicated block (14 lines × 2) (src/prefect/cli/server.py)
- Duplicated block (14–16 lines × 3) (src/prefect/server/database/_migrations/versions/sqlite/2024_01_22_120214_265eb1a2da4c_create_deployment_schedule_and_add_.py)
- Duplicated block (15 lines × 2) (src/prefect/infrastructure/provisioners/coiled.py)
- Duplicated block (15 lines × 2) (src/prefect/server/database/_migrations/versions/postgresql/2024_04_03_112409_aeea5ee6f070_automations_models.py)
- Duplicated block (16 lines × 2) (src/prefect/task_engine.py)
- Duplicated block (16 lines × 2) (src/prefect/task_engine.py)
- Duplicated block (16–18 lines × 2) (src/prefect/server/database/_migrations/versions/postgresql/2022_07_21_133134_e085c9cbf8ce_remove_flow_runners.py)
- …and 55 more
New (527)
- Documentation: no build, run or usage example (src/integrations/prefect-bitbucket/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-dask/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-databricks/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-dbt/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-docker/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-email/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-github/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-gitlab/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-kubernetes/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-kubernetes/integration_tests/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-redis/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-shell/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-snowflake/README.md)
- Documentation: no build, run or usage example (src/integrations/prefect-sqlalchemy/README.md)
- Duplicated block (10 lines × 3) (src/integrations/prefect-bitbucket/prefect_bitbucket/repository.py)
- Duplicated block (11 lines × 4) (src/prefect/server/database/_migrations/versions/postgresql/2024_04_03_112409_aeea5ee6f070_automations_models.py)
- Duplicated block (13 lines × 2) (src/integrations/prefect-github/prefect_github/repository.py)
- Duplicated block (17 lines × 2) (src/integrations/prefect-github/prefect_github/mutations.py)
- Duplicated block (18 lines × 2) (src/integrations/prefect-github/prefect_github/mutations.py)
- Duplicated block (18 lines × 2) (src/integrations/prefect-github/prefect_github/mutations.py)
- …and 507 more
Changes since last survey
- 114 commits — 91 feature/other, 23 fixes
By area
- src/prefect — 38 commits
- (root) — 16 commits
- docs/v3 — 15 commits
- src/integrations — 13 commits
- ui-v2/src — 8 commits
- .github/workflows — 6 commits
- docs/integrations — 4 commits
- tests/server — 4 commits
- ui-v2/package-lock.json — 3 commits
- tests/events — 2 commits
- tests/_internal — 1 commit
- tests/blocks — 1 commit
- tests/cli — 1 commit
- tests/concurrency — 1 commit
- ui-v2/e2e — 1 commit
Notable commits
- fix: Fix 500 from flow run resume when run input has an integer beyond 64 bits (#23617)
- fix: Fix CodSpeed import benchmark timeout and fail the job on benchmark failures (#23897)
- fix: Fix IndexError in prefect block type inspect/ls on empty description (#23762)
- fix: Fix SQLAlchemy cursor reuse for parameterized fetches (#23847)
- fix: Fix RemoteFileSystem uploading all files when ignore file excludes everything (#23786)
- fix: Fix visit_collection for defaultdict, Counter, deque, and frozenset (#23794)
- fix: Fix automation trigger "Format" button submitting the wizard form (#23776)
- fix: Fix broken examples in the store-flow-code guide (#23801)
- fix: Fix early closure of async generator flows (#23886)
- fix: Fix flaky UI E2E tests reading stale default task run names (#23829)
- fix: Fix flaky artifact ls tests that assert a sub-second "updated" time (#23866)
- fix: Fix flaky test_read_task_workers by resetting the in-memory task worker tracker (#23903)
- fix: Fix flaky test_concurrency_context_releases_slots_async (#23891)
- fix: Fix prefect-aws's ECS worker CLI on typer 0.29 (#23846)
- fix: Fix retry restart of async generator tasks closed during finalizer failure (#23739)
- fix: fix parallel tasks failing when using kwargs (#23820)
- fix: fix parent transactions committing after child commit failure (#23790)
- fix: fix(prefect-redis): keep messages published while a publisher flush is in flight (#23792)
- fix: fix(schedules): keep RDATE/EXDATE instants in RRuleSchedule.from_rrule() (#23436)
- fix: fix(ui-v2): make the subflow run count respect the state filter (#23912)
- …and 94 more
Architecture
- Unchanged — 0 containers · 1 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
PrefectHQ/prefect was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 17 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 2b95707e76d52051e8ad8bd280ad6c8f50e36b5a — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.85+preprod-eba6548ac50f.