PrefectHQ/prefect
53.6
Weak · 19 September 2026
381k
lines of production code
Python
with TypeScript
1
measurement over time
What this system is
This system is a workflow orchestration platform that manages the execution, scheduling, and monitoring of data pipelines and automated tasks. It provides a client-server architecture with a lightweight SDK for defining flows, a robust CLI for deployment management, and a modular integration ecosystem supporting major cloud providers, databases, and container runtimes. The platform handles infrastructure provisioning, concurrency control, and event-driven automations to ensure reliable, observable, and scalable task execution across diverse environments.
How it got here
2021–2024 — Prefect 3.0 architecture and integrations
123 changes.
This period centered on the foundational development of Prefect 3.0, featuring a comprehensive architectural rewrite of the core SDK, server, and CLI to support modern Python and async patterns. It involved migrating numerous third-party integrations into the core repository, standardizing them on Pydantic v2, and introducing new capabilities such as server-side automations, event streaming, and a modular infrastructure provisioning system.
2025–2026 — Client modularization and integration expansion
57 changes.
The project refactored the orchestration client into dedicated modules for variables, logs, deployments, and other resources to improve code organization. Simultaneously, it expanded integration capabilities with new dbt orchestration, cloud-specific workers, and file bundling infrastructure, while stabilizing experimental APIs and adding comprehensive test coverage.
Features
Add ECS worker infrastructure CloudFormation templates
Added CloudFormation templates for deploying Prefect ECS workers, including an events-only stack for capturing task state changes via SQS and a full service stack for configuring worker tasks, networking, secrets, and auto-scaling.
_src/integrations/prefect-aws/prefect\aws/templates · high confidence
Add S3 deployment steps for pushing and pulling code
The \prefect\_aws.deployments.steps\ module now provides \push\_to\_s3\ and \pull\_from\_s3\ functions, allowing users to configure S3 as a code storage backend in their deployment flows. \push\_to\_s3\ uploads the current working directory to a specified S3 bucket and folder, respecting \.prefectignore\ patterns, while \pull\_from\_s3\ downloads files from a specific S3 folder back to the local environment. Both steps utilize the \AwsCredentials\ block for authentication and support custom client parameters.
_src/integrations/prefect-aws/prefect\aws/deployments · high confidence
Add benchmarking infrastructure and performance tests
The \benches\ directory now provides a structured benchmarking suite using \pytest-benchmark\ and \codspeed\ to track performance regressions. This includes a runner script (\\_\main\\_.py\) and specific benchmarks for CLI startup times (e.g., \prefect --help\), Python import speeds (e.g., \prefect\, \prefect.flows\), flow and task execution overhead, and database query performance for \count\_flow\_runs\ and \read\_block\_schemas\. A configuration file (\cli-bench.toml\) is also added to support the \python-cli-bench\ tool for measuring CLI command startup latency.
benches · high confidence
Added GitHub GraphQL schema definitions for the Prefect GitHub integration
The Prefect GitHub integration now includes generated schema files (\graphql\_schema.json\ and \graphql\_schema.py\) that define the types, enums, and mutations for the GitHub GraphQL API. This addition provides the necessary type definitions to support API interactions within the integration.
_src/integrations/prefect-github/prefect\github/schemas · high confidence
Added TLA+ model for deployment concurrency leases
Added a formal TLA+ model in the \formal/tla/deployment-concurrency\ directory to verify the safety of the deployment concurrency lease protocol. The model and its associated configuration files define invariants such as \NoForeignRelease\ and \RenewWinsAgainstStaleScan\, and include counterexample configurations that demonstrate specific failure modes (e.g., duplicate reapers, acquisition aborts) to ensure the protocol correctly handles concurrency and lease state transitions.
formal · high confidence
Added local telemetry stack and database monitoring for load testing
The load\_testing area now includes a local OpenTelemetry stack (Prometheus, Jaeger, and OpenTelemetry Collector) to enable performance investigation and debugging. Users can start this stack via the new \local-telemetry/start\ script and run the server with tracing enabled using \run-server.sh\, which supports both SQLite and PostgreSQL backends. Additionally, a new \track\_cnx.py\ script has been added to monitor PostgreSQL database connections in real-time, highlighting stuck reads, long transactions, and blocked queries to assist in diagnosing performance issues during load tests.
_load\testing · high confidence
Adds GCS bundle upload and download steps for Prefect deployments
Users can now configure deployment build and pull steps to store and retrieve project code in Google Cloud Storage. The new \push\_to\_gcs\ step uploads the current working directory to a specified GCS bucket (respecting \.prefectignore\ patterns), and \pull\_from\_gcs\ downloads code from a GCS bucket to the local environment. Both steps support authentication via application default credentials, service account info dictionaries, or service account file paths.
_src/integrations/prefect-gcp/prefect\gcp/deployments · high confidence
Automated symlink creation for AGENTS.md to CLAUDE.md
A new hook script has been added to automatically create symlinks from CLAUDE.md to AGENTS.md files within the project directory. This ensures that Claude Code can access AGENTS.md files through the standard CLAUDE.md entry point, with logic to handle existing symlinks and skip directories containing regular files named CLAUDE.md.
.claude/hooks · high confidence
Azure Blob Storage deployment steps now support Service Principal authentication
The \prefect\_azure.deployments.steps\ module for pushing and pulling code to Azure Blob Storage has been updated to support Service Principal authentication. Users can now provide \tenant\_id\, \client\_id\, and \client\_secret\ in their credentials configuration to use \ClientSecretCredential\, in addition to the existing support for connection strings and default Azure credentials. This allows for more flexible and secure authentication options when configuring deployment steps in \prefect.yaml\.
_src/integrations/prefect-azure/prefect\azure/deployments · high confidence
Azure Container Instances worker now available
The Azure Container Instances (ACI) worker has been added to the prefect-azure integration, allowing users to execute flow runs in Azure ACI containers. This new worker class supports custom ARM templates for flexible infrastructure configuration, accepts Docker registry credentials for image authentication, and includes retry logic for transient Azure API errors to improve reliability.
_src/integrations/prefect-azure/prefect\azure/workers · high confidence
ECS observer provides structured failure diagnoses for stopped tasks
The ECS observer now automatically diagnoses infrastructure failures for stopped tasks by analyzing EventBridge event details. A new diagnostics module parses stop codes and container exit statuses to provide actionable resolution hints for common issues such as container image pull failures, tasks that failed to start, essential containers exiting, or tasks terminated by the underlying infrastructure (e.g., spot interruptions). This allows users to quickly identify the root cause of ECS task failures without needing to manually inspect logs or make additional AWS API calls.
_src/integrations/prefect-aws/prefect\aws/observers · high confidence
Initial packaging and documentation infrastructure for prefect-kubernetes
The prefect-kubernetes integration package is now properly structured for distribution and documentation. A LICENSE file (Apache 2.0) and MANIFEST.in have been added to ensure correct packaging, while a README provides basic project information and links to the documentation. Additionally, a justfile has been introduced to streamline local development tasks, including running tests and generating API reference documentation using mdxify.
src/integrations/prefect-kubernetes · high confidence
Initial release of prefect-redis integration
This change introduces the \prefect-redis\ package to the repository, providing the foundational structure for a Redis storage integration. The addition includes the Apache 2.0 license, package manifest configuration, and a README, establishing the legal and distribution framework for the new component. It also adds a \justfile\ that configures the \mdxify\ tool (version 0.2.45 or higher) to automatically generate API reference documentation for the \prefect\_redis\ module, ensuring that technical documentation is kept in sync with the codebase.
src/integrations/prefect-redis · high confidence
Initial repository scaffolding and build configuration
This change introduces the foundational structure for the Prefect project, including the root Dockerfile (with multi-stage builds for Python, V1/V2 UIs, and SQLite), a new .dockerignore, and a .prefectignore. It establishes the development workflow via a justfile task runner and a .pre-commit-config.yaml (using ruff, codespell, mypy, and uv-lock). The repository now includes AGENTS.md and REVIEW.md for contributor and AI guidance, a LICENSE (Apache 2.0), SECURITY.md, CODE\_OF\_CONDUCT.md, and a compat-tests submodule. Node versioning is standardized via .nvmrc (24.20.0), and a hatch\_build.py hook enforces packaged UI bundles during sdist/wheel builds.
(repo-wide) · high confidence
Initial scaffolding of the UI v2 development environment
This change introduces the foundational configuration files for the new React-based UI v2, including a \.nvmrc\ pinning the Node.js version to 24.20.0, an \.npmrc\ to handle peer dependency conflicts, and a \.env\ file for local API URL overrides. It also adds Storybook branding assets (dark and light mode logos) and a \.claude/settings.json\ file to automatically run linting checks after code edits.
ui-v2 · high confidence
Introduce Asset model and materialization decorator
Users can now define and track materialized data using the new Asset class, which supports metadata properties like name, URL, description (capped at 2500 characters), and owners, along with a key validated for allowed characters and length. Additionally, the @materialize decorator allows tasks to explicitly declare which assets they produce, enabling better lineage tracking and dependency management within flows.
src/prefect/assets · high confidence
Introduce CDK infrastructure for ECS Prefect workers
Adds AWS CDK stacks to define and deploy the infrastructure for Prefect ECS workers, separating concerns into an \EcsServiceStack\ (managing the ECS service, IAM roles, and secrets) and an \EcsEventsStack\ (handling EventBridge rules and SQS queues for task state monitoring). The infrastructure is configured to deploy to existing ECS clusters and VPCs, supports conditional secret creation for API keys and auth strings, and includes a deployment circuit breaker with automatic rollback for ECS service updates. A \justfile\ is provided to synthesize CloudFormation templates and validate them, ensuring deterministic output by disabling CDK version reporting and bootstrap version rules.
src/integrations/prefect-aws/infra · high confidence
Introduce ECSWorker for running Prefect flow runs on AWS ECS
The \prefect-aws\ integration now includes a dedicated ECSWorker, allowing users to create work pools of type \ecs\ and execute flow runs as Amazon ECS tasks. This new component handles task definition registration, caching, and execution via AWS Fargate or EC2 launch types, integrating with existing AWS credentials and settings to manage infrastructure lifecycle.
_src/integrations/prefect-aws/prefect\aws/workers · high confidence
Introduce Redis integration for storage, messaging, and concurrency
This release adds the \prefect-redis\ integration, providing a \RedisDatabase\ block for use as a result storage backend (with optional per-key TTL), a \RedisLockManager\ for serializable caching, and Redis-backed implementations for worker cleanup queues, concurrency lease storage, event causal ordering, and pub/sub messaging. It also includes prebuilt \redis\_set\ and \redis\_get\ tasks, supports Redis Sentinel and Cluster URL schemes, and adds configurable settings for publisher/consumer behavior and connection timeouts.
_src/integrations/prefect-redis/prefect\redis · high confidence
Introduce WebSocket-based Worker Channel with REST fallback
Prefect workers now use a dedicated WebSocket connection (the Worker Channel) for real-time communication with the server, enabling features like work-pool snapshots, heartbeats, and cleanup delivery. If the WebSocket connection is unavailable or fails, the worker automatically falls back to the existing REST API for heartbeats and work-pool synchronization, ensuring continuous operation. This change introduces a new internal module (\src/prefect/workers/\_worker\_channel\) that manages the WebSocket lifecycle, protocol handshake, and state transitions between healthy, fallback-retrying, and disabled states.
_src/prefect/workers/\_worker\channel · high confidence
Introduce client-side events subsystem with automations and interleaved flow run streaming
The \src/prefect/events\ module now provides a complete client-side event system for emitting, subscribing to, and defining automations on Prefect events. Users can now define and trigger automations using various action types (such as \RunDeployment\, \PauseDeployment\, \SendNotification\, and \CallWebhook\) and triggers (including event, metric, and compound triggers). The subsystem introduces a \FlowRunSubscriber\ that allows users to watch a flow run by interleaving real-time events and logs in a single stream. Event emission is managed by an \EventsWorker\ that automatically attaches related resources (like flows, deployments, and work pools) to events, and includes robust checkpointing and retry logic for WebSocket connections to ensure reliable delivery.
src/prefect/events · high confidence
Introduce database-backed event storage and querying
Added new modules in \src/prefect/server/events/storage\ that provide the infrastructure for persisting and retrieving events from a database. This includes utilities for paginating event lists via encoded tokens, processing time-based event counts with interval backfilling, and executing SQL queries against the \Event\ ORM model to support filtering, distinct resource counting, and ordered retrieval.
src/prefect/server/events/storage · high confidence
Introduce deployment steps framework for pull and utility actions
The \src/prefect/deployments/steps\ module is introduced, providing a structured framework for defining and executing deployment actions. This includes the \git\_clone\ step for pulling code from Git repositories (with support for sparse checkout, credentials blocks, and retries), the \set\_working\_directory\ step for managing working directories, and utility steps like \run\_shell\_script\ (supporting shell operators and environment variable expansion) and \pip\_install\_requirements\ for installing dependencies. These steps are orchestrated by the core \run\_step\ and \run\_steps\ functions, which handle input resolution, package installation, and execution, enabling users to define complex pull and build workflows in their deployment YAML files.
src/prefect/deployments/steps · high confidence
Introduce dynamic runtime context for flows, tasks, and deployments
Users can now access dynamic attributes of the current execution context via the new \prefect.runtime\ module. This includes \prefect.runtime.flow\_run\ (exposing ID, name, tags, parameters, run count, parent IDs, and API/UI URLs), \prefect.runtime.task\run\ (exposing ID, name, tags, parameters, run count, task name, and API/UI URLs), and \prefect.runtime.deployment\ (exposing ID, name, version, parameters, and flow run ID). These attributes are populated from the active context or fetched from the API if the context is unavailable, and all modules support mocking via environment variables prefixed with \PREFECT\\RUNTIME\\_\ for testing purposes.
src/prefect/runtime · high confidence
Introduce experimental Service Level Agreement (SLA) definitions and client
Adds new experimental SLA types—TimeToCompletion, Frequency, and Lateness—along with a client API to apply these agreements to deployments. Users can now define specific performance targets for their flows, and the client provides methods to create, update, or delete these SLAs via the backend, returning a summary of changes made.
_src/prefect/\experimental/sla · high confidence
Introduce experimental Snowpark Container Services (SPCS) worker
Users can now deploy Prefect flows on Snowflake using the new experimental SPCS worker, which manages job services within Snowpark Container Services. This worker allows you to configure compute pools, resource limits (CPU, memory, GPU), container images, and external access integrations, enabling flow execution directly inside Snowflake's container environment.
_src/integrations/prefect-snowflake/prefect\snowflake/experimental · high confidence
Introduce internal infrastructure for safe concurrency, control sessions, and observability
This change establishes the \prefect.\_internal\ package as a central location for private utilities, introducing several key capabilities: a \SafeLogger\ that avoids logging locks to prevent deadlocks during complex concurrency handling; an internal Attempt Control Session protocol (with negotiation and outcome receipts) to allow runners to gracefully relinquish or cancel flow runs; a centralized registry of infrastructure exit codes with human-readable explanations and resolution hints; and new modules for configuring Logfire observability with sampling settings and collecting OpenTelemetry resource metrics (CPU/memory) during flow runs. It also includes utilities for lazy-loading modules, validating bundle launchers, and installing packages via \uv\ when available.
_src/prefect/\internal · high confidence
Introduce per-node dbt orchestration with lifecycle hooks and cross-run caching
The prefect-dbt core integration now includes a new execution engine that supports running dbt builds with per-node or per-wave Prefect task scheduling. This adds the PrefectDbtOrchestrator for wave-based or per-node execution, the PrefectDbtRunner for wrapping the dbt Python API, and a comprehensive lifecycle hook system (DbtHookMixin) allowing users to register callbacks for run start, run end, and post-model events. The engine features cross-run caching for dbt nodes based on file content and dependencies, source freshness integration to skip stale nodes, and detailed artifact generation including markdown summaries and dbt-compatible run\_results.json files. Additionally, it provides an adapter connection pool to reuse database connections across dbt invocations for improved performance.
_src/integrations/prefect-dbt/prefect\dbt/core · high confidence
Introduce schema\_tools for deployment parameter hydration and validation
The \src/prefect/utilities/schema\tools\ package is introduced to handle the hydration of \\\_prefect\_kind\ template structures (such as Jinja, JSON, and workspace variables) in deployment parameters and automation action payloads, alongside JSON Schema validation. This change provides a dedicated API (\hydrate\, \validate\) that resolves placeholders into real Python values and validates parameter specs, including specific handling for type preservation in Jinja templates and prevention of SSRF vulnerabilities during schema validation by using a non-fetching registry for remote references.
_src/prefect/utilities/schema\tools · high confidence
Introduce server-side base schema models and JSON serialization utilities
The server now exposes a dedicated \prefect.server.utilities.schemas\ package containing foundational Pydantic models (\PrefectBaseModel\, \IDBaseModel\, \ORMBaseModel\, etc.) and JSON serialization helpers (\orjson\_dumps\). \PrefectBaseModel\ configures Pydantic to ignore extra fields for forward compatibility and uses \orjson\ for efficient JSON serialization, providing a consistent base for all server-side schema definitions.
src/prefect/server/utilities/schemas · high confidence
Introduce typed flow run input and response system
Users can now send and receive structured, type-checked data between flows at runtime using the new \RunInput\ class and \send\_input\/\receive\_input\ functions. This feature allows flows to exchange custom data models (e.g., \NumberData\) with automatic JSON serialization, schema storage, and support for two-way communication channels where receivers can respond to senders.
src/prefect/input · high confidence
Introduces a new locking subsystem with filesystem and memory implementations
Prefect now includes a dedicated locking module (\src/prefect/locking\) that provides a \LockManager\ protocol and two concrete implementations: \FileSystemLockManager\, which uses atomic lock files with PID-based stale lock recovery for cross-process safety, and \MemoryLockManager\, which uses in-memory threading locks for single-process scenarios. This change adds the underlying infrastructure for transaction record locking, enabling features like distributed cache isolation and race-condition-free code pulls that depend on this new locking capability.
src/prefect/locking · high confidence
Introduces abstract messaging interfaces and an in-memory implementation with dead-letter queue support
The \prefect.server.utilities.messaging\ module now provides abstract base classes for \Publisher\ and \Consumer\, along with a concrete in-memory implementation (\memory.py\) that supports topic-based subscriptions, message deduplication via a configurable cache, and automatic routing of failed messages to a dead-letter queue after a configurable number of retries. This change establishes the foundational abstractions for messaging in Prefect, allowing for future broker-specific implementations (such as Redis) while providing a functional, testable in-memory system for local development and event handling.
src/prefect/server/utilities/messaging · high confidence
Introduces configurable causal ordering for server events
Adds a new \src/prefect/server/events/ordering\ module that manages the partial causal ordering of events, ensuring they are processed in the order they occurred. This change introduces two concrete implementations: an in-memory version (\memory.py\) using TTL caches to prevent memory leaks, and a database-backed version (\db.py\) for persistent state. The system allows administrators to select the implementation via the \server.events.causal\_ordering\ setting, enabling better control over event sequencing and scalability in multi-server deployments.
src/prefect/server/events/ordering · high confidence
Introduces in-memory cleanup queue for worker channel delivery
The server now includes a new \worker\_communication\ module featuring an in-memory cleanup queue (\WorkerCleanupQueue\) that manages the lifecycle of worker cleanup messages. This component provides storage for enqueueing, reserving, acknowledging, and releasing cleanup operations, including support for dead-letter queues and lease expiry handling, ensuring reliable delivery and idempotency for worker channel communications.
_src/prefect/server/worker\communication · high confidence
Introduces infrastructure provisioners for push work pools
Users can now automatically provision the necessary cloud resources and credentials when creating push work pools for Coiled, Modal, Azure Container Instances, Amazon ECS, and Google Cloud Run. The system detects missing dependencies (such as the \coiled\ or \modal\ packages) and offers to install them, then handles the creation of service accounts, IAM policies, and credential blocks required for the work pool to function.
src/prefect/infrastructure/provisioners · high confidence
Introduces new Blocks module with abstract interfaces and built-in block implementations
The \src/prefect/blocks\ package has been restructured to provide a clear separation between core block logic, abstract capability interfaces, and concrete built-in implementations. The new \abstract.py\ module defines base classes for \CredentialsBlock\, \NotificationBlock\, \JobBlock\, and others to standardize how blocks interact with external systems. Concrete implementations are now organized into specific files: \system.py\ provides the \Secret\ block for secure value storage, \notifications.py\ contains \SlackWebhook\ and Apprise-based notification blocks, \webhook.py\ offers a generic \Webhook\ block for HTTP calls, and \redis.py\ implements \RedisStorageContainer\ for file-system-like storage. The \core.py\ module handles the underlying server interaction, schema registration, and secret serialization, while \\_\init\\_.py\ ensures these built-in blocks are automatically registered upon import.
src/prefect/blocks · high confidence
Introduces private SDK generation engine for typed deployment clients
Adds a new internal \prefect.\_sdk\ module that provides the core infrastructure for automatically generating typed Python SDKs from workspace deployments. This includes data models for flows, deployments, and work pools; a fetcher to retrieve deployment metadata from the Prefect API; a schema converter to translate JSON schemas into Python type annotations; a template renderer using Jinja2 to produce the final SDK code; and utilities for handling naming conventions and union types. This change establishes the backend components required for the SDK generation feature but does not expose a public API or CLI command in this location.
_src/prefect/\sdk · high confidence
Introduces server-side automations and event schemas
Adds the foundational Pydantic schema models for Prefect's new server-side automations system. This includes the \AutomationCore\ model and a hierarchy of trigger types (\EventTrigger\, \MetricTrigger\, \CompoundTrigger\, \SequenceTrigger\) that define the criteria for triggering automations. It also introduces \ResourceSpecification\ for filtering resources by labels and \RelatedResource\ to support querying and matching against related resources in event data. Additionally, the \Event\ schema is updated to use UUIDv7 for event IDs and enforces limits on the number of labels per resource and related resources.
src/prefect/events/schemas · high confidence
Introduces server-side automations and event streaming capabilities
This change adds the core server-side infrastructure for Prefect Automations, including the event pipeline, trigger evaluation logic, and action execution framework. Users can now define automations that react to specific events (reactive triggers) or time-based conditions (proactive triggers), with support for complex filtering, causal event ordering, and various action types such as running deployments, sending notifications, or calling webhooks. Additionally, the server now supports streaming events to clients via WebSocket, allowing real-time monitoring of automation activity and system events.
src/prefect/server/events · high confidence
Introduces server-side schema models for Automations and Events
This change adds the foundational Pydantic schema models for the server's automation and event systems. It defines the data structures for Automations, including Triggers (such as Compound and Sequence triggers), Actions, and their associated states, as well as the core Event, Resource, and RelatedResource models used to describe system occurrences. Additionally, it introduces lifecycle event builders for domain objects like Variables, Flows, and Blocks, enabling the system to emit structured events when these objects are created, updated, or deleted.
src/prefect/server/events/schemas · high confidence
Introduction of the \`prefect-client\` lightweight package
A new \prefect-client\ package is introduced as a minimal installation of Prefect designed for interacting with remote servers or Prefect Cloud. This package excludes CLI and server-side components to reduce installation size, making it suitable for lightweight or resource-constrained environments like Lambda functions. The \client/\ directory now contains the build infrastructure (\build\_client.sh\, \Dockerfile\, \pyproject.toml\) that strips server code from the main source tree, along with documentation (\README.md\, \AGENTS.md\) and smoke tests (\client\_flow.py\, \client\_deploy.py\) to validate the reduced dependency set.
client · high confidence
New AI agent skill for writing documentation
Added a \write-docs\ skill in \.claude/skills/write-docs\ that provides AI agents with comprehensive guidance for creating and updating Prefect documentation. The skill includes a detailed guide covering page types (Get Started, Concepts, How-to Guides, etc.), navigation registration, Mintlify component usage, and code block formatting, along with starter templates for concept and how-to pages to ensure consistent structure and tone.
.claude/skills/write-docs · high confidence
New CLI for managing ECS worker infrastructure
Users can now deploy and manage AWS ECS worker infrastructure directly from the command line using the new \prefect-aws ecs-worker\ subcommand. This CLI provides commands to deploy CloudFormation stacks for ECS services, allowing configuration of work pools, VPCs, subnets, and worker scaling parameters, while handling AWS credential validation and stack tagging.
_src/integrations/prefect-aws/prefect\_aws/\cli · high confidence
New DockerImage class with optional BuildKit/buildx support
Users can now use the new \prefect.docker.DockerImage\ class to configure and build Docker images for deployments. This class introduces a \build\_backend\ parameter that allows switching from the default \docker-py\ backend to \buildx\ (via \python-on-whales\), enabling advanced features like build secrets, SSH forwarding, and multi-platform builds. It also adds a \stream\_progress\_to\ parameter to control build and push output visibility, and automatically applies the default Docker build namespace if none is specified in the image name.
src/prefect/docker · high confidence
New UI-specific API endpoints for flow, task, and schema data
Prefect introduces a new \/ui\ API namespace containing dedicated endpoints to support the user interface. This includes \/ui/flow\_runs/history\ for retrieving flow run history with work pool filtering, \/ui/flow\_runs/count-task-runs\ for bulk task run counts, \/ui/flows/next-runs\ and \/ui/flows/count-deployments\ for flow-specific metrics, \/ui/task\_runs/dashboard/counts\ for dashboard time-series data, \/ui/task\_runs/count\ for state-based counts, and \/ui/task\_runs/{id}\ for detailed task run information. Additionally, \/ui/schemas/validate\ provides a dedicated endpoint for JSON schema validation, separating these UI-centric data retrieval and validation paths from the general API.
src/prefect/server/api/ui · high confidence
New \`prefect transfer\` CLI command for migrating resources between profiles
Users can now use the \prefect transfer\ command to move resources (such as flows, deployments, work pools, automations, and variables) from one Prefect profile to another. The command automatically resolves dependencies between resources, builds a dependency graph to ensure correct transfer order, and handles concurrent execution. It provides interactive confirmation, progress reporting, and detailed event logging for transfer start and completion, including success, failure, and skip counts.
src/prefect/cli/transfer · high confidence
New automation client with configurable filtering
The Prefect client now includes a dedicated \AutomationClient\ (and async variant) for managing automations, introducing a \read\_automations\ method that accepts configurable filter, sort, limit, and offset parameters for precise querying. This replaces or supplements previous mechanisms by allowing users to filter automations directly via the API using \AutomationFilter\ and \AutomationSort\ schemas, while also providing helper methods like \find\_automation\ for lookup by ID or name.
_src/prefect/client/orchestration/\automations · high confidence
New background services for event persistence, trigger evaluation, and debugging
The Prefect server now includes dedicated background services to handle core event automation workflows. The EventPersister service consumes events from the bus and persists them to the database using configurable batching, flushing intervals, and memory safeguards. The ReactiveTriggers service evaluates reactive automation triggers as events arrive, while a new periodic service handles proactive trigger evaluation. Additionally, an EventLogger service is provided for debugging, which prints incoming events to the console. These services are built on a unified Service base class and integrate with the server's messaging infrastructure.
src/prefect/server/events/services · high confidence
New bundle upload and execution steps for AWS, Azure, and GCP integrations
This change introduces new \bundles\ subpackages to the AWS, Azure, and GCP integrations, providing concrete steps to upload and execute flow bundles on each cloud provider. For AWS, the \prefect\_aws.bundles\ module adds \upload\_bundle\_to\_s3\ and \execute\_bundle\_from\_s3\ functions (with CLI wrappers) to manage bundles in S3. For Azure, \prefect\_azure.bundles\ adds \upload\_bundle\_to\_azure\_blob\_storage\ and \execute\_bundle\_from\_azure\_blob\_storage\ to handle bundles in Azure Blob Storage. For GCP, \prefect\_gcp.bundles\ adds \upload\_bundle\_to\_gcs\ and \execute\_bundle\_from\_gcs\ to manage bundles in Google Cloud Storage. Each execution step downloads the bundle (and any included files), extracts them, and runs the bundle using Prefect's core execution logic, falling back to experimental internal paths if the stable Prefect bundle APIs are not available in the installed version.
_src/integrations/prefect-aws/prefect\_aws/bundles, src/integrations/prefect-azure/prefect\_azure/bundles, src/integrations/prefect-gcp/prefect\gcp/bundles · high confidence
New concurrency lease storage subsystem with in-memory and filesystem backends
Prefect introduces a new \lease\_storage\ module under \src/prefect/server/concurrency\ that manages concurrency limits via a lease-based system. This change adds a \ConcurrencyLeaseStorage\ protocol defining operations for creating, reading, renewing, and revoking leases, along with methods to list active lease IDs and identify holders for specific limits. Two concrete implementations are provided: an in-memory singleton (\memory.py\) for single-process or testing scenarios, and a filesystem-based storage (\filesystem.py\) that persists leases as JSON files with atomic writes to prevent race conditions. A factory function \get\_concurrency\_lease\_storage()\ allows selecting the backend via settings. This infrastructure supports the broader shift from simple slot counting to lease-based concurrency management, enabling more robust handling of concurrency limits across distributed or long-running tasks.
src/prefect/server/concurrency · high confidence
New dbt Cloud integration with per-node orchestration and asset materialization
The \prefect\_dbt.cloud\ package is now available, providing a complete integration for orchestrating dbt Cloud jobs. Users can authenticate via the new \DbtCloudCredentials\ block and interact with the dbt Cloud Administrative and Metadata APIs through dedicated clients. A key addition is the \DbtCloudExecutor\, which enables per-node orchestration by creating ephemeral dbt Cloud jobs for individual DAG nodes, allowing for fine-grained control and parallel execution. The integration also includes tasks and flows for triggering job runs, polling for completion, and retrieving run artifacts. Additionally, it supports automatic Prefect asset materialization, where successful dbt runs automatically emit asset events for downstream dependency tracking.
_src/integrations/prefect-dbt/prefect\dbt/cloud · high confidence
New dedicated log client for flow and task run operations
Users can now interact with logs through a dedicated \LogClient\ and \LogAsyncClient\ located in the orchestration module. These clients provide specific methods to create logs for flow or task runs and to read logs using filters, limits, offsets, and sorting options, streamlining log management within the Prefect client interface.
_src/prefect/client/orchestration/\logs · high confidence
New deployment steps for building and pushing Docker images
The \prefect\_docker.deployments.steps\ module has been introduced, providing \build\_docker\_image\ and \push\_docker\_image\ functions that can be used within \prefect.yaml\ files to define default or specific build steps for deployments. This new capability includes support for auto-generating Dockerfiles (with an option to persist them), caching of build outputs, and a choice of build backends (\docker-py\ or \buildx\ via python-on-whales) to enable advanced features like BuildKit, secrets, and multi-platform builds. Image name validation is also enforced to prevent errors caused by invalid characters.
_src/integrations/prefect-docker/prefect\docker/deployments · high confidence
New examples for AI agents, asset pipelines, and concurrency patterns
Added several new example scripts to the examples directory: an AI data analyst using pydantic-ai and durable execution, a database cleanup workflow with human or AI approval, a social analytics dashboard using Prefect Assets, a per-worker task concurrency pattern using Global Concurrency Limits, a resume-flow-run-on-PR-merge automation, and updated examples for API-sourced ETL, dbt orchestration, and web scraping.
examples · high confidence
New file bundling infrastructure for deployments
Prefect introduces a new file bundling system in \src/prefect/bundles\ to handle including local files with deployments. This system collects files based on user patterns (including gitignore-style exclusions via \.prefectignore\), packages them into a sidecar zip archive, and uses content-addressed storage keys (SHA256 hashes) to deduplicate identical file sets across deployments. The implementation includes path validation to prevent directory traversal and symlink loops, and provides utilities to extract these bundles during flow execution.
src/prefect/bundles · high confidence
New filesystem utility module with path filtering and context managers
A new \src/prefect/utilities/filesystem\ module has been introduced, providing core utilities for file system operations. This includes \filter\_files\ for pathspec-based file filtering compatible with \.gitignore\ patterns, \tmpchdir\ for scoped working-directory changes with thread safety and UNC path support on Windows, and \get\_open\_file\_limit\ for platform-specific open-file limit introspection. The module also adds helpers for creating default ignore files, normalizing paths, determining if a path is local or remote via \fsspec\, and converting paths to display formats.
src/prefect/utilities/filesystem · high confidence
New internal concurrency API for managing synchronous and asynchronous calls
Prefect introduces a new internal concurrency module (\prefect.\_internal.concurrency\) that provides a unified API for seamlessly executing both synchronous and asynchronous functions without blocking threads. This system uses a \Call\ abstraction to capture function calls and context, routing them to either dedicated worker threads or a global event loop via \Portal\ implementations. It includes specialized waiters (\SyncWaiter\, \AsyncWaiter\) that allow threads to wait for results while still processing callbacks, and introduces thread-safe async primitives like \Event\ and \Future\. The module also implements robust fork handling for multiprocessing safety on POSIX systems and includes utilities for cancellation, stack inspection, and service lifecycle management.
_src/prefect/\internal/concurrency · high confidence
New processutils module for subprocess management and environment sanitization
A new \processutils\ module has been introduced to centralize cross-platform subprocess primitives, including \run\_process\ for async execution with signal forwarding, \sanitize\_subprocess\_env\ for safely handling \None\ values in environment variables before launch, and helpers like \command\_to\_string\ for platform-neutral command serialization. This module also provides robust output streaming via \consume\_process\_output\ and \stream\_text\, while explicitly documenting behavioral details such as the silent replacement of non-UTF-8 bytes and the removal of quotes from Windows Python paths in \get\_sys\_executable\.
src/prefect/utilities/processutils · high confidence
New scripts for documentation generation, link checking, and integration management
This change introduces a suite of new automation scripts in the \scripts/\ directory to support the Prefect 3.x documentation and release workflows. Key additions include \all\_links\_should\_be\_ok.py\ for auditing HTTP links in documentation, \generate\_api\_ref.py\ and \generate\_mintlify\_openapi\_docs.py\ for building API reference pages, \generate\_settings\_ref.py\ for auto-generating settings documentation, and \generate\_example\_pages.py\ for rendering code examples. It also adds \backfill\_release\_notes.py\ and \prepare\_integration\_release\_notes.py\ for managing release note history, a \collections-manager\ CLI for bulk operations on integration packages, and \generate-lower-bounds.py\ for testing dependency compatibility. The \entrypoint.sh\ script is updated to use \uv\ for installing extra pip packages, and \check\_pyright\_report.py\ ensures type-checking runs are valid.
scripts · high confidence
New server utilities module for database, security, and worker channel management
The \src/prefect/server/utilities\ package has been introduced to centralize core server infrastructure. This includes \database.py\ for platform-independent ORM utilities (such as cross-dialect UUID generation and timezone-aware timestamps), \encryption.py\ for Fernet-based configuration encryption, and \http.py\ for redacting sensitive HTTP headers. Security is enhanced via \subscriptions.py\, which enforces WebSocket authentication using \hmac.compare\_digest\ and requires the 'prefect' subprotocol when auth is configured, and \user\_templates.py\, which implements a sandboxed Jinja2 environment with strict limits on loop counts and range sizes to prevent template injection attacks. Operational reliability is improved through \postgres\_listener.py\, which normalizes DSNs for multihost support and URL-encodes credentials, and \worker\_channel.py\, which manages worker heartbeats, work-pool snapshots, and cleanup dispatching with Prometheus metrics.
src/prefect/server/utilities · high confidence
New templating utility module for placeholder resolution
A new \src/prefect/utilities/templating\ module has been introduced to handle \{{ }}\ placeholder detection and substitution within nested structures. This module provides entry points to find placeholders (classifying them as standard variables, block document references, or environment variables) and apply resolved values back into templates. It distinguishes between inline block placeholders (embedded in text) and whole-string block references, ensuring correct type coercion and error handling for malformed block references.
src/prefect/utilities/templating · high confidence
New utilities package with documentation and core helper modules
The \src/prefect/utilities\ package has been introduced to centralize general-purpose helpers and cross-cutting tools used throughout the Prefect SDK. This location now includes an \AGENTS.md\ file defining the scope and subpackages (such as \callables\, \asyncutils\, and \templating\), alongside core modules like \annotations.py\ (providing \unmapped\, \quote\, \opaque\, and \freeze\ wrappers for flow/task parameters), \collections.py\ (offering \visit\_collection\, \flatten\, and \isiterable\), \dispatch.py\ (for dynamic type registration), \hashing.py\ (stable hashing utilities), \importtools.py\ (dynamic imports and script loading), and \pydantic.py\ (Pydantic v1/v2 compatibility shims). A deprecated \compat.py\ module is also present, warning users to use \importlib.metadata\ directly.
src/prefect/utilities · high confidence
OpenTelemetry instrumentation for Prefect runs
Prefect now automatically instruments flow and task runs with OpenTelemetry spans, enabling distributed tracing integration. The new \RunTelemetry\ class manages span lifecycle, capturing run names, IDs, tags, and parameters as attributes. Trace context is propagated via run labels (specifically \\_\_OTEL\_TRACEPARENT\), allowing nested runs to maintain correct parent-child relationships in the trace hierarchy. This feature can be disabled by setting the \PREFECT\_CLOUD\_ENABLE\_ORCHESTRATION\_TELEMETRY\ setting to false.
src/prefect/telemetry · high confidence
Prefect AWS integration migrated to core with Pydantic 2 and new AWS capabilities
The \prefect-aws\ integration has been promoted from experimental to a core, generally available component, bringing significant structural and functional updates. The codebase has been migrated to Pydantic 2, updating all credential and parameter models (such as \AwsCredentials\ and \AwsClientParameters\) to use Pydantic v2 syntax and validators. This release introduces AWS AssumeRole support, allowing users to configure \assume\_role\_arn\ and \assume\_role\_kwargs\ within the \AwsCredentials\ block for temporary credential management. Additionally, it adds a new \GlueJobBlock\ for executing AWS Glue jobs, an \ecs\ decorator for binding flows to ECS work pools, and IAM authentication support for RDS PostgreSQL via a new plugin hook. The integration also includes new async tasks for AWS Batch and Client Waiters, and adds a \py.typed\ file to enable static type checking for users.
_src/integrations/prefect-aws/prefect\aws · high confidence
Prefect Azure integration now supports Azure Database for PostgreSQL with managed identity authentication
The prefect-azure integration now allows Prefect's own server database (Azure Database for PostgreSQL) to authenticate using Azure managed identity (Microsoft Entra ID) instead of a static password. This is enabled via the new \integrations.azure.postgres.managed\_identity\ settings (controlled by \PREFECT\_AZURE\_POSTGRES\_MANAGED\_IDENTITY\_ENABLED\ and \PREFECT\_AZURE\_POSTGRES\_MANAGED\_IDENTITY\_CLIENT\_ID\). When enabled, the integration automatically injects an async token-fetching password callable and an SSL context into the database connection, ensuring tokens are refreshed transparently and securely without requiring a stored database password.
_src/integrations/prefect-azure/prefect\azure · high confidence
Scaffolded Prefect UI v2 project structure and tooling
The UI location has been scaffolded with a new project structure for the Prefect UI v2, including configuration files for ESLint, PostCSS with Tailwind, and VS Code development settings. The entry point \index.html\ now serves the application with favicons that adapt to the user's system color scheme preference (light/dark mode). Additionally, new public assets have been added, including decorative SVG grid patterns for both light and dark themes to enhance the visual design of the interface.
ui · high confidence
prefect-aws integration promoted to core with ECS worker infrastructure and Docker images
The prefect-aws integration is now a core package, providing pre-built Docker images for simplified deployment and new ECS worker infrastructure to run Prefect flows on AWS Fargate. This update also introduces experimental support for AWS IAM authentication for RDS PostgreSQL databases, allowing automatic token generation for database connections, and includes updated documentation and API reference generation workflows.
src/integrations/prefect-aws · high confidence
prefect-azure package structure and managed identity database authentication
The prefect-azure integration package now includes a Dockerfile for building container images with pinned versions of prefect and prefect-azure, along with standard packaging files (LICENSE, MANIFEST.in) and updated documentation. A key new capability is managed identity authentication for the Prefect server's Azure Database for PostgreSQL connection: users can enable this by setting PREFECT\_PLUGINS\_ENABLED=true and PREFECT\_INTEGRATIONS\_AZURE\_POSTGRES\_MANAGED\_IDENTITY\_ENABLED=true, allowing the server to acquire short-lived Microsoft Entra ID tokens via DefaultAzureCredential instead of storing database passwords.
src/integrations/prefect-azure · high confidence
prefect-bitbucket package structure and documentation tooling added
The prefect-bitbucket integration package now includes its own standalone structure with a LICENSE (Apache 2.0), MANIFEST.in, and README.md. Additionally, a justfile has been added to facilitate local testing and the generation of API reference documentation using mdxify.
src/integrations/prefect-bitbucket · high confidence
prefect-databricks integration package scaffolding and documentation tooling
The prefect-databricks integration package now includes standard distribution files (LICENSE, MANIFEST.in, README.md) and a justfile to automate API reference documentation generation using mdxify. This establishes the package structure for distribution and ensures consistent, up-to-date API documentation is available for users.
src/integrations/prefect-dask, src/integrations/prefect-databricks · high confidence
prefect-dbt integration package structure and documentation setup
The prefect-dbt integration package now includes its own LICENSE (Apache 2.0), MANIFEST.in, README, and a justfile to streamline local development and documentation generation. The justfile configures mdxify to automatically generate API reference documentation for the prefect\_dbt module, ensuring that users have up-to-date SDK references available in the docs. This change standardizes the package structure and improves the developer experience for maintaining and consuming the prefect-dbt integration.
(repo-wide) · high confidence
prefect-ray integration package structure and documentation tooling
The prefect-ray integration package now includes its own standalone directory structure with a LICENSE file (Apache 2.0), a MANIFEST.in for packaging, a README.md, and a justfile to automate API reference documentation generation using mdxify. This change establishes the foundational package metadata and build configuration for the prefect-ray integration, ensuring proper licensing, distribution, and automated documentation generation for the integration's public API.
src/integrations/prefect-ray · high confidence
prefect-shell integration package initialization and documentation setup
The prefect-shell integration package has been added to the repository, including its source code structure, Apache 2.0 license, and build manifest (MANIFEST.in). This change also introduces automated API reference documentation generation for the package using mdxify, ensuring that the shell command integration tools are properly documented and distributed.
src/integrations/prefect-shell · high confidence
prefect-snowflake integration package structure and licensing
The prefect-snowflake integration package now includes its own LICENSE (Apache 2.0), MANIFEST.in, README, and justfile. This clarifies usage rights and provides local tooling for testing and API reference generation, ensuring the package is self-contained and properly licensed for distribution.
src/integrations/prefect-docker, src/integrations/prefect-snowflake · high confidence
Security
Security fix for SSRF via remote JSON schema references
The JSON schema validation logic in \src/prefect/\_internal/schemas\ has been hardened to prevent Server-Side Request Forgery (SSRF) attacks. A new internal registry (\non\_fetching\_registry\) is now used during validation, which disables the default behavior of fetching remote \$ref\ URLs over the network. If a schema contains an external reference, validation will now fail with an error instead of attempting a network request, while internal document references continue to resolve normally.
_src/prefect/\internal/schemas · high confidence
Architecture
Prefect REST API restructured into modular route files
The Prefect REST API has been reorganized from a monolithic structure into a modular package under \src/prefect/server/api\. This change introduces lazy-loading for all API modules (such as \admin\, \artifacts\, \automations\, \block\_types\, \deployments\, \events\, \logs\, \variables\, and \workers\) to improve startup performance and reduce memory usage. It also adds a private helper module (\\_ui\_static.py\) to manage UI static file copying and error logging during server startup, and introduces a new internal \OrchestrationClient\ (\clients.py\) that allows server-side services to interact with the API using the same authentication and CSRF handling as external clients.
src/prefect/server/api · high confidence
Prefect client schemas are reorganized into a modular package
The \prefect.client.schemas\ module has been restructured from a single monolithic file into a modular package with separate files for actions, objects, filters, responses, schedules, sorting, events, and the worker channel protocol. This change introduces lazy loading for schema imports to improve startup performance and reduces circular import risks. Users interacting with the client API will see no functional change, but the internal organization is now cleaner and more maintainable.
src/prefect/client/schemas · high confidence
Prefect server schemas are reorganized into a modular package structure
The server-side Pydantic schemas have been reorganized from a single monolithic module into a structured package under \src/prefect/server/schemas\. This change introduces lazy loading via \\_\getattr\\\ in the package \\\init\\_.py\ to improve import performance, and splits the schema definitions into distinct modules for actions (API inputs), core models, filters, sorting, states, statuses, schedules, and UI-specific responses. This modularization supports the separation of server and client schemas and enables more targeted updates to specific schema areas without reloading the entire server module.
src/prefect/server/schemas · high confidence
Prefect-dbt integration migrated to core repository with lazy-loading API
The prefect-dbt package has been moved into the core Prefect repository, restructuring the module layout to expose core components like PrefectDbtRunner and PrefectDbtSettings directly at the top level while keeping database-specific configurations (such as SnowflakeTargetConfigs) in submodules. To maintain performance and backward compatibility, the package now uses a lazy-loading mechanism via \_\getattr\\_ for CLI-related attributes like DbtCliProfile and TargetConfigs, ensuring that optional dependencies are only imported when actually accessed. This change also introduces utility functions for handling dbt CLI arguments and formatting asset keys, supporting the integration's new position within the main codebase.
_src/integrations/prefect-dbt/prefect\dbt · high confidence
Refactored concurrency limit client into a dedicated module
The concurrency limit client logic has been reorganized into a new \src/prefect/client/orchestration/\_concurrency\_limits\ package, separating the implementation from the main orchestration client. This change introduces a dedicated \ConcurrencyLimitClient\ class that handles CRUD operations for tag-based concurrency limits, including creating, reading, listing, resetting, and deleting limits, as well as managing concurrency slots. This structural change improves code organization and maintainability for concurrency-related API interactions.
_src/prefect/client/orchestration/\_concurrency\limits · high confidence
Refactored orchestration client into modular, domain-specific sub-clients
The orchestration client has been restructured from a single monolithic implementation into a modular architecture with dedicated sub-clients for specific resource domains (e.g., blocks, events, deployments, automations, logs, variables, concurrency limits, flows, and work pools). This change introduces new internal modules under \src/prefect/client/orchestration/\ (such as \\_blocks\_documents/client.py\, \\_events/client.py\, and \routes.py\) that encapsulate the HTTP request logic and API route definitions for each domain. For users, this represents a significant internal refactoring that improves code organization and maintainability, while preserving the existing public API surface through the main \PrefectClient\ and \SyncPrefectClient\ classes which now aggregate these specialized sub-clients.
src/prefect/client/orchestration · high confidence
Runner refactored into single-responsibility services
The \Runner\ implementation has been restructured from a monolithic class into a thin facade delegating to extracted, single-responsibility services. This change introduces dedicated modules for process lifecycle (\ProcessManager\), state transitions (\StateProposer\), cancellation control (\CancellationManager\), hook execution (\HookRunner\), event emission (\EventEmitter\), concurrency limiting (\LimitManager\), and deployment tracking (\DeploymentRegistry\). The refactoring also introduces a \ProcessStarter\ strategy pattern to handle different execution modes (direct subprocess, engine command, workspace-resolved, and bundle execution) and implements an internal TCP loopback \ControlChannel\ to manage cancellation intent and outcome receipts between the runner and child processes. Legacy methods on the \Runner\ facade are now deprecated in favor of these new internal components.
src/prefect/runner · high confidence
prefect-gcp integration migrated to core Prefect repository
The \prefect-gcp\ integration has been moved from its standalone repository into the core \prefect\ codebase, consolidating GCP support (BigQuery, Cloud Storage, Secret Manager, Vertex AI, and Cloud Run workers) alongside the main SDK. This change updates import paths and module structures to reflect the new location within the \prefect\ package, ensuring that users accessing GCP features do so through the unified core distribution rather than a separate integration package.
_src/integrations/prefect-gcp/prefect\gcp · high confidence
Behavioural changes
Artifact creation and updates now emit system events
The artifact client now automatically emits \prefect.artifact.created\ and \prefect.artifact.updated\ events whenever an artifact is created or modified. This enables external systems and workflows to react to artifact lifecycle changes via the Prefect events mechanism.
_src/prefect/client/orchestration/\artifacts · high confidence
Build process now generates analytics configuration
The build tooling has been updated to automatically generate an analytics configuration file containing an Amplitude API key during the build process. This change ensures that analytics settings are embedded into the project at build time, allowing the application to track usage metrics based on the environment-provided API key.
tools · high confidence
Centralized type definitions and validation logic in src/prefect/types
The \src/prefect/types\ package has been introduced to centralize type aliases, validation rules, and helper functions previously scattered across the codebase. This change consolidates datetime handling (including timezone-aware coercion and Python 3.13 compatibility via \whenever\ or \pendulum\), standardizes name validation (enforcing banned characters and length limits for variables, assets, and keys), and introduces self-validating types for concurrency leases and entrypoints. Users benefit from stricter input validation, consistent behavior for time-related fields across different Python versions, and cleaner error messages when invalid names or values are provided.
src/prefect/types · high confidence
Clear error when passing FlowRun or TaskRun objects as task arguments
Prefect now raises a clear error when a \FlowRun\ or \TaskRun\ object is passed as a task argument without the \opaque\ annotation. Previously, these Pydantic models would be recursively inspected by \visit\_collection\, causing the embedded \.state\ attribute to be mistakenly treated as an upstream task dependency. Users must now pass individual fields (e.g., \flow\_run.id\) or wrap the object with \opaque()\ to skip dependency traversal.
src/prefect/utilities/engine · high confidence
Concurrency module refactored to use lease-based slot acquisition with renewal and strict mode
The \prefect.concurrency\ module has been rewritten to use a lease-based model for managing concurrency slots, replacing the legacy slot API. The \concurrency()\ context manager now acquires a lease that is actively renewed by a background daemon thread (renewing at 75% of the lease duration with exponential backoff retries) to prevent slot starvation during long-running tasks. A new \strict\ parameter allows users to raise an error if a named concurrency limit does not exist, instead of the previous default behavior of logging a warning. Additionally, a \raise\_on\_lease\_renewal\_failure\ parameter lets users control whether a renewal failure terminates the task or merely logs a warning. The module now emits structured events for slot acquisition and release, and includes client-side caching to avoid redundant API calls for limits known to not exist.
src/prefect/concurrency · high confidence
Database migration infrastructure reorganized and documented
The database migration system has been reorganized: the migration module is now located at \prefect.server.database.\_migrations\ (renamed from \prefect.server.database.migrations\), and a new \MIGRATION-NOTES.md\ file provides a comprehensive history of schema changes, including details on new tables (such as \deployment\_version\ and \events\), added indexes (e.g., on \flow\_run.deployment\_id\ and \event\_resources\), and column modifications (like adding \parameters\ to deployment schedules). The migration environment (\env.py\) has been updated to handle SQLite-specific constraints, such as disabling foreign keys during migrations and using batch mode for schema alterations, while also introducing a dedicated migration timeout for PostgreSQL to prevent long-running index builds from timing out.
_src/prefect/server/database/\migrations · high confidence
Databricks integration now supports Service Principal authentication
The Databricks credentials block now supports authenticating via Service Principal (OAuth 2.0) in addition to the existing Personal Access Token method. Users can configure \client\_id\ and \client\_secret\ (with an optional \tenant\_id\ for Azure Databricks) to enable OAuth-based access, with automatic token caching and refresh handled internally. The integration also includes a restored \jobs\_runs\_submit\_and\_wait\_for\_completion\ flow and updated telemetry attribution headers to comply with Databricks partner requirements.
_src/integrations/prefect-databricks/prefect\databricks · high confidence
Deploy command refactored to cyclopts with improved YAML validation and configuration handling
The \prefect deploy\ and \prefect init\ CLI commands have been migrated from Typer to the cyclopts framework, reorganizing the codebase into a modular structure under \src/prefect/cli/deploy/\. This change introduces Pydantic-based models (\PrefectYamlModel\) for validating \prefect.yaml\ files, resulting in more detailed and user-friendly error messages for invalid configurations. The deployment configuration now supports single-step mappings in action sections (build, push, pull) and preserves explicit empty \pull: \[\]\ definitions instead of forcing a default working directory. Additionally, the CLI now supports ISO 8601 duration strings for interval schedules, a \replaces\ field for renaming schedule slugs, and the \day\_or\ parameter for cron schedules.
src/prefect/cli/deploy · high confidence
Deployments module restructured with lazy loading and new entry points
The \src/prefect/deployments\ package has been reorganized to improve import performance and clarify the public API. The module now uses lazy loading via \\_\getattr\\_\ and \TYPE\_CHECKING\ guards to defer imports of heavy dependencies like the Prefect client and Pydantic models until they are actually used. New entry points have been introduced: \initialize\_project\ in \base.py\ for scaffolding \prefect.yaml\ files, and \arun\_deployment\ in \flow\_runs.py\ for asynchronous deployment triggering. The \runner.py\ module now exposes \RunnerDeployment\ with enhanced schedule handling, concurrency limits, and support for module-path entrypoints. Additionally, an \AGENTS.md\ file has been added to document the deployment lifecycle, step system, and known pitfalls for developers working within this module.
src/prefect/deployments · high confidence
Deprecate experimental plugin imports in favor of stable API
Imports from the \prefect.\_experimental.plugins\ path now emit a \DeprecationWarning\ and redirect users to the stable \prefect.plugins\ module. This change reflects the graduation of the plugin system from experimental to general availability, ensuring that existing code using the experimental path continues to function while encouraging migration to the public API.
_src/prefect/\experimental/plugins · high confidence
Deprecation of experimental GCP bundle modules
The GCP bundle modules previously located under \prefect\_gcp.experimental.bundles\ (including \upload\, \execute\, and their submodules) have been deprecated and now emit warnings directing users to migrate to the stable \prefect\_gcp.bundles\ paths. The code in this location now acts as a shim, re-exporting the functionality from the new stable location while alerting users that the old import paths will be removed in a future release.
_src/integrations/prefect-gcp/prefect\gcp/experimental/bundles · high confidence
Deprecation of experimental GCP decorators
The \prefect\_gcp.experimental\ module and its \decorators\ submodule have been deprecated and now redirect to \prefect\_gcp.decorators\. Importing from the old experimental paths will trigger a deprecation warning, indicating that these imports will be removed in a future release. Users should update their code to import \cloud\_run\ and \vertex\_ai\ directly from \prefect\_gcp.decorators\.
_src/integrations/prefect-gcp/prefect\gcp/experimental · high confidence
Deprecation of experimental Kubernetes decorator import paths
The \prefect\_kubernetes.experimental\ and \prefect\_kubernetes.experimental.decorators\ modules have been deprecated. Importing the \kubernetes\ decorator from these experimental paths now triggers a \DeprecationWarning\, directing users to migrate to the stable \prefect\_kubernetes.decorators\ module. The experimental paths will be removed in a future release.
_src/integrations/prefect-docker/prefect\_docker/experimental, src/integrations/prefect-kubernetes/prefect\kubernetes/experimental · high confidence
Deprecation of experimental import paths in Prefect Azure integration
The \prefect\_azure.experimental\ module and its submodules (\decorators\, \bundles\, \bundles.execute\, \bundles.upload\) now emit deprecation warnings when imported, indicating that these components have moved to their stable counterparts in \prefect\_azure.decorators\ and \prefect\_azure.bundles\. Users relying on the old experimental import paths will see warnings advising them to update their imports to the new locations before the old paths are removed in a future release.
_src/integrations/prefect-azure/prefect\azure/experimental · high confidence
Deprecation of experimental import paths in prefect-aws
The \prefect\_aws.experimental\ module and its submodules (\decorators\, \bundles.execute\, \bundles.upload\) now emit deprecation warnings when imported, indicating that these components have moved to stable locations (\prefect\_aws.decorators\ and \prefect\_aws.bundles\). Users should update their imports to use the new paths, as the old experimental paths will be removed in a future release.
_src/integrations/prefect-aws/prefect\aws/experimental · high confidence
Deprecation of legacy bundle import paths
The bundle implementation has graduated to general availability at \prefect.bundles\, and the legacy \prefect.\_experimental.bundles\ path is now deprecated. Importing from the old location or running \python -m prefect.\_experimental.bundles.execute\ will now emit a \DeprecationWarning\ and forward to the new GA modules, ensuring backward compatibility for existing work pool storage configurations while encouraging migration to the stable API.
_src/prefect/\experimental/bundles · high confidence
Deprecation of the prefect\_dbt.cli module in favor of prefect\_dbt.core
The \prefect\_dbt.cli\ module is now deprecated and will be removed in a future release; users should migrate to \prefect\dbt.core\. The \\\init\\_.py\ file in this location now emits a \UserWarning\ directing users to the new module, while still re-exporting the existing public API classes (such as \DbtCliProfile\, \DbtCoreOperation\, and various config classes) to maintain backward compatibility during the transition.
_src/integrations/prefect-dbt/prefect\dbt/cli · high confidence
Drop Python 3.8 support in Prefect 3.x
The Prefect 3.x line no longer supports Python 3.8, aligning with the language's end-of-life status. Users must upgrade to Python 3.9 or later to use this version of the library.
src/prefect · high confidence
Improved BitBucket authentication for self-hosted instances and special characters
The BitBucket integration now supports username and password authentication for self-hosted BitBucket Server instances that do not include 'bitbucketserver' in their hostname, resolving previous connectivity issues for these environments. Additionally, credentials (usernames and tokens) are now properly URL-encoded when embedded in git clone URLs, ensuring that special characters in tokens or usernames no longer cause authentication failures or git errors.
_src/integrations/prefect-bitbucket/prefect\bitbucket · high confidence
Initial SQLite database schema and migration history
This change introduces the foundational Alembic migration files for the SQLite database backend, establishing the initial schema and the complete historical upgrade path. The initial migration creates core tables for flows, logs, concurrency limits, and saved searches, while subsequent migrations progressively add support for agents, work queues, block storage (renamed from block\_data to block, then to block\_document), block schemas, flow run alerts, and block schema capabilities. This ensures that SQLite deployments can be initialized and upgraded through all historical schema changes, maintaining parity with the ORM models.
_src/prefect/server/database/\migrations/versions/sqlite · high confidence
Introduce FlexibleScheduleList type alias for schedule definitions
A new \FlexibleScheduleList\ type alias has been added to the client types module, allowing schedules to be defined using a sequence of deployment schedule creation objects, dictionaries, or existing schedule types. This change supports the ability to define per-schedule parameters via \.serve\ and \.deploy\ commands by providing a unified type for flexible schedule inputs.
src/prefect/client/types · medium confidence
Introduce automation change notifications and composite trigger race-condition handling
The \src/prefect/server/events/models\ module now includes logic to emit lifecycle events (created, updated, deleted) for automations and manages composite trigger state. For PostgreSQL, automation changes trigger a \NOTIFY\ on a dedicated channel to update in-memory caches, while SQLite updates the cache directly after commit. Additionally, a new \composite\_trigger\_child\_firing\ module serializes concurrent evaluations of compound triggers using PostgreSQL advisory locks and tracks child firing states to prevent race conditions where multiple transactions might otherwise miss firing the parent trigger.
src/prefect/server/events/models · high confidence
Introduce internal Pydantic v2 compatibility layer for function validation
Prefect adds a new internal module at src/prefect/\_internal/pydantic to support Pydantic v2, including a ValidatedFunction class that validates function arguments using Pydantic v2 models, schema generation utilities for both v1 and v2, and helpers to detect v1/v2 types in function signatures. This enables flow parameters and function signatures to work correctly with Pydantic v2 constructs and resolves forward reference and container-type annotation issues that previously caused errors or deprecation warnings.
_src/prefect/\internal/pydantic · high confidence
Introduces Pydantic 2 data models for Cloud Run V2 jobs
The \prefect\_gcp\ integration now uses Pydantic 2 models to define the structure of Cloud Run V2 job data. This change introduces new data classes, such as \JobV2\ and \SecretKeySelector\, which replace previous implementation details with strict schema validation for job attributes and secret injection configurations. Users benefit from more robust type checking and validation when interacting with Cloud Run V2 resources.
_src/integrations/prefect-gcp/prefect\gcp/models · high confidence
Introduces structured server settings models
The server configuration is now organized into a hierarchical set of Pydantic models under \src/prefect/settings/models/server\, replacing the previous flat structure. This change introduces dedicated settings groups for the API (including CSRF protection, CORS, and WebSocket ping intervals), database (PostgreSQL connection tuning, mTLS, and search path), concurrency (lease storage and duration), events (retention, caching, and messaging), services (docket, event persister, and database vacuum), tasks, and the UI (V2 default and promotional content). Users can now configure these server behaviors through a more granular and type-safe settings interface.
src/prefect/settings/models/server · high confidence
Migrate prefect-docker integration to core repository with Pydantic 2 and new container management tasks
The \prefect-docker\ integration has been migrated into the core Prefect repository, bringing the Docker worker, host settings, and registry credentials blocks into the main codebase. This update migrates the integration to Pydantic 2 for configuration validation and introduces new async Prefect tasks for managing Docker containers and images, including \create\_docker\_container\, \get\_docker\_container\_logs\, \start\_docker\_container\, \stop\_docker\_container\, and \pull\_docker\_image\. The Docker worker now supports an \IfPossible\ image pull policy, resolves relative bind-mount volume sources against the worker's current working directory, and allows passing \container\_create\_kwargs\ for finer control over container creation. Additionally, a \@docker\ decorator is provided to bind flows directly to a Docker work pool.
_src/integrations/prefect-docker/prefect\docker · high confidence
Migrate prefect-slack integration to core with Pydantic 2 and async support
The prefect-slack integration has been moved into the core codebase, bringing the SlackCredentials and SlackWebhook blocks, along with send\_chat\_message and send\_incoming\_webhook\_message tasks, up to date with the latest released version. This update migrates the credential and block definitions to Pydantic 2 and adopts async\_dispatch for the SlackWebhook notify method, enabling consistent asynchronous notification sending while maintaining synchronous fallbacks.
_src/integrations/prefect-slack/prefect\slack · high confidence
Migration helpers for Prefect 3.0 infrastructure module changes
The \src/prefect/infrastructure\ module now includes \\_\init\\_.py\ and \base.py\ files that provide actionable error messages for users upgrading to Prefect 3.0. These files utilize a migration helper to intercept imports of moved or removed objects, guiding users through the transition rather than failing with opaque errors.
src/prefect/infrastructure · high confidence
New deployment client module with branching, versioning, and pause/resume capabilities
The deployment client logic has been reorganized into a new module (\src/prefect/client/orchestration/\_deployments\), introducing support for deployment versioning and branching via new \version\_info\, \branch\, \base\, and \root\ parameters in \create\_deployment\. The client now exposes explicit \pause\_deployment\ and \resume\_deployment\ methods (with the previous \\_set\_deployment\_paused\_state\ marked as deprecated) and normalizes empty \parameter\_openapi\_schema\ values to ensure valid OpenAPI format. Additionally, concurrency options are serialized using \exclude\_unset\ to avoid sending unset \grace\_period\_seconds\ fields.
_src/prefect/client/orchestration/\deployments · high confidence
New internal compatibility utilities for Prefect 3.0 migration and async handling
This change introduces a new \prefect.\_internal.compatibility\ package containing utilities to support the Prefect 3.0 upgrade and modernize internal async patterns. It adds \async\_dispatch\, a decorator that allows methods to automatically run synchronously or asynchronously based on the execution context, replacing the older \sync\_compatible\ approach. It also provides \migration.py\ with \MOVED\_IN\_V3\ and \REMOVED\_IN\_V3\ mappings to guide users through import path changes and removed classes (like \Deployment\ and \PrefectAgent\) with helpful error messages. Additional helpers include \deprecated\_paths.py\ for managing deprecated module re-exports, \backports.py\ for Python version-specific features (like TOML parsing), and \starlette.py\ to maintain compatibility with renamed HTTP status codes in newer Starlette versions.
_src/prefect/\internal/compatibility · high confidence
New structured settings system with TOML profiles and pydantic models
Prefect replaces its legacy flat settings with a new hierarchical system built on Pydantic models, allowing settings to be organized into logical groups (such as API, server, and client) and accessed via nested object paths (e.g., \settings.server.api.host\). Users can now configure Prefect using \prefect.toml\ or \pyproject.toml\ files, and manage environment-specific configurations through named profiles stored in \profiles.toml\. The system maintains backward compatibility by supporting legacy environment variable names and providing a \temporary\_settings\ context manager that accepts both old-style setting objects and new dotted-path string keys.
src/prefect/settings · high confidence
PostgreSQL database schema migrations for block and flow run structures
This update applies a series of PostgreSQL database migrations to the Prefect server schema. It introduces tables for block types, schemas, and documents, replacing the legacy block data structure with a more robust schema-based model including checksums and capabilities. It also adds support for flow run notification policies and queues, renames alert tables to notification tables, and backfills state names for flow and task runs to improve query performance and data consistency.
_src/prefect/server/database/\migrations/versions/postgresql · high confidence
Prefect CLI is now powered by cyclopts
The Prefect command-line interface has migrated from Typer to cyclopts, introducing native lazy loading of command modules to significantly improve CLI startup performance. This change restructures the CLI codebase (e.g., \\_app.py\, \\_utilities.py\) to use cyclopts for command registration and argument parsing while retaining Rich for console output. Users benefit from faster command execution and a more robust internal architecture, with no changes to the external command syntax or behavior.
src/prefect/cli · high confidence
Prefect Cloud CLI rewritten with Cyclopts and new management commands
The \prefect cloud\ command-line interface has been rebuilt using the Cyclopts framework, replacing the previous implementation. This change introduces new subcommands for managing Cloud resources: \prefect cloud asset\ (list and delete assets), \prefect cloud webhook\ (list, get, create, rotate, toggle, and update webhooks), and \prefect cloud ip-allowlist\ (enable, disable, list, add, and remove IP allowlist entries). The core \prefect cloud login\ workflow has also been updated to support interactive workspace selection, profile switching, and environment variable handling, while maintaining JSON output support across the new commands.
src/prefect/cli/cloud · high confidence
Prefect Email integration migrated to Pydantic v2 and core with inline image support
The \prefect-email\ integration has been refactored to use Pydantic v2, updating credential validation and configuration handling. The \EmailServerCredentials\ block now exposes a \verify\ option to control SSL certificate verification, allowing users to disable strict validation when necessary. Additionally, the \email\_send\_message\ task now supports embedding images directly into the email body via the new \inline\_images\ parameter, enabling richer HTML email content alongside existing attachment capabilities.
_src/integrations/prefect-email/prefect\email · high confidence
Prefect GCP workers migrated to core repository with Pydantic 2 and transient error retries
The GCP worker implementations (Cloud Run, Cloud Run v2, and Vertex AI) have been migrated from the separate prefect-gcp package into the core Prefect repository. This update upgrades the codebase to Pydantic 2, replacing custom type annotations with standard Optional types and updating field validators. Additionally, the Cloud Run v2 worker now includes robust retry logic for transient HTTP errors during job creation, submission, and execution polling, improving reliability in unstable network conditions.
_src/integrations/prefect-gcp/prefect\gcp/workers · high confidence
Prefect GitHub integration migrated to core repository with Pydantic v2 and security hardening
The \prefect-github\ integration has been moved into the core Prefect repository and rewritten to use Pydantic v2, introducing the \GitHubCredentials\ and \GitHubRepository\ blocks. This update includes several security and behavioral improvements: the \GitHubRepository\ block now preserves symlinks when cloning directories, uses a shared sanitizer to prevent GitHub token leakage in error messages, and guards against git argument injection. Additionally, async tasks now use \@async\_dispatch\ instead of the deprecated \@sync\_compatible\ decorator, and type hints have been updated to use \Optional\ for clarity.
_src/integrations/prefect-github/prefect\github · high confidence
Prefect GitLab integration moved to core with Pydantic v2 and credential formatting fixes
The prefect-gitlab integration has been migrated into the core Prefect codebase, upgrading its data models to Pydantic v2 and replacing the previous async compatibility layer with async\_dispatch. This update introduces a dedicated GitLabCredentials block to manage authentication, which now correctly formats personal access tokens with the required 'oauth2:' prefix while preserving existing deploy token formats to prevent authentication failures. Additionally, the integration now uses a shared output sanitizer to ensure that sensitive credentials are stripped from git command error messages before they are surfaced to the user.
_src/integrations/prefect-gitlab/prefect\gitlab · high confidence
Prefect Kubernetes integration restructured for Prefect 3.x
The \prefect-kubernetes\ package has been migrated to the core Prefect repository and updated for Prefect 3.x, introducing a new \@kubernetes\ decorator to bind flows to a Kubernetes work pool and exposing \KubernetesClusterConfig\ at the top level. The integration now uses \kubernetes-asyncio\ for all API interactions, replaces the old \@sync\_compatible\ pattern with \@async\_dispatch\, and adds structured pod-failure diagnostics that emit matchable event labels for automated handling.
_src/integrations/prefect-kubernetes/prefect\kubernetes · high confidence
Prefect SQLAlchemy integration restructured with Pydantic v2 and split connectors
The prefect-sqlalchemy integration has been migrated to Pydantic v2 and restructured to provide separate synchronous (SqlAlchemyConnector) and asynchronous (AsyncSqlAlchemyConnector) database connectors. The credential model now uses a ConnectionComponents class to build URLs from individual fields (driver, database, username, password, host, port, query), and the database parameter is now optional. New driver enums (AsyncDriver, SyncDriver) explicitly list supported database dialects, including Oracle support via oracledb.
_src/integrations/prefect-sqlalchemy/prefect\sqlalchemy · high confidence
Prefect Snowflake integration migrated to core repository with Pydantic 2 and new authentication support
The \prefect-snowflake\ package has been moved into the core Prefect repository, bringing the \SnowflakeCredentials\ and \SnowflakeConnector\ blocks up to date with Pydantic 2. This update introduces support for Workload Identity Federation (WIF) via a new \workload\_identity\_provider\ field and fixes validation for private key formats. Additionally, the \SnowflakeConnector\ now handles schema aliasing more robustly to prevent errors when loading previously saved configurations, and includes fixes for cursor management and fetch operations to ensure reliable database interactions.
_src/integrations/prefect-snowflake/prefect\snowflake · high confidence
Prefect client SDK restructured with new architecture and attribution headers
The \src/prefect/client\ module has been reorganized into a new structure featuring domain-specific orchestration submodules, a dedicated \CloudClient\, and WebSocket subscription support. A key behavioral addition is the automatic inclusion of attribution headers (e.g., \X-Prefect-Worker-Id\, \X-Prefect-Flow-Id\) in API requests to improve usage tracking and rate limit debugging. The HTTP client now includes robust retry logic with jitter, CSRF support, and informative error wrapping. Additionally, the client automatically starts a local server subprocess if no API URL is specified, and WebSocket subscriptions now provide clearer authentication error messages.
src/prefect/client · high confidence
Prefect server background services now run as distributed perpetual functions via Docket
Background services (scheduler, foreman, late runs, pause expirations, cancellation cleanup, db vacuum, repossessor, and cleanup reconciler) have been migrated from the legacy loop-based model to a Docket-powered perpetual service architecture. This change introduces a new \Service\ base class and a \perpetual\_service\ decorator that registers these services with Docket for distributed, high-availability scheduling. As a result, services are now managed by Docket's task engine, which provides automatic retries, error isolation, and deduplication via deterministic keys. Operators can control which services run in ephemeral or webserver modes via new settings, and services like DB vacuum now use dedicated database connections without statement timeouts to prevent premature termination during bulk maintenance.
src/prefect/server/services · high confidence
Prefect server models are rewritten to use SQLAlchemy 2 and emit lifecycle events
The server's database models have been completely rewritten to use the SQLAlchemy 2 ORM interface, replacing the previous \declarative\_mixin\ approach with \DeclarativeBase\. This change introduces a new eventing system where all domain objects (such as deployments, blocks, concurrency limits, and artifacts) now emit lifecycle events (created, updated, deleted) via the \PrefectServerEventsClient\. Additionally, the models now support client-side schema validation for block registration and utilize UUIDv7 for time-oriented object IDs.
src/prefect/server/models · high confidence
Prefect settings are reorganized into a structured Pydantic model hierarchy
The configuration system has been refactored from a flat structure into a nested hierarchy of Pydantic models (e.g., \ClientSettings\, \FlowsSettings\, \TasksSettings\, \PluginsSettings\). This change introduces new, structured settings paths such as \PREFECT\_CLIENT\_CUSTOM\_HEADERS\ and \PREFECT\_TASKS\_DEFAULT\_NO\_CACHE\, while maintaining backward compatibility through validation aliases for legacy environment variable names (e.g., \PREFECT\_CLIENT\_ENABLE\_METRICS\). The new structure also standardizes default behaviors, such as setting the default local storage path to \$PREFECT\_HOME/storage\ and enabling OS-level resource metric collection by default.
src/prefect/settings/models · high confidence
Prefect v1 concurrency now supports both synchronous and asynchronous execution
The concurrency v1 module has been refactored to provide explicit synchronous and asynchronous entry points. Users can now import \concurrency\ from \prefect.concurrency.v1.sync\ for use in standard functions or \prefect.concurrency.v1.asyncio\ for async contexts. This change replaces the previous \@sync\_compatible\ decorator pattern with a dedicated \@async\_dispatch\ mechanism, ensuring that slot acquisition and release logic correctly handles both sync and async clients without relying on implicit runtime detection.
src/prefect/concurrency/v1 · high confidence
Prefect-Dask integration migrated to core repository with new client and task runner implementations
The \prefect-dask\ package has been moved into the core Prefect repository, introducing a new \PrefectDaskClient\ that wraps Dask futures to integrate with Prefect's task engine, allowing Prefect \Task\ objects to be submitted directly to Dask. The \DaskTaskRunner\ has been updated to support lazy client loading, custom cluster class instantiation, and performance report generation, while utility functions \get\_dask\_client\ and \get\_async\_dask\_client\ are now available to help users distribute work across Dask workers within tasks.
_src/integrations/prefect-dask/prefect\dask · high confidence
Prefect-Ray integration migrated to core with new context and future handling
The prefect-ray integration has been moved into the core codebase, introducing a new \RayTaskRunner\ and \PrefectRayFuture\ implementation. This update includes a new \remote\_options\ context manager that allows users to dynamically pass keyword arguments (such as CPU/GPU limits) to Ray \@remote\ calls within a flow scope. The task runner now uses lazy imports for the \ray\ library to prevent startup issues and ensures the Ray driver is only shut down in the task runner's exit if it was the one that started it, preventing conflicts when running on existing Ray workers.
_src/integrations/prefect-ray/prefect\ray · high confidence
Prefect-dbt config models migrated to Pydantic v2 with profiles.yml support
The \prefect\_dbt.cli.configs\ module has been rewritten to use Pydantic v2, introducing new \TargetConfigs\, \BigQueryTargetConfigs\, \PostgresTargetConfigs\, and \SnowflakeTargetConfigs\ models. This update adds a \from\_profiles\_yml\ class method to \TargetConfigs\, allowing users to instantiate configuration objects directly from a dbt \profiles.yml\ file. The config generation logic now respects Pydantic \serialization\alias\ fields (such as \schema\\ mapping to \schema\) and correctly handles Snowflake private key secrets. Additionally, BigQuery and Postgres config exporters now explicitly map Prefect connector fields to the specific keys expected by dbt profiles, ensuring compatibility with the dbt CLI.
_src/integrations/prefect-dbt/prefect\dbt/cli/configs · high confidence
Refactored Flow and FlowRun CRUD methods in client
The flow client implementation has been restructured into a dedicated module (\src/prefect/client/orchestration/\_flows\), introducing explicit \FlowClient\ and \FlowAsyncClient\ classes. This change consolidates flow-related operations such as creating, reading, filtering, and deleting flows, while also documenting the default server limit behavior for client queries to improve clarity for users interacting with the API.
_src/prefect/client/orchestration/\flows · medium confidence
Refactored Variable CRUD operations into dedicated client classes
The variable management logic has been reorganized into new \VariableClient\ and \VariableAsyncClient\ classes within the orchestration client module. This change introduces specific methods for creating, reading, updating, and deleting variables (including reading by name and filtering), replacing the previous implementation structure. Users interacting with variables via the client will now use these dedicated classes, which handle HTTP requests to the \/variables/\ endpoints and map responses to \Variable\ schema objects, while preserving existing error handling for conflicts and not-found scenarios.
_src/prefect/client/orchestration/\variables · high confidence
Refactored block schema and type client methods
The client methods for managing block schemas and block types have been reorganized into dedicated modules (\\_blocks\_schemas\ and \\_blocks\_types\). This change improves code structure and maintainability without altering the external API behavior for users interacting with block definitions.
_src/prefect/client/orchestration/\_blocks\_schemas, src/prefect/client/orchestration/\_blocks\types · high confidence
Refactored database layer with unified interface and SQLAlchemy 2.0 support
The database module has been restructured to provide a unified \PrefectDBInterface\ that abstracts backend-specific SQLAlchemy configurations, ORM models, and query components for both PostgreSQL and SQLite. This change introduces SQLAlchemy 2.0-style ORM models using \DeclarativeBase\, replaces the legacy \OrionDBInterface\ with the new \PrefectDBInterface\, and adds configurable connection pool settings (such as pool size, max overflow, and application name) via server settings. The refactoring also includes thread-safe Alembic migration commands and improved type safety across the database layer.
src/prefect/server/database · high confidence
Refactored flow run client into a dedicated module
The flow run client logic has been reorganized into a new \src/prefect/client/orchestration/\_flow\runs\ package, splitting the implementation into \\\init\\_.py\ and \client.py\. This change introduces a dedicated \FlowRunClient\ class that handles creating, updating, and deleting flow runs, including support for specifying work pools, work queues, and job variables during creation. This modularization separates flow run orchestration details from the broader client interface.
_src/prefect/client/orchestration/\_flow\runs · high confidence
Refactored orchestration engine into modular policy and rule components
The orchestration logic in \src/prefect/server/orchestration\ has been restructured to improve modularity and observability. The core state-transition rules are now organized into distinct policy classes (\CoreFlowPolicy\, \CoreTaskPolicy\, \GlobalFlowPolicy\, \GlobalTaskPolicy\) that define the priority order of execution. New universal transforms handle bookkeeping tasks such as ensuring unique state timestamps (\EnsureFlowRunStateTimestampIsUnique\) and managing concurrency lease releases (\\_release\_concurrency\_lease\). Additionally, a new instrumentation policy (\InstrumentFlowRunStateTransitions\) automatically emits Prefect Server Events for every flow run state change, enhancing observability without requiring user configuration.
src/prefect/server/orchestration · high confidence
Refactored work pool client into a dedicated module
The work pool client logic has been reorganized into a new \src/prefect/client/orchestration/\_work\pools\ package, splitting the implementation into \\\init\\_.py\ and \client.py\. This change consolidates methods for managing work pools and workers—such as reading work pools, creating pools, filtering workers, and sending heartbeats—into a single, structured client class, improving code maintainability without altering the external API surface.
_src/prefect/client/orchestration/\_work\pools · high confidence
Reintroduce Databricks job models for Prefect integration
The \prefect-databricks\ integration now includes generated Pydantic models for Databricks Jobs (based on the 2.1 AWS API specification). This restores the data structures required to define and manage Databricks jobs within Prefect flows, ensuring compatibility with the Databricks API for job configuration, cluster settings, and permissions.
_src/integrations/prefect-databricks/prefect\databricks/models · high confidence
Restructured logging module with new public API and configuration entry points
The logging infrastructure has been reorganized into the \src/prefect/logging\ package, introducing a cleaner public API and more robust configuration management. Users can now access the standard logger via \prefect.logging.get\_logger()\ and the run-context-aware logger via \prefect.logging.get\_run\_logger()\, along with testing utilities like \LogEavesdropper\ and \disable\_run\_logger()\. The module now includes a dedicated \ensure\_logging\_setup()\ function to safely initialize logging in remote execution environments (such as Dask or Ray workers) where the standard import path might not trigger configuration. Additionally, the logging configuration system has been enhanced to support incremental updates, allowing user-defined handlers to be preserved across multiple setup calls, and the \logging.yml\ template now supports environment variable overrides for granular control over log levels, formatters, and handlers.
src/prefect/logging · high confidence
SDK telemetry and analytics infrastructure added
Prefect now collects anonymous usage data to improve the product. This change introduces an internal analytics module that tracks SDK milestones (such as first import, first flow run, and first deployment) and provides a public API for integration libraries to emit their own telemetry events. Telemetry is opt-out: it respects the DO\_NOT\_TRACK environment variable, is automatically disabled in CI environments, and displays a one-time notice to new users in interactive terminals. Existing users are detected via local artifacts and will not see the notice or receive onboarding events.
_src/prefect/\internal/analytics, src/prefect/analytics · high confidence
Server package exports are lazily loaded to improve startup performance
The \prefect.server\ package now uses lazy loading for its submodules (\models\, \orchestration\, \schemas\, \services\). This change reduces the initial import overhead and memory footprint when the server package is imported, as submodules are only loaded when they are first accessed. This is a performance optimization for server startup and does not change the public API.
src/prefect/server · high confidence
Shell command execution now kills entire process trees and preserves sync output logs
The \prefect-shell\ integration has been updated to improve reliability and observability when running shell commands. On POSIX systems, spawned shell processes are now isolated in their own process groups, ensuring that when a flow run is cancelled or cleaned up, the entire process tree (including any detached child processes like background sleeps) is terminated rather than leaving orphaned processes running. Additionally, the \shell\_run\_command\ task now correctly preserves and streams output logs to the API even when executed in synchronous contexts, ensuring that users can see the full output of their shell commands in the Prefect UI regardless of how the task is invoked.
_src/integrations/prefect-shell/prefect\shell · high confidence
Support for AnyIO 4.14 task group interface
The async utilities in \src/prefect/utilities/asyncutils\ have been updated to support the AnyIO 4.14 task group interface. This change ensures compatibility with the latest AnyIO version, allowing users to leverage new concurrency primitives and interface updates without breaking existing async workflows.
src/prefect/utilities/asyncutils · medium confidence
Workers module restructured with new cleanup, healthcheck, and configuration components
The \src/prefect/workers\ package has been reorganized to support the work-pool execution model. A new \BaseWorker\ abstract base class now handles core lifecycle, heartbeating, and attribution, while \ProcessWorker\ delegates flow execution to the \FlowRunExecutor\. A dedicated cleanup subsystem (\\_cleanup.py\, \\_cleanup\_handlers.py\) manages idempotent infrastructure teardown for cancelled runs, and a lightweight healthcheck webserver (\server.py\) provides a \/health\ endpoint for monitoring worker responsiveness. Configuration is standardized via \BaseJobConfiguration\ and \ProcessJobConfiguration\, which handle environment variable coercion and command preparation.
src/prefect/workers · high confidence
Workers now skip in-process retries when fetching scheduled flow runs
The SQL queries used to retrieve scheduled flow runs for workers have been updated to explicitly exclude runs with an in-process retry type. By filtering out these runs, workers will no longer attempt to execute flow runs that are already being retried within the same process, preventing duplicate execution attempts.
src/prefect/server/database/sql · high confidence
prefect-gcp integration moved to core repository with build and documentation tooling
The \prefect-gcp\ integration has been migrated into the core \prefect\ repository. This change introduces a Dockerfile for building container images with pinned versions of \prefect\ and \prefect-gcp\, along with a smoke test to verify installation. Additionally, standard package metadata files (LICENSE, MANIFEST.in) and a \justfile\ for running tests and generating API reference documentation via \mdxify\ have been added to support the integration's lifecycle within the main codebase.
src/integrations/prefect-gcp · high confidence
prefect-slack integration added to core with Python 3.10+ requirement
The prefect-slack integration has been migrated into the core repository, bringing with it official support for Prefect 3.x and a minimum Python version of 3.10. This change includes the addition of standard package metadata files (LICENSE, MANIFEST.in, README.md) and an automated API reference documentation generation setup via mdxify, ensuring the integration is now maintained as a first-party component with updated licensing and documentation infrastructure.
src/integrations/prefect-slack · high confidence
Test coverage
Add benchmarking suite for PrefectDbtOrchestrator; Added comprehensive test coverage for prefect-dbt core orchestration components; Added comprehensive test suite for ECS Worker; Added comprehensive test suite for prefect-gcp integration; Added integration test harness for Kubernetes worker; Added proxy-test script to validate WebSocket connections through HTTP proxies; Added standard test suites for Blocks and Worker Cleanup Queues; Added test project flow for prefect-docker integration; Added test suite for prefect-azure integration; Added tests for AWS ECS decorator and bundle execution; Added tests for Azure Blob Storage bundle execution and upload; Added tests for Azure Blob Storage deployment steps; Added tests for ECS worker CLI, observer diagnostics, and crash detection; Added tests for GCP infrastructure decorators; Added tests for GCS bundle upload and execution; Added tests for Prefect AWS deployment steps; Added tests for dbt CLI configuration classes; Added tests for dbt Cloud integration components; Added tests for prefect-dbt CLI commands and credentials; Added tests for the @docker decorator and its configuration options; Added tests for the Kubernetes flow decorator; Added unit tests for the experimental SPCS worker; Comprehensive test suite for prefect-redis integration; Expanded test coverage for Prefect AWS integration; Extensive test suite updates and fixes; Initial test suite for prefect-snowflake integration; New integration test suite for Prefect orchestration behaviors; New testing utilities and fixtures for local server testing.
Dependencies
Update Node.js to v24
The project's Node.js runtime requirement has been upgraded to version 24.
(dependencies) · high confidence
Vendor croniter library for cron expression handling
The croniter library has been vendored into src/prefect/\_vendor/croniter. This adds the croniter module and its associated classes and functions (such as CroniterError, croniter, and croniter\_range) directly into the Prefect codebase, enabling internal cron expression parsing and scheduling logic without relying on an external package dependency at runtime.
_src/prefect/\vendor · high confidence
Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Baseline
- First survey — no prior run to compare against. CAI 54.
Lenses
- Code Health 84
- Architecture 61
- Maturity 70
- Readiness 45
- Security 57
- Domain Modelling 100
- Accessibility 57
Changes since last survey
- 300 commits — 233 feature/other, 67 fixes
By area
- ui-v2/src — 63 commits
- src/prefect — 60 commits
- (root) — 36 commits
- docs/v3 — 31 commits
- src/integrations — 28 commits
- .github/workflows — 21 commits
- ui-v2/package-lock.json — 21 commits
- ui-v2/e2e — 6 commits
- ui/package-lock.json — 4 commits
- tests/cli — 3 commits
- tests/server — 3 commits
- docs/docs.json — 2 commits
- docs/integrations — 2 commits
- integration-tests/test_concurrency_leases.py — 2 commits
- load_testing/local-telemetry — 2 commits
- tests/events — 2 commits
- ui-v2/AGENTS.md — 2 commits
- (repo) — 1 commit
- .github/CODEOWNERS — 1 commit
- .github/dependabot.yml — 1 commit
Notable commits
- fix: fix(server): fix database locked when marking deployments ready (#22967)
- fix: (ui-v2) Fix inconsistent deployment status display on flow detail page (#22901)
- fix: Add deployments-list regression test for the quick run enum dropdown (#22955)
- fix: Bugfix: automations SDK filter model (#22945)
- fix: Fix CI pyright check that silently analyzed 0 files (#22827)
- fix: Fix Docker work pool editing and block/env serialization in UI v2 (#23067)
- fix: Fix ProcessWorker auto-uv after pull steps (#22857)
- fix: Fix V2 run-list wrapping and toolbar overflow at constrained widths (#23115)
- fix: Fix always-true shell assertions in prefect-shell tests (#23144)
- fix: Fix as_completed timeout enforcement in worker threads and on Windows (#22607)
- fix: Fix codspell error on unit test TEST_BLOCK_CODE_BAD_SYNTAX on Windows (#22642)
- fix: Fix dashboard flow run query ordering (#22917)
- fix: Fix date-dependent failure in interval schedule form test (#23125)
- fix: Fix db_vacuum retention-override full scan of event_resources (#22536)
- fix: Fix docstring Args entries that name a parameter the function does not take (#22733)
- fix: Fix flaky blocks E2E test: scope block cleanup to a single test (#22892)
- fix: Fix flaky deployment detail e2e tests: scope details assertions to the sidebar (#22880)
- fix: Fix flaky e2e test: scope deployment name assertion to breadcrumb in runs tab test (#22878)
- fix: Fix lateness label on flow run cards in ui-v2 (#22896)
- fix: Fix nested deployment parameter rendering (#22760)
- …and 280 more
Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
PrefectHQ/prefect was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 19 September 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 96d33e5326214a20de35000b01b859951f63e2e0 — the exact code this score is about.
- Scored under rubric-2026.09.15 — the same rubric and the same method as every other entry in this index.
- Measured by watchdog.canine.dev using codehealth-analyzer preprod-5d04157a340d.