comet-ml/opik
46.9
Weak · 17 November 2026
655.6k
lines of production code
TypeScript
with Python, Java
5
measurements over time
What this system is
Features
Add Ollama LLM provider support
The backend now supports Ollama as a configurable LLM provider. This change introduces the Ollama service provider and its Guice module, registering it under the OLLAMA provider type. Since Ollama uses an OpenAI-compatible API, the implementation reuses the existing CustomLlmClientGenerator, allowing users to configure Ollama endpoints (typically including the /v1 suffix) for inference and evaluation tasks.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/ollama · high confidence
Add OpenTelemetry instrumentation utilities for async operations
The backend now includes a new utility class, InstrumentAsyncUtils, which provides methods to create and manage OpenTelemetry spans for asynchronous reactive operations. This enables distributed tracing for specific product operations by automatically handling span creation, context propagation, and cleanup on a bounded elastic scheduler.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/instrumentation · high confidence
Add TypeScript SDK evaluation reference documentation
Added new documentation pages for the Opik TypeScript SDK evaluation module, covering datasets, the evaluate and evaluatePrompt functions, metrics, models, experiments, and test suites. These pages provide API references, usage examples, and configuration details for evaluating LLM applications using the SDK.
apps/opik-documentation/documentation/fern/docs-v2/reference/typescript-sdk/evaluation · high confidence
Added Java infrastructure for Python RQ queue integration
New classes in the \opik-backend\ queue infrastructure package now enable the Java service to produce and manage jobs for the Python RQ (Redis Queue) worker. This includes data models for job definitions and statuses, a mapper that formats timestamps with microsecond precision to ensure compatibility with Python RQ's parsing, and utilities to build the specific Redis HASH structures required by the queue. A dedicated queue configuration class is also added to manage job TTL settings.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/queues · high confidence
Anthropic provider integration with adaptive-thinking support
The Anthropic LLM provider is now fully integrated into the backend, introducing support for the latest Claude models (including Sonnet 4.5, Opus 4.5/4.6, and Haiku 4.5). The implementation enforces strict parameter validation for Anthropic's adaptive-thinking models, automatically gating temperature and top\_p sampling parameters to prevent API errors. It also handles extended thinking configurations by managing \max\_tokens\ and \thinking\_budget\_tokens\ to ensure valid request structures, while supporting multimodal content and custom base URLs.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/antropic · high confidence
Backend utility library for async, JSON, and error handling
The backend now includes a new set of utility classes in the \com.comet.opik.utils\ package to standardize common operations. \AsyncUtils\ manages Reactor context propagation for user and workspace identity and provides safe lifecycle handling for ClickHouse v2 client futures to prevent connection leaks. \JsonUtils\ centralizes the Jackson \ObjectMapper\ configuration, including stream-read constraints and custom deserializers for OpenAI messages and durations. \ChunkedOutputHandlers\ formats Server-Sent Events (SSE) with the \data: \ prefix and \\[DONE\]\ marker for OpenAI SDK compatibility. \ErrorUtils\ handles ClickHouse JSON path parsing errors gracefully, and \HttpStatusRetryability\ defines which HTTP statuses are transient for retry logic. Additional utilities include \FileNameUtils\ for safe Content-Disposition headers, \SentinelTranslation\ for ClickHouse sentinel-to-null mapping, and \RetryUtils\ for database and HTTP error retries.
apps/opik-backend/src/main/java/com/comet/opik/utils · high confidence
Cookbook pages now feature direct links to run notebooks in Google Colab
The documentation now includes a client-side banner on specific cookbook pages (Quickstart, Hallucination Metric, Moderation Metric, and Dynamic Tracing Control) that provides one-click buttons to open the corresponding Jupyter notebooks in Google Colab or view them on GitHub. This is implemented via a new JavaScript snippet (\add\_cookbook\_banner.js\) injected into the Fern documentation site, which detects the current page and dynamically inserts a callout with the appropriate notebook links.
apps/opik-documentation/documentation/fern · high confidence
Cost Intelligence documentation added
Added comprehensive documentation for the Cost Intelligence feature, covering the installation overview and three rollout paths (MDM, managed settings, and macOS app), a detailed data privacy and security page specifying that only counts and metadata are collected, a guide on reducing coding agent spend through policy-driven savings, and a page for querying spend via the Opik MCP server.
_apps/opik-documentation/documentation/fern/docs-v2/cost\intelligence · high confidence
Database schema evolution for core Opik features
This update introduces a comprehensive Liquibase migration suite that establishes and evolves the application's database schema to support key platform capabilities. It adds tables and columns for prompt library versioning (including Jinja2/Mustache types and tags), LLM provider configuration (supporting custom providers, base URLs, and headers), and dataset versioning with Git-like tagging and evaluation suite support. The schema also enables online scoring automation rules with span-level evaluators, project-scoped alerts with webhook integrations (Slack/PagerDuty), workspace dashboards, and trace thread tracking. Additionally, it introduces visibility flags for projects and datasets, boolean feedback types, and optimized indexing for performance-critical queries.
apps/opik-backend/src/main/resources/liquibase/db-app-state · high confidence
Docker Compose profiles and local development overrides
The Docker Compose deployment now supports service profiles to start specific combinations of services for different development scenarios. Users can start only infrastructure services, backend services, the full Opik suite, or enable optional components like Guardrails and OpenTelemetry using the \--profile\ flag. Additionally, new override files (\docker-compose.local-be.yaml\, \docker-compose.local-be-fe.yaml\) allow developers to run the backend and frontend locally while using Docker Compose for other services, with port mappings exposed via \docker-compose.override.yaml\.
deployment · high confidence
Document online evaluation rules, Agent as a Judge, and Jev classifier support
The documentation now includes a dedicated page for online evaluation rules, explaining how to define scoring metrics for production traces via the UI or REST API. It details two data ingestion methods for trace and span rules: 'Agent as a Judge', which uses tools to read trace data dynamically, and 'Variable mapping', which points prompt variables to specific trace fields. The entry also documents the integration of Jev as a classifier judge for fast, low-cost pass/fail checks, specifying its constraints (Boolean scores, text-only prompts) and configuration via OpenRouter.
apps/opik-documentation/documentation/fern/docs-v2/production/online-evaluation · high confidence
Documentation for LLM Guardrails feature
Added comprehensive documentation for the new LLM Guardrails capability, covering the five guard types (PII, Topic, Prompt Injection, LLM Judge, and Custom Classifier), how to define and use stored policies, and how to fine-tune custom models for specific checks. The new pages also detail the self-hosted Guardrails server configuration, including installation, environment variables, and API usage. Also in this area: Improved code block readability in documentation.
apps/opik-documentation/documentation/fern/docs-v2/guardrails · high confidence
Documentation for data anonymization and updated permission model
Added a new 'Data Anonymization' page explaining how Opik Enterprise masks sensitive values (like emails and phone numbers) in trace data for users without the 'Original data view' permission, and updated the 'Roles and Permissions' and 'Administration Overview' pages to reflect the new permission structure and the preview status of this feature.
apps/opik-documentation/documentation/fern/docs-v2/administration · high confidence
Documentation for the Opik Admin Dashboard and its sections
Added documentation pages for the Opik Admin Dashboard, covering how to access the central hub for organization management and detailing the specific sections: Workspaces (creating, viewing, deleting), Users (viewing, role changes, removal), Service Accounts (creation, API key management, usage), Opik Usage (tracking span consumption, filtering, CSV export), Organization Settings (restricting user invitations), and Billing. These pages guide administrators on managing organization-wide configuration, access control, and usage monitoring.
apps/opik-documentation/documentation/fern/docs-v2/administration/admin-dashboard · high confidence
Documentation: Changelog entries for Opik releases from September 2024 through October 2025
The documentation now includes weekly changelog entries covering Opik releases from 2024-09-30 through 2025-10-21. These entries document new features and enhancements across the Opik Dashboard (such as the Playground, Prompt Library, Agent Optimizer, Thread-level LLM-as-a-Judge metrics, Annotation Queues, and Dark Mode), SDK updates (including integrations with LangChain, Google ADK, Anthropic, and many others, plus new methods like \get\_or\_create\_dataset\ and \search\_traces\), and deployment improvements. This provides a comprehensive, date-ordered record of product changes for users. Also in this area: Documentation updates for prompt environments, MCP server setup, and alert custom fields; Improved code block readability in documentation.
apps/opik-documentation/documentation/fern/docs-v2/changelog · high confidence
Initial Opik backend application entry point
The Opik backend application entry point (OpikApplication) is introduced, establishing the core server bootstrap. This includes configuring Jackson JSON serialization (snake\_case naming, non-null inclusion, date/duration formatting), registering Guice modules for infrastructure components (database, Redis, caching, rate limiting, LLM providers like OpenAI/Gemini/Ollama, auth, and jobs), and applying Jackson stream-read constraints to protect against oversized payloads. It also sets up Liquibase for database migrations, multi-part form support, and specific exception mappers for JSON and OAuth errors.
apps/opik-backend/src/main/java/com/comet/opik · high confidence
Initial Python SDK documentation site
The Python SDK documentation site is now available, providing users with installation instructions, usage guides for tracing and evaluation, and a complete API reference. The site includes dedicated sections for the Opik client, CLI configuration, and the track decorator, alongside integration documentation for major frameworks such as LangChain, OpenAI, Anthropic, and Bedrock.
apps/opik-documentation/python-sdk-docs/source · high confidence
Initial backend resource configuration and data files
The backend application now includes its foundational resource files, establishing the initial configuration and data layer. This includes a custom application banner, a comprehensive default LLM model registry (\llm-models-default.yaml\) defining supported models and capabilities, and detailed pricing and context window data (\model\_prices\_and\_context\_window.json\ and \model\_prices\_overrides.json\) for cost tracking. Additionally, the REST API documentation template (\openapi\_template.yml\) is provided, and the Quartz scheduler is configured via \quartz.properties\ to use daemon threads and a RAM-based job store.
apps/opik-backend/src/main/resources · high confidence
Initial backend scaffolding and configuration
The Opik backend module is introduced with its foundational build and runtime configuration. This includes a Dockerfile that upgrades the base image to Amazon Corretto 25 (Java 25) and hardens the image by removing Perl to address [CVE redacted]. The service is configured via a new \config.yml\ that defines connection parameters for MySQL and ClickHouse, along with toggles for the new trace/span data model migration. Additionally, the entrypoint script now supports OpenTelemetry Java agent integration and virtual threads, while a new \ObservabilityTraceRecorder\ component enables internal backend flows to be recorded as hidden traces and spans for monitoring purposes.
apps/opik-backend · high confidence
Initial database schema for the app analytics store
This change introduces the foundational Liquibase migration scripts for the new app analytics database (ClickHouse). It defines the core data model required for analytics, including tables for spans, traces, feedback scores, dataset items, experiments, and experiment items. It also adds support for newer capabilities such as comments, attachments, guardrails, optimizations, trace threads, and annotation queues, while refining data types like decimal precision and timestamp resolution to support accurate cost tracking and high-frequency updates.
apps/opik-backend/src/main/resources/liquibase/db-app-analytics · high confidence
Introduce Agent Blueprint and Configuration domain models
Added new domain records (AgentBlueprint, AgentConfig, AgentConfigEnv, AgentConfigValue) and their associated DAOs and services to support the Agent Configuration Management System. This change introduces the data structures for managing agent blueprints (including mask types), configuration values with typed support (string, integer, float, boolean, prompt, prompt\_commit), and environment bindings, laying the groundwork for the backend logic to create, update, and query agent configurations.
apps/opik-backend/src/main/java/com/comet/opik/domain · high confidence
Introduce Evolutionary Optimizer with DEAP-based genetic algorithms
The Opik Optimizer SDK now includes an EvolutionaryOptimizer that uses genetic algorithms to evolve and improve prompts over multiple generations. This new capability supports both single-objective and multi-objective optimization (balancing performance and prompt length), featuring operations like crossover, mutation, and selection via the DEAP library. It allows users to iteratively refine prompts by evaluating candidate populations against a dataset, with options for adaptive mutation rates, semantic crossover, and early stopping based on stagnation.
_sdks/opik\optimizer · high confidence
Introduce Local Runner API data models
Added a comprehensive set of Java record and enum types in the backend runner package to support the Local Runner feature. These models define the request and response structures for agent execution (including job creation, status updates, and result reporting), runner registration and heartbeat communication, and a command bridge for file and execution operations. The changes also introduce specific enums for runner and job statuses, command types, and parameter presence, establishing the data contract for the new local execution capability.
apps/opik-backend/src/main/java/com/comet/opik/api/runner · high confidence
Introduce Opik Guardrails Backend service with PII, topic, prompt injection, and custom classifier validations
A new backend service for Opik Guardrails is introduced, providing an API to validate text against PII, restricted topics, prompt injection, and user-trained custom classifiers. The service is packaged with Dockerfiles for both GPU (CUDA) and CPU-only environments, runs as a non-root user, and exposes a \/api/v1/guardrails/validations\ endpoint for combined checks and a \/custom/train\ endpoint to fine-tune and serve custom binary classifiers via LoRA adapters.
apps/opik-guardrails-backend · high confidence
Introduce Opik pytest plugin for LLM unit testing
The Opik pytest plugin is now available, allowing users to automatically track and evaluate LLM-based tests. By marking test functions with the \@llm\_unit\ decorator and running pytest with the \--opik\ flag (or enabling it in pytest.ini), test inputs, expected outputs, and metadata are captured as traces. Upon test completion, results are logged to a 'tests' dataset and an experiment is created in Opik, with a summary printed to the terminal showing pass/fail counts and a link to the Opik UI.
sdks/python/src/opik/plugins/pytest · high confidence
Introduce anonymous usage reporting and product analytics infrastructure
The backend now collects anonymous usage data and product analytics events to help improve the product. This includes a daily usage report sent at midnight (tracking total/daily users, traces, experiments, and datasets), an installation/startup event sent once per instance, and a first-trace-created event sent once per workspace. Product analytics events (e.g., first trace created) are also tracked when enabled. All reporting is opt-out: it requires usage reporting or analytics to be enabled in configuration, uses a persistent anonymous ID per installation, and is fully asynchronous and non-blocking to avoid impacting API traffic.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/bi · high confidence
Introduce backend cost calculation service with provider-specific pricing and caching logic
The backend now includes a new CostService that calculates span and trace costs based on token usage, supporting a wide range of providers (OpenAI, Anthropic, Bedrock, Google, Azure, etc.) via a centralized model price table. The service handles provider name normalization, model name variations (dot-to-hyphen, date suffixes, provider prefixes), and applies tiered pricing for long contexts. It also implements provider-specific logic for caching discounts (e.g., Anthropic, OpenAI, Google), audio token billing, and reasoning token billing, ensuring accurate cost tracking for LLM interactions.
apps/opik-backend/src/main/java/com/comet/opik/domain/cost · high confidence
Introduce data anonymization and agent configuration APIs
The Python SDK now includes a new anonymization subsystem (opik.anonymizer) with an abstract Anonymizer base class, a RecursiveAnonymizer implementation, and a factory for creating anonymizers, allowing users to redact sensitive data before it is sent to the backend. Additionally, a new agent configuration API is exposed under opik.api\_objects.agent\_config, providing a ConfigManager, a Config data class, a blueprint system, and an agent\_config\_context manager to programmatically manage and scope agent configurations.
sdks/python · high confidence
Introduce data retention policy configuration
Users can now define data retention policies at the organization, workspace, or project level. This change adds the core API models (RetentionLevel, RetentionPeriod, and RetentionRule) that support specifying retention durations (14 days, 60 days, 400 days, or unlimited) and applying rules to past data, laying the groundwork for automated data cleanup.
apps/opik-backend/src/main/java/com/comet/opik/api/retention · high confidence
Introduce dedicated data access and mapping layers for automation rules and evaluator logs
The backend now includes new DAOs and mappers in the evaluators domain to support the underlying data model for automation rules (including annotation queue routers and various evaluator types like LLM-as-Judge and Python metrics) and their associated event logs. This change adds the persistence logic for storing and retrieving rule configurations, project associations, and log entries, enabling the features that allow users to configure, enable, and monitor automated evaluation rules.
apps/opik-backend/src/main/java/com/comet/opik/domain/evaluators · high confidence
Introduce dedicated trace thread management and lifecycle support
This change establishes trace threads as first-class entities within the system, introducing a new domain layer to manage their full lifecycle. Users can now explicitly open and close threads, with the backend automatically handling the creation of thread records and resolving external thread IDs to internal model IDs. The update includes mechanisms for auto-closing inactive threads based on configurable workspace-level timeouts, as well as support for online scoring and sampling of threads upon closure. Additionally, the underlying data access has been optimized for performance, replacing expensive deduplication queries with more efficient aggregation strategies to handle high-volume trace data.
apps/opik-backend/src/main/java/com/comet/opik/domain/threads · high confidence
Introduce extensible logging table factory for user-facing logs
The backend now uses a new \UserLogTableFactory\ to manage database access for user-facing log entries. This factory maps specific log types, such as automation rule evaluator logs and alert event logs, to their respective Data Access Objects, enabling the system to persist these distinct log categories into the underlying storage.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/log/tables · high confidence
Introduce guardrails validation schemas and provider model sync infrastructure
This change introduces the backend schema definitions for the new guardrails feature, adding Pydantic models for PII, topic, prompt injection, and custom classifier validations in \opik-guardrails-backend/opik\_guardrails/schemas.py\. It also adds a new Python script \scripts/sync\_provider\_models.py\ to automatically synchronize LLM provider model definitions (from OpenRouter, OpenAI, Anthropic, Gemini, and Vertex AI) into backend Java enums and frontend TypeScript files, ensuring the playground only exposes runnable models. Additionally, a utility script \scripts/count\_rule\_tokens.py\ is added to report token usage for agent rules.
python · high confidence
Introduce opik-python-backend service with isolated code execution and Optimization Studio support
This change introduces the new opik-python-backend service, which provides a sandboxed environment for running Python code (via Docker containers or isolated subprocesses) and powers the server-side logic for Optimization Studio. The service includes a multi-stage Dockerfile (using uv for dependency management and libexpat for security), an entrypoint that manages the Docker daemon and Gunicorn workers, and core application code for handling online scoring evaluations and asynchronous optimization jobs via Redis. It also ships with demo data generation capabilities and comprehensive documentation for the isolated executor architecture.
apps/opik-python-backend · high confidence
Introduce project-scoped event infrastructure and agent blueprint configuration models
The backend now includes a new event base class, EntityProjectEvent, which allows events for spans, traces, comments, and feedback scores to carry an optional project ID for downstream scoping. Additionally, new domain models for AgentBlueprint, AgentConfigEnv, and AgentConfigValue are added to support agent configuration management, including environment variables and typed configuration values. Also in this area: Introduce Agent Blueprint and Configuration domain models.
apps/opik-backend/src/main/java/com/comet/opik/domain/retention · high confidence
Introduce structured alert and webhook event models
The backend now uses dedicated data models to represent alert and webhook events, enabling more structured and reliable delivery of notifications to external endpoints. AlertEvent defines the core context for an alert (type, workspace, user, and a generic payload), while WebhookEvent provides a typed, serializable structure for the actual webhook payload, including metadata, retry configuration, and target URL details. Additionally, MetricsAlertPayload introduces a specific DTO for metrics-based alerts, exposing both scalar fields for backward compatibility and a list of contributing conditions to support complex AND-logic threshold evaluations.
apps/opik-backend/src/main/java/com/comet/opik/api/events/webhooks · high confidence
Introduces a new, unified filter API for all backend entities
The backend now exposes a standardized filtering mechanism across all major entities, including Alerts, Annotation Queues, Automations, Dashboards, Datasets, Experiments, Optimizations, Prompts, Spans, and Traces. This change replaces previous ad-hoc filtering with a consistent structure where each entity has a dedicated Field enum (defining available columns like ID, Name, Tags, and timestamps) and a corresponding Filter class. Users can now apply uniform operators (such as equals, contains, is\_empty) and value types to query these resources, with specific support for dynamic fields in Dataset Items and Experiments Comparison.
apps/opik-backend/src/main/java/com/comet/opik/api/filter · high confidence
Introduces configurable rate limiting for API endpoints
The backend now enforces rate limits on API requests using a new interceptor-based infrastructure. Limits can be applied at the workspace level, per user (via API key or CIPX device ID), or via custom buckets that support placeholders for workspace ID, API key, and client IP. The system exposes standard rate-limit headers in responses and records OpenTelemetry metrics for entities created or updated, allowing users and operators to monitor and control request throughput.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/ratelimit · high confidence
Introduces event-based alerting with debouncing, project scoping, and guardrail filtering
The backend now supports a new event-driven alerting system that aggregates incoming events (such as trace guardrails triggers, prompt changes, and experiment finishes) into debounced buckets stored in Redis before sending consolidated webhook notifications. This change introduces project-scoped alerting, allowing alerts to be restricted to specific projects via a new project\_id column or legacy trigger configs, and adds filtering for guardrail types so alerts can target specific guardrail categories. The system includes services for managing alert buckets, evaluating events against alert rules, logging alert events to the database, and sending webhook payloads, providing a foundation for real-time alerting on operational events.
apps/opik-backend/src/main/java/com/comet/opik/domain/alerts · high confidence
Introduces new backend event subscribers and services for Agent Insights, annotation queue routing, and attachment processing
This change adds several new event-driven components to the backend to support new and refined capabilities. The Agent Insights feature now includes a dedicated subscriber (AgentInsightsReportSubscriber) that consumes report triggers from a Redis stream, handling concurrency limits and budget exhaustion by cancelling auto-rollouts. Annotation queue automation is gated behind a service toggle and implemented via a listener-subscriber pair (AnnotationQueueRoutingListener/Subscriber) that buffers feedback score events, evaluates automation rules against production SDK-sourced traces and threads, and routes matches to queues. Additionally, a new AgenticScoringService extracts shared agentic tool-loop orchestration and attachment handling logic from online scorers into a reusable service, and an AttachmentUploadListener decouples attachment processing from the main request flow with OpenTelemetry metrics.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/events · high confidence
Introduction of attachment API data models
The backend now includes the core data structures required to support attachments on traces and spans. This change introduces new API records for managing the attachment lifecycle, including request and response models for starting and completing multipart uploads, deleting attachments, and searching by criteria. It also defines the \Attachment\ and \AttachmentInfo\ records that represent the metadata (such as file name, size, and MIME type) and the \EntityType\ enum to distinguish between trace and span attachments.
apps/opik-backend/src/main/java/com/comet/opik/api/attachment · high confidence
MCP OAuth 2.1 Authorization Server endpoints
The backend now exposes a new set of OAuth 2.1 resources for the MCP integration, including a dynamic client registration endpoint (RFC 7591), an authorization endpoint with PKCE support, a consent submission flow, token issuance and revocation (RFC 6749/7009), and an access token validation endpoint. These resources are conditionally enabled via the \mcpOAuth.enabled\ configuration flag and include metadata discovery (RFC 8414) and CSRF protection for the consent screen.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/oauth · high confidence
MCP OAuth dynamic client registration and connection tracking
The backend now supports RFC 7591 dynamic client registration for MCP hosts, allowing them to register their metadata (name, logo, software ID) and redirect URIs. This change introduces persistent tracking of user connections via a new \mcp\_client\_connections\ table, which records the host's display metadata and determines if a connection is new or a re-authorization. It also includes a background scrub job to clean up expired authorization codes and tokens, and sanitizes client metadata to prevent injection in logs and UI rendering.
apps/opik-backend/src/main/java/com/comet/opik/domain/mcpoauth · high confidence
Native Slack and PagerDuty alert integrations
The backend now supports sending alert notifications directly to Slack and PagerDuty via native webhook integrations. This change introduces payload mappers and data models that transform internal webhook events into Slack Block Kit messages and PagerDuty Events API v2 structures, ensuring alerts for events like trace errors, cost/latency thresholds, and prompt changes are formatted and delivered to these external services.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/events/webhooks/slack · high confidence
New /is-alive and /ver health endpoints with critical-check gating
The backend now exposes a new /is-alive endpoint that returns a simple healthy/unhealthy status for liveness probes. Unlike the existing /health-check endpoint, /is-alive only considers health checks marked as 'critical' in the configuration to determine the server's status, allowing non-critical checks (such as the optional ClickHouse read-only client for Agent Insights) to fail without causing the server to appear down to external monitors. Additionally, a new /ver endpoint is available to retrieve the current application version.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/health · high confidence
New API data models for automation rule evaluators
The backend now exposes a structured, polymorphic API for automation rule evaluators, introducing a sealed hierarchy that distinguishes between LLM-as-Judge and Python-based user-defined metrics, each scoped to traces, spans, or threads. These models support multi-project assignment, configurable sampling rates, and explicit enable/disable toggles, allowing users to define and manage evaluation rules with precise control over which data is scored and how.
apps/opik-backend/src/main/java/com/comet/opik/api/evaluators · high confidence
New API models for Agent Insights, Annotation Queues, and Alerts
The backend API layer introduces new data models to support Agent Insights, Annotation Queues, and Alerting. For Agent Insights, records like \AgentInsightsJob\, \AgentInsightsIssue\, and \AgentInsightsEnrollment\ define the structure for job status, issue details (including severity and occurrence counts), and project enrollment. Annotation Queue support is expanded with models for queue configuration (\AnnotationQueue\), automation rules (\AnnotationQueueAutomation\), and item sources (\AnnotationQueueItem\), enabling the system to manage and route items for review. Additionally, Alerting capabilities are defined through \Alert\, \AlertTrigger\, and \AlertTriggerConfig\ models, which specify event types (such as trace errors or cost thresholds), trigger conditions, and notification channels (Webhook, Slack, PagerDuty).
apps/opik-backend/src/main/java/com/comet/opik/api · high confidence
New Agent Config, Annotation Queue, and CLI infrastructure in TypeScript SDK
The TypeScript SDK introduces a new Agent Config system allowing users to manage versioned configuration blueprints and masks via the ConfigManager, with automatic injection of configuration metadata into traces and spans. It also adds support for Annotation Queues, enabling users to add, remove, and retrieve traces or threads for review. Additionally, a new CLI entry point is provided to wrap the Python Opik CLI via \npx\, including analytics tracking and \uv\ detection.
sdks/typescript · high confidence
New Agent Optimizer documentation hub in docs-v2
The documentation site now includes a dedicated Agent Optimizer section under Development, providing a complete user guide for the \opik-optimizer\ SDK. This new content covers the full optimization workflow, including a Quickstart for running your first optimization, an overview of supported algorithms (such as HRPO, GEPA, and MetaPrompt), and a guide to the no-code Optimization Studio UI. It also includes an FAQ, a changelog tracking SDK versions, and a known issues page with workarounds for dependency conflicts (e.g., \pyrate-limiter\ 4.x, \tqdm\ \>= 4.71).
apps/opik-documentation/documentation/fern/docs-v2/development/optimization-runs · high confidence
New Comet plugin components for assistant, billing, and workspace management
The Comet frontend plugin now includes a suite of new UI components and providers. The assistant sidebar is supported by dedicated components for error states, loading animations, and prewarming, along with a debug info display that shows the Ollie version. Workspace management features have been added, including a Collaborators tab for managing members and roles, an invite users popover with autocomplete, and a sidebar workspace selector. Additionally, a billing link component restricts access to the Ollie credits page for organization admins only, and a retention banner alerts users when they approach their span quota limits.
apps/opik-frontend/src/plugins/comet · high confidence
New Cursor extension for Opik integration
The Cursor extension for Opik is now available, allowing you to automatically save Cursor chat sessions to Opik. You can share these chat sessions with your team or on social media. The extension automatically uploads new Cursor chats to Opik and makes them available in the \cursor\ project. Existing chats are not uploaded automatically. To import them, run \*\Opik: Import Historical Cursor Traces\\* from the command palette. You can view conversations in the \thread\ tab and token usage in the metrics tab.
extensions · high confidence
New E2E test infrastructure and coverage taxonomy
The \tests\_end\_to\_end\ directory now includes a structured foundation for end-to-end testing, featuring a capability taxonomy (\taxonomy.yaml\) that defines product areas and capabilities, and a tagging system (\TESTING-TAGS.md\) that enforces how tests declare their scope and frequency. This is supported by tooling (\tag\_lint.py\, \reconcile.py\) to validate tags against the taxonomy and automatically update coverage metrics. The suite also introduces environment configuration for cloud and local deployments, global setup/teardown for workspace cleanup, and golden agent fixtures to support deterministic testing of the Agent Playground and Ollie features.
_tests\_end\_to\end · high confidence
New LLM provider infrastructure and Gemini thinking support
The backend now includes a new LLM provider infrastructure layer in the \com.comet.opik.infrastructure.llm\ package. This introduces a configurable LLM model registry service (\LlmModelRegistryService\) that loads model definitions from YAML (classpath, remote CDN, or local override) and a factory (\LlmProviderFactoryImpl\) that routes model requests to the correct provider. It also adds native support for Gemini thinking parameters (\GeminiThinkingParams\), allowing users to configure thinking levels and budgets for Gemini 3+ models. Additionally, it provides a delegating HTTP client builder pattern for timeout configuration and logging utilities for streaming responses. Also in this area: 7 commits (3 fixes) modifying apps/opik-backend/src; AI-generated trace labels and optional custom alert fields; View dataset versions by hash and normalize LLM request parameters; Agent Insights guidance, issue closure notes, and auto-run reliability.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm · medium confidence · unverified
New OpenTelemetry mapping infrastructure for attribute routing and processing
The backend now includes a dedicated mapping layer for OpenTelemetry data, introducing classes to define, find, and apply rules for various integrations (such as GenAI, LiveKit, Claude Code, and LangFuse). This infrastructure enables precise routing of span attributes into specific fields like input, output, metadata, usage, cost, and tags, while also handling the extraction of span events and the normalization of usage metrics and cost values.
apps/opik-backend/src/main/java/com/comet/opik/domain/mapping · high confidence
New Opik Python SDK E2E driver service
A new FastAPI-based bridge service has been added to the e2e test infrastructure to drive the Opik Python SDK during end-to-end testing. This service exposes REST endpoints for creating and managing core resources—projects, datasets, experiments, test suites, prompts, traces, threads, and annotation queues—and provides specialized evaluation routes for experiments, test suites, and threads. It handles SDK client construction with workspace and API key context, manages JSON request compression settings, and implements deterministic evaluation logic (such as fixed-score metrics and tautological tasks) to allow test suites to assert on SDK behavior without external dependencies.
_tests\_end\_to\end/e2e/services/opik-sdk-driver · high confidence
New Prompt Library and MCP server documentation
The documentation now includes a dedicated Prompt Library section covering how to create, fetch, and version prompts using the Python and TypeScript SDKs, as well as the Opik UI. It details the distinction between text and chat prompts, version control mechanics, and the new Prompt Playground for testing variants against datasets. Additionally, a new MCP server guide explains how to connect coding assistants to Opik traces using a single \npx opik mcp configure\ command, enabling agents to read, score, and fix code based on real execution data. Also in this area: Documentation updates for prompt environments, MCP server setup, and alert custom fields; Improved code block readability in documentation.
_apps/opik-documentation/documentation/fern/docs-v2/prompt\engineering · high confidence
New Redis-based distributed locking, rate limiting, and caching infrastructure
The backend now uses Redis for core distributed coordination and caching capabilities. A new distributed lock service provides semaphore-based locking with metrics to monitor wait times and held permits, ensuring bounded cardinality for observability. A Redis-backed rate limiter enforces workspace-level limits with automatic retry on transient errors. A new cache manager abstracts Redis operations for both reactive and non-reactive contexts, supporting pattern-based eviction and TTL management. Additionally, a lenient UUID deserializer and fault-tolerant stream codec prevent Redis stream wedging from legacy message formats or undecodable payloads, improving reliability during rolling upgrades.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/redis · high confidence
New TypeScript SDK documentation for OQL, CLI setup, and prompt management
Added reference documentation for the TypeScript SDK, introducing the Opik Query Language (OQL) for filtering data in search operations, the \opik-ts\ CLI tool for automated project configuration and tracing setup, and comprehensive guides for managing versioned prompt templates with Mustache and Jinja2 support.
apps/opik-documentation/documentation/fern/docs-v2/reference/typescript-sdk · high confidence
New advanced evaluation documentation pages
The documentation site now includes new advanced guides for evaluation workflows. Users can learn how to set up and manage Annotation Queues for subject matter expert review, evaluate agent trajectories and multi-turn agents using simulation, and resume interrupted evaluations to recover from crashes or interruptions. Additional pages cover exporting full experiment results to CSV via the SDK, manually logging experiments using the REST API, and managing datasets with versioning and tagging. Also in this area: Improved code block readability in documentation.
apps/opik-documentation/documentation/fern/docs-v2/evaluation/advanced · high confidence
New advanced tracing documentation pages added
The documentation site now includes a comprehensive set of new advanced tracing guides covering user feedback logging, cost tracking, agent graph visualization, chat conversation threading, distributed tracing (including OpenTelemetry bridging), multimodal media attachments, general trace logging, offline fallback with message replay, and SDK configuration. These pages provide code examples for Python and TypeScript SDKs, UI instructions, and configuration details for these specific tracing capabilities. Also in this area: 3 commits (3 fixes) fixing apps/opik-documentation; Documentation updates for prompt environments, MCP server setup, and alert custom fields; Improved code block readability in documentation.
apps/opik-documentation/documentation/fern/docs-v2/tracing/dashboards · medium confidence · unverified
New advanced tracing documentation pages added
The documentation site now includes a comprehensive set of new advanced tracing guides covering user feedback logging, cost tracking, agent graph visualization, chat conversation threading, distributed tracing (including OpenTelemetry bridging), multimodal media attachments, general trace logging, offline fallback with message replay, and SDK configuration. These pages provide code examples for Python and TypeScript SDKs, UI instructions, and configuration details for these specific tracing capabilities. Also in this area: Documentation updates for prompt environments, MCP server setup, and alert custom fields; Improved code block readability in documentation.
apps/opik-documentation/documentation/fern/docs-v2/tracing/advanced · high confidence
New alerts documentation with custom fields and webhook configuration
Added comprehensive documentation for the Alerts feature, detailing how to configure automated webhook notifications for events like trace errors and feedback scores. The guide covers creating alerts with specific triggers (thresholds, time windows, project scoping) and configuring destinations including native integrations for Slack and PagerDuty, as well as general webhooks. It also introduces support for optional custom fields in alert configurations for AWS EventBridge integration, allowing users to add key/value pairs to alert payloads for custom workflows.
apps/opik-documentation/documentation/fern/docs-v2/production/alerts · high confidence
New annotation-based caching infrastructure with deadlock prevention
The backend introduces a new caching layer in the infrastructure package, providing @Cacheable, @CachePut, and @CacheEvict annotations to declaratively manage Redis cache interactions. This system includes a CacheInterceptor and CacheManager that support both reactive (Mono/Flux) and synchronous execution paths, allowing services to cache method results and evict entries without manual cache management. The implementation specifically addresses nested cache deadlocks by enforcing a pattern where only the actual implementation method is annotated, preventing nested Mono.block() calls and Reactor threading violations that previously caused Redis timeouts and thread pool exhaustion.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/cache · high confidence
New attachment management infrastructure for traces and spans
The backend now includes a dedicated attachment domain layer that handles the full lifecycle of file attachments associated with traces and spans. This change introduces services to automatically detect and strip base64-encoded images from payload inputs/outputs during ingestion (AttachmentStripperService), persisting them to S3/MinIO instead. It provides APIs for uploading (including multipart uploads with presigned URLs), downloading, listing, and deleting these attachments. A reinjection service (AttachmentReinjectorService) allows clients to retrieve the original base64 data by replacing stored references in the JSON payloads. The implementation includes a new DAO for database operations, utility classes for reference pattern matching, and OpenTelemetry metrics for monitoring attachment processing.
apps/opik-backend/src/main/java/com/comet/opik/domain/attachment · high confidence
New background jobs for Agent Insights, metrics alerting, and system maintenance
This change introduces several new scheduled jobs in the backend to automate reporting, alerting, and maintenance tasks. Agent Insights now includes an auto-first-run job that triggers an initial diagnostic for enrolled projects once they meet a trace threshold, alongside a daily report job for enabled projects. Metrics-based alerting is handled by a new job that evaluates cost and latency thresholds and sends webhook notifications. Additional maintenance jobs manage annotation queue routing buffer flushing, ClickHouse partition health metrics, dataset version migration, experiment denormalization, and local runner reaping.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/jobs · high confidence
New benchmark packages for HotpotQA, HoVer, IFBench, and PUPA
The Opik Optimizer SDK now includes dedicated benchmark packages for four additional evaluation datasets: HotpotQA, HoVer, IFBench, and PUPA. The HotpotQA package provides a multi-hop retrieval agent with BM25-based Wikipedia search and specific exact-match/F1 metrics. The HoVer, IFBench, and PUPA packages supply specialized LLM-based judge metrics for claim verification, instruction-following compliance, and privacy-preserving quality/leakage evaluation, respectively. These additions expand the range of supported benchmarks available for optimizer testing and evaluation.
(repo-wide) · high confidence
New benchmarking infrastructure and algorithm implementations in the Optimizer SDK
The \opik\_optimizer\ SDK now includes a comprehensive benchmarking framework to evaluate and compare optimization strategies. This adds a manifest-based configuration system (\benchmarks/core/manifest.py\) for defining tasks, datasets, and metrics, along with a planning engine (\benchmarks/core/planning.py\) and checkpoint management (\benchmarks/core/state.py\) to track run progress and resume failed executions. The update also introduces new optimization algorithms, including a Few-Shot Bayesian Optimizer with columnar search space support and a Hierarchical Reflective Optimizer (HAPO) with improved prompt generation and root-cause analysis. Additionally, it provides specific agent implementations for benchmarking, such as a multi-hop retrieval agent for HotpotQA and an integration for Google's ADK framework, alongside scripts to automate changelog and documentation generation.
_opik\optimizer · high confidence
New contributor guides for Opik components and local development
The documentation site now includes dedicated guides for contributing to the Agent Optimizer SDK, Backend, Frontend, Python SDK, and TypeScript SDK, along with a comprehensive Local Development Setup guide. These pages provide step-by-step instructions for setting up development environments (including JDK 25 and Node.js 18+ requirements), running tests, and following coding standards, enabling external contributors to effectively work on the Opik platform.
apps/opik-documentation/documentation/fern/docs-v2/contributing/guides · high confidence
New database infrastructure components for ClickHouse integration and data mapping
This change introduces a suite of new database infrastructure classes in the \infrastructure.db\ package to support enhanced ClickHouse integration and data persistence. Key additions include \JsonEachRowBulkInsert\ for high-performance bulk inserts using the ClickHouse v2 client, replacing older R2DBC binding methods to improve throughput. The \DatabaseAnalyticsModule\ has been updated to configure and provide read-only ClickHouse clients for Agent Insights and Custom Charts, along with configurable health check timeouts. Additionally, a comprehensive set of column mappers and argument factories (e.g., \AbstractEnumColumnMapper\, \AbstractJsonColumnMapper\, and specific mappers for \PromptVersion\, \Dashboard\, \AlertType\, and \AgentInsights\) have been added to handle serialization, deserialization, and type mapping for various domain entities, ensuring robust handling of enums, JSON structures, and encrypted configurations in the database layer.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/db · high confidence
New demo data generator, Python code executor, and Optimization Studio types
The backend now includes a new demo data generator that seeds a dedicated 'Opik Demo Agent Observability' project with traces, spans, and feedback scores, ensuring UUIDv7 timestamps stay within the ingestion window to prevent silent data loss. A new Python code executor base class is introduced to handle metric scoring with configurable parallelism, timeouts, and fast-fail behavior (HTTP 503) when the execution pool is saturated. Additionally, new types and context objects for Optimization Studio are added to extract and validate scoring health and finish reasons from optimization runs, enabling the UI to display accurate run status and error details.
_opik\backend · high confidence
New development scripts and tooling for local Opik workflows
This change introduces a suite of new shell and PowerShell scripts to streamline local development and CI processes. The primary addition is the \dev-runner\ script (available as \dev-runner.sh\ and \dev-runner.ps1\), which automates the startup, stopping, and verification of the Opik backend, frontend, and Docker infrastructure, including support for multi-worktree isolation, database migrations, and optional integrations like the Comet EM/Platform stack and Ollie. Additional scripts include \generate\_openapi.sh\ and \start\_openapi\_server.sh\ for managing OpenAPI specifications and SDK generation via Fern, \check\_public-fe-plugins.sh\ to prevent committing private frontend plugins, and migration validation scripts (\check\_backend\_migration\_conflicts.sh\, \check\_clickhouse\_migrations\_cluster.sh\) to enforce database migration standards. Utility scripts for AI agent interoperability (\sync-codex.sh\, \convert-frontmatter.sh\, \convert-mcp.sh\) and security scanning (\analyze\_trivy\_report.sh\) are also added, along with comprehensive documentation in \scripts/README.md\.
scripts · high confidence
New documentation for Anonymizers and Gateway integrations
Added new documentation pages for the Anonymizers feature, detailing how to protect sensitive information in LLM applications using rules-based (regex, function) and custom anonymizers, and for the Gateway feature, listing supported LLM gateway integrations such as Opik LLM Gateway, Kong, LiteLLM, and others.
apps/opik-documentation/documentation/fern/docs-v2/production/gateway-guardrails · high confidence
New documentation for Opik 2.0 evaluation capabilities
The documentation site now includes a complete evaluation section covering two main approaches: Test Suites (assertion-based testing via SDK, UI, or Ollie) and Datasets & Metrics (quantitative scoring). New guides explain how to build and run test suites, evaluate conversation threads using the \evaluate\_threads\ SDK function, and apply specific metrics like hallucination and moderation detection. Best practices for evaluating AI agents, including step-level evaluation and tool selection quality, are also documented. Also in this area: 5 commits (2 fixes) modifying apps/opik-documentation; Improved code block readability in documentation.
apps/opik-documentation/documentation/fern/docs-v2/evaluation · medium confidence · unverified
New documentation for Opik Agent Optimizer workflows
Added a comprehensive set of documentation pages for the Opik Agent Optimizer, covering core concepts, LLM provider configuration, dataset and metric definition, and optimization strategies for prompts, agents, multimodal inputs, and MCP tools.
apps/opik-documentation/documentation/fern/docs-v2/development/optimization-runs/optimization · high confidence
New documentation for Opik Bounty Program and AI optimization cookbooks
The documentation site now includes a dedicated page for the Opik Bounty Program, detailing how to participate via Algora (currently paused), and adds three new optimization cookbooks: an ARC-AGI optimizer tutorial, a multimodal agent optimizer tutorial (self-driving car example), and a synthetic data optimizer guide using the MetaPromptOptimizer. These pages provide high-level overviews, link to the corresponding SDK scripts and notebooks, and explain how to run the examples locally or in Colab.
apps/opik-documentation/documentation/fern/docs-v2/contributing/developer-programs, apps/opik-documentation/documentation/fern/docs-v2/development/optimization-runs/cookbooks · high confidence
New documentation for Opik evaluation metrics and configuration
The documentation site now includes detailed guides for the Opik evaluation metrics, covering advanced configuration options such as asynchronous scoring, temperature control, and log-probability handling. New pages introduce specific metrics including Agent Task Completion, Agent Tool Correctness, Compliance Risk, Answer Relevance, Context Precision, and Context Recall, along with conversational metrics like Knowledge Retention and Conversation Degeneration. The documentation also provides instructions for creating custom single-turn and multi-turn conversation metrics, as well as integrating custom LLM providers via LiteLLM.
apps/opik-documentation/documentation/fern/docs-v2/evaluation/metrics · high confidence
New documentation for Opik optimization algorithms and benchmarks
Added comprehensive documentation for the Opik optimization algorithms, including guides for the Evolutionary, Few-Shot Bayesian, GEPA, HRPO, MetaPrompt, and Parameter optimizers. The new content covers quickstart code examples, configuration options, model support, and selection criteria to help users choose the right optimizer. Additionally, a new benchmarks page was added to compare algorithm performance across common datasets and explain how to reproduce results locally.
apps/opik-documentation/documentation/fern/docs-v2/development/optimization-runs/algorithms · high confidence
New documentation for SAML, OIDC, and JWT authentication configuration
Added comprehensive setup guides for SAML SSO, OIDC SSO, and JWT authentication in the Opik documentation. The new pages detail how to configure SAML identity providers (including Okta and Azure AD) with specific guidance on matching SP Entity ID and ACS URLs, explain OIDC setup with required scopes and endpoint configuration, and describe JWT authentication for programmatic access using JWKS URIs or static public keys.
apps/opik-documentation/documentation/fern/docs-v2/administration/authentication · high confidence
New domain event types for trace, span, experiment, and scoring operations
The backend now exposes a comprehensive set of new domain event classes in the \com.comet.opik.api.events\ package to support asynchronous processing, automation rules, and distributed scoring. These include lifecycle events for core entities (\TracesCreated\, \TracesDeleted\, \SpansCreated\, \SpansDeleted\, \CommentsCreated\, \DatasetsDeleted\, \ExperimentsCreated\, \OptimizationsCreated\), specialized messages for online scoring and LLM-as-Judge automation (\TraceToScoreLlmAsJudge\, \SpanToScoreLlmAsJudge\, \TraceThreadToScoreLlmAsJudge\, \TraceToScoreUserDefinedMetricPython\), and infrastructure events for cost intelligence (\TraceCostIntelligenceChanged\) and attachment uploads (\AttachmentUploadRequested\). A new \RedisSubscriberMessage\ interface and \EntityProjectEvent\ base class standardize how these events carry workspace context and entity references for downstream consumers.
apps/opik-backend/src/main/java/com/comet/opik/api/events · high confidence
New infrastructure configuration classes for backend services
The backend now includes a comprehensive set of new configuration classes in the infrastructure package to manage various operational aspects. These include settings for agent insights reporting, annotation queue routing, dataset exports and versioning, bulk insert behaviors, and ClickHouse analytics connectivity. Additionally, new configurations handle authentication timeouts and retries, CORS, custom charts workspace allowlists, and cipx token validation. These changes introduce new configuration keys that operators must define in their deployment configs to enable or tune these specific backend features.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure · high confidence
New infrastructure for automatic event listener registration and observability
The backend now includes a new event infrastructure in the \infrastructure.events\ package that automatically discovers and registers event listeners at startup, eliminating the need for manual registration. This system introduces OpenTelemetry-based observability for event processing, specifically tracking listener invocation counts and error rates via new metrics (\opik.event.invoked\ and \opik.event.error\), and enriches event traces with parent trace IDs for better debugging.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/events · high confidence
New integration documentation for Google ADK, AG2, Agent Spec, Agno, AISuite, Anthropic, AutoGen, Bedrock, and BeeAI
The documentation site now includes dedicated integration guides for a broad set of new frameworks and providers. Users can now follow setup instructions for Google Agent Development Kit (ADK), AG2, Agent Spec, Agno, AISuite, Anthropic, AutoGen, AWS Bedrock, and BeeAI (TypeScript). These pages cover installation, environment configuration for Opik Cloud, Enterprise, and self-hosted deployments, and provide code examples for tracing agent workflows, LLM calls, and tool usage.
apps/opik-documentation/documentation/fern/docs-v2/integrations · high confidence
New internal system endpoints for usage metrics, BI events, and analytics
This change introduces several new internal REST endpoints under /v1/internal to support system monitoring and analytics. UsageResource adds endpoints to retrieve daily trace and span counts per workspace, a detailed span usage breakdown by workspace/project/user, and BI event information for traces, experiments, datasets, and spans. AnalyticsQueriesResource provides internal endpoints to execute free-form read-only SQL against ClickHouse, gated by the Ollie and Custom Charts toggles, allowing workspace-scoped queries for Agent Insights and Custom Charts. AgentInsightsEnrollmentResource adds an internal endpoint to manage enrollment in the auto-first-run rollout for Agent Insights.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/internal · high confidence
New observability documentation pages for agent development and debugging
The documentation site now includes dedicated pages for the Agent Playground (agent\_sandbox.mdx), debugging agents with Ollie (debug\_agents.mdx), Diagnostics (diagnostics.mdx), and a Development Overview (development/overview.mdx). These pages guide users through running agents locally with full tracing, using Ollie for root-cause analysis and code fixes, and leveraging automated daily scans to detect recurring agent issues. The content also clarifies the distinction between \opik connect\ (for Ollie access) and \opik endpoint\ (for playground execution), and introduces the Diagnostics feature which groups repeated failures into single issues with suggested fixes.
apps/opik-documentation/documentation/fern/docs-v2/observability · high confidence
New private API endpoints for agent configuration, insights, alerts, annotation queues, assertions, and attachments
The backend now exposes a suite of new private REST endpoints under \/v1/private/\ to support advanced agent and data management features. \AgentConfigsResource\ allows creating and managing optimizer configurations and blueprints (including applying masks and removing keys). \AgentInsightsResource\ and \AgentInsightsJobsResource\ provide endpoints to retrieve, update, and trigger agent insights reports and issues. \AlertResource\ handles the creation, updating, listing, and batch deletion of workspace alerts. \AnnotationQueuesResource\ supports the full lifecycle of annotation queues, including batch creation and item locking. \AssertionResultsResource\ enables batch ingestion of assertion results for traces and spans. Finally, \AttachmentResource\ provides endpoints for uploading, downloading, and listing file attachments, including multipart upload support.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/priv · high confidence
New project and workspace metrics API endpoints with group-by breakdowns
The backend now exposes dedicated API contracts for project and workspace-level metrics, enabling users to query aggregated data across multiple projects. This change introduces request and response models for KPI cards (count, errors, avg\_duration, total\_cost), project metrics (supporting trace, span, and thread types like duration, cost, and token usage), and workspace span metrics. A key capability is the 'Breakdown' feature, which allows metrics to be grouped by dimensions such as tags, metadata, model, provider, error type, and guardrail name, with the backend handling the SQL generation and validation for these groupings.
apps/opik-backend/src/main/java/com/comet/opik/api/metrics · high confidence
New redirect endpoints for project, dataset, experiment, and optimization links
A new REST resource at /v1/session/redirect has been added, exposing four GET endpoints (/projects, /datasets, /experiments, /optimizations) that accept trace/dataset/experiment/optimization IDs, workspace names, and encoded paths. These endpoints generate and return 303 redirects to the corresponding resource URLs, enabling SDK-generated links to resolve correctly without requiring the base URL to be known or configured via environment variables.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/session · high confidence
New self-host configuration documentation for analytics, ingestion, and model management
This update adds comprehensive documentation for several self-host configuration areas. It introduces guides for enabling and configuring anonymous usage statistics reporting, including the specific data points collected. It details the automated dataset versioning migration process for upgrades to version 1.9.92+, covering Liquibase schema changes, MySQL counter migrations, and lazy reconciliation options. New pages explain how to configure large CSV uploads (up to 2GB) with appropriate timeout and disk space settings, and how to set up the LLM model registry with classpath defaults, optional remote CDN fetching, and local overrides. Additionally, it documents the new UUIDv7 ingestion validation feature for bounding client-supplied timestamps, the MCP OAuth authorization server for coding agent sign-in, and the post-cutover SQL querying patterns for the ClickHouse traces table.
apps/opik-documentation/documentation/fern/docs-v2/self-host/configure · high confidence
New self-host documentation site (v2)
The self-host documentation has been restructured into a new v2 site under \documentation/fern/docs-v2/self-host\. This update introduces a comprehensive Platform Architecture overview, a dedicated Helm chart migration guide for moving from Bitnami to official images, and specific operational guides for ClickHouse cold-tier migration, backup strategies, and scaling. It also includes updated local deployment instructions and a self-host changelog to track breaking changes.
apps/opik-documentation/documentation/fern/docs-v2/self-host · high confidence
New shared chart components for legends and tooltips
The frontend now includes new shared React components for charting: ChartHorizontalLegend, ChartVerticalLegend, LegendItem, and ChartTooltipContent. These components provide consistent, interactive legend and tooltip rendering for LineChart and RadarChart, supporting features like line highlighting on hover, custom label actions, and configurable tooltip content.
(repo-wide) · high confidence
Operator runbook and scripts for the spans table cutover to partitioned v2
This location introduces the operational tooling and documentation for migrating the ClickHouse \spans\ table to a weekly-partitioned, sharding-ready structure (\spans\_local\_v2\). It includes a comprehensive runbook (\README.md\) detailing the backfill, delta, deletion replay, and reconciliation steps, alongside shell scripts (\backfill.sh\, \clickhouse-client-docker.sh\) and SQL reference statements (\000001\_backfill\_spans\_local\_v2.sql\, \000002\_delete\_partition\_scope.sql\). The changes specifically address the scale of the spans table by implementing partition-scoped deletion replays to prevent ZooKeeper failures, adjusting insert block sizes and partition spread limits to handle large uncompressed rows, and mapping legacy \parent\_span\_id\ values to the new schema's fixed-string requirements.
apps/opik-backend/data-migrations/spans-local-v2-cutover · high confidence
Optional usage limit enforcement for API methods
The backend now supports optional enforcement of usage quotas on API endpoints. A new \UsageLimitInterceptor\ checks for the \@UsageLimited\ annotation on methods; if present, it verifies against the user's quota limits (such as \OPIK\_SPAN\_COUNT\) and returns a 402 Payment Required error if the limit is exceeded. This feature is controlled by configuration (\usageLimit\), allowing administrators to enable or disable quota checks, which is particularly useful for self-hosted installations or specific plan tiers.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/usagelimit · high confidence
Python SDK introduces lightweight analytics worker and structured result types
The Python SDK now includes a dedicated background thread for sending anonymous usage analytics, ensuring that reporting never blocks or slows down user code. This worker batches events and handles network failures gracefully, while a new lightweight \ScoreResult\ dataclass provides a clean, dependency-free container for metric evaluation outcomes. Additionally, the SDK adds support for attaching files to traces and spans via a new \Attachment\ model and an \AttachmentsExtractor\ that automatically detects and extracts base64-encoded images from input and output data.
opik · high confidence
Python SDK load-test suite and cutover rehearsal tooling
The \tests\_load\ directory now provides a structured pytest-driven load-test suite for the Python SDK and CLI-based rehearsal tools for backend data-cutover operations. The SDK suite (\suite/python\_sdk/\) exercises ingestion performance across eight scenarios: high-rate trace/span ingestion, heavy payloads (up to 1 MB per trace/span), explicit and implicit attachment uploads, burst and spread traffic patterns, concurrent multi-threaded writes, and dataset version insertion integrity. Each test logs metrics (logging, flush, verify timings) to JSON files and verifies that all submitted items are retrievable, with configurable volume scaling via \--load-scale\. Additionally, the \tests/cutover\_common/\ module supplies shared helpers for local cutover rehearsals, including ClickHouse client construction, UUIDv7 generation, and a delete-traffic generator that exercises the deletion-events bridge and resurrection guards during cutover simulations.
_tests\load · high confidence
Standardize AI editor configuration with .agents/ directory and symlinked surfaces
The repository now uses a single canonical \.agents/\ directory for AI editor rules and instructions, replacing scattered agent-specific files. A new \Makefile\ provides commands (\make cursor\, \make codex\, \make claude\) to sync these rules to editor-specific surfaces (\.cursor\, \.codex\, \.claude/\) via symlinks and file conversion. \.cursorignore\ and \.gitignore\ have been updated to manage these new directories and generated artifacts, ensuring consistent agent behavior across Cursor, Codex, and Claude.
(repo-wide) · high confidence
Support for JSON trace ingestion in OpenTelemetry
The backend now accepts OpenTelemetry trace data in JSON format, in addition to the existing Protobuf support. This is enabled by a new \OtelJsonMessageBodyReader\ that converts incoming JSON payloads into Protobuf messages, allowing users to send traces using JSON-encoded bodies without needing to switch to the binary Protobuf format.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/otel · high confidence
Support for OpenAI Responses API and improved quota error handling
The OpenAI LLM provider now supports the modern OpenAI Responses API (in addition to the traditional Chat Completions API) for both Playground usage and LLM-as-a-Judge evaluations, allowing users to select the API pipeline mode in their provider configuration. This change introduces new backend components to translate between the Chat Completions wire format and the Responses API, including support for structured output (strict JSON schema) and tool calling. Additionally, the provider now correctly treats 'insufficient\_quota' (HTTP 429) errors as non-retryable, preventing error storms when an API key is out of credits, and improves error mapping to surface the correct HTTP status codes (e.g., 402) instead of generic 500s.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/openai · high confidence
Support for OpenRouter Decisions API models in online evaluation
The backend now supports OpenRouter models served via the Decisions API (specifically TypeSafe Jev) for online evaluation rules. These models, which return calibrated probabilities for typed questions rather than free-form text, are handled by a new dedicated client that bypasses the standard chat completions path. The change includes a new model enum for these decision models, a specific error message handler for the Decisions endpoint, and the necessary Guice wiring to register the service provider and client.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/openrouter · high confidence
Traces cutover migration scripts for analytics database
Added the SQL runbook and driver scripts for the traces-local-v2 cutover in the analytics database. This includes the backfill (000001), delta-insert and deletion-replay (000002), and the exchange-and-wrap (000003) steps to migrate trace data to a new partitioned schema with a Distributed wrapper. It also provides the rollback procedures (000004), covering reverse-replay of deletes, sentinel repair for non-nullable columns, and stage-specific table swaps and unwraps to restore the original state or recover from wrap failures.
apps/opik-backend/data-migrations/traces-local-v2-cutover/scripts/db-app-analytics · high confidence
Traces cutover migration tooling and scripts
This change introduces the operational scripts and driver logic for the traces cutover migration, enabling the transition of trace data from the legacy \traces\ table to the new \traces\_local\_v2\ schema. The new tooling includes \backfill.sh\ for copying historical data in resumable, time-batched windows; \delta\_replay.sh\ to capture and replay changes made during the migration window; \estimate.sh\ to project backfill duration based on live throughput probes; and \exchange\_and\_wrap.sh\ to perform the atomic table swap and optionally apply a Distributed table wrapper. These scripts provide the necessary controls for operators to safely execute, monitor, and verify the data migration process.
apps/opik-backend/data-migrations/traces-local-v2-cutover/scripts · high confidence
User-facing logs are now persisted to ClickHouse
The backend now captures user-facing log events and stores them in ClickHouse via a new asynchronous logging infrastructure. This change introduces a dedicated appender that buffers log events and flushes them in batches to ensure non-blocking performance, along with utilities to correctly propagate MDC (Mapped Diagnostic Context) across reactive threads so that log context is preserved. Users can now access these persisted logs through the new automation rule evaluator logs endpoint.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/log · high confidence
Security
Upgrade frontend Docker base image to fix [CVE redacted]
The frontend Docker build image has been upgraded to patch the nghttp2-libs vulnerability ([CVE redacted]), improving the security posture of the deployed application.
apps/opik-frontend · high confidence
Behavioural changes
3 commits (1 fix) modifying apps/opik-backend/src/main/java/com/comet/opik/infrastructure/inference
A change to existing behaviour in apps/opik-backend/src/main/java/com/comet/opik/infrastructure/inference — 3 commits (1 fix), 26 files: “\[OPIK-9015\] \[BE\] fix: reuse connections for guardrail decide calls (\#9490)”; “\[OPIK-9015\] \[BE\] feat: add the synchronous decide call to the inference client (\#9287)”; “\[NA\] \[BE\] feat: ai-labels opik-inference client and tasks (\#8966)” (refs \#9490, \#9287, \#8966).
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/inference · medium confidence · unverified
7 commits (3 fixes) modifying apps/opik-backend/src
A change to existing behaviour in apps/opik-backend/src — 7 commits (3 fixs), 40 files: “\[issue-3493\] \[BE\] \[FE\] \[SDK\] feat: filter traces and threads by comments (\#8724)”; “\[NA\] \[BE\]\[FE\] chore: sync provider model definitions (\#9365)”; “\[OPIK-4331\] \[BE\]\[FE\] fix: let Bedrock providers call Claude (Anthropic Messages) and Nova (Converse) (\#9294)”; and 4 more (refs \#8724, \#9365, \#9294, \#9364, \#9309, …). Also in this area: View dataset versions by hash and normalize LLM request parameters; Added test infrastructure and coverage for backend infrastructure components.
(repo-wide) · medium confidence · unverified
8 commits (0 fixes) modifying apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels
A change to existing behaviour in apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels — 8 commits, 30 files: “\[NA\] \[BE\] feat(ai-labels): job gauges and a submitted counter for the tick's digital twin (\#9167)”; “\[NA\] \[BE\] test: drop the sweep lock in AiLabelsTickTest instead of waiting it out (\#9129)”; “\[NA\] \[BE\] feat: ai-labels render duration and CPU metrics (\#9091)”; and 5 more (refs \#9167, \#9129, \#9091, \#8967, \#8965, …).
apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels · medium confidence · unverified
8 commits (4 fixes) modifying apps/opik-backend/src/main/java/com/comet/opik/domain/llm/langchain4j
A change to existing behaviour in apps/opik-backend/src/main/java/com/comet/opik/domain/llm/langchain4j — 8 commits (4 fixs), 7 files: “\[OPIK-8790\] \[BE\] fix: send base64 data-URI images, video and audio to Vertex AI as inline data (\#8954)”; “\[OPIK-3058\] \[BE\] \[FE\] Playground message formatting (\#4057)”; “\[OPIK-3383\] \[BE\] Add video MIME type auto-detection for Gemini (\#4384)”; and 5 more (refs \#8954, \#4057, \#4384, \#4207, \#4396, …).
apps/opik-backend/src/main/java/com/comet/opik/domain/llm/langchain4j · medium confidence · unverified
Backend sorting logic now supports dynamic keys and specific null handling for experiment scores
The backend introduces a new SortingQueryBuilder component that enables sorting on dynamic fields (such as JSON metadata or experiment scores) by generating SQL with bound parameters and handling null values explicitly. For experiment scores specifically, the logic now uses a pre-computed aggregation alias and checks for key existence to determine null direction, ensuring that missing scores are handled consistently during sort operations.
apps/opik-backend/src/main/java/com/comet/opik/domain/sorting · high confidence
Cursor extension introduces structured trace processing and deduplication
The Cursor extension now reads conversation data directly from the local SQLite database and processes it into structured Opik traces. This change adds logic to classify chat bubbles (user, tool, error, thinking) and order them correctly, even when branches are edited away. It implements a request ledger to deduplicate traces and prevent duplicate uploads across edits, using deterministic UUIDv7 identifiers. The extension also handles usage enrichment, ensuring token counts and costs are accurately attributed to the correct traces and revisions.
extensions/cursor · high confidence
Data migration for dataset item \`data\` field structure
A new data migration script (1.0.3) is provided to backfill the \data\ field in existing dataset item rows to match the new dynamic fields structure. This migration updates legacy rows by merging \input\, \expected\_output\, and \metadata\ into the \data\ map column, ensuring consistency for installations that generated datasets prior to this release. Users must manually execute the provided ClickHouse SQL scripts outside peak hours to avoid resource contention.
apps/opik-backend/data-migrations/1.0.3 · high confidence
Enhanced multimodal support and refined LLM request handling
The backend now supports richer multimodal inputs in the Playground and online evaluations, allowing users to include images, videos, audio, and files alongside text in LLM prompts. To ensure compatibility with provider-specific constraints, the system now normalizes requests by dropping sampling parameters (temperature, top\_p, penalties) for OpenAI reasoning models and enforcing Anthropic's exclusive sampling rules for Claude models. Additionally, error handling has been improved to preserve the provider's original HTTP status codes and classify retries based on status rather than exception type, while also closing Vertex AI clients to prevent thread leaks. Also in this area: 2 commits (1 fix) modifying opik-backend; Refactored LLM provider routing and multimodal message handling.
apps/opik-backend/src/main/java/com/comet/opik/domain/llm · medium confidence · unverified
Fix StringTemplate memory leak and add template error logging
The backend now uses a new TemplateUtils factory to create StringTemplate instances, preventing memory leaks caused by templates being cached in a shared singleton group. This change ensures each template is ephemeral and garbage-collectible after use. Additionally, a new TemplateLogger has been introduced to capture and log template compilation and runtime errors, while suppressing specific debug-level warnings for undefined attributes in conditional SQL fragments.
apps/opik-backend/src/main/java/com/comet/opik/utils/template · high confidence
Improved OpenTelemetry provider mapping and cost tracking for GenAI spans
The backend now includes dedicated mapping rules and provider resolvers for OpenTelemetry spans, ensuring accurate attribution and cost calculation. New files define specific attribute mappings for integrations including Claude Code, GenAI, LiveKit, LangFuse, LiteLLM, Logfire, OpenInference, Pydantic, and Smolagents. Additionally, a provider resolution chain (ElasticInferenceServiceResolver, GenAiProviderAliasResolver, GoogleProviderResolver, VertexAnthropicResolver) normalizes provider names from various semantic conventions into Opik's canonical pricing keys, fixing issues where unmapped providers resulted in zero-cost spans.
apps/opik-backend/src/main/java/com/comet/opik/domain/mapping/otel · high confidence
Improved parameter parsing and JSON upload format handling
The backend now includes new infrastructure components to handle date/time parameters and JSON upload formats more robustly. Instant and LocalDate parameters are now parsed with specific error handling, supporting ISO-8601 formats and milliseconds since epoch for Instants, while providing clear error messages for invalid inputs. Additionally, a new MessageBodyReader ensures that JsonUploadFormat values from multipart form data are correctly parsed regardless of case sensitivity, addressing potential issues with Jersey's default enum provider.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/web · high confidence
Introduce pluggable template parser architecture with unescaped Mustache rendering
The backend now uses a new \TemplateParser\ interface with concrete implementations for Mustache, Python-style format strings, and Jinja2. This change introduces unescaped rendering for Mustache templates to ensure that JSON structures and special characters (like \=\ and backticks) are passed to LLM judges exactly as written, matching the frontend preview and Python parser behavior. It also adds robust error handling for malformed templates during variable extraction and provides a stub for future Jinja2 support.
apps/opik-backend/src/main/java/com/comet/opik/domain/template · high confidence
Introduce reactive webhook delivery with Redis stream decoupling
The backend now decouples webhook triggering from alert generation by publishing events to a Redis stream via the new WebhookPublisher, which is then consumed and sent asynchronously by the new WebhookHttpClient. This change enables reliable, retry-capable delivery of webhook notifications (including those for Slack and PagerDuty integrations) with configurable timeouts, destination validation, and encrypted secret handling, ensuring that alerting logic is not blocked by external HTTP response times.
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/events/webhooks · high confidence
Introduces a unified sorting framework for backend list endpoints
The backend now uses a centralized sorting infrastructure (SortingFactory, SortableFields, and SortingField) to standardize how list endpoints handle sorting. This change introduces specific sorting factories for key resources, enabling users to sort lists of Traces, Spans, Experiments, Datasets, Prompts, Prompt Versions, Projects, Dashboards, Alerts, Annotation Queues, Automation Rules, and Agent Insights Issues by their respective columns (such as name, timestamps, duration, cost, and status). The new system also supports sorting by dynamic JSON fields (e.g., feedback scores) and ensures that invalid or unsupported sort requests are handled gracefully without causing server errors.
apps/opik-backend/src/main/java/com/comet/opik/api/sorting · high confidence
Introduces structured request/response models and robust error handling for Python evaluators
The backend now uses dedicated record classes (PythonEvaluatorRequest, PythonEvaluatorResponse, PythonEvaluatorErrorResponse, PythonScoreResult) to serialize and deserialize Python evaluation payloads, ensuring snake\_case JSON mapping and ignoring unknown fields. PythonEvaluatorService implements non-blocking, buffered response processing with retry logic and null-safe error extraction, preventing NullPointerExceptions when the Python evaluation service returns empty or malformed bodies and surfacing truncated error messages to the user.
apps/opik-backend/src/main/java/com/comet/opik/domain/evaluators/python · high confidence
New API request and response models for Connect session management
The backend now exposes specific data structures for the Connect API, introducing ActivateRequest, CreateSessionRequest, and CreateSessionResponse. These models define the expected input for activating runners (requiring a runner name and HMAC tag) and creating sessions (requiring a project ID, activation key, optional TTL, and runner type), as well as the resulting session and runner identifiers returned to the client.
apps/opik-backend/src/main/java/com/comet/opik/api/connect · high confidence
New API request validation rules for dataset, experiment, and provider operations
The backend now enforces stricter validation on several API request types to prevent invalid configurations and improve error reporting. Dataset item batch updates and deletions now require either specific item IDs or filter criteria (with a required dataset ID for filters), but never both simultaneously. Experiment bulk uploads validate that item-level trace projects match the request-level project name and reject records containing both evaluate task results and traces. Provider configuration validation ensures API keys are present when required, prevents mixing API keys with auth configs, and enforces naming requirements for custom LLM and Bedrock providers. Additional validation covers metric breakdown configurations (requiring sub-metrics for duration, feedback, and token usage metrics), duration precision (seconds or higher), and execution policy thresholds.
apps/opik-backend/src/main/java/com/comet/opik/api/validation · high confidence
New ClickHouse readiness probes and connection-leak fix
The backend now includes several new ClickHouse readiness health checks: a basic connectivity probe, a toggle-gated check for the Distributed cluster definition, a toggle-gated check for the cold S3 storage disk, and topology assertions for traces and spans that fail readiness if the distributed-wrap flag disagrees with the actual table structure. Additionally, the shared health-check base class was rewritten to stop cancelling ClickHouse v2 client futures on timeout, which previously caused connection leaks and pod unready states.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/db/healthchecks · high confidence
New Semgrep rules to prevent SQL injection and datetime wrapping bugs
Added two new Semgrep pre-commit hooks for Java SQL code: \sql-query-clause-splice\ (ERROR) blocks SQL strings using \%s\ for clause splicing to prevent injection, and \sql-narrow-datetime-on-id\ (ERROR) flags narrow datetime conversions on UUIDv7-derived IDs to prevent far-future date wrapping. A secondary reporting rule, \sql-query-format-slot\ (WARNING), alerts on any other \%s\ format slots in SQL literals. These rules apply to production code in \apps/opik-backend/src/main/java\ and exclude test sources.
.semgrep · high confidence
New agent rules for deployment, code style, and security
The repository now includes structured agent rules in \.agents/rules/\ that guide development workflows. The \agents.mdc\ file defines domain routing (e.g., backend vs. frontend skills) and mandates the use of the V2 frontend structure, noting the removal of V1. The \code-style.mdc\ file enforces concise commenting and prohibits over-engineering. The \git-workflow.mdc\ file standardizes branch naming, commit formats, and PR body structures, including specific conventions for Jira key references to prevent unintended linking. Finally, the \security.mdc\ file explicitly forbids building SQL from formatted strings and requires parameterized queries, alongside general rules against hardcoding secrets.
.agents/rules · high confidence
New contribution guidelines and pull request policies for external contributors
The documentation site now includes a dedicated 'Contribution Overview' page that outlines how users can help, including reporting issues, improving documentation, and setting up local development. A key addition is the new pull request policy for external contributors, which limits them to a maximum of 3 open pull requests at a time and requires responses to reviews within 2 weeks. The page also details project architecture, developer tooling, and new rules for AI-assisted contributions, requiring disclosure in PR descriptions.
apps/opik-documentation/documentation/fern/docs-v2/contributing · high confidence
New error handling infrastructure with structured metrics and validation
The backend introduces a new set of exception classes and mappers in the error package to standardize API error responses and improve observability. This includes specific exceptions for conflicts (ConflictException, EntityAlreadyExistsException, IdentifierMismatchException) and a new InvalidUUIDException for UUIDv7 ingestion validation, which maps to HTTP 400 errors. A JsonProcessingExceptionMapper now distinguishes between structural JSON errors (400) and size-limit violations (413), recording metrics for ingestion size guards. Additionally, generic exception mappers (OpikGenericExceptionMapper, OpikWebApplicationExceptionMapper) have been added to count 5xx server errors under the opik.errors.count metric, while maintaining existing logging behavior.
apps/opik-backend/src/main/java/com/comet/opik/api/error · high confidence
Online evaluation runs are persisted as hidden monitoring traces
Online evaluation runs are now recorded as hidden monitoring traces and spans in the backend, ensuring that LLM-as-judge scoring activity is visible in observability without cluttering the default traces view or triggering recursive self-evaluation. The system creates a dedicated parent trace per evaluation and child spans for LLM calls, tool executions, and preparation steps, capturing token usage and provider details for accurate cost tracking. This change introduces a sealed hierarchy of evaluated subjects (traces, spans, and threads) to support different evaluation targets and integrates with the OpenRouter Decisions API for specific model routing.
apps/opik-backend/src/main/java/com/comet/opik/domain/evaluation · high confidence
Opik documentation v2 site launch with new core pages
The documentation site has been updated to version 2, introducing a new home page, a quickstart guide for Python and TypeScript SDKs, and a dedicated FAQ page covering rate limits, size limits, and API keys. A new 'Ollie' page details the AI assistant and its integration via \opik connect\, while an upgrade guide explains the shift to project-scoped data in Opik 2.0. The reference overview now explicitly lists the Ruby OpenTelemetry SDK alongside Python, TypeScript, and REST API docs.
apps/opik-documentation/documentation/fern/docs-v2 · high confidence
Pre-computed experiment aggregates for faster experiment views
The backend now uses pre-computed aggregation tables in ClickHouse to serve experiment details, groupings, and dataset item comparisons. This change introduces a new aggregation pipeline that computes and stores metrics (such as pass rates, feedback scores, and trace/span statistics) in dedicated tables, allowing the API to read from these aggregates instead of performing expensive real-time joins across raw trace and span data. Users will experience significantly faster load times for experiment pages and comparisons, especially for large datasets, as the system now relies on these denormalized aggregates for most read operations.
apps/opik-backend/src/main/java/com/comet/opik/domain/experiments · high confidence
Provider-specific token usage parsing for accurate pricing
The backend now parses token usage data using provider-specific formats (Anthropic, OpenAI, Gemini, and Bedrock Converse) instead of a generic approach. This ensures that cache, audio, and reasoning tokens are correctly interpreted according to each provider's native schema, resulting in more accurate cost calculations for spans.
_apps/opik-backend/src/main/java/com/comet/opik/domain/token\usage · high confidence
Python sandbox executor optimized for performance and security
The Python sandbox executor now starts significantly faster by compiling standard library and application bytecode at build time and pre-warming essential imports, while also reducing image size by shipping only compiled \.pyc\ files. To improve security, the container runs as a non-root user (UID 1001) and strips package managers from the runtime environment. Additionally, the scoring runner now uses lightweight stub modules to delay heavy \opik\ package imports until necessary, preventing timeout issues during metric evaluation.
apps/opik-sandbox-executor-python · high confidence
Refactored Gemini LLM provider infrastructure with structured thinking and video support
The Gemini LLM integration has been restructured into a new modular package (GeminiChatModelMapper, GeminiClientGenerator, GeminiLangChainMapper, etc.) to improve maintainability and extend capabilities. This change introduces explicit support for Gemini's 'thinking' configuration (allowing users to control reasoning depth via thinkingLevel/thinkingBudget) while suppressing raw thought traces by default to prevent them from leaking into the final response text. It also adds native video URL handling, automatically converting video content to image content for the Gemini API and detecting MIME types when necessary. The update expands the supported model list to include Gemini 3.x and 3.5 variants, removes deprecated models (like gemini-1.0-pro), and standardizes error handling and client generation for both streaming and non-streaming chat completions.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/gemini · high confidence
Refactored authentication infrastructure with Redis caching and granular permission support
The authentication layer in the backend has been restructured to improve performance and support fine-grained access control. A new Redis-based caching mechanism (AuthCredentialsCacheService) now stores and retrieves user credentials, workspace metadata, and quotas, significantly reducing latency for repeated authentication requests. The system now supports granular workspace permissions via the WorkspaceUserPermission enum and the @RequiredPermissions annotation, allowing endpoints to enforce specific access rights. Additionally, the authentication filter has been extended to validate CIPX device tokens for machine-to-machine ingestion and MCP OAuth tokens, broadening the supported identity types beyond standard API keys and session cookies.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/auth · high confidence
Refactored backend filtering infrastructure to support dynamic fields and robust JSON path handling
The filtering logic in the analytics backend has been restructured to introduce a strategy-based approach for building queries across all entity types (traces, spans, experiments, datasets, prompts, etc.). This change adds comprehensive support for filtering on dynamic fields (such as metadata and custom dataset columns) by correctly resolving JSON paths for ClickHouse, ensuring that special characters in keys are handled safely. It also implements sentinel-aware filtering for time-based fields (end\_time, duration, ttft) to correctly exclude rows with missing or null values, and introduces a new ENUM\_LEGACY field type to maintain compatibility with legacy source data while standardizing filter behavior.
apps/opik-backend/src/main/java/com/comet/opik/domain/filter · high confidence
Refactored project stats mapping and merging logic
The backend now uses a dedicated StatsMapper to map database rows into ProjectStats objects, explicitly handling fields such as thread count, guardrails failed count, and error counts (recent, past period, and total). A new StatsMerger component combines trace/span statistics with separate feedback-score statistics, ensuring that feedback data is spliced into the correct position within the stats list without resurrecting projects that lack trace data. This change centralizes the mapping logic and clarifies how different statistical aggregates are combined for project-level views.
apps/opik-backend/src/main/java/com/comet/opik/domain/stats · high confidence
Restructured MCP server documentation into five dedicated pages
The MCP server documentation has been reorganized from a single page into five distinct sections: an overview, advanced setup, tools reference, use cases, and FAQ. This change introduces new pages for advanced configuration (covering hosted vs. local servers and manual setup), a detailed reference for the five MCP tools (\read\, \list\, \write\, \schema\, \read\_skill\), practical use-case prompts for debugging and evaluation, and a troubleshooting FAQ. The previous single-page content has been migrated and expanded across these new files to provide more granular guidance for users.
_apps/opik-documentation/documentation/fern/docs-v2/prompt\engineering/mcp-server · high confidence
SDK configuration flow improvements
The SDK's configuration process has been enhanced to provide a smoother onboarding experience. The \opik configure\ command now intelligently suggests the most recently created project during setup, and the configuration flow has been hardened to better handle interactive environments and API key validation, ensuring that users are guided more effectively when setting up their workspace and API credentials.
sdks · high confidence
Structured output now uses native JSON response formats where supported
The backend now automatically selects the most reliable method for obtaining structured JSON from LLMs based on provider capabilities. For providers that support native JSON response formats (OpenAI, Gemini, OpenRouter, and Vertex AI), the system now uses the provider's built-in JSON schema enforcement via the \ToolCallingStrategy\. For providers that do not support this feature (Anthropic, Bedrock, Ollama, and custom LLMs), it falls back to appending detailed text instructions to the user message via the new \InstructionStrategy\. This change improves reliability and reduces parsing errors for supported models while maintaining compatibility with others.
apps/opik-backend/src/main/java/com/comet/opik/domain/llm/structuredoutput · high confidence
Updated REST API overview with specific usage examples for Open-Source and Cloud
The REST API overview documentation now provides distinct usage examples for the Open-Source platform and Opik Cloud, clarifying the differences in base URLs and authentication headers. Users can now see specific cURL commands for local development (localhost:5173) versus cloud/on-premise deployments, including the correct header format for API keys without the Bearer prefix.
apps/opik-documentation/documentation/fern/docs-v2/reference/rest-api · high confidence
Vertex AI integration now properly manages client lifecycle and supports new Gemini models
The Vertex AI backend infrastructure has been refactored to prevent thread leaks by ensuring the Google GenAI client is created and closed per request, addressing previous stability issues. This change also introduces support for a broader range of Gemini models (including Gemini 2.5, 3, and 3.5 variants) and enables native thinking-level configuration for Vertex Gemini 3. Additionally, a workaround is implemented to handle requests that lack user or AI messages, preventing 500 errors during generation.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/vertexai · high confidence
Workspace metadata tracking and legacy feedback scores support
The backend now tracks workspace-level metadata, including the timestamp of the first reported trace and a flag indicating whether legacy feedback scores exist. This enables the system to correctly handle legacy data during stats queries by conditionally including the legacy feedback\_scores table, and ensures accurate audit trails for workspace initialization.
apps/opik-backend/src/main/java/com/comet/opik/domain/workspaces · high confidence
Fixes
12 commits (8 fixes) fixing apps/opik-documentation/documentation/fern/docs-v2/administration/workspace-settings
A fix in apps/opik-documentation/documentation/fern/docs-v2/administration/workspace-settings — 12 commits (8 fixs), 5 files: “\[OPIK-8661\] \[DOCS\] Document recent SDK/FE changes and replace retired Claude model ids (\#8773)”; “\[OPIK-9002\] \[BE\] fix: keep the shared Responses API transport open, parse OpenAI errors with extra fields, keep provider header values out of logs (\#9245)”; “\[OPIK-4331\] \[BE\]\[FE\] fix: let Bedrock providers call Claude (Anthropic Messages) and Nova (Converse) (\#9294)”; and 9 more (refs \#8773, \#9245, \#9294, \#9214, \#9000, …).
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/freemodel, apps/opik-backend/src/main/java/com/comet/opik/infrastructure/net, apps/opik-documentation/documentation/fern/docs-v2/administration/workspace-settings · medium confidence · unverified
Exclude demo project usage from billing and BI reports
Usage reports and daily billing counts now correctly exclude demo projects. Previously, demo projects (created per signup) caused query performance issues and inaccurate counts because their IDs were unbounded. This change moves the exclusion logic from SQL queries to Java, filtering out demo project IDs after aggregation to ensure accurate workspace and user-level usage totals for billing and business intelligence.
apps/opik-backend/src/main/java/com/comet/opik/domain/utils · high confidence
Fix HTTP 500 errors for error message serialization in streaming endpoints
Added custom JAX-RS MessageBodyWriters for \ErrorMessage\, \InternalErrorMessage\, \ValidationErrorMessage\, and \JsonNode\ types to ensure they are correctly serialized to JSON when the response content type is \application/octet-stream\. This prevents HTTP 500 errors that previously occurred in streaming endpoints when the framework could not find a suitable writer for these error types.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/json · high confidence
Fixes outbound HTTP double-decompression and adds liveness detection for dead connection pools
The backend now prevents ZipException errors when calling external services that return gzip-compressed responses by reconciling Content-Encoding headers after Apache HttpClient's transparent decompression. Additionally, a new liveness health check detects when the shared outbound HTTP connection pool has shut down, ensuring Kubernetes restarts the pod to restore connectivity rather than leaving it serving failed requests.
apps/opik-backend/src/main/java/com/comet/opik/infrastructure/http · high confidence
Test coverage
1 commit adding/updating tests in apps/opik-backend/src/test/java/com/comet/opik/infrastructure/log; 15 commits adding/updating tests in apps/opik-backend/src/test/java/com/comet/opik/infrastructure/llm/customllm; 3 commits adding/updating tests in apps/opik-backend/src/test/java/com/comet/opik/infrastructure/inference; 7 commits adding/updating tests in apps/opik-backend/src/test/java/com/comet/opik/infrastructure/llm/openrouter; Add tests for alerting, encryption, and UUIDv7 time filtering; Add trace test assertion utilities; Added Liquibase test fixtures for topology-aware DDL patterns; Added MySQL query plan-shape regression tests; Added Podam manufacturers for backend test data generation; Added Podam test data factories for backend unit tests; Added end-to-end test for MySQL RDS IAM authentication; Added end-to-end tests for CORS configuration; Added integration and unit tests for MCP OAuth client lifecycle and security; Added integration test for project metadata cache behavior; Added integration tests for internal analytics and usage endpoints; Added integration tests for the backend analytics and usage reporting infrastructure; Added span test assertion utilities; Added test infrastructure and coverage for backend infrastructure components; Added test infrastructure and utilities for backend integration tests; Added test infrastructure and validation for test timeouts; Added test utility clients for backend API resources; Added tests for Anthropic model catalog consistency and client generation; Added tests for AttachmentUploadRequested event; Added tests for ClickHouse health check connection release and topology readiness; Added tests for ClickHouse partition metrics, error classification, and ingestion size guards; Added tests for LLM infrastructure components; Added tests for LLM usage extraction and online evaluation recording; Added tests for LLM-as-Judge evaluator filter deserialization and multimodal message serialization; Added tests for MCP OAuth resource endpoints and bundle wiring; Added tests for OpikUserMessage serialization; Added tests for Optimization Log Sync Service; Added tests for Redis infrastructure reliability and configuration; Added tests for SortingFactory null/blank field handling and server-side bind key generation; Added tests for TemplateUtils memory leak prevention; Added tests for alert debouncing and Redis stream configuration; Added tests for alert notification publishing logic; Added tests for annotation queue automation, locking, and routing logic; Added tests for authentication infrastructure components; Added tests for backend utility classes; Added tests for cost calculation logic; Added tests for database migration safety and schema parity; Added tests for evaluator domain logic and online scoring infrastructure; Added tests for event processing and online scoring infrastructure; Added tests for experiment aggregation and denormalization logic; Added tests for filter query builder sentinel handling, v2 client integration, and JSON path utilities; Added tests for rate limiting infrastructure; Added tests for retention policy enforcement and velocity estimation; Added tests for the Is Alive health check resource; Added tests for the cache infrastructure; Added tests for the session redirect resource; Added unit and integration tests for backend background jobs; Added unit tests for ClickHouse v2 client bulk inserts and UUIDv7 validation; Added unit tests for Mustache and Python template parsers; Added unit tests for Ollie state service upload, download, and delete operations; Added unit tests for OpenTelemetry mapping utilities; Added unit tests for Python Evaluator Service; Added unit tests for StatsMapper project and score statistics; Added unit tests for ValidationErrorMessageBodyWriter; Added unit tests for alert configuration, streaming responses, and UUIDv7 time filtering; Added unit tests for attachment stripping and reference extraction; Added unit tests for backend error mappers; Added unit tests for the pytest plugin hooks; Added validation script for dataset versioning migration; Added validation tests for backend API constraints; Centralized dataset item test assertions; Test infrastructure: ClickHouse config alignment and test stability improvements.
Dependencies
Add integration test requirements for new and updated SDK libraries
Added explicit dependency specifications for Python SDK library integration tests, including \google-adk\ (with legacy version constraints), \harbor\, \litellm\ (pinned below 1.75.0), \opentelemetry\ components, \pydantic-ai\, \pyagentspec\, \anthropic\, \boto3\, \cerebras\_cloud\_sdk\, \crewai\ (v1+), \google-genai\, and \groq\. These files define the isolated environments required to test the corresponding SDK integrations.
(dependencies) · high confidence
Housekeeping
New advanced optimization documentation for Opik v2
The documentation site now includes a new 'Advanced' section for optimization runs, covering API references, chaining optimizers, custom metrics, extending the optimizer framework, multiple completions (n parameter), sampling controls, and prompt customization. This content is part of the Opik 2.0 documentation refactor and replaces the previous v1 documentation.
apps/opik-documentation/documentation/fern/docs-v2/development/optimization-runs/advanced · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 44.7 → 46.9 (+2.2)
- Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 68.5 → 63.0 (-5.5)
- Architecture 64.3 → 67.3 (+3.0)
- Maturity 76.5 → 79.7 (+3.2)
- Readiness 34.1 → 33.9 (-0.2)
- Security 72.3 → 74.5 (+2.2)
- Accessibility 44.0 → 47.8 (+3.8)
- Performance 64.9 → 68.6 (+3.7)
Resolved (1444)
- AgentConfigServiceImpl.generateUpdateDescription (cognitive 23) (apps/opik-backend/src/main/java/com/comet/opik/domain/AgentConfigService.java)
- AgentConfigServiceImpl.generateUpdateDescription (cyclomatic 18) (apps/opik-backend/src/main/java/com/comet/opik/domain/AgentConfigService.java)
- AiLabelStateFitter.fit (cognitive 28) (apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels/AiLabelStateFitter.java)
- AiLabelsTickService.submit (cognitive 18) (apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels/AiLabelsTickService.java)
- AiLabelsTickService.turn (cognitive 17) (apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels/AiLabelsTickService.java)
- AlertServiceImpl.validateThresholdConfigs (cognitive 27) (apps/opik-backend/src/main/java/com/comet/opik/domain/AlertService.java)
- AttachmentUtils.collectAttachmentReferences (cognitive 17) (apps/opik-backend/src/main/java/com/comet/opik/domain/attachment/AttachmentUtils.java)
- AttachmentUtils.containsAttachmentReference (cognitive 22) (apps/opik-backend/src/main/java/com/comet/opik/domain/attachment/AttachmentUtils.java)
- AutomationRuleEvaluatorServiceImpl.update (cyclomatic 16) (apps/opik-backend/src/main/java/com/comet/opik/domain/evaluators/AutomationRuleEvaluatorService.java)
- AwsEventStreamParser.parse (cognitive 20) (apps/opik-backend/src/main/java/com/comet/opik/infrastructure/llm/customllm/AwsEventStreamParser.java)
- Backend.run (cognitive 22) (scripts/check_new_models.py)
- BlockRow.from (cognitive 22) (apps/opik-backend/src/main/java/com/comet/opik/domain/CipxSpendBlockDAO.java)
- ClassTooLong: AgentConfigServiceImpl (apps/opik-backend/src/main/java/com/comet/opik/domain/AgentConfigService.java)
- ClassTooLong: AiLabelsTickService (apps/opik-backend/src/main/java/com/comet/opik/domain/ailabels/AiLabelsTickService.java)
- ClassTooLong: AlertServiceImpl (apps/opik-backend/src/main/java/com/comet/opik/domain/AlertService.java)
- ClassTooLong: AnnotationQueueDAOImpl (apps/opik-backend/src/main/java/com/comet/opik/domain/AnnotationQueueDAO.java)
- ClassTooLong: AutomationRuleEvaluatorServiceImpl (apps/opik-backend/src/main/java/com/comet/opik/domain/evaluators/AutomationRuleEvaluatorService.java)
- ClassTooLong: BaseRedisSubscriber (apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/events/BaseRedisSubscriber.java)
- ClassTooLong: DatasetItemDAOImpl (apps/opik-backend/src/main/java/com/comet/opik/domain/DatasetItemDAO.java)
- ClassTooLong: DatasetItemServiceImpl (apps/opik-backend/src/main/java/com/comet/opik/domain/DatasetItemService.java)
- …and 1424 more
New (638)
- AuthConfigSection.AuthConfigSection (cognitive 22) (apps/opik-frontend/src/v2/pages-shared/llm/ManageAIProviderDialog/AuthConfigSection.tsx)
- Duplicated block (10 lines × 3) (sdks/python/src/opik/integrations/cerebras/chat_completion_chunks_aggregator.py)
- Duplicated block (14 lines × 5) (sdks/python/src/opik/integrations/cerebras/_streamed_tool_calls.py)
- Duplicated block (19 lines × 3) (sdks/python/src/opik/integrations/cerebras/chat_completion_chunks_aggregator.py)
- Duplicated block (23 lines × 3) (sdks/python/src/opik/integrations/cerebras/_streamed_tool_calls.py)
- Duplicated block (29 lines × 3) (sdks/python/src/opik/integrations/cerebras/chat_completion_chunks_aggregator.py)
- Duplicated block (7 lines × 4) (sdks/python/src/opik/integrations/cerebras/_streamed_tool_calls.py)
- Edited copies of one member (3 members, 2+ corresponding lines) (sdks/python/src/opik/integrations/cerebras/_streamed_tool_calls.py)
- Edited copies of one member (3 members, 2+ corresponding lines) (sdks/python/src/opik/integrations/cerebras/_streamed_tool_calls.py)
- Edited copies of one member (3 members, 21+ corresponding lines) (sdks/python/src/opik/integrations/langchain/provider_usage_extractors/langchain_run_helpers/langchain_usage.py)
- Edited copies of one member (3 members, 23+ corresponding lines) (sdks/opik_optimizer/src/opik_optimizer/algorithms/meta_prompt_optimizer/reporting.py)
- Edited copies of one member (3 members, 23+ corresponding lines) (sdks/python/src/opik/api_objects/search_helpers.py)
- Edited copies of one member (3 members, 23+ corresponding lines) (sdks/python/src/opik/integrations/anthropic/messages_create_decorator.py)
- Edited copies of one member (3 members, 30+ corresponding lines) (sdks/python/src/opik/integrations/cerebras/_streamed_tool_calls.py)
- Edited copies of one member (3 members, 4+ corresponding lines) (sdks/python/src/opik/evaluation/metrics/llm_judges/hallucination/metric.py)
- Edited copies of one member (4 members, 15+ corresponding lines) (sdks/python/src/opik/evaluation/metrics/llm_judges/answer_relevance/templates.py)
- Edited copies of one member (4 members, 24+ corresponding lines) (sdks/python/src/opik/evaluation/metrics/conversation/llm_judges/conversational_coherence/metric.py)
- Edited copies of one member (4 members, 25+ corresponding lines) (tests_end_to_end/test-helper-service/routes/projects.py)
- Edited copies of one member (4 members, 29+ corresponding lines) (sdks/python/src/opik/evaluation/metrics/llm_judges/structure_output_compliance/metric.py)
- Edited copies of one member (4 members, 9+ corresponding lines) (tests_end_to_end/test-helper-service/routes/ai_providers.py)
- …and 618 more
Changes since last survey
- 91 commits — 57 feature/other, 34 fixes
By area
- apps/opik-backend — 28 commits
- sdks/python — 21 commits
- apps/opik-frontend — 13 commits
- sdks/typescript — 9 commits
- .github/workflows — 6 commits
- deployment/helm_chart — 5 commits
- apps/opik-documentation — 4 commits
- apps/opik-sandbox-executor-python — 2 commits
- (root) — 1 commit
- .github/repoguard.yaml — 1 commit
- tests_end_to_end/e2e — 1 commit
Notable commits
- fix: [DND-1973] [BE] fix: pin jetty/hc.client5 loggers to INFO (log-hygiene hardening) (#9379)
- fix: [DND-2054] [HELM] fix: render bundled-Redis REDIS_URL only when redis.enabled (#9493)
- fix: [DND-2054] [HELM] fix: skip recursive fsGroup chown on ClickHouse volumes (#9450)
- fix: [NA] [DOCS] docs: fix the dead Testing Guide link in the local development guide (#9561)
- fix: [NA] [SDK] fix: stop HTML-escaping Jinja2 prompt values in the TS SDK (#9398)
- fix: [OPIK-6910] [HELM] fix: cold-tier migration job waits for every replica (#9471)
- fix: [OPIK-6910] [HELM] fix: require S3_OBJECT_DISK_PATH when the cold tier is backed up (#9495)
- fix: [OPIK-8173] [BE] fix: remove the dashboard slug so creates and renames no longer fail with 409 (#9189)
- fix: [OPIK-8546] [FE] fix: count annotators by person, not by score key (#9474)
- fix: [OPIK-8656] [FE] fix: keep the suite pass criteria when adding traces to a test suite (#9482)
- fix: [OPIK-8801] [CI] fix: speed up e2e image cleanup and stop orphaning e2e manifests (#8962)
- fix: [OPIK-8802] [CI] fix: skip E2E image cleanup on fork PR close (#8960)
- fix: [OPIK-8927] [BE] fix: cap Gemini 4 Argon's max output tokens at Google's 32,768 (#9432)
- fix: [OPIK-8934] [SDK] fix: Prompt() and ChatPrompt() use the default project (#9458)
- fix: [OPIK-8934] [SDK] fix: scope remaining by-name lookups to the project (#9485)
- fix: [OPIK-9015] [BE] fix: reuse connections for guardrail decide calls (#9490)
- fix: [OPIK-9040] [SDK] fix: report traces the SDK drops at shutdown, on 429 and on queue overflow (#9298)
- fix: [OPIK-9055] [BE] [FE] fix: project name length follow-ups (#9469)
- fix: [OPIK-9055] [BE] fix: reject project names over 150 characters (#9308)
- fix: [OPIK-9058] [BE] fix: read a thread page's messages from its first and last traces only (#9314)
- …and 71 more
Architecture
- Containers 0 added · 0 removed · contexts 0 added · 1 removed · edges 0 added · 0 removed
Removed bounded contexts (1)
- opik-backend
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
comet-ml/opik was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 17 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit d5eb86a81e6b7cd7fe9642f485cbef5d36d467df — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.85+preprod-eba6548ac50f.