Evil0ctal/Douyin_TikTok_Download_API
59.4
Adequate · 17 November 2026
100.8k
lines of production code
Python
with TypeScript
5
measurements over time
What this system is
Features
Added i18n integrity check script to catch missing translations and key mismatches
A new \check-i18n.mjs\ script has been added to \web/scripts\ to enforce locale integrity during development and CI. This tool validates that English and Chinese locale files contain identical keys, ensures all message strings are valid ICU format, verifies that every backend error code has corresponding console translations, and checks that all literal translation keys used in the source code actually exist in the locale catalogs. This prevents silent fallbacks, raw key display, and missing error messages for users.
web/scripts · high confidence
Initial database schema and support for content archives, media downloads, watchlists, and demo operations
The database schema is established through a series of migrations that create the foundational tables and structures required for the application. The initial migration sets up TimescaleDB hypertables for time-series data (request logs, identity events, content snapshots) and continuous aggregates for health metrics. Subsequent migrations introduce tables for archiving current-state data on authors and content, tracking media downloads on local disk, and managing a watchlist for scheduled re-collection of content. Operational changes include enforcing a constraint to allow only one live download per post at a time to prevent file conflicts, adding a 'demo' role to the users table to support public trial instances, and introducing encrypted columns to store and display demo credentials securely. Also in this area: Identity rotation controls, download file deletion, and proxy error visibility; Fixes for transcription reliability, download profile removal, and MCP result accuracy; Add offline transcription sidecar and API.
src/dtk/db/migrations/versions · high confidence
Initial release of the dtk console web interface
This change introduces the dtk console, a React single-page application that serves as the primary web interface for the dtk v5 API. The console provides a responsive, accessible UI with support for dark and light themes, and includes built-in internationalization for English and Chinese. It features a comprehensive design system with reusable components (such as data tables, modals, and status badges), a robust API client that handles asynchronous task polling and error states, and a setup gate to manage initial configuration. The application is built with Vite and TypeScript, enforcing strict linting rules for color tokens and translation coverage to ensure consistency and maintainability.
web · high confidence
Initial release of the v5 API route surface
The API now exposes a complete v5 route table, replacing the previous implementation. This introduces dedicated endpoints for system health, one-time setup, console authentication (with rate-limiting and spray protection), content parsing and platform-specific reads (Douyin/TikTok), local media downloads with disk-capacity guards, and a searchable archive with keyset pagination. The API also includes bilingual OpenAPI documentation (selectable via ?lang=), iOS Shortcut metadata, and a unified task submission model that returns 202 Accepted for asynchronous operations. Also in this area: Douyin bookmarks endpoint added and role-based access tightened; Identity rotation controls, download file deletion, and proxy error visibility; Fixes for transcription reliability, download profile removal, and MCP result accuracy; Add offline transcription sidecar and API.
src/dtk/api/routes · high confidence
Introduce Alembic migration environment for TimescaleDB
Added the Alembic environment configuration (env.py) and migration script template (script.py.mako) to support database migrations with TimescaleDB. The configuration ensures that continuous aggregates are created outside of transaction blocks to comply with database constraints, and automatically excludes TimescaleDB internal schemas and objects from autogeneration to prevent accidental drops. It also supports both synchronous and asynchronous database connections for migration execution.
src/dtk/db/migrations · high confidence
Introduce async SQLAlchemy data access layer with TimescaleDB support
The \src/dtk/db\ package now provides the core database infrastructure, including ORM models, asynchronous repositories, and migration utilities. Users benefit from a new typed async data access layer that manages transactions via \AsyncSession\ and supports TimescaleDB hypertables for append-only time-series data (request logs, identity events, content snapshots). The package exposes repositories for all primary entities (User, ApiKey, Identity, Proxy, Task, etc.) and provides a Python-based migration runner that locates Alembic revisions relative to the package, ensuring migrations work correctly from installed wheels.
src/dtk/db · high confidence
Introduce backend internationalization and HTTP transport capabilities
This change adds the core infrastructure for backend internationalization (i18n) and a new HTTP transport layer. The i18n module (\src/dtk/i18n\) enables language negotiation via query parameters or Accept-Language headers, supports English and Chinese, and provides localized rendering for error messages and catalog text with robust fallbacks. The transport module (\src/dtk/transport\) introduces a new HTTP client implementation that constructs browser-consistent headers (including User-Agent and Chromium-specific client hints) based on identity fingerprints, supporting browser emulation and request classification.
src/dtk/i18n, src/dtk/transport · high confidence
Introduce browser-rpc service and core identity infrastructure
This change introduces the \browser-rpc\ service, a new Dockerized component that manages browser contexts, identity minting, and platform-specific signing for Douyin and TikTok. It defines a strict backend contract (\BrowserBackend\) to decouple browser logic from service policy, implements geo-alignment to match browser locale and timezone with proxy exit locations, and adds configuration for warm signing contexts and concurrent minting limits. Concurrently, it establishes the core application infrastructure by adding an async SQLAlchemy engine, defining the full database schema (including encrypted credentials and TimescaleDB hypertables for time-series logging), and implementing identity cookie importing with multi-format detection and security masking.
dtk · high confidence
Introduce browser-rpc service for identity minting and signing
A new headless-browser RPC service has been added to the Docker infrastructure to handle identity minting and request signing. This service drives a real browser through an identity's own proxy to obtain guest cookies and fingerprints, and runs the platform's JavaScript to sign requests when the native algorithm is unavailable. It includes robust geo-alignment to match the proxy exit's locale and timezone, and provides specific HTTP endpoints (/rpc/mint, /rpc/sign, /rpc/health) for internal communication, ensuring that browser-based operations are isolated from the main application logic. Also in this area: Support for signed POST requests with body and content-type.
_docker/browser\rpc · high confidence
Introduce hardened, multi-service Docker deployment with optional browser and media components
The docker directory now provides a complete, single-host deployment stack via Docker Compose, introducing three distinct container images: a main Python image (serving API, worker, and migration roles) built with reproducible stages for the React console and Python dependencies; an optional browser-rpc image for headless identity minting using a pinned CloakBrowser backend; and an optional Go-based media downloader image built on scratch for minimal attack surface. The deployment enforces strict security hardening, including non-root execution (uid 10001), read-only root filesystems, dropped capabilities, and resource ceilings (memory, CPU, and PID limits) for all services. A mandatory DTK\_SECRET\_KEY gate prevents startup without encryption credentials, and the stack includes internal networking for data stores (Postgres, Redis) to isolate them from external access. Also in this area: Transcription sidecar support and CloakBrowser version update; Transcriber stability, security, and network isolation improvements; Add offline transcription sidecar and API.
docker · high confidence
Introduce internal transcription API with FunASR ONNX pipeline
Added a new internal FastAPI application in \docker/transcriber/app.py\ that exposes an offline transcription service using the FunASR ONNX model (paraformer-zh-large-onnx-int8). The service handles audio decoding via ffmpeg with a configurable timeout to prevent hanging workers, manages job queuing and worker concurrency, and supports text normalization with a configurable failure policy (fallback or fail). It includes lifecycle management for jobs, progress callbacks, and logging, providing the backend infrastructure for the transcription feature.
docker/transcriber · high confidence
Introduce pluggable browser backend architecture with Cloak and Fake implementations
The browser RPC service now supports a pluggable backend architecture, allowing operators to select between a real browser implementation (Cloak) and a lightweight in-memory stand-in (Fake) via configuration. This change introduces a strict registry that prevents silent fallbacks to the Fake backend, ensuring that service health checks accurately reflect the operational state of the identity pool. The Cloak backend provides a Playwright-based adapter for generating real platform signatures, while the Fake backend enables local development and CI testing by simulating browser behavior without requiring a full Chromium installation.
_docker/browser\rpc/backends · high confidence
Introduces core foundation contracts and toolchain
This change establishes the foundational infrastructure for the application by adding the \src/dtk/core\ module. It introduces a two-layer configuration system that supports hot-reloadable runtime settings alongside bootstrap environment variables, and implements AES-256-GCM encryption for credentials at rest. The update also provides an async SQLAlchemy database engine, a structured logging system with automatic redaction of sensitive data, and a Redis client with atomic Lua script support. Additionally, it defines a comprehensive set of shared enumerations for platforms, content types, task states, and user roles, along with a stable error code hierarchy to standardize API responses.
src/dtk/core · high confidence
Introduction of the dtk console v5 React SPA
The web frontend has been replaced with a new React Single Page Application (SPA) for the dtk v5 API, built to static files and served by the API container without a Node runtime in production. This new console implements a comprehensive design system with strict linting rules (enforcing design tokens, i18n parity, and accessibility), a robust API client with async-first task polling, and a suite of reusable UI components including a responsive DataTable, CodeBlock, and various form controls. The application supports internationalization (English and Chinese), theme toggling, and a setup gate that manages the initial configuration flow.
web/src/components · high confidence
Native signing support with browser fallback
The signing module now includes a native signer that generates signatures without requiring a browser, significantly reducing latency for supported endpoints. For endpoints that require browser-specific session values (such as visitor IDs or tokens not available natively), the system falls back to a browser-rpc signer. This change introduces a configurable signing mode that allows hot-reloading of the signing strategy, ensuring that requests are signed efficiently while maintaining compatibility with platform-specific security requirements.
src/dtk/signing · high confidence
New API foundation with localized console, demo mode, and secure proxy handling
The API module now provides the core application factory, establishing a uniform response envelope and middleware stack that supports language negotiation for both API errors and the served console interface. A new demo role restricts public visitors to read-only operations, while a new mechanism allows operators to selectively expose specific endpoints to unauthenticated callers. Additionally, caller-supplied proxy URLs are now strictly validated and pinned to public addresses in public mode to prevent server-side request forgery.
src/dtk/api · high confidence
New console-triggered maintenance jobs for backup, diagnostics, and identity management
The worker now exposes a dedicated dispatch layer for ten console-triggered maintenance operations, separating them from the upstream platform registry to prevent routing failures. This includes jobs for running a six-step self-check (diagnose), creating and restoring database backups, minting new guest identities, testing identity sessions and probes, sending test notifications, probing proxies, downloading media to disk, transcribing media, and sweeping the archive for availability and backfilling history. Each job runs asynchronously with its own database session scope, ensuring long-running tasks do not block request connections, and returns structured results consumed by the corresponding console pages.
src/dtk/worker/ops · high confidence
New frontend library and React hooks for the console
The web/src/lib directory now contains the core frontend infrastructure for the console, introducing a suite of React hooks and utility modules. This includes a typed API client (web/src/lib/api.ts) that handles response envelopes, error codes, and automatic polling for asynchronous tasks, alongside a centralized endpoint registry (web/src/lib/endpoints.ts). New hooks provide standardized data fetching (useApiQuery), session management (useSession), and UI utilities such as clipboard copying (useCopy), focus trapping for accessibility (useFocusTrap), and responsive media queries (useMediaQuery). The library also adds locale-aware formatting (web/src/lib/format.ts) and internationalization support (web/src/lib/i18n.ts) to ensure consistent display of data and messages across languages.
web/src/lib · high confidence
New operational module for health, backup, diagnostics, and secure masking
A new \dtk.ops\ package has been introduced to provide self-hosting users with essential operational capabilities, including system health checks (liveness/readiness), backup and restore management, diagnostic reporting, notification delivery, and data retention maintenance. This module also introduces a robust masking layer (\dtk.ops.masking\) that redacts credentials, tokens, and secrets from logs, CLI output, and stored probe errors to prevent accidental exposure. Additionally, it includes safe database execution helpers (\dtk.ops.\_sql\) that use savepoints to prevent optional SQL queries from poisoning transactions, ensuring that operational status pages remain functional even when specific database features (like TimescaleDB extensions) are unavailable.
src/dtk/ops · high confidence
New service layer for content archival, caching, and media downloads
This change introduces a new \dtk.services\ package that centralizes core data handling capabilities. It adds an archive module to persist parsed content metadata beyond the immediate task result, a Redis-backed response cache with in-flight request coalescing to reduce redundant upstream calls, and a download service to manage media file retrieval and storage. Additionally, it provides services for managing user collections, handling demo account provisioning, and storing runtime configuration with hot-reload support. Also in this area: Douyin bookmarks endpoint added and role-based access tightened; Identity rotation controls, download file deletion, and proxy error visibility; Fixes for transcription reliability, download profile removal, and MCP result accuracy; Add offline transcription sidecar and API.
src/dtk/services · high confidence
Behavioural changes
41 commits (11 fixes) modifying src/dtk/api/routes/admin
A change to existing behaviour in src/dtk/api/routes/admin — 41 commits (11 fixs), 13 files: “feat(console): say before a login is imported that the account could be banned”; “fix(proxies): say why a proxy failed, probe SOCKS, and stop calling no route a bug (\#781)”; “fix: report identity health, read share text, sign with a jar, warn without failing”; and 38 more (refs \#781, \#789).
(repo-wide) · medium confidence · unverified
Centralized URL resolution and short-link expansion with SSRF protection
The application now uses a dedicated URL handling module to parse, normalize, and expand short links (e.g., from Douyin or TikTok) into canonical resource identifiers. This change introduces a centralized chokepoint that validates every redirect hop against a host allowlist to prevent Server-Side Request Forgery (SSRF) attacks, ensuring that short links cannot redirect to internal or unauthorized addresses. It also corrects previous behavior where valid multi-hop share links were incorrectly flagged as redirect loops, allowing users to successfully resolve and view content from these platforms.
src/dtk/urls · high confidence
Native pure-Python signing replaces browser dependency
The signing module now includes a native, pure-Python implementation for TikTok and Douyin signatures (X-Dynosaur, X-Gnarly, A-Bogus, X-Bogus, and x-secsdk-web-signature), eliminating the need for a browser to generate request signatures. This change shifts the browser's role to identity minting only, while the native signer handles all signature algorithms locally using ported algorithms and inline cryptographic implementations (such as SM3).
src/dtk/signing/native · high confidence
New design system and base styles for the console
The console now uses a dedicated design system with CSS tokens for colors, typography, and spacing, supporting both dark and light themes. This change fixes a usability issue where selected text was nearly invisible by defining explicit selection colors, and introduces consistent page rhythms and form layouts to prevent elements from running off-screen on mobile devices.
web/src/styles · high confidence
Platform adapter discovery and TikTok page-size enforcement
The platform subsystem now automatically discovers available adapters by scanning subpackages for an \\adapter.ADAPTER\\ object, removing the need for manual registration mappings. Additionally, a new paging module enforces platform-specific page-size ceilings (35 for TikTok, 50 for Douyin) to prevent requests that platforms refuse, ensuring that TikTok requests exceeding 35 items are capped to avoid silent data absence errors.
src/dtk/platforms · high confidence
Fixes
11 commits (6 fixes) fixing web/src/pages/identities
A fix in web/src/pages/identities — 11 commits (6 fixs), 16 files: “fix(console): the health cell keeps its label on one line”; “feat(console): say before a login is imported that the account could be banned”; “feat(console): an Identities page that follows the platform and the login”; and 8 more.
web/src/pages/identities · medium confidence · unverified
Test coverage
102 commits adding/updating tests in tests/integration; Added contract test scaffolding for live platform monitoring; Added full-stack smoke test script; Added test infrastructure with isolated policy registry fixture; Added unit tests for platform absence handling, API middleware, and CLI behavior; Initial test suite for the browser-rpc service.
Dependencies
Dependency updates and new service manifests for v5.1.3
This release updates the web console to React 19.3, Vite 8, and ESLint 10, while upgrading the Python backend to use SQLAlchemy 2.1, uvicorn 0.54, and redis 8.1. It also introduces new dependency manifests for the browser-rpc service (pinning FastAPI, uvicorn, and pydantic) and the Go-based downloader (using only the standard library for security isolation). Also in this area: Version bump to 5.1.5; Pin CloakBrowser dependencies and enable SOCKS support in httpx; Pin transcriber dependencies and update web source-map-js; Add transcription dependencies and update source-map-js.
(dependencies) · high confidence
Release of dtk version 5.1.3
The dtk package has been updated to version 5.1.3, as reflected in the module's version identifier. This release includes the standard package initialization and metadata for the self-hosted Douyin/TikTok data API. Also in this area: Douyin bookmarks endpoint added and role-based access tightened; Add offline transcription sidecar and API.
src/dtk · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 58.7 → 59.4 (+0.7)
- Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 58.6 → 56.7 (-1.9)
- Architecture 79.3 → 79.3 (0.0)
- Maturity 73.9 → 76.2 (+2.3)
- Readiness 58.8 → 61.6 (+2.8)
- Security 73.8 → 73.8 (0.0)
- Domain Modelling 54.1 → 54.1 (0.0)
- Accessibility 65.9 → 76.5 (+10.6)
- Performance 100.0 → 100.0 (0.0)
Resolved (24)
- Change-coupling hub: policies.py → content.py, base.py, adapter.py, adapter.py, endpoints.py, params.py, parser.py (src/dtk/scheduler/policies.py)
- Duplicated block (5 lines × 2) (src/dtk/platforms/tiktok/parser.py)
- Duplicated block (5 lines × 4) (src/dtk/platforms/douyin/parser.py)
- Duplicated block (7 lines × 3) (src/dtk/db/migrations/versions/0001_initial_schema.py)
- Duplicated block (8 lines × 2) (src/dtk/db/migrations/versions/0001_initial_schema.py)
- Duplicated block (8–9 lines × 2) (src/dtk/signing/native/tokens.py)
- Hotspot: src/dtk/ops/diagnose.py (src/dtk/ops/diagnose.py)
- Hotspot: src/dtk/scheduler/scheduler.py (src/dtk/scheduler/scheduler.py)
- Hotspot: src/dtk/services/archive.py (src/dtk/services/archive.py)
- Hotspot: src/dtk/services/download_requests.py (src/dtk/services/download_requests.py)
- Hotspot: web/src/pages/Scheduler.tsx (web/src/pages/Scheduler.tsx)
- Hotspot: web/src/pages/downloadRuns.tsx (web/src/pages/downloadRuns.tsx)
- Leaked secret: hardcoded-credential (web/src/pages/Diagnose.tsx)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/dtk/platforms/douyin/parser.py)
- No ADRs found
- Projects may be oversized for their cohesion
- Repeated repair: src/dtk/core/errors.py (src/dtk/core/errors.py)
- Secret: generic-api-key (crawlers/tiktok/web/config.yaml)
- Secret: generic-api-key (crawlers/tiktok/web/config.yaml)
- Secret: generic-api-key (crawlers/tiktok/web/config.yaml)
- …and 4 more
New (57)
- Change-coupling hub: policies.py → content.py, base.py, adapter.py, adapter.py, endpoints.py, params.py (src/dtk/scheduler/policies.py)
- Duplicated block (5 lines × 3) (src/dtk/platforms/douyin/parser.py)
- Edited copies of one member (3 members, 11+ corresponding lines) (src/dtk/api/routes/admin/identities.py)
- Edited copies of one member (3 members, 27+ corresponding lines) (src/dtk/platforms/douyin/parser.py)
- Edited copies of one member (3 members, 43+ corresponding lines) (src/dtk/api/routes/content.py)
- Edited copies of one member (3 members, 5+ corresponding lines) (docker/browser_rpc/geo.py)
- Edited copies of one member (3 members, 7+ corresponding lines) (src/dtk/db/repositories.py)
- Edited copies of one member (3 members, 8+ corresponding lines) (src/dtk/platforms/douyin/parser.py)
- Edited copies of one member (4 members, 21+ corresponding lines) (src/dtk/api/routes/discovery.py)
- Edited copies of one member (4 members, 7+ corresponding lines) (src/dtk/mcp/tools.py)
- Edited copies of one member (7 members, 40+ corresponding lines) (src/dtk/api/routes/content.py)
- Edited copy of a member (10 corresponding lines) (src/dtk/platforms/douyin/parser.py)
- Edited copy of a member (10 corresponding lines) (src/dtk/platforms/tiktok/parser.py)
- Edited copy of a member (11 corresponding lines) (src/dtk/mcp/tools.py)
- Edited copy of a member (13 corresponding lines) (src/dtk/identity/importing.py)
- Edited copy of a member (13 corresponding lines) (src/dtk/platforms/douyin/parser.py)
- Edited copy of a member (13 corresponding lines) (src/dtk/platforms/douyin/parser.py)
- Edited copy of a member (13 corresponding lines) (src/dtk/services/archive.py)
- Edited copy of a member (14 corresponding lines) (src/dtk/api/routes/downloads.py)
- Edited copy of a member (14 corresponding lines) (src/dtk/api/routes/system.py)
- …and 37 more
Changes since last survey
- 11 commits — 5 feature/other, 6 fixes
By area
- src/dtk — 6 commits
- documents/en — 2 commits
- (root) — 1 commit
- .github/RELEASE_v5.1.11.md — 1 commit
- web/src — 1 commit
Notable commits
- fix: fix(console): the single-post form follows a short link instead of refusing it
- fix: fix(db): retention blanks a task payload once, not on every pass
- fix: fix(downloads): a post whose media is withheld fails with that reason
- fix: fix(tiktok): a page of shop videos is not read as a schema change
- fix: fix(tiktok): one post without media no longer fails its whole list page
- fix: fix(tiktok): the post search cursor carries the search id page two needs
- change: chore: release 5.1.11
- change: docs: pin the real build sha in the 5.1.11 release notes
- change: feat(build): apt and PyPI mirrors for the images built on this machine
- change: feat(errors): MEDIA_WITHHELD for a post whose media the platform will not give out
- change: perf(sm3): faster SM3 _expand -- A-Bogus signing bench 9.5% faster (#821)
Architecture
- Unchanged — 0 containers · 1 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
Evil0ctal/Douyin_TikTok_Download_API was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 17 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 3c045e11aecd58ab076562f1246931b9dd5d909b — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.84+preprod-d645ca9bbedd.