📚 Audit-log series — stack on master (merge bottom → top)
- #10603 — part 1: Tier-1 audited dbUtils +
AuditAction enum → master (reland of #10514, reverted in #10584)
- #10515 — part 2:
auditedTransaction (branded AuditedTx) → #10603
- #10516 — part 3: Tier-2 pipeline + coverage middleware → #10515
- #10517 — part 4: read API (internal-dashboard) → #10516
Supersedes the production-based stack (#10355 / #10457 / #10510 / #10511 / #10513).
You are here: #10516
Description
Part 3 (on master, stacks on #10515). The async Tier-2 path: non-critical control-plane events go through a durable queue into ClickHouse, plus blanket coverage via a global after-middleware.
- Queue → sink → ClickHouse:
queue/ (AuditQueue port + Vercel send), sink/ (ClickHouse async_insert), record_audit.ts (resolve actor → publish), consumer route app/api/queues/audit/route.ts (handleCallback → sink), vercel.json trigger.
- CH
audit_logs DDL appended to clickhouse/schema.sql (prod SharedReplacingMergeTree) + docker-init (local), with MATERIALIZED is_internal/credential_id/actor_* + 3yr TTL.
- Coverage after-middleware (
server/middleware/audit_after.ts) wired into getAuthRouter: map-gated, best-effort, never alters the response. The route→action map (actions.ts) reuses the AuditAction enum from Tier-1's schema.ts (duplicate enum removed).
- Adds
@vercel/queue.
How did I test this PR
⚠️ Static-only (worktree has no deps). Two follow-ups for CI/review:
pnpm install must regenerate pnpm-lock.yaml for the new @vercel/queue dep — I couldn't run it here, so frozen-lockfile CI will fail until it's regenerated.
- Known minor:
deriveTargetId picks the last path param, so auth_configs/{nanoid}/{status} records {status} as the target — needs a per-route override (carried over from the earlier middleware PR).
🤖 Generated with Claude Code
Update (chorm port)
audit_logs is now defined in chorm (packages/db/src/clickhouse/audit_logs.ts, exported via @composio/db/clickhouse) as the source of truth; createTableSql(audit_logs) matches clickhouse/schema.sql exactly and doubles as the provisioning path.
- Sort key fixed:
ORDER BY (org_id, project_id, createdAt, id) — the previous (project_id, …) key left org-wide reads unindexed. Changed while the table exists nowhere.
- Sink writes through chorm's typed
insert() (InferInsert excludes MATERIALIZED columns by construction); async_insert settings injected at the service layer over the existing singleton client. Hand-rolled AuditClickHouseRowSchema removed.
@clickhouse/client bumped ^1.11 → ^1.19 in apollo to dedupe with chorm; lockfile regenerated for @vercel/queue (the known CI follow-up).