webhooks: add Pipedrive triggers (24 slugs, v2 envelope)

@anshugarg15checks n/achecks…webhook/pipedrive-triggers → master30 files · +1470 −277updated 1mo ago

▸Description

Description

Adds the full Pipedrive Webhooks v2 trigger catalog: 24 slugs covering create / change / delete for 8 entities (deal, person, organization, activity, note, lead, product, user). The 3 triggers that shipped before this PR (PIPEDRIVE_NEW_DEAL_TRIGGER, PIPEDRIVE_NEW_NOTE_TRIGGER, PIPEDRIVE_NEW_ORGANIZATION_TRIGGER) keep their original contract verbatim — same slug, same flat Optional[str] payload schema, same JSON {"webhook_id": <id>} hook_id format, same display_name. The other 21 are net-new and follow <entity>_<action> naming with the modern envelope design.

Key design + implementation decisions

Single base class, two override hooks (_base.py is shared by all 24 triggers):

_build_hook_id(webhook_id) — default returns USER_LEVEL_HOOK_ID (apollo routes via webhook_endpoint_id). Legacy 3 override to return the JSON {"webhook_id": <id>} string that matches their persisted trigger_instance.webhookId DB rows. Apollo's broadcast filters by exact WHERE webhookId = :hook_id, so changing this shape silently breaks existing subscriptions — DO NOT touch the legacy overrides without a DB migration.
_transform(body) — default produces {meta, data, previous} envelope (used by 21 new triggers). Legacy 3 override with their original flat-shape str(...)-coerced transform.

No flag, no parallel hierarchies. Legacy and new classes coexist in one inheritance tree because they override exactly the methods that differ. Adding new triggers later → inherit the defaults. Touching the base → legacy untouched because they override what matters.

Payload schema design for new 21 triggers:

Each new trigger has its own inline *Payload Pydantic class (no shared _payloads.py consolidation). The structural similarity across 21 classes (meta + data + previous) IS NOT shared via a common module because the field descriptions differ per action — data on create.deal says "the newly-created deal" while data on delete.deal says "always null; see previous for the deleted deal's last-known state". These descriptions are customer-facing API documentation (OpenAPI spec, SDK type generation, dashboard help text), and templating / consolidating them produces generic-per-entity strings that lose the action-specific clarity.
data and previous are both Optional[<Entity>Data] = None uniformly across all 21 new triggers. Same shape regardless of action — only null-placement varies (create → previous is null, change → previous is the diff, delete → data is null). Customer writes one handler that works across create/change/delete; the field description on each specific trigger tells them what to expect for that action.
Wire fidelity: data: null on delete events is preserved end-to-end (not normalized to {}). previous's name comes from Pipedrive's wire envelope verbatim — semantically inconsistent across actions (diff on change, full last state on delete) but kept as-is so Composio's docs use the same vocabulary as Pipedrive's.

Customer payload curation strips 7 provider infra meta fields (webhook_id, webhook_owner_id, attempt, version, correlation_id, permitted_user_ids, type) and keeps 10 stable fields (id, action, entity, entity_id, company_id, timestamp, user_id, host, change_source, is_bulk_edit). The *Data sub-models use ConfigDict(extra="allow") so new provider fields pass through to the customer instead of being silently dropped — Pipedrive does not publish a versioned schema.

WebhookSpec: registration_type: user_level, signature: null (Pipedrive staff verbatim: "We currently do not offer any signing options."), handshake: null (no challenge on subscribe), event_parser: single with dedup on body.meta.id (per-event UUID stable across Pipedrive's 3-retry attempts).

Backwards compat (the rule that drove half the design): existing pipedrive trigger_instances on prod have specific persisted state — webhookId column values in JSON shape, and customer downstream workflows wired to the flat Optional[str] payload field names. The 3 legacy classes preserve both contracts verbatim, including the bug-for-bug behavior of str(None) producing the literal string "None", cc_email mapped to email, etc. Fixing those bugs would be a customer-visible breaking change; defer to a deprecation cycle on dedicated new slugs if ever needed.

Authz model

See in-PR apps/pipedrive/WEBHOOK_TRIGGER_AUTHZ.md. Customer-facing security boundary verbatim:

Composio creates a webhook in Pipedrive bound to the connecting user's user_id via POST /webhooks with the user's OAuth token (admin or webhooks:full scope). Pipedrive enforces a per-event permission filter at delivery time — "Webhooks about objects are not sent if the selected user does not have permitted access to them." — meaning Composio receives only events for objects the connecting user can see in the Pipedrive UI; when the user loses permission on a resource, events for that resource silently stop flowing. Token revocation is detected within ≤1 h via the auth-refresh sweeper, and instantly via Pipedrive's OAuth DELETE uninstall callback when wired. Pipedrive does not sign webhook deliveries; the security boundary on the receiver path relies on the unguessability of Composio's per-CA webhook URL, the provider's strict user_id permission filter, and (when configured) HTTP Basic Auth.

How did I test this PR

Live-stack e2e (apollo + thermos + mercury + cloudflared, real Pipedrive sandbox)

Ran the full webhook-trigger e2e runbook against a real Pipedrive company (anshutech.pipedrive.com, API-token-auth CA cloned from staging into local DB). Setup: local hermes started with OVERWRITE_APOLLO_URL=<cloudflared tunnel>, 21 trigger_instances upserted via POST /api/v3/trigger_instances/<SLUG>/upsert, real provider events fired by hitting Pipedrive's REST API for every entity × action where physically possible.

Per-trigger validation matrix

Trigger slug	Shape	Hook_id format	Verified
`PIPEDRIVE_NEW_DEAL_TRIGGER`	FLAT (legacy)	`{"webhook_id": "<id>"}`	✓
`PIPEDRIVE_NEW_NOTE_TRIGGER`	FLAT (legacy)	`{"webhook_id": "<id>"}`	✓
`PIPEDRIVE_NEW_ORGANIZATION_TRIGGER`	FLAT (legacy)	`{"webhook_id": "<id>"}`	✓
`PIPEDRIVE_DEAL_UPDATED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_DEAL_DELETED`	ENVELOPE (`data: null`)	`__user_level__`	✓
`PIPEDRIVE_NEW_PERSON`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_PERSON_UPDATED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_PERSON_DELETED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_ORGANIZATION_UPDATED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_ORGANIZATION_DELETED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_NEW_ACTIVITY`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_ACTIVITY_UPDATED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_ACTIVITY_DELETED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_NOTE_UPDATED`	ENVELOPE	`__user_level__`	✓
`PIPEDRIVE_NOTE_DELETED`	ENVELOPE	`__user_level__`	✓

21/24 validated against real Pipedrive traffic. Pydantic schemas parse field-for-field for every captured event.

3 user. triggers could not be exercised*: the test company has only the OAuth-installing user. create.user would invite a real human via email (intrusive). delete.user would expel the only admin (destructive). change.user — PUT /users/{id} only accepts active_flag; Pipedrive blocks self-deactivation; no other writable user fields exist. For these 3, code-only validation was performed: classes load, match() returns true on a synthesized v2 envelope with entity=user, *Payload schemas instantiate. They use the same envelope shape + dispatch as every other entity's verified triggers.

Discovered bugs (all fixed in this PR)

Four real bugs surfaced from running events against the live stack — none were visible in the original implementation or in provider docs:

Lead entity_id is a UUID string, not int. Initial PipedriveEventMeta.entity_id was typed Optional[int]. First create.lead event failed Pydantic validation: "Input should be a valid integer, unable to parse string as an integer" on value '87b689b0-4c8e-11f1-9e94-f1d2fd3c13dd'. Pipedrive uses int ids for every other entity but UUID strings for leads — provider docs do not flag this. Fix: relaxed type to Optional[Union[int, str]] in _base.py.
delete.<entity> envelope inverts data and previous. First delete.deal capture showed Pipedrive actually sends data: null (empty) and previous: {full last-state of the resource}. Provider docs do not document this. Fix: per-trigger payload descriptions on the delete classes explain it; _data() helper preserves null wire-faithfully (does NOT coerce to {}).
The pre-existing 3 transforms had silent bugs. The original new_deal.py / new_note.py / new_organisation.py coerced every field with str(...), which turns None into the literal string "None", drops type information, etc. NOT FIXED IN THIS PR — those bugs ARE the customer contract for those 3 triggers; fixing them is a customer-visible breaking change. Documented but preserved verbatim for backwards compat.
activity.due_time and activity.duration are objects, not strings. First change.activity capture failed Pydantic with "Input should be a valid string ... input_value={'timezone_id': None, 'value': ''}". Pipedrive serializes activity time fields as a {timezone_id, value} object, not a plain HH:MM string as the docs imply. Fix: relaxed due_time and duration types in PipedriveActivityData.

Operational notes

Apollo /api/v3.1/webhook_ingress is slow in pnpm dev (~10s render time → Pipedrive's 30s timeout aborts the delivery silently). Known dev-mode quirk. For local testing, registered Pipedrive webhook URLs were swapped from /api/v3.1/... to /api/v3/... (same handler, faster route in dev). Production uses both paths backed by the same handler.
Pipedrive caps webhooks at 40 per user per app. With 24 trigger slugs in the catalog, a single CA enabling every trigger uses 24/40 of the budget.
No webhook signing. Pipedrive does not provide HMAC signatures. Receiver-path security relies on the unguessability of Composio's per-CA webhook URL, the provider's user_id permission filter, and (optionally) HTTP Basic Auth.
Test data cleanup: all test webhooks created on the Pipedrive sandbox during e2e were deleted after capture; all test deals/persons/organizations/activities/notes/leads/products were deleted.

🤖 Generated with Claude Code

loading diff…