# Receive data from other systems Most write paths into a Homespun app authenticate as the agent or a signed-in human. An **inbound catch-hook** is the exception: it gives an external system a secret URL to POST JSON to, and that JSON is written into one of your app’s collections with no agent online. Use it for Stripe events, a Zapier or Make push, a Home Assistant automation, or an email router that turns a message into a webhook. ## Declare a hook in the manifest [Section titled “Declare a hook in the manifest”](#declare-a-hook-in-the-manifest) Hooks are declared in `x-homespun-manifest.ingest`, validated at deploy exactly like `webhooks`, and materialized into a secret URL. There is no dashboard-created or agent-created hook: you add one by editing the manifest and redeploying. ``` "ingest": [ { "name": "stripe-payments", "collection": "payments", "mode": "upsert", "upsertOn": "external_id", "map": { "external_id": "id", "amount": "data.object.amount", "customer": "data.object.customer" }, "dedupeKey": "id", "wake": true } ] ``` Field by field: * `name` (required): unique within the app, `[a-z0-9][a-z0-9_-]{0,63}`. This names the hook and survives redeploys: the same name keeps the same URL, so a sender you configured weeks ago keeps working. * `collection` (required): the declared collection the incoming row is written into. * `mode`: `append` (default) writes a new row per delivery; `upsert` merges onto an existing row. * `upsertOn`: required when `mode` is `upsert`, and must name one of the collection’s unique fields. A redelivery with the same key updates the row in place. * `map` (optional): target row field to a dot-path into the JSON body. Absent means the raw default row `{ hook, payload, receivedAt }`, with the whole body under `payload`. A path that resolves to nothing omits that field, it never writes a null. * `dedupeKey` (optional): where the value that dedupes a redelivery comes from, either a dot-path into the JSON body (the default form, e.g. `"id"`) or `header:`, a reference to a request header (e.g. `"header:x-github-delivery"`), for a sender whose replay id lives in a header rather than the body: GitHub puts its delivery id in `X-GitHub-Delivery` and its webhook bodies carry no equivalent stable id of their own. `` is a lowercase HTTP header name; the lookup is case-insensitive regardless of how the header arrives. A second delivery carrying a value already seen is acked `200` and recorded as `dropped_duplicate`, not written again. Set this for any sender that retries (Stripe redelivers). This only ever guards an ACCEPTED delivery: a `failed` delivery releases its dedupe value, so retrying the SAME logical write under the SAME `dedupeKey` value after a failure (a fixed schema violation, or a stale `if_match`, see below) is a real write attempt, never a silently dropped duplicate. With no `dedupeKey` at all, every delivery (including an exact redelivery) writes a new row: the dedupe value is null, and null values never collide. A `header:` dedupeKey never resolves during `homespun ingest backfill`, since a backfill replays stored bodies with no live request headers to read; a body-path `dedupeKey` still works there. * `handshake`: set to `echo` to answer a Slack or Microsoft Graph URL-verification challenge. * `verify` (optional): opt into body-signature verification. `{ "scheme": "github" }` requires a valid GitHub HMAC-SHA256 signature over the raw body (see below). Strictly opt-in: without it the URL secret is the only check. * `wake`: `true` auto-wakes a dormant app on delivery (default `false`). ## Avoid lost updates with if\_match [Section titled “Avoid lost updates with if\_match”](#avoid-lost-updates-with-if_match) A backend that reads a row, computes for a while, and writes the result back through a `mode: "upsert"` hook has no way to tell “nobody touched this row since I read it” unless the write can carry the version it read. Send a top-level `if_match` in the POST body, the exact field name and type the direct `PATCH /v1/apps/:id/collections/:name/:key` route’s body already uses: ``` { "id": "cust_42", "status": "reviewed", "if_match": 3 } ``` The delivery is checked against the row’s current `version` (the same counter a PATCH or DELETE’s `if_match` checks) before it writes: * Match: the write lands, and the row’s `version` increments, exactly like a matching PATCH. * Mismatch: the write is refused with a retryable `409 conflict`, not acked `200`, so a compute-then-write-back backend learns synchronously to re-read the row and retry rather than believing a stale write landed. This is the ONE other sender-visible failure past the guard checks, alongside the `503` a full app returns (see “What a sender sees” below). * A malformed `if_match` (not a non-negative integer) is rejected as `invalid_request` and journaled `failed`, the same as any other malformed payload; it is never silently ignored, which would defeat the whole point of asking for it. **`if_match` is checked only for a `mode: "upsert"` hook**, and only once the delivery’s natural key actually matches a live row. A `mode: "append"` hook always writes a brand-new row, so there is never an existing version for it to be stale against; an `if_match` key in an append body is inert. The very first delivery for a natural key (nothing to match yet) also ignores it, for the same reason: a fresh row has no prior version to compare against. **Everywhere `if_match` is not sent, the hook behaves exactly as before: last-write-wins.** This is the default for every hook that does not use the field, and for every `append` hook regardless. Omitting it is not a regression; it is today’s behaviour, unchanged. **A corrective retry after a `409` should reuse the SAME `dedupeKey` value, if the rule declares one.** It is the same logical write (re-read, recompute, resend), so reusing its identifier is the natural choice, and it is safe: a `failed` delivery (a version conflict included) releases its dedupe slot, so the retry is a real write attempt against the row’s current version, not a `dropped_duplicate` that never reaches the write. Only an `accepted` delivery keeps its slot. ## Nullable fields [Section titled “Nullable fields”](#nullable-fields) A mapped field arrives one of two ways, and the collection schema has to allow both: * The source path is **absent** from the body: the field is omitted from the row (it is never written as `null`). * The source path is **present with an explicit `null`**: the field is written as `null`. So a sender that sends `null` for a field, rather than leaving it out, needs that field typed as nullable in your collection schema, or the delivery fails validation. GitHub is the classic case. A `workflow_job` webhook sends `"conclusion": null`, `"completed_at": null`, and `"runner_name": null` on the `queued` and `in_progress` events, with the real values arriving only on `completed`. A collection that types those as a plain `"string"` accepts every `completed` delivery and rejects every `queued` / `in_progress` one with `row_schema_violation`, even though the payloads look identical in shape. Type any field a sender can send as `null` with a nullable JSON Schema type: ``` { "conclusion": { "type": ["string", "null"] } } ``` When a delivery does fail this way, the `failed` delivery’s detail names the field and the received value, for example `/conclusion must be string (received: null)`, so the cause is visible in the delivery log without diffing a working payload against a failing one. Fix the schema, redeploy, and replay the failed deliveries (see below). ## Verify GitHub signatures [Section titled “Verify GitHub signatures”](#verify-github-signatures) By default the URL secret is the whole authentication. For a GitHub webhook you can require a signature over the body as well, so a leaked URL alone cannot post forged deliveries. Add `verify` to the rule and provision a signing secret: ``` "ingest": [ { "name": "gh-issues", "collection": "issues", "verify": { "scheme": "github" } } ] ``` 1. Deploy the manifest. The hook is **fail-closed** until you provision its signing secret: every delivery gets `401` until you complete step 2. 2. Mint the signing secret and copy the value it shows once: Terminal window ``` homespun ingest signing-secret set --app --name gh-issues ``` 3. In the GitHub webhook settings, paste that value into **Secret**, and set **Content type** to `application/json` (a `application/x-www-form-urlencoded` webhook gets `415`, since the endpoint accepts JSON only). The relay reads GitHub’s `X-Hub-Signature-256` header, recomputes the HMAC over the exact bytes GitHub sent, and compares in constant time. A missing, malformed, or wrong signature (and a hook whose signing secret is not set) all return one uniform `401`, and the rejected request is **not** recorded in the delivery log. Rotate the secret with `homespun ingest signing-secret set` again: the previous value keeps verifying for a grace window so deliveries do not drop while you update GitHub. ## The URL [Section titled “The URL”](#the-url) After deploy, read back the hook’s full secret URL: Terminal window ``` homespun ingest list --app ``` Each hook returns a `url` shaped `https:///v1/ingest//`. The secret in the URL is the whole authentication: no header, no cookie. Hand that URL to the owner to paste into the external system, and treat it like a password. If it leaks, rotate it (the old URL stops working immediately, no redeploy needed): Terminal window ``` homespun ingest rotate --app --name stripe-payments ``` An owner can also see the URL, a ready-made `curl` test, the rotate button, and the delivery log in the app’s dashboard under **Inbound hooks**. ## What a sender sees [Section titled “What a sender sees”](#what-a-sender-sees) The endpoint is fire-and-forget. Past a handful of guard checks it always acks `200 {"ok":true}` and records the outcome in the delivery log, so a sender never retries into a mapping bug forever. The failures a sender can see are `429` (rate or hourly cap), `404` (wrong URL, uniform so it is not an existence oracle), `415` (not `application/json`), `413` (body over the size cap), `503` (the app is at its row/storage quota, see the delivery journal for the retry story), and `409` (an `if_match` version mismatch, see “Avoid lost updates with if\_match” above: re-read the row and retry). Once accepted, the outcome is one of: * `accepted`: the row was written (and flows through feed replay, live delivery to a connected agent, notify rules, and outbound webhooks like any other write). * `dropped_duplicate`: a redelivery of a `dedupeKey` value an ACCEPTED delivery already used; no row written. A `failed` delivery’s value does not count as seen (it released its slot), so a corrective retry under the same value is never mistaken for this. * `failed`: the write was rejected. For most reasons (a schema violation, an append-only conflict, a malformed `if_match`) the sender still got its `200`; the failure is yours to see in the log. Two reasons are NOT acked `200`, because a silent success would be actively wrong: the app being at its row/storage quota (`503`, see above) and an `if_match` version mismatch (`409`, see “Avoid lost updates with if\_match” above). Both are still journaled `failed` here, so the log reads the same either way. ## Inspect and replay [Section titled “Inspect and replay”](#inspect-and-replay) List recent deliveries, filter by hook or status: Terminal window ``` # via the HTTP API (agent-or-owner auth) GET /v1/apps//ingest/deliveries?hook=stripe-payments&status=failed&limit=25 ``` Each delivery keeps a capped snapshot of the received payload. That powers the debugging loop: when a delivery `failed` because your `map` was wrong, fix the manifest, redeploy, and **replay** the stored payload through the corrected rule: Terminal window ``` POST /v1/apps//ingest/deliveries//replay ``` Replay respects the current manifest: a hook renamed or removed since the original delivery gives a clean error rather than writing into the wrong place. A replay bypasses dedupe (you asked for it explicitly), so an `append` hook can produce a duplicate row on replay, while an `upsert` hook just merges onto the same key again. The delivery journal is pruned automatically (by age and a per-app cap), so it is a rolling window for debugging, not a permanent store: the collection rows are the durable record.