Webhook Integration and Reliability: Signature Verification, Retries and Idempotency

Webhooks let you "subscribe" to events in a system; when an event occurs, the platform pushes data to your server automatically instead of you polling an API. After investigating more than 100 webhook providers, webhooks.fyi points out that while webhooks are universal in concept, they are largely unstandardized API contracts whose security controls and operational experience vary widely. This guide combines webhooks.fyi's best practices with the official Stripe and GitHub documentation to lay out a reliable integration approach. For detailed signature code, see our Webhook Signature Verification Implementation.

A real cautionary tale: a team integrating payment callbacks checked only the event type, never the signature. An attacker forged a batch of payment_intent.succeeded events, the system marked unpaid orders as paid and triggered shipping, and the damage was only discovered later — the endpoint was effectively "naked". That kind of incident is entirely preventable with the three steps below: verify, dedupe, respond fast.

Event Models: Understand the Delivery Rules First

  • Webhooks are event-driven: you express interest once when creating the webhook, then receive data only when events occur (GitHub's examples include triggering CI on push, PR review notifications, and deploying to production).
  • Each event typically carries an event ID, event type, and event object (data.object), such as Stripe's payment_intent.succeeded.
  • Events are not guaranteed to arrive in order: Stripe explicitly states it does not guarantee delivery in the order events were generated. Creating a subscription produces customer.subscription.created, invoice.created, invoice.paid, and more, so handlers must not depend on ordering and may need to fetch missing objects via the API.

Signature Verification: Security First

Webhook endpoints without signature verification are a major risk — attackers can forge events to trigger "fulfill orders, grant access, modify records" operations. Both Stripe and GitHub use HMAC signatures:

  1. The platform computes HMAC-SHA256 over timestamp + "." + raw request body with the endpoint secret.
  2. The signature travels in a header (e.g. Stripe-Signature: t=...v1=...).
  3. Your server recomputes with the same secret and compares using a constant-time comparison to prevent timing attacks.
  4. Check the timestamp against the current time (Stripe's default tolerance is 5 minutes) to defend against replay attacks.

Always verify with official libraries, and verify using the raw request body — if your framework rewrites the body, verification will fail.

In Node.js, for example, you can verify the signature and check the timestamp for replay protection as follows:

const crypto = require('crypto');

function verifySignature(payload, signatureHeader, secret) {
  const [tsPart, sigPart] = signatureHeader.split(',');
  const timestamp = tsPart.split('=')[1];
  const expected = sigPart.split('=')[1];
  const signed = `${timestamp}.${payload}`;
  const actual = crypto
    .createHmac('sha256', secret)
    .update(signed)
    .digest('hex');

  // Constant-time comparison plus a time window check (anti-replay)
  const ok = crypto.timingSafeEqual(
    Buffer.from(actual), Buffer.from(expected)
  ) && (Date.now() / 1000 - Number(timestamp)) < 300;

  return ok;
}

In production, prefer the platform's official SDK (such as stripe.webhooks.constructEvent); the code above is for understanding the mechanics.

Automatic Retries and Idempotency

  • Automatic retries: Stripe retries failed deliveries over roughly three days with decreasing frequency; GitHub also retries a limited number of times. A 2xx response counts as success; 4xx/5xx, timeouts, and TLS errors all trigger retries.
  • Idempotent deduplication: the same event may be delivered more than once. The documented best practice is to record processed event IDs and skip duplicates, never re-run business logic such as double-billing.
  • Respond 2xx quickly: Stripe explicitly requires returning a success status before executing complex logic that could time out, then moving heavy processing to a background queue to avoid retry storms. For async processing, see our Background Jobs and Message Queues Guide.

Stripe's default retry cadence is roughly: the first retry about a minute after the initial failure, with intervals stretching to hours over an overall window of about three days. A pragmatic way to make delivery idempotent is to keep processed event IDs in a unique database index:

CREATE TABLE processed_events (
  event_id   VARCHAR(64) PRIMARY KEY,
  processed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Before handling an event, run INSERT ... ON CONFLICT DO NOTHING; if the insert affects zero rows, the event was already processed, so return 2xx and skip.

One more easily overlooked case: a retried delivery can arrive after the business state has already changed. For example, a user cancels an order and the platform later retries a "payment succeeded" event; if the handler blindly updates state from the event, it flips the cancelled order back to paid. A safer approach is to record the event ID together with the business state and compare the current state before acting, so stale events cannot overwrite newer state.

Security Hardening Checklist

  • HTTPS is mandatory: production endpoints must be HTTPS; Stripe requires TLS v1.2+.
  • IP allowlisting: receive events only from the platform's fixed IP ranges, filtered at the firewall.
  • Subscribe only to what you need: do not listen to every event; it reduces noise and attack surface.
  • Rotate secrets regularly: rotate immediately on suspected compromise, with an overlap window for old and new secrets.
  • Exempt CSRF on the webhook route: Django/Rails frameworks validate CSRF tokens by default; exempt the webhook route because signature verification already authenticates the request.

Scenario-Based Advice

  • Payment callbacks: pair with our Stripe Payment Integration Guide and align events like checkout.session.completed with local order state.
  • CI/CD triggers: GitHub webhooks drive pipelines; see GitHub Actions CI/CD Guide.
  • Building your own webhook platform: provide signatures, retries, and delivery logs for your consumers, following webhooks.fyi's provider best practices. Delivery logs matter most — record each request's URL, body, response code, and duration so both sides can quickly tell whether an event was "never sent" or "handled with an error".

16IDC perspective

Webhooks are the "reverse API": your server shifts from consumer to callee, and with it comes the reliability responsibility. Get signature verification, idempotent deduplication, and fast responses right, and integrations like payments, CI, and notifications become rock-solid. More backend engineering practices live in the Backend Integration category.

Source: https://webhooks.fyi/
Reference: webhooks.fyi best practices https://webhooks.fyi/guide/best-practices/; Stripe webhook docs https://docs.stripe.com/webhooks; GitHub webhook events https://docs.github.com/en/webhooks