Payment Webhook System

Overview

Production-grade webhook handling system for Stripe: signature verification, idempotent processing, retry logic, event logging, and manual replay. Ensures no payment events are missed even during outages.

Use Cases

• Payment confirmations: Fulfill orders only after webhook confirms payment
• Subscription lifecycle: Handle renewals, upgrades, cancellations, payment failures
• Dispute handling: Track chargebacks and refund requests
• Compliance: Audit trail of all payment events for accounting

When to Use This Pattern

Use this pattern when you need to:

• Process payment events reliably (never miss a webhook)
• Handle webhook retries from Stripe (automatic exponential backoff)
• Prevent duplicate processing of the same event
• Log all webhook events for debugging and compliance
• Manually replay failed webhooks
• Monitor webhook delivery health

Pro Tips

Critical concepts for webhook reliability:

1. Always verify webhook signatures - Prevents spoofed webhooks from attackers
2. Webhooks are asynchronous - User might see success before webhook processes
3. Stripe retries failed webhooks - 3 days with exponential backoff (immediate, 1h, 6h, 24h...)
4. Idempotency is critical - Same event may be delivered multiple times
5. Return 200 ASAP - Process async, don't block webhook response

Implementation Phases

Phase 1: Database Schema for Event Logging

Track every webhook event:

model WebhookEvent {
  id              String   @id @default(uuid())
  stripeEventId   String   @unique // Stripe's event ID (evt_...)
  type            String   // payment_intent.succeeded, invoice.paid, etc.
  data            Json     // Full event payload
  processed       Boolean  @default(false)
  processingError String?  // Error message if processing failed
  attempts        Int      @default(0)
  lastAttemptAt   DateTime?
  createdAt       DateTime @default(now())
  updatedAt       DateTime @updatedAt

  @@index([type, processed])
  @@index([createdAt])
}

Why log every event:

• Debugging: See exactly what Stripe sent
• Audit trail: Compliance and accounting
• Manual replay: Reprocess failed events
• Analytics: Track webhook delivery health

Phase 2: Webhook Endpoint with Signature Verification

Secure webhook handler (app/api/webhooks/stripe/route.ts):

import Stripe from 'stripe'
import { NextResponse } from 'next/server'
import { headers } from 'next/headers'
import { prisma } from '@/lib/prisma'

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!, {
  apiVersion: '2024-11-20.acacia',
})

export async function POST(request: Request) {
  const body = await request.text()
  const signature = headers().get('stripe-signature')

  if (!signature) {
    return NextResponse.json(
      { error: 'Missing stripe-signature header' },
      { status: 400 }
    )
  }

  let event: Stripe.Event

  try {
    // Verify webhook signature
    event = stripe.webhooks.constructEvent(
      body,
      signature,
      process.env.STRIPE_WEBHOOK_SECRET!
    )
  } catch (err) {
    console.error('⚠️  Webhook signature verification failed:', err)
    return NextResponse.json(
      { error: 'Invalid signature' },
      { status: 400 }
    )
  }

  console.log(`✅ Webhook received: ${event.type}`)

  try {
    // Process webhook asynchronously
    await processWebhookEvent(event)

    // Return 200 immediately (Stripe expects fast response)
    return NextResponse.json({ received: true })
  } catch (error) {
    console.error('❌ Webhook processing failed:', error)

    // Still return 200 to Stripe (we logged the error, will replay manually)
    // If you return 4xx/5xx, Stripe will retry indefinitely
    return NextResponse.json({ received: true })
  }
}

async function processWebhookEvent(event: Stripe.Event) {
  // Check if event already processed (idempotency)
  const existingEvent = await prisma.webhookEvent.findUnique({
    where: { stripeEventId: event.id },
  })

  if (existingEvent?.processed) {
    console.log(`⏭️  Event ${event.id} already processed, skipping`)
    return
  }

  // Log event to database
  const webhookEvent = await prisma.webhookEvent.upsert({
    where: { stripeEventId: event.id },
    update: {
      attempts: { increment: 1 },
      lastAttemptAt: new Date(),
    },
    create: {
      stripeEventId: event.id,
      type: event.type,
      data: event.data.object as any,
      attempts: 1,
      lastAttemptAt: new Date(),
    },
  })

  try {
    // Route to appropriate handler
    await handleWebhookEvent(event)

    // Mark as processed
    await prisma.webhookEvent.update({
      where: { id: webhookEvent.id },
      data: {
        processed: true,
        processingError: null,
      },
    })

    console.log(`✅ Event ${event.id} processed successfully`)
  } catch (error) {
    // Log error but don't throw (return 200 to Stripe)
    const errorMessage = error instanceof Error ? error.message : 'Unknown error'

    await prisma.webhookEvent.update({
      where: { id: webhookEvent.id },
      data: {
        processingError: errorMessage,
      },
    })

    console.error(`❌ Event ${event.id} processing failed:`, errorMessage)

    // Optional: Send alert to Sentry, Slack, etc.
    await sendWebhookErrorAlert(event, errorMessage)
  }
}

Phase 3: Event Handlers by Type

Route webhook events to specific handlers:

async function handleWebhookEvent(event: Stripe.Event) {
  switch (event.type) {
    // Payment Intents
    case 'payment_intent.succeeded':
      await handlePaymentSuccess(event.data.object as Stripe.PaymentIntent)
      break

    case 'payment_intent.payment_failed':
      await handlePaymentFailure(event.data.object as Stripe.PaymentIntent)
      break

    // Subscriptions
    case 'customer.subscription.created':
    case 'customer.subscription.updated':
      await handleSubscriptionUpdate(event.data.object as Stripe.Subscription)
      break

    case 'customer.subscription.deleted':
      await handleSubscriptionCanceled(event.data.object as Stripe.Subscription)
      break

    case 'customer.subscription.trial_will_end':
      await handleTrialEnding(event.data.object as Stripe.Subscription)
      break

    // Invoices
    case 'invoice.payment_succeeded':
      await handleInvoicePaid(event.data.object as Stripe.Invoice)
      break

    case 'invoice.payment_failed':
      await handleInvoicePaymentFailed(event.data.object as Stripe.Invoice)
      break

    // Disputes and Refunds
    case 'charge.dispute.created':
      await handleDisputeCreated(event.data.object as Stripe.Dispute)
      break

    case 'charge.refunded':
      await handleRefund(event.data.object as Stripe.Charge)
      break

    default:
      console.log(`⚠️  Unhandled webhook event type: ${event.type}`)
  }
}

// Example handler
async function handlePaymentSuccess(paymentIntent: Stripe.PaymentIntent) {
  // Update payment record
  await prisma.payment.update({
    where: { stripePaymentId: paymentIntent.id },
    data: {
      status: 'succeeded',
      stripeCustomerId: paymentIntent.customer as string,
      receiptUrl: paymentIntent.charges.data[0]?.receipt_url,
    },
  })

  // Fulfill order (grant access, send download link, etc.)
  if (paymentIntent.metadata.orderId) {
    await fulfillOrder(paymentIntent.metadata.orderId)
  }

  // Send receipt email
  await sendReceiptEmail(paymentIntent)
}

async function handleSubscriptionUpdate(subscription: Stripe.Subscription) {
  await prisma.subscription.upsert({
    where: { stripeSubscriptionId: subscription.id },
    update: {
      status: subscription.status,
      stripePriceId: subscription.items.data[0].price.id,
      currentPeriodStart: new Date(subscription.current_period_start * 1000),
      currentPeriodEnd: new Date(subscription.current_period_end * 1000),
      cancelAtPeriodEnd: subscription.cancel_at_period_end,
    },
    create: {
      stripeSubscriptionId: subscription.id,
      userId: subscription.metadata.userId,
      status: subscription.status,
      stripePriceId: subscription.items.data[0].price.id,
      currentPeriodStart: new Date(subscription.current_period_start * 1000),
      currentPeriodEnd: new Date(subscription.current_period_end * 1000),
    },
  })
}

Phase 4: Manual Replay System

Admin endpoint to replay failed webhooks:

// app/api/admin/replay-webhook/route.ts
import { NextResponse } from 'next/server'
import { getServerSession } from 'next-auth'

export async function POST(request: Request) {
  const session = await getServerSession()

  // Only admins can replay webhooks
  if (session?.user?.role !== 'admin') {
    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
  }

  const { eventId } = await request.json()

  const webhookEvent = await prisma.webhookEvent.findUnique({
    where: { id: eventId },
  })

  if (!webhookEvent) {
    return NextResponse.json({ error: 'Event not found' }, { status: 404 })
  }

  try {
    // Reconstruct Stripe event from logged data
    const stripeEvent: Stripe.Event = {
      id: webhookEvent.stripeEventId,
      type: webhookEvent.type,
      data: { object: webhookEvent.data },
    } as any

    // Reprocess event
    await handleWebhookEvent(stripeEvent)

    return NextResponse.json({ success: true })
  } catch (error) {
    console.error('Webhook replay failed:', error)
    return NextResponse.json(
      { error: 'Replay failed' },
      { status: 500 }
    )
  }
}

Admin dashboard to view and replay webhooks:

// app/admin/webhooks/page.tsx
'use client'

import { useState, useEffect } from 'react'

export default function WebhooksDashboard() {
  const [events, setEvents] = useState<any[]>([])

  useEffect(() => {
    fetch('/api/admin/webhooks')
      .then(res => res.json())
      .then(data => setEvents(data))
  }, [])

  const replayEvent = async (eventId: string) => {
    await fetch('/api/admin/replay-webhook', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ eventId }),
    })

    alert('Event replayed')
  }

  return (
    <div className="p-6">
      <h1 className="text-2xl font-bold mb-6">Webhook Events</h1>

      <table className="w-full">
        <thead>
          <tr className="border-b">
            <th className="text-left p-2">Event Type</th>
            <th className="text-left p-2">Status</th>
            <th className="text-left p-2">Attempts</th>
            <th className="text-left p-2">Created</th>
            <th className="text-left p-2">Actions</th>
          </tr>
        </thead>
        <tbody>
          {events.map(event => (
            <tr key={event.id} className="border-b">
              <td className="p-2">{event.type}</td>
              <td className="p-2">
                {event.processed ? (
                  <span className="text-green-600">✓ Processed</span>
                ) : (
                  <span className="text-red-600">✗ Failed</span>
                )}
              </td>
              <td className="p-2">{event.attempts}</td>
              <td className="p-2">
                {new Date(event.createdAt).toLocaleString()}
              </td>
              <td className="p-2">
                {!event.processed && (
                  <button
                    onClick={() => replayEvent(event.id)}
                    className="text-blue-600 hover:underline"
                  >
                    Replay
                  </button>
                )}
              </td>
            </tr>
          ))}
        </tbody>
      </table>
    </div>
  )
}

Phase 5: Monitoring and Alerts

Webhook health monitoring:

// lib/webhook-monitoring.ts
export async function getWebhookHealth() {
  const last24Hours = new Date(Date.now() - 24 * 60 * 60 * 1000)

  const stats = await prisma.webhookEvent.groupBy({
    by: ['processed'],
    where: { createdAt: { gte: last24Hours } },
    _count: true,
  })

  const total = stats.reduce((sum, s) => sum + s._count, 0)
  const successful = stats.find(s => s.processed)?._count || 0
  const failed = stats.find(s => !s.processed)?._count || 0

  return {
    total,
    successful,
    failed,
    successRate: total > 0 ? (successful / total) * 100 : 0,
  }
}

// Send alert if success rate drops below 95%
export async function checkWebhookHealth() {
  const health = await getWebhookHealth()

  if (health.successRate < 95 && health.total > 10) {
    await sendAlert({
      title: '🚨 Webhook Health Alert',
      message: `Webhook success rate: ${health.successRate.toFixed(1)}%
                Failed: ${health.failed}/${health.total} events`,
      severity: 'high',
    })
  }
}

// Run health check every hour
// (use cron job, Vercel Cron, or Inngest scheduled function)

Edge Cases to Handle

Critical Edge Cases

Webhook endpoint down during maintenance:

• Stripe retries for 3 days with exponential backoff
• Webhooks queued during downtime will replay when back online
• Solution: Monitor webhook dashboard for failed events
• Manually replay if needed after maintenance window

Event delivered out of order:

• Example: subscription.updated arrives before subscription.created
• Cause: Network delays, Stripe retry logic
• Solution: Use upsert instead of create in handlers
• Check event timestamp (event.created) if order matters

Duplicate event delivery:

• Stripe may send same event multiple times (rare but possible)
• Cause: Network timeouts, Stripe internal retries
• Solution: Check stripeEventId uniqueness before processing
• Use database unique constraint on stripeEventId

Processing takes longer than webhook timeout:

• Stripe expects response within 30 seconds
• Long-running tasks (email, file processing) block webhook
• Solution: Queue tasks asynchronously (Inngest, Bull, etc.)
• Return 200 immediately, process in background

Webhook signature mismatch:

• Cause: Wrong webhook secret, request modified in transit
• Solution: Verify webhook secret environment variable
• Check request isn't going through proxy that modifies body
• Test with Stripe CLI: stripe listen --forward-to localhost:3000/api/webhooks/stripe

Testing vs production webhooks:

• Test mode events go to test webhook endpoint
• Live mode events go to live webhook endpoint
• Solution: Use same endpoint, different webhook secrets
• Check event.livemode flag to separate concerns

Tech Stack Recommendations

Minimum Viable Stack

• Framework: Next.js API routes
• Database: PostgreSQL or MySQL (webhook event log)
• Payment Provider: Stripe
• Monitoring: Console logs + Stripe webhook dashboard

Production-Grade Stack

• Framework: Next.js 15 with App Router
• Database: Prisma + PostgreSQL
• Queue: Inngest or Trigger.dev (async processing)
• Monitoring: Sentry (error tracking) + DataDog (metrics)
• Alerts: Slack or PagerDuty for webhook failures
• Testing: Stripe CLI for local webhook testing

Full Implementation Prompt

Copy this prompt to use with Claude Code:

I need to implement a production-grade webhook handling system for Stripe payments. Before we start, help me understand my current setup.

First, let's review my environment:

1. Show me where webhook endpoints are currently defined (if any)
2. What's my database setup? (Need to add webhook event logging)
3. Do I have a job queue system? (For async processing)
4. What monitoring tools am I using? (Sentry, LogRocket, etc.)
5. How do I handle admin tasks? (Need admin dashboard for replaying webhooks)

Then let's discuss these critical webhook concepts:

• Why signature verification is non-negotiable (security)
• Why we return 200 even on errors (Stripe retry logic)
• Why idempotency matters (duplicate events)
• Why we log every event (debugging, compliance, replay)
• How to handle events arriving out of order

Then we'll implement in phases: Phase 1: Database schema for webhook event logging Phase 2: Webhook endpoint with signature verification Phase 3: Event handlers for each webhook type Phase 4: Idempotent processing logic Phase 5: Manual replay system for failed events Phase 6: Admin dashboard for webhook monitoring Phase 7: Health checks and alerting

After implementation, let's test:

• Use Stripe CLI to forward test webhooks
• Simulate processing failures (throw error in handler)
• Check webhook event log in database
• Manually replay failed event
• Verify idempotency (send duplicate event)
• Test signature verification failure (wrong secret)

Sound good? Let's start by reviewing your current webhook setup (if any) and database schema.

Related Feature Specs

• Payment Flow - One-time payment webhooks
• Stripe Subscriptions - Subscription lifecycle webhooks
• Admin Dashboard - Webhook monitoring UI
• Invoice Generation - Generate invoices from webhooks

Success Criteria

You've successfully implemented this when:

✅ All webhooks verify signatures before processing ✅ Every webhook event is logged to database ✅ Idempotency prevents duplicate processing ✅ Failed events can be manually replayed ✅ Webhook handlers route to appropriate functions ✅ Health monitoring alerts on high failure rate ✅ Admin dashboard shows webhook status ✅ Testing with Stripe CLI works locally

Common Mistakes to Avoid

❌ Not verifying webhook signatures (security risk) ❌ Returning 4xx/5xx on processing errors (infinite retries) ❌ Not logging events (no debugging, no audit trail) ❌ Processing same event multiple times (missing idempotency check) ❌ Blocking webhook response with slow operations (use queue) ❌ No manual replay system (can't recover from failures) ❌ Not monitoring webhook health (silent failures) ❌ Using wrong webhook secret (test vs live mode)

Implementation Checklist

Setup:

• Get webhook secret from Stripe Dashboard
• Store webhook secret in environment variables
• Create webhook endpoint in Stripe Dashboard
• Install Stripe CLI for local testing

Database:

• Create WebhookEvent model with all fields
• Add unique constraint on stripeEventId
• Add indexes on type, processed, createdAt
• Test database connection

Webhook Endpoint:

• Create API route at /api/webhooks/stripe
• Verify webhook signature
• Return 400 if signature invalid
• Log received event to database
• Check for duplicate events (idempotency)
• Route to appropriate handler
• Return 200 immediately (don't block)

Event Handlers:

• Handle payment_intent.succeeded
• Handle payment_intent.payment_failed
• Handle customer.subscription.created
• Handle customer.subscription.updated
• Handle customer.subscription.deleted
• Handle invoice.payment_succeeded
• Handle invoice.payment_failed
• Handle charge.dispute.created
• Handle charge.refunded

Error Handling:

• Log processing errors to database
• Send alerts on webhook failures
• Still return 200 to Stripe (prevent infinite retries)
• Track attempt count

Replay System:

• Create admin API endpoint for replay
• Add authentication (admin only)
• Fetch event from database
• Reconstruct Stripe event
• Reprocess event
• Update database status

Admin Dashboard:

• Create webhook events table view
• Show event type, status, attempts, timestamp
• Add "Replay" button for failed events
• Add filters (by type, status, date)
• Show event details (full payload)

Monitoring:

• Implement webhook health check function
• Calculate success rate (last 24 hours)
• Set alert threshold (< 95% success)
• Send alerts to Slack/PagerDuty
• Create cron job to run health checks

Testing:

• Install Stripe CLI: stripe listen --forward-to localhost:3000/api/webhooks/stripe
• Trigger test payment: stripe trigger payment_intent.succeeded
• Verify event logged in database
• Check processing marked as successful
• Trigger processing error (throw exception)
• Verify error logged, event marked failed
• Test manual replay from admin dashboard
• Test duplicate event (verify idempotency)
• Test signature verification failure

Last Updated: 2025-12-04 Difficulty: Intermediate Estimated Time: 3-4 hours Prerequisites: Payment Flow spec, database basics, Stripe webhook fundamentals

Need help with webhook reliability? Book a consultation for architecture review and monitoring setup.