Webhooks

Webhooks allow your application to receive real-time notifications when events occur in the Essence Platform. Instead of polling the API, your application receives HTTP POST requests when something changes.

Overview

Webhooks are powered by the Unified Webhook Management Platform, providing:

• Real-time Events - Instant notifications when data changes
• Reliable Delivery - Automatic retries with exponential backoff
• Secure Verification - HMAC-SHA256 signature verification
• Event Filtering - Subscribe only to events you care about
• Delivery Monitoring - Track delivery success and failures
• Custom Headers - Add your own headers for routing

How Webhooks Work

sequenceDiagram
    participant Essence as Essence Platform
    participant Webhook as Webhook Service
    participant Your as Your Endpoint

    Essence->>Webhook: Event occurs (e.g., vault created)
    Webhook->>Webhook: Generate signature
    Webhook->>Your: POST /webhooks/essence<br/>X-Essence-Signature: hmac_sha256=...
    Your->>Your: Verify signature
    Your->>Your: Process event
    Your-->>Webhook: 200 OK
    alt If request fails
        Webhook->>Webhook: Wait (exponential backoff)
        Webhook->>Your: Retry (up to 5 times)
    end

Available Events

Vault Events

Event Type	Description	Payload
vault.created	New vault created	Vault object
vault.updated	Vault modified	Vault object + changes
vault.deleted	Vault deleted	Vault ID
vault.quota.warning	Storage quota at 80%	Vault object + usage
vault.quota.exceeded	Storage quota at 100%	Vault object + usage

Document Events

Event Type	Description	Payload
document.uploaded	New document added	Document object
document.updated	Document metadata changed	Document object + changes
document.deleted	Document removed	Document ID + vault ID
document.shared	Document shared	Share object
document.share.accessed	Shared document accessed	Access log
document.share.expired	Share link expired	Share ID

Consent Events

Event Type	Description	Payload
consent.granted	User granted consent	Consent object
consent.revoked	User revoked consent	Consent ID + reason
consent.expired	Consent period ended	Consent object
consent.renewed	Consent renewed	Consent object

User Events

Event Type	Description	Payload
user.created	New user registered	User object
user.updated	User profile changed	User object + changes
user.deleted	User account deleted	User ID
user.login	User logged in	Login details

Instance Events

Event Type	Description	Payload
instance.health.degraded	Instance health issue	Health status
instance.health.recovered	Instance recovered	Health status
instance.quota.warning	Instance quota warning	Usage stats

Webhook Registration

Creating a Webhook

import { EssenceClient } from '@essence/sdk';

const client = new EssenceClient({
  apiKey: process.env.ESSENCE_API_KEY
});

// Register webhook
const webhook = await client.webhooks.create({
  url: 'https://your-app.com/webhooks/essence',
  events: [
    'vault.created',
    'document.uploaded',
    'document.shared'
  ],
  description: 'Production webhook for vault and document events',
  secret: 'your_webhook_secret_here', // Used for signature verification
  headers: {
    'X-Custom-Header': 'your-value'
  },
  active: true
});

console.log('Webhook created:', webhook.id);

Using API Directly

curl -X POST "https://api.essence.digital/api/v1/webhooks" \
  -H "X-Api-Key: essence_sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-app.com/webhooks/essence",
    "events": ["vault.created", "document.uploaded"],
    "description": "Production webhook",
    "secret": "your_webhook_secret_here"
  }'

Using GraphQL

mutation CreateWebhook {
  createWebhook(input: {
    url: "https://your-app.com/webhooks/essence"
    events: ["vault.created", "document.uploaded"]
    description: "Production webhook"
    secret: "your_webhook_secret_here"
  }) {
    success
    webhook {
      id
      url
      events
      active
      createdAt
    }
  }
}

Configuration Options

Option	Type	Required	Description
url	string	Yes	HTTPS endpoint to receive events
events	string[]	Yes	Array of event types to subscribe to
description	string	No	Human-readable description
secret	string	Yes	Secret key for signature verification (min 32 chars)
headers	object	No	Custom headers to include in requests
active	boolean	No	Enable/disable webhook (default: true)
metadata	object	No	Custom metadata for your use

Event Payload Structure

All webhook events follow this structure:

interface WebhookEvent {
  // Event metadata
  id: string;                    // Unique event ID
  type: string;                  // Event type (e.g., "vault.created")
  timestamp: string;             // ISO 8601 timestamp
  apiVersion: string;            // API version (e.g., "v1")

  // Event data
  data: {
    object: any;                 // The resource object
    previous?: any;              // Previous state (for updates)
    changes?: string[];          // Changed fields (for updates)
  };

  // Context
  context: {
    userId: string;              // User who triggered event
    instanceId: string;          // Instance where event occurred
    requestId: string;           // Original request ID
    ipAddress?: string;          // IP address (if applicable)
  };
}

Example: Vault Created Event

{
  "id": "evt_1234567890abcdef",
  "type": "vault.created",
  "timestamp": "2025-11-06T10:30:00Z",
  "apiVersion": "v1",
  "data": {
    "object": {
      "id": "vault-123",
      "name": "Medical Records",
      "description": "Patient health information",
      "category": "health",
      "userId": "user-456",
      "instanceId": "instance-789",
      "documentCount": 0,
      "totalSize": 0,
      "isActive": true,
      "createdAt": "2025-11-06T10:30:00Z",
      "updatedAt": "2025-11-06T10:30:00Z"
    }
  },
  "context": {
    "userId": "user-456",
    "instanceId": "instance-789",
    "requestId": "req_abc123",
    "ipAddress": "192.168.1.1"
  }
}

Example: Document Uploaded Event

{
  "id": "evt_9876543210fedcba",
  "type": "document.uploaded",
  "timestamp": "2025-11-06T10:35:00Z",
  "apiVersion": "v1",
  "data": {
    "object": {
      "id": "doc-789",
      "vaultId": "vault-123",
      "fileName": "lab_results.pdf",
      "sizeBytes": 1048576,
      "mimeType": "application/pdf",
      "encryptionMetadata": {
        "algorithm": "AES-256-GCM",
        "keyDerivation": "PBKDF2"
      },
      "uploadedAt": "2025-11-06T10:35:00Z"
    }
  },
  "context": {
    "userId": "user-456",
    "instanceId": "instance-789",
    "requestId": "req_xyz789"
  }
}

Signature Verification

All webhook requests include an X-Essence-Signature header for verification.

Signature Format

X-Essence-Signature: hmac_sha256=<signature>

Verifying Signatures

Node.js/TypeScript

import crypto from 'crypto';

function verifyWebhookSignature(
  payload: string | Buffer,
  signature: string,
  secret: string
): boolean {
  // Extract signature from header (format: "hmac_sha256=<signature>")
  const signatureParts = signature.split('=');
  if (signatureParts[0] !== 'hmac_sha256') {
    return false;
  }

  const receivedSignature = signatureParts[1];

  // Compute expected signature
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');

  // Use timing-safe comparison
  return crypto.timingSafeEqual(
    Buffer.from(receivedSignature),
    Buffer.from(expectedSignature)
  );
}

// Express.js example
import express from 'express';

const app = express();

app.post(
  '/webhooks/essence',
  express.raw({ type: 'application/json' }), // Get raw body
  (req, res) => {
    const signature = req.headers['x-essence-signature'] as string;
    const secret = process.env.WEBHOOK_SECRET!;

    // Verify signature
    if (!verifyWebhookSignature(req.body, signature, secret)) {
      console.error('Invalid webhook signature');
      return res.status(401).send('Invalid signature');
    }

    // Parse and process event
    const event = JSON.parse(req.body.toString());
    console.log('Received event:', event.type);

    // Handle event
    handleWebhookEvent(event);

    // Respond quickly (process async if needed)
    res.status(200).send('OK');
  }
);

Python

import hmac
import hashlib

def verify_webhook_signature(payload: bytes, signature: str, secret: str) -> bool:
    """Verify webhook signature"""
    # Extract signature
    if not signature.startswith('hmac_sha256='):
        return False

    received_signature = signature.split('=', 1)[1]

    # Compute expected signature
    expected_signature = hmac.new(
        secret.encode('utf-8'),
        payload,
        hashlib.sha256
    ).hexdigest()

    # Timing-safe comparison
    return hmac.compare_digest(received_signature, expected_signature)

# Flask example
from flask import Flask, request

app = Flask(__name__)

@app.route('/webhooks/essence', methods=['POST'])
def handle_webhook():
    signature = request.headers.get('X-Essence-Signature', '')
    secret = os.environ['WEBHOOK_SECRET']

    # Verify signature
    if not verify_webhook_signature(request.data, signature, secret):
        return 'Invalid signature', 401

    # Parse and process event
    event = request.json
    print(f'Received event: {event["type"]}')

    # Handle event
    handle_webhook_event(event)

    return 'OK', 200

Using SDK Helper

import { verifyWebhookSignature } from '@essence/sdk';

app.post('/webhooks/essence', express.raw({ type: 'application/json' }), (req, res) => {
  const signature = req.headers['x-essence-signature'] as string;
  const secret = process.env.WEBHOOK_SECRET!;

  if (!verifyWebhookSignature(req.body, signature, secret)) {
    return res.status(401).send('Invalid signature');
  }

  const event = JSON.parse(req.body.toString());
  handleWebhookEvent(event);

  res.status(200).send('OK');
});

Event Handling

Basic Event Handler

async function handleWebhookEvent(event: WebhookEvent) {
  console.log(`Processing event: ${event.type} (${event.id})`);

  switch (event.type) {
    case 'vault.created':
      await handleVaultCreated(event.data.object);
      break;

    case 'document.uploaded':
      await handleDocumentUploaded(event.data.object);
      break;

    case 'document.shared':
      await handleDocumentShared(event.data.object);
      break;

    case 'consent.granted':
      await handleConsentGranted(event.data.object);
      break;

    case 'consent.revoked':
      await handleConsentRevoked(event.data.object);
      break;

    default:
      console.log(`Unhandled event type: ${event.type}`);
  }
}

async function handleVaultCreated(vault: any) {
  console.log(`New vault created: ${vault.name} (${vault.id})`);

  // Example: Send welcome email
  await sendEmail({
    to: vault.userEmail,
    subject: 'Vault Created',
    body: `Your vault "${vault.name}" has been created successfully.`
  });

  // Example: Initialize analytics
  await analytics.track('vault_created', {
    vaultId: vault.id,
    category: vault.category
  });
}

async function handleDocumentUploaded(document: any) {
  console.log(`Document uploaded: ${document.fileName}`);

  // Example: Trigger document processing
  await processDocument(document.id);

  // Example: Update statistics
  await updateStorageStats(document.vaultId);
}

Asynchronous Processing

For long-running tasks, respond quickly and process asynchronously:

import { Queue } from 'bull';

const webhookQueue = new Queue('webhooks', {
  redis: { host: 'localhost', port: 6379 }
});

// Webhook endpoint - respond quickly
app.post('/webhooks/essence', express.raw({ type: 'application/json' }), async (req, res) => {
  const signature = req.headers['x-essence-signature'] as string;

  if (!verifyWebhookSignature(req.body, signature, process.env.WEBHOOK_SECRET!)) {
    return res.status(401).send('Invalid signature');
  }

  const event = JSON.parse(req.body.toString());

  // Queue for async processing
  await webhookQueue.add('process-event', event);

  // Respond immediately
  res.status(200).send('OK');
});

// Worker - process events asynchronously
webhookQueue.process('process-event', async (job) => {
  const event = job.data;
  await handleWebhookEvent(event);
});

Testing Webhooks Locally

1. Using ngrok

# Install ngrok
npm install -g ngrok

# Start your local server
node server.js # Running on http://localhost:3000

# Create tunnel
ngrok http 3000

# Use the HTTPS URL for webhook
# https://abc123.ngrok.io/webhooks/essence

2. Using localhost.run

# Create tunnel (no installation needed)
ssh -R 80:localhost:3000 localhost.run

# Use the provided URL for webhook

3. Using webhook.site

For testing without code:

1. Visit webhook.site
2. Copy your unique URL
3. Register webhook with that URL
4. Trigger events and inspect payloads

4. Testing with SDK

// Manually trigger a test webhook
const testResult = await client.webhooks.test(webhookId);
console.log('Test delivery status:', testResult.status);

Retry and Failure Handling

Retry Policy

The Unified Webhook Management Platform automatically retries failed deliveries:

Attempt	Delay	Total Time
1	Immediate	0s
2	1 minute	1m
3	5 minutes	6m
4	15 minutes	21m
5	1 hour	1h 21m
6	4 hours	5h 21m

Successful Response

Return 200 OK within 10 seconds to acknowledge receipt:

app.post('/webhooks/essence', async (req, res) => {
  // Verify signature...

  // Respond immediately
  res.status(200).send('OK');

  // Process asynchronously
  processEventAsync(event);
});

Handling Failures

// Monitor webhook failures
const webhook = await client.webhooks.get(webhookId);
console.log('Failed deliveries:', webhook.stats.failedDeliveries);
console.log('Last error:', webhook.stats.lastError);

// Get recent delivery attempts
const attempts = await client.webhooks.getDeliveryAttempts(webhookId, {
  page: 1,
  pageSize: 20,
  status: 'failed'
});

attempts.forEach(attempt => {
  console.log(`Attempt ${attempt.attemptNumber}: ${attempt.statusCode} - ${attempt.error}`);
});

Manual Retry

// Retry a specific failed delivery
await client.webhooks.retryDelivery(webhookId, deliveryId);

// Retry all failed deliveries
await client.webhooks.retryAllFailed(webhookId);

Troubleshooting

Common Issues

1. Signature Verification Fails

// Check you're using the raw body (not parsed JSON)
app.post('/webhooks', express.raw({ type: 'application/json' }), (req, res) => {
  // req.body is a Buffer - correct!
});

// ❌ Wrong: Using express.json()
app.post('/webhooks', express.json(), (req, res) => {
  // req.body is parsed object - signature will fail!
});

2. Webhook Endpoint Not Accessible

# Test your endpoint is publicly accessible
curl -X POST https://your-app.com/webhooks/essence \
  -H "Content-Type: application/json" \
  -d '{"test": true}'

# Check firewall rules allow incoming connections
# Ensure HTTPS is properly configured

3. Request Timeout

// Respond within 10 seconds
app.post('/webhooks/essence', async (req, res) => {
  // ❌ Bad: Long synchronous processing
  await processEventForMinutes(event);
  res.send('OK');

  // ✅ Good: Quick response + async processing
  res.send('OK');
  processEventAsync(event);
});

4. Duplicate Events

// Handle idempotency
const processedEvents = new Set<string>();

async function handleWebhookEvent(event: WebhookEvent) {
  // Check if already processed
  if (processedEvents.has(event.id)) {
    console.log(`Event ${event.id} already processed, skipping`);
    return;
  }

  // Process event
  await processEvent(event);

  // Mark as processed
  processedEvents.add(event.id);

  // Store in database for persistence
  await db.processedEvents.create({ id: event.id, processedAt: new Date() });
}

Debugging Tools

// Enable webhook logging
const webhook = await client.webhooks.update(webhookId, {
  metadata: { debug: true }
});

// View recent deliveries
const deliveries = await client.webhooks.getDeliveries(webhookId, {
  page: 1,
  pageSize: 10
});

deliveries.forEach(delivery => {
  console.log(`Delivery ${delivery.id}:`, {
    status: delivery.status,
    statusCode: delivery.statusCode,
    duration: delivery.durationMs,
    attempt: delivery.attemptNumber,
    error: delivery.error
  });
});

Best Practices

1. Verify Signatures Always

// ✅ Always verify signatures
if (!verifyWebhookSignature(payload, signature, secret)) {
  return res.status(401).send('Invalid signature');
}

2. Respond Quickly

// ✅ Respond within 10 seconds
res.status(200).send('OK');
processEventAsync(event);

3. Handle Idempotency

// ✅ Store processed event IDs
await db.processedEvents.create({ id: event.id });

4. Monitor Failures

// ✅ Set up alerts for failures
if (webhook.stats.failureRate > 0.1) {
  await sendAlert('High webhook failure rate');
}

5. Use HTTPS Only

// ✅ Always use HTTPS endpoints
url: 'https://your-app.com/webhooks/essence'

// ❌ Never use HTTP
url: 'http://your-app.com/webhooks/essence'

6. Implement Circuit Breaker

// ✅ Temporarily disable webhooks if endpoint is down
if (consecutiveFailures > 10) {
  await client.webhooks.update(webhookId, { active: false });
  await sendAlert('Webhook disabled due to repeated failures');
}

Managing Webhooks

List All Webhooks

const webhooks = await client.webhooks.list();
webhooks.forEach(webhook => {
  console.log(`${webhook.description}: ${webhook.url}`);
});

Update Webhook

await client.webhooks.update(webhookId, {
  events: ['vault.created', 'vault.updated', 'vault.deleted'],
  description: 'Updated webhook configuration'
});

Disable Webhook

await client.webhooks.update(webhookId, { active: false });

Delete Webhook

await client.webhooks.delete(webhookId);

Rotate Secret

// Generate new secret
const newSecret = crypto.randomBytes(32).toString('hex');

// Update webhook
await client.webhooks.update(webhookId, { secret: newSecret });

// Update your environment
process.env.WEBHOOK_SECRET = newSecret;

Next Steps

• Tutorial: Webhook Integration - Complete webhook setup guide
• API Reference - Full webhook API documentation
• Best Practices - Security and reliability tips
• Error Codes - Webhook error reference

---

Need help? Join our Discord community or email support.