Skip to main content
AgentMark’s webhook allows you to receive events from AgentMark and respond to them. AgentMark supports helpers that automatically handle tracing, formatting, and more. AgentMark handles the following events:
  • Prompt Runs - When a prompt is run in the AgentMark platform
  • Dataset Runs - When a dataset is run in the AgentMark platform
  • Alerts - When an alert is triggered or resolved in the AgentMark platform

​
Webhook Endpoint

import { NextRequest, NextResponse } from "next/server";
import { AgentMarkSDK } from "@agentmark/sdk";
import { verifySignature } from "@agentmark/shared-utils";
import {
  createAgentMarkClient,
  VercelAIModelRegistry,
  VercelAIToolRegistry,
} from "@agentmark/vercel-ai-v4-adapter";
import { openai } from "@ai-sdk/openai";
import { WebhookHelper } from "@agentmark/vercel-ai-v4-webhook-helper";
import { EvalRegistry } from "@agentmark/agentmark-core";

export const dynamic = "force-dynamic";

// Initialize AgentMark SDK
const sdk = new AgentMarkSDK({
  apiKey: process.env.AGENTMARK_API_KEY,
  appId: process.env.AGENTMARK_APP_ID,
});

// Required for tracing dataset runs
sdk.initTracing({ disableBatch: true });

// Initialize eval registry (optional, for dataset evaluations)
const evalRegistry = new EvalRegistry();

evalRegistry.register("correctness", (_params) => {
  return {
    score: 0.95,
    label: "correct",
    reason: "The output matches the expected result",
  };
});

// Set up model registry
const modelRegistry = new VercelAIModelRegistry();

modelRegistry.registerModels(
  ["gpt-4o", "gpt-4o-mini", "gpt-4-turbo"],
  (name: string) => openai(name)
);

// Set up tool registry (optional, for prompts with tools)
const toolRegistry = new VercelAIToolRegistry<any>();

const checkWeather = async (input: string) => {
  return { current_weather: "sunny" };
};

toolRegistry.register("check_weather", checkWeather);

// Create AgentMark client
const agentmark = createAgentMarkClient({
  modelRegistry,
  loader: sdk.getFileLoader(),
  toolRegistry,
});

// Create Webhook Handler
export async function POST(request: NextRequest) {
  const payload = await request.json();
  const headers = request.headers;
  const xAgentmarkSign = headers.get("x-agentmark-signature-256");

  try {
    // Verify signature
    if (!xAgentmarkSign) {
      throw new Error("Missing x-agentmark-signature-256 header");
    }

    if (!(await verifySignature(
      process.env.AGENTMARK_WEBHOOK_SECRET!,
      xAgentmarkSign,
      JSON.stringify(payload)
    ))) {
      throw new Error("Invalid signature");
    }

    const event = payload.event;
    const webhookHelper = new WebhookHelper(agentmark, evalRegistry, sdk);

    if (event.type === "prompt-run") {
      const response = await webhookHelper.runPrompt(event.data);
      if (response.type === "stream") {
        return new Response(response.stream, {
          headers: { ...response.streamHeaders },
        });
      }
      return NextResponse.json(response);
    }

    if (event.type === "dataset-run") {
      const response = await webhookHelper.runDataset(event.data);
      return new Response(response.stream, {
        headers: {
          ...response.streamHeaders,
        },
      });
    }
    
    if (event.type === "alert") {
      // Alerts are not handled by the helper, process manually
      console.log("Alert received:", event.data);
      return NextResponse.json({ message: "alert processed" });
    }

    throw new Error(`Unknown event type: ${event.type}`);
  } catch (error) {
    console.error("Webhook error:", error);
    return NextResponse.json(
      { message: "Internal server error" },
      { status: 500 }
    );
  }
}

​
Advanced Configuration

​
Custom Evaluators and Tools

The complete example above includes EvalRegistry and ToolRegistry. To learn more about registering custom evaluators and tools:
  • Registries Guide - Learn how to configure EvalRegistry, ToolRegistry, and ModelRegistry
  • Webhook Helpers - Understand how WebhookHelper works with registries

​
Model Registration

The example above shows basic text model registration with OpenAI. See the Registries documentation for more advanced examples including speech and image models.

​
Further Reading

For more details on helpers, event types, and security best practices, please refer to the main Webhook documentation. If you’re not using a custom framework, see our event-specific guides.

​
Security Best Practices

  • Always verify the x-agentmark-signature-256 header
  • Use environment variables for sensitive data
  • Implement proper error handling
  • Return appropriate HTTP status codes
  • Use HTTPS for your endpoint

​
Common Issues

  • Signature Verification: Ensure your webhook verifies the x-agentmark-signature-256 header
  • Error Handling: Implement proper error handling and return appropriate status codes
  • Response Format: Follow the expected response format for each event type
  • Header Naming: Always use streamHeaders (plural) for streaming responses