Jobs API

Overview

A job is one store analysis run. Creating a job kicks off the full scraping and analysis pipeline (sitemap → audit → product indexing → PageSpeed → export). This section covers job creation, lookup, status polling, retry, and the real-time WebSocket progress stream.

Base URL: https://shopsniffer.com/api

POST /jobs Stable

Create a store analysis job. Starts the full pipeline.

Auth: API key, Better Auth session, or x402 payment.

bash
curl -X POST https://shopsniffer.com/api/jobs \
  -H "X-API-Key: ss_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "allbirds.com",
    "webhook_url": "https://your-server.com/hooks/shopsniffer",
    "notify_email": "you@example.com"
  }'

Request body

json
{
  "job": {
    "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "domain": "allbirds.com",
    "slug": "allbirds-com",
    "status": "pending",
    "created_at": "2026-04-12T10:30:00Z",
    "user_id": "usr_better_auth_id"
  }
}

Response fields

Error responses

json
{ "error": "Invalid domain format. Please enter a valid domain like 'store.com'." }

json
{ "error": "This domain does not appear to be a Shopify store." }

json
{ "error": "This Shopify store is password-protected and cannot be scanned." }

json
{ "error": "Unauthorized" }

json
{
  "error": "Payment Required",
  "x402": {
    "version": "1",
    "accepts": [{
      "scheme": "exact",
      "network": "base",
      "maxAmountRequired": "9990000",
      "resource": "https://shopsniffer.com/api/jobs",
      "payTo": "0x767131d92c41D56546eC72fecD0F1d63900fa9D9",
      "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913"
    }]
  }
}

See x402 payment for handling the 402 flow.

GET /jobs Stable

Look up a job by ID or by most recent for a domain. No authentication required.

bash
curl "https://shopsniffer.com/api/jobs?id=a1b2c3d4-e5f6-7890-abcd-ef1234567890"

json
{
  "job": {
    "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "domain": "allbirds.com",
    "slug": "allbirds-com",
    "status": "completed",
    "created_at": "2026-04-12T10:30:00Z",
    "completed_at": "2026-04-12T10:33:42Z",
    "product_count": 156,
    "collection_count": 24,
    "page_count": 18,
    "shop_meta": {
      "shop_name": "Allbirds",
      "currency": "USD",
      "theme_name": "Custom Theme",
      "theme_id": 123456789,
      "detected_apps": ["Klaviyo", "Judge.me", "Afterpay"]
    },
    "insights": {
      "top_vendors": [{ "name": "Allbirds", "count": 156 }],
      "price_range": { "min": 28, "max": 160, "avg": 98.5 },
      "product_types": [{ "type": "Shoes", "count": 82 }]
    },
    "downloads": {
      "csv": "products.csv",
      "productsJson": "products.json",
      "collectionsJson": "collections.json",
      "pagesJson": "pages.json"
    }
  }
}

Errors:

json
{ "error": "Provide id or domain parameter" }

json
{ "error": "Job not found" }

GET /jobs/:id/status Stable

Lightweight status check for polling. Returns only status and progress counts to minimize bandwidth. No authentication required.

bash
curl https://shopsniffer.com/api/jobs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/status

json
{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "status": "processing",
  "domain": "allbirds.com",
  "product_count": 89,
  "collection_count": 24,
  "page_count": 18,
  "created_at": "2026-04-12T10:30:00Z",
  "updated_at": "2026-04-12T10:31:47Z"
}

For real-time progress updates, use the WebSocket endpoint instead of polling — no rate limits, push-based updates, lower latency.

POST /jobs/:id/retry Stable

Retry a failed or completed job. Creates a new job with a new UUID; the original is left unchanged. Auth: Better Auth session.

json
{
  "job": {
    "id": "new-retry-uuid",
    "domain": "allbirds.com",
    "slug": "allbirds-com",
    "status": "pending",
    "created_at": "2026-04-12T11:00:00Z"
  }
}

Errors: 400 if the job isn't in a retryable state, 401 if unauthenticated, 404 if the job doesn't exist.

GET /ws/:jobId WebSocket

Real-time job progress stream via WebSocket. Push-based: no polling, sub-second latency, backed by a Cloudflare Durable Object that tracks per-step progress.

Protocol: WebSocket upgrade. Send Upgrade: websocket header.

Auth: None — the endpoint is keyed by job ID.

typescript
const ws = new WebSocket(`wss://shopsniffer.com/api/ws/${jobId}`);

ws.addEventListener("message", (event) => {
  const msg = JSON.parse(event.data);
  // { type: "status", status: "processing", step: "indexing-products", progress: 0.42 }
  console.log(msg);
});

ws.addEventListener("close", () => {
  console.log("Job complete or connection closed");
});

Message shape

Messages are JSON with a type discriminator:

json
{ "type": "status", "status": "processing", "step": "indexing-products", "progress": 0.42 }

json
{ "type": "step", "step": "pagespeed", "completed": true }

json
{ "type": "complete", "job_id": "a1b2c3d4-…", "report_url": "https://shopsniffer.com/report/a1b2c3d4-…" }

The connection closes after a complete or error message.

Status lifecycle

Next steps