Background workflows & cron

Overview

ShopSniffer runs four scheduled background workflows on Cloudflare Workflows, orchestrated by wrangler.jsonc cron triggers. This page is the reference for when each workflow fires, what it does, and what retention policies apply to the data it produces.

If you're a user of monitoring, most of what happens here is invisible — daily scans Just Work. This page exists for developers integrating with the data.

Cron schedule

All times are UTC and configured in wrangler.jsonc under triggers.crons.

MonitorShopsWorkflow Stable

Fires: Daily at 12:00 UTC.

Picks up every tracked store (across all users), groups them by effective scan frequency, and kicks off a ScrapeShopWorkflow for each one that's due. Results:

• A new completed job per store
• A snapshot row in the jobs table
• Diff computation vs the previous snapshot → rows in shop_changes
• Inventory deltas (sales vs restocks) → rows in inventory_events
• A new changelog entry per job
• Webhook deliveries and email notifications for any subscribers

Effective frequency

Each user picks their own scan frequency per store via PATCH /api/store-detail/:slug/frequency. The workflow uses the most aggressive frequency across all subscribers to that store — if any user wants daily, everyone tracking that store gets daily updates.

The "effective frequency" is recomputed every time any subscriber changes theirs. If the last daily subscriber downgrades to weekly, the store immediately drops to weekly cadence on the next cron tick.

AutoPublishBlogWorkflow Internal

Fires: Weekly Monday at 10:00 UTC.

Generates a weekly blog post about a trending or recently-analyzed store and publishes it to the public blog at https://shopsniffer.com/blog. Admin-only — not configurable by users.

AutoPublishReportsWorkflow Internal

Fires: Daily at 14:00 UTC.

Scans for completed reports older than 30 days that haven't been auto-published yet, and promotes them to public SEO content under https://shopsniffer.com/stores/:slug. This is how the public store directory gets populated.

Jobs you create while signed in are owned by your user ID, but the report data itself may be auto-published as public SEO content after 30 days. If you don't want a report you generated to appear in the public directory, delete it before the 30-day mark. This doesn't apply to stores you track for monitoring — those keep their per-user scoping.

CleanupSnapshotsWorkflow Internal

Fires: Weekly Sunday at 03:00 UTC.

Enforces retention on historical snapshot data:

• Job records — kept indefinitely; only metadata rows, very small
• Product/collection/page items (job_items, shop_items) — kept indefinitely so diffs always have a baseline
• R2 export files (CSV, JSON) — kept for 90 days per job; older files are deleted to control R2 storage costs
• Screenshots — kept for 90 days per job
• Inventory snapshots — kept for 180 days (needed for velocity calculations)

After cleanup, the job record itself still exists — only the heavy downloadable artifacts are removed. The report API will return a 410 Gone if you try to fetch a pruned file.

What fires in what order

Observability

All workflows emit structured logs to Cloudflare Workers Logs. Failed runs are tracked in the admin dashboard (admin-only).

For end-user status, check:

• GET /api/store-detail/:slug → summary.latest_job.created_at shows when the last scan ran
• GET /api/changelog/:jobId → diffs for any individual job
• GET /api/store-detail/:slug/analytics → 365-day daily-changes heatmap

Next steps