Approach · workflow · in production
Audits as Crons
I convert any manual check for system drift, compliance, or state anomalies into automated cron jobs, ensuring only actionable deviations surface.
Any manual check I would perform to detect drift, validate compliance, or identify state anomalies across my product portfolio is automated as a scheduled cron job, ensuring only actionable deviations surface.
What it is
This approach formalizes routine operational checks into automated, recurring tasks. Instead of periodically logging into various dashboards—Firebase, Vercel, Stripe, Mercury—to manually verify expected states or identify unexpected changes, I configure dedicated cron jobs. These jobs execute at defined intervals, running specific audit logic. The output is a structured report, typically stored in Firebase, which I review for anomalies. The goal is to shift from reactive, manual inspection to proactive, automated detection, reducing the cognitive load required to operate a multi-product studio.
Why I do it this way
Operating multiple products, such as Total Formula 1, Pregnancy Power Hour, and Inky, with AI as the primary team, demands an efficient operational model. Manual audits become a significant bottleneck and a source of human error. By automating these checks, I ensure consistency and thoroughness across all my ventures. This frees up my time from repetitive verification tasks, allowing focus on development and strategic work. It's a core component of how I manage Multiple Revenue Streams, One Back Office, ensuring that each product maintains its integrity without requiring constant direct oversight. This proactive stance means issues are identified and flagged before they escalate, often before any user impact.
How it works in practice
I use Vercel Cron Jobs to schedule these audits. Each cron job triggers a serverless function, which then orchestrates an AI agent (often powered by Claude Code or Gemini) to perform the actual audit. For example, a nightly cron might:
- Financial Audit: Query Stripe for recent transactions, cross-referencing against expected revenue patterns or checking for failed payments that require follow-up. It might also connect to Mercury's API to verify account balances against a predetermined threshold, flagging any unexpected dips.
- Infrastructure Audit: Check Vercel deployments for unauthorized changes or configuration drift from Docs as Source specifications. It might also scan Firebase security rules to ensure no unintended permissions have been introduced or modified, comparing the live ruleset against a version-controlled baseline.
- Content Audit: For a product like Inky, an agent might verify content generation limits for Credit Packs over Subscriptions users, ensuring no overages or underages based on credit consumption.
All audit results are written to a `systemAudits/{date}` collection in Firebase. Each entry includes the audit type, timestamp, status, and any detected anomalies. Only critical `status: 'red'` items trigger a direct Slack notification via a Resend-powered email-to-Slack webhook, ensuring I only receive alerts for truly actionable issues. This contrasts with my CI Local Before CI Remote approach, which catches issues before deployment; audits-as-crons catch issues after deployment and during runtime.
Where this breaks down
- False Positives: Overly sensitive audit logic can generate noise, leading to alert fatigue. Tuning the detection thresholds and anomaly definitions requires iteration.
- Initial Setup Cost: Each new audit requires defining the check logic, agent prompts, integration points, and notification thresholds. This initial investment can be substantial, especially for complex systems.
- API Rate Limits: Frequent or extensive audits against external services (Stripe, Vercel, Mercury) can quickly consume API rate limits, potentially impacting other operational tasks. Careful scheduling and batching are necessary.
- Agent Hallucinations: While rare with structured audit tasks, an AI agent might misinterpret an instruction or incorrectly flag a non-issue, requiring manual verification. This is mitigated by clear, specific prompts and validation steps within the audit function.
FAQs
- How do you handle audit failures?
- Critical audit failures (`status: 'red'`) trigger immediate Slack alerts. Less severe issues are logged in Firebase for daily review. The goal is to identify and address the root cause, often by adjusting agent logic or system configuration.
- What's the cost overhead of running these crons and agents?
- Vercel's cron and serverless function costs are minimal for the frequency I run. AI agent calls (Claude/Gemini) are the primary variable cost, but for structured audit tasks, they are efficient. The cost is justified by preventing larger issues.
- How do you prevent alert fatigue?
- I tune the audit logic to only flag true anomalies. Critical issues directly alert Slack. Non-critical findings are aggregated in Firebase for asynchronous review, ensuring I only get paged for what demands immediate attention.
I run this and four other brands. Want to see the operator playbook in detail?
Get the Builder's Playbook →