Is this Apify actor build safe to deploy?
Deploy Guard is the release-safety layer for any Apify actor. Point it at a build and it auto-discovers a test suite, runs it, compares the output against a trusted baseline, and returns one unambiguous releaseAction — deploy / halt / review — plus a routable decision enum and confidence score your CI/CD pipeline or agent branches on. $0.35 per suite run.
Most scraper bugs don't crash — they quietly return the wrong data. A run finishes SUCCEEDED while the dataset empties out, a field goes all-null, or a selector breaks. Deploy Guard catches that before your customers do: it fingerprints the root cause (Selector Breakage, Bot Protection, Login Wall) with deterministic confidence — no LLM — and accumulates releaseMemory so each run knows your actor better than the last.
Every field answers one question — should I trust this release?
One unambiguous releaseAction — deploy / halt / review / misconfigured — plus a routable decision enum (act_now / monitor / ignore) and a confidence score. Parse one field in CI; exit non-zero unless it's deploy. No prose to interpret.
Zero-config. Point Deploy Guard at an actor with no test cases and it probes the actor, infers the output contract, and generates the suite for you. You don't have to hand-write assertions to get a verdict.
With enableBaseline, every run is scored against a trusted baseline built from prior runs — result-count collapse, null-rate shifts, schema drift, field disappearance, and flaky-test signals. Catches the regression a single pass can't see.
Run it on a schedule and it accumulates the actor's operational record: first seen, total runs, availability, regression count, average runtime / results / compute, and the fields it has learned are critical. After 100 runs it knows your actor better than any one-off test — and a fresh tool can't reproduce it.
It doesn't just say "failed" — it fingerprints why, deterministically and with no LLM: Selector Breakage (97%) → update CSS selectors; Bot Protection (94%) → enable residential proxies; Login Wall, Pagination Bug, Runtime Explosion. Each with a priority-ranked remediation step.
The dangerous case: a run finishes SUCCEEDED but returns empty or broken data. Deploy Guard flips that to halt — result count dropped 99% vs baseline, a required field went all-null, a price flipped from number to string — before it reaches production.
Under the verdict: presets (canary, scraper-smoke, api-actor, contact-scraper, ecommerce-quality, store-readiness) and custom test cases run through six assertion types.
Assert that your actor returns at least N results (minResults) or no more than N results (maxResults). Catches pagination bugs, empty outputs, and runaway scrapers that return unbounded data.
Verify that specified fields exist with non-null values in at least one output item. Catches missing extractors, changed CSS selectors, and broken API endpoints that silently drop fields.
Assert that fields match expected JavaScript types: string, number, boolean, object, or array. Catches silent type coercion issues like prices stored as strings instead of numbers.
Set a maxDuration per test case in seconds. If a 10-second test starts taking 60 seconds, the assertion fails — catching performance regressions from target site changes or proxy issues.
Assert that fields are never null, undefined, empty string, or empty array. Catches scrapers that technically succeed but return blank data — a common failure mode that passes result count checks.
If one test case crashes or times out, the remaining cases still run and produce results. One bad scenario does not block the rest of the suite — each case runs as an independent actor call.
There are 4 common approaches to testing Apify actors with multiple input scenarios. Each has trade-offs in automation level, assertion coverage, and cost.
| Method | Time per scenario | Assertion types | Automation | Cost |
|---|---|---|---|---|
| Deploy Guard | 10-60 seconds (depends on actor) | 6 types: counts, fields, types, duration, empty values, isolation | Fully automated (define cases once, rerun via API) | $0.35/suite + actor compute |
| Manual Apify Console runs | 5-15 minutes | Visual inspection only (no structured assertions) | Fully manual per scenario | Free (time cost only) |
| Custom API test script | 3-6 hours (initial setup), seconds per run | Unlimited (custom code, but you build each assertion) | Automated after development | Free (development time cost) |
| Jest / Vitest unit tests | Milliseconds per test | Function-level assertions (does not test full actor on platform) | Automated (local only, no platform integration) | Free |
No single testing method catches every issue — the most reliable approach combines unit tests for logic with end-to-end platform tests for integration and performance.
{
"actorName": "ryanclinton/google-maps-email-extractor",
"releaseAction": "halt",
"decision": "act_now",
"confidence": 0.96,
"verdictReasonCodes": ["RESULT_COUNT_COLLAPSE", "SILENT_FAILURE"],
"failureFingerprint": {
"type": "Selector Breakage",
"confidence": 0.97,
"remediation": "Update CSS selectors — output shape changed vs baseline"
},
"baseline": {
"trusted": true,
"medianResults": 312,
"thisRun": 4,
"resultCountDeltaPct": -98.7
},
"decisionDrivers": [
"Result count dropped 99% vs trusted baseline (312 → 4)",
"Run status SUCCEEDED but required field 'email' went 100% null"
],
"tests": { "total": 3, "passed": 2, "failed": 1 },
"releaseMemory": { "totalRuns": 142, "regressionCount": 3, "availability": 0.99 }
}Connect your Apify API token and enter the actor ID to test
Deploy Guard runs each test case on your account with its own input and assertions
Get a structured pass/fail report with per-case timing, assertion details, and error messages — results cached for free
There are several approaches to testing Apify actors across multiple input scenarios, from fully manual to fully automated. The right choice depends on your team size, deployment frequency, and how critical your scrapers are to downstream systems.
Run the actor manually in the Apify Console with each input scenario, then inspect the output visually. Works for ad-hoc checks but does not scale — testing 5 scenarios manually takes 25-75 minutes and provides no structured assertions or regression history.
Best for: one-off debugging of a specific actor input before deployment.
Write a Node.js or Python script that calls the Apify API to run the actor with different inputs and checks the output dataset programmatically. Requires 3-6 hours of initial development and ongoing maintenance as assertions evolve.
Best for: teams with custom CI/CD pipelines who need assertions not covered by standard testing tools.
Standard JavaScript test frameworks that validate individual functions in isolation. Fast (milliseconds per test) and free, but cannot test the full actor on the Apify platform — they miss proxy issues, API rate limits, platform timeouts, and deployment-specific failures.
Best for: testing pure data transformation logic and utility functions during development.
Schedule the actor to run periodically via Apify Schedules and set up webhook alerts for failures. Catches hard failures but provides no assertion-level validation — a scraper can 'succeed' while returning incorrect data.
Best for: uptime monitoring and hard failure detection on production scrapers.
Automated multi-case test suites with 6 assertion types, per-case error isolation, and structured JSON output. Up to 20 test cases per suite at a flat $0.35. No scripting required — define cases as JSON input, trigger via UI or API.
Best for: developers who maintain multiple actors and need repeatable, assertion-based end-to-end testing.
Each approach has trade-offs in setup time, assertion depth, and maintenance burden. The right choice depends on your deployment frequency and how critical your scrapers are to downstream data pipelines.
Every suite run executes on your own Apify account at the flat pay-per-event rate of $0.35 per suite— not per test case. A suite with 20 test cases costs the same $0.35 as a suite with 1 test case. You also pay standard Apify compute costs for the individual actor runs. ApifyForge has no platform fee or subscription. Apify's free plan includes $5/month in credits, enough for approximately 14 suite runs per month.
Deploy Guard answers one question — is this build safe to deploy? It runs a test suite against your actor (auto-discovered if you don't supply one), validates the output with 6 assertion types (minResults, maxResults, requiredFields, fieldTypes, maxDuration, noEmptyValues), compares the run against a trusted baseline for drift and flakiness, and returns one releaseAction (deploy / halt / review / misconfigured) plus a routable decision enum and confidence score. It's the decision layer, not just a pass/fail report.
Parse releaseAction (or the decision enum) from the synchronous run result and exit non-zero unless it's deploy / act_now. Any failing assertion, a breaking baseline drift, or a cold-start surfaces in verdictReasonCodes and flips the verdict to halt or review automatically — your pipeline never has to reason about individual test failures. The output is additive-only and deterministic, so it's safe to branch a build on.
With enableBaseline, Deploy Guard scores each run against a baseline built from prior runs — so it catches a result-count collapse, a null-rate spike, schema drift, or a flaky test that a single pass would miss. Run it on a schedule and it accumulates releaseMemory: the actor's operational record (first seen, total runs, availability, regression count, average runtime / results / compute, and the fields it has learned are critical). It compounds — after 100 runs Deploy Guard knows your actor far better than a one-off test, and a baselinePolicy (minRunsForTrusted, maxBaselineAgeHours) tunes when a baseline is trusted.
A silent failure is a run that finishes SUCCEEDED but returns empty or broken data — a selector breaks, a login wall appears, pagination drops results. Deploy Guard compares against the baseline and flips the verdict to halt when that happens. It then fingerprints the root cause deterministically (no LLM): Selector Breakage (97%) → update CSS selectors, Bot Protection (94%) → enable residential proxies, Login Wall → refresh credentials, each with a priority-ranked remediation step.
Deploy Guard supports up to 20 test cases per suite at the flat $0.35 fee. Each test case runs as an independent actor call with its own input and assertions. Since the price is per suite rather than per test case, adding more cases improves cost efficiency — 20 test cases at $0.35 works out to $0.0175 per case.
The 6 assertion types are: (1) minResults — actor must return at least N items, (2) maxResults — actor must return no more than N items, (3) requiredFields — specified fields must exist with non-null values, (4) fieldTypes — fields must match expected JavaScript types (string, number, boolean, object, array), (5) maxDuration — run must complete within N seconds, (6) noEmptyValues — fields must not be null, undefined, empty string, or empty array.
Each suite run costs $0.35, charged as a pay-per-event (PPE) fee on your own Apify account. ApifyForge has no platform fee or subscription. Apify's free tier includes $5/month in credits, enough for approximately 14 suite runs per month. You also pay standard Apify compute costs for the actor runs themselves.
Deploy Guard uses per-case error isolation. If one test case crashes, times out, or returns an actor error, the remaining test cases still run and produce results. The failed case is marked with passed: false and includes the specific error message, while passing cases report their assertions normally.
Yes. Deploy Guard runs as an Apify actor, so you can trigger it via the Apify API from any CI/CD system (GitHub Actions, GitLab CI, Jenkins). Pass your test cases as JSON input, then check the output dataset for any failed assertions. The structured JSON output makes it straightforward to parse programmatically.
Deploy Guard validates a single actor against multiple input scenarios in one run — it answers 'does my actor handle these inputs correctly?' ApifyForge Regression Tests, by contrast, runs the same actor twice (before and after a code change) and compares outputs to detect regressions. Test Runner checks correctness; Regression Tests check stability.
No. Unit tests validate individual functions in isolation, while Deploy Guard validates the full actor end-to-end on the Apify platform. They serve different purposes: unit tests catch logic errors during development, while Test Runner catches integration issues, API changes, and platform-level failures that only appear when the actor runs in production.