Output Completeness Monitor

Detect silent data quality degradation across your Apify actor fleet. When an actor stops throwing errors but starts returning fewer results than it used to — 100 results last week, 30 this week — that is the hardest kind of failure to catch. This actor catches it automatically by comparing current result counts against historical baselines for every actor in your account.

Why this matters: Actors break in two ways. Loud failures throw errors and show up in your run history as FAILED. Silent failures succeed with status SUCCEEDED but return partial or empty data. A site changes its pagination, an API starts rate-limiting without returning errors, or a selector stops matching — the run completes fine, but your data is incomplete. This actor detects those silent failures.

Features

• Automatic baseline comparison — Compares recent successful run output counts against historical averages for each actor
• Configurable sensitivity — Set your own degradation threshold (default 50% drop) and minimum runs required for analysis
• Trend detection — Identifies whether degradation is declining, stable, volatile, or improving over time
• Smart data splitting — Works even when all runs are within the analysis window by splitting data proportionally
• Fleet-wide coverage — One run checks every actor in your account, regardless of fleet size
• Actionable recommendations — Each degraded actor gets a specific recommendation based on severity and trend

Use cases

Scheduled data quality monitoring

Run weekly on a schedule. Get alerted when any scraper starts returning fewer results than its baseline. Catch site changes before your downstream data consumers notice gaps.

Post-deployment validation

After pushing actor updates, run this to verify output volume hasn't dropped. Compare before and after by adjusting the hoursBack window.

SLA compliance

If you deliver data to clients who expect consistent volumes, use this to verify your actors maintain output levels. A 50% drop in results could mean a 50% drop in data delivery.

Multi-actor fleet management

When you run dozens or hundreds of actors, manually checking output counts is impractical. This actor surfaces degradation across your entire fleet in one dataset.

Scraper maintenance prioritization

Sort degraded actors by drop percentage to prioritize which scrapers need immediate attention versus which can wait.

Input

Example input

{
    "apiToken": "apify_api_YOUR_TOKEN_HERE",
    "hoursBack": 168,
    "degradationThreshold": 0.5,
    "minRunsToAnalyze": 3
}

Output

Each actor with detected degradation produces one record. The final record is always a fleet summary.

Degradation report example

{
    "actorName": "website-contact-scraper",
    "actorId": "BCq991ez5HObhS5n0",
    "status": "DEGRADED",
    "currentAvgResults": 3.2,
    "historicalAvgResults": 8.5,
    "dropPercentage": 62,
    "recentRuns": 5,
    "totalSuccessfulRuns": 18,
    "trend": "declining",
    "latestRunResults": 2,
    "recommendation": "Result count dropped 62% and continues to decline — possible site structure change, rate limiting, or partial blocking",
    "checkedAt": "2026-03-18T14:30:00.000Z"
}

Fleet summary example

{
    "type": "summary",
    "totalActors": 294,
    "actorsAnalyzed": 180,
    "actorsDegraded": 4,
    "actorsHealthy": 162,
    "actorsInsufficientData": 14,
    "worstDegradation": "website-contact-scraper (62% drop)",
    "checkedAt": "2026-03-18T14:30:00.000Z"
}

Output fields — Degradation report

Output fields — Fleet summary

How to use the API

Python

from apify_client import ApifyClient

client = ApifyClient(token="YOUR_API_TOKEN")

run = client.actor("ryanclinton/output-completeness-monitor").call(
    run_input={
        "apiToken": "YOUR_API_TOKEN",
        "hoursBack": 168,
        "degradationThreshold": 0.5,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("type") == "summary":
        print(f"Fleet: {item['actorsAnalyzed']} analyzed, {item['actorsDegraded']} degraded")
    else:
        print(f"[{item['status']}] {item['actorName']}: {item['dropPercentage']}% drop ({item['trend']})")
        print(f"  {item['recommendation']}")

JavaScript / Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('ryanclinton/output-completeness-monitor').call({
    apiToken: 'YOUR_API_TOKEN',
    hoursBack: 168,
    degradationThreshold: 0.5,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

const summary = items.find(i => i.type === 'summary');
console.log(`Fleet: ${summary.actorsAnalyzed} analyzed, ${summary.actorsDegraded} degraded`);

const degraded = items.filter(i => i.status === 'DEGRADED');
degraded.forEach(r => {
    console.log(`${r.actorName}: ${r.dropPercentage}% drop (${r.trend})`);
    console.log(`  ${r.recommendation}`);
});

cURL

curl -X POST "https://api.apify.com/v2/acts/ryanclinton~output-completeness-monitor/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "apiToken": "YOUR_API_TOKEN",
    "hoursBack": 168,
    "degradationThreshold": 0.5,
    "minRunsToAnalyze": 3
  }'

How it works

1. Fetches all your actors via the Apify API using your token
2. Pulls run history for each actor (up to 200 most recent runs)
3. Filters to successful runs only — ignores FAILED, TIMED-OUT, and ABORTED runs since those are a separate problem
4. Splits runs into two periods — recent (within hoursBack) and historical (before hoursBack). If all runs fall within the window, splits proportionally (60% historical, 40% recent)
5. Computes averages — average dataset item count per successful run in each period
6. Detects degradation — flags actors where currentAvg / historicalAvg falls below the degradationThreshold
7. Analyzes trends — splits the full run history into halves and compares averages. High coefficient of variation indicates volatility
8. Generates recommendations — based on drop severity (30%, 50%, 80%) and trend direction

Understanding the results

Drop percentage

• 30-50% drop — Gradual degradation. May indicate soft rate limiting, reduced content on target sites, or pagination issues
• 50-80% drop — Significant degradation. Likely a site structure change, API endpoint modification, or partial blocking
• 80%+ drop — Near-total failure. Almost certainly a breaking change: new anti-bot measures, complete site redesign, or API deprecation

Trend values

Limitations

• 200 run limit per actor — Only fetches the 200 most recent runs per actor. Very high-frequency actors may not have enough historical data outside the analysis window.
• Successful runs only — Only compares SUCCEEDED runs. If an actor starts failing entirely, use the Actor Health Monitor instead.
• Zero-result baseline — Actors that historically return zero results are skipped, since there is no meaningful baseline to compare against.
• No per-field analysis — Compares total result counts, not individual field completeness. An actor could return the same number of items but with missing fields — this monitor would not catch that.
• Stats availability — Relies on run stats (outputItems, resultCount, datasetItemCount) being populated. Some very old runs may not have these stats.

FAQ

Q: How is this different from the Actor Health Monitor? A: The Actor Health Monitor catches loud failures — runs that crash, timeout, or return errors. This monitor catches silent failures — runs that succeed but return less data than expected. Use both together for complete fleet coverage.

Q: What if an actor intentionally returns different amounts of data? A: Set the degradationThreshold lower (e.g., 0.2) for actors with naturally variable output. Or increase minRunsToAnalyze to smooth out variability.

Q: How much does it cost to run? A: $0.05 per completeness check. One run covers your entire fleet. A weekly check costs about $0.22/month.

Q: Can I monitor specific actors only? A: Currently it monitors all actors in your account. Filter the output dataset to focus on specific actors.

Q: What if I just deployed a new actor with only 2 runs? A: It will be skipped (default minRunsToAnalyze is 3). Once it accumulates enough runs, it will be included in future checks.

Q: Does this work with PPE-priced actors? A: Yes. It compares dataset item counts regardless of pricing model.

Integrations

Use this actor with:

• Zapier for automated workflows when degradation is detected
• Make for complex remediation automations
• Google Sheets for tracking output trends over time
• The Apify API for programmatic access
• Actor Health Monitor for complete fleet monitoring (failures + degradation)

Related actors

• ryanclinton/actor-health-monitor — Monitor actor failures, diagnose root causes, track trends
• ryanclinton/cost-watchdog — Monitor and control Apify spending
• ryanclinton/actor-portfolio-analytics — Analyze your actor portfolio performance

Pricing

• $0.05 per completeness check — one check covers your entire fleet

Changelog

v1.0.0 (2026-03-18)

• Initial release
• Baseline comparison: current vs historical result counts
• Trend detection with declining/stable/improving/volatile classification
• Smart data splitting for actors with limited historical runs
• Fleet-wide summary with worst degradation identification
• Actionable recommendations based on severity and trend