Pipeline Builder

Actor Pipeline Builder validates multi-actor data pipelines before you run them, catching field mapping errors and type mismatches at design time. Built for Apify developers who chain two or more actors together, it reads each actor's live schema from the Apify API and checks that every output field flows correctly into the next stage's input. The result is a validated pipeline report, a cost estimate, and complete TypeScript orchestration code — ready to copy into your project.

Manual pipeline debugging means running each actor, inspecting datasets, discovering a field name mismatch, and repeating the cycle. This actor replaces that loop by fetching actor schemas directly via the Apify API and performing cross-stage validation in a single run. You get field mapping checks, missing-connection warnings, per-stage memory and timeout configuration, and auto-generated Actor.call() chain code — all without triggering a single billable run of your target actors.

What data can you extract?

Why use Actor Pipeline Builder?

Chaining actors manually is where most pipeline projects stall. You wire stage 1's output into stage 2's input, run the whole chain, and discover 30 minutes later that businessUrl should have been website — or that stage 3 receives no data at all because no field mapping was defined. Repeating this cycle across a 3- or 4-stage pipeline burns hours and wastes compute credits.

This actor validates the entire pipeline at design time, before any actor runs. It fetches live schemas directly from the Apify API using each actor's latest build, checks every field mapping you've defined, and generates copy-paste TypeScript code that correctly wires the stages together. A typical pipeline validation costs $0.40 and takes under 60 seconds.

• Scheduling — schedule validation runs on a cron to catch schema drift when actors are updated
• API access — trigger pipeline builds from CI/CD pipelines, Python scripts, or any HTTP client
• Proxy rotation — not applicable (this actor calls the Apify API, not external websites)
• Monitoring — get Slack or email alerts when a pipeline validation fails after an actor update
• Integrations — pipe the generated code into a webhook, GitHub Action, or deployment workflow

Features

• Live schema fetching — reads each actor's input and output schema from its latest published build via the Apify API, so validation always reflects current field definitions
• Cross-stage field mapping validation — for every fieldMapping entry, checks that the source field exists in the previous stage's output schema and the target field exists in the next stage's input schema
• Missing mapping detection — flags any stage transition where no fieldMapping is defined, warning that upstream data will not be passed downstream
• Parallel actor info fetching — all stages are fetched concurrently using Promise.all(), keeping validation fast regardless of pipeline length
• TypeScript Actor.call() code generation — produces a complete, runnable orchestration script using Apify's Actor.call() pattern with dataset forwarding via apifyClient.dataset().listItems()
• Per-stage resource configuration — accepts memory (MB) and timeout (seconds) per stage, embedding these values in the generated code
• PPE price aggregation — reads each actor's pay-per-event pricing from the Apify API and sums the cost across all stages
• Three-tier cost projection — outputs cost per run, cost at 100 runs/month, and cost at 1,000 runs/month
• Structured error reporting — separates hard errors (actor not found) from soft warnings (missing or unverifiable mappings) in the output
• Minimum 2-stage enforcement — rejects single-stage inputs with a clear error message, since pipeline validation requires at least one stage transition
• Spending limit support — respects Apify's eventChargeLimitReached signal, stopping charges cleanly when a budget cap is hit

Use cases for actor pipeline builder

Automated lead generation pipeline design

Sales and growth teams assembling multi-actor pipelines — for example, Google Maps scraping followed by email extraction followed by email verification — need to confirm that each actor's output fields connect correctly before committing to a long orchestration run. Pipeline Builder validates the full chain in one step and generates the orchestration code, so the team can hand a working script to a developer or deploy it directly via the Apify API.

CI/CD pipeline schema regression testing

Engineering teams maintaining actor workflows need to know when an upstream actor update breaks a downstream stage. Scheduling Pipeline Builder to run after each actor deployment catches schema drift before it causes silent data loss in production. The structured JSON output integrates directly with CI systems — a non-zero errors array can fail the build automatically.

Rapid pipeline prototyping for agencies

Apify consultants and agencies building data pipelines for clients need to prototype and cost different actor combinations quickly. Pipeline Builder lets you swap actors in and out of the stages array, compare the generated code and cost estimates, and present a validated design to the client — without running any actors or spending compute credits.

Onboarding complex multi-actor workflows

Developer teams new to the Apify ecosystem often struggle to understand how to pass data between actors. Pipeline Builder's generated TypeScript code shows exactly how to chain Actor.call() calls, forward datasets between stages, and apply field mappings — serving as a working reference implementation alongside the validation report.

Pipeline cost forecasting

Operations managers and finance teams need to forecast monthly compute spend before committing to a pipeline architecture. Pipeline Builder's cost projection output — per run, at 100 runs, at 1,000 runs — gives concrete numbers based on each actor's actual PPE pricing, not guesswork.

How to build and validate an actor pipeline

1. List your pipeline stages — Enter an array of stage objects in the stages input field. Each stage needs at minimum an actorId (e.g., ryanclinton/google-maps-email-extractor). Add a fieldMapping object to specify how the previous stage's output fields map to this stage's input fields.
2. Configure resources per stage — Optionally set memory (in MB, default 512) and timeout (in seconds, default 120) for each stage. These values are embedded in the generated code.
3. Run the actor — Click "Start" and wait. A 3-stage pipeline typically validates in under 30 seconds. The actor charges $0.40 when complete.
4. Download your report and code — Open the Dataset tab to find the full validation report as JSON. Copy the generatedCode field into your project as a starting point for your orchestration script.

Input parameters

Stage object format

Each entry in the stages array follows this structure:

Input examples

Three-stage lead generation pipeline (Maps → Email → Verify):

{
  "stages": [
    {
      "actorId": "ryanclinton/google-maps-email-extractor",
      "memory": 1024,
      "timeout": 300
    },
    {
      "actorId": "ryanclinton/email-pattern-finder",
      "fieldMapping": {
        "urls": "website"
      },
      "memory": 512,
      "timeout": 120
    },
    {
      "actorId": "ryanclinton/bulk-email-verifier",
      "fieldMapping": {
        "emails": "emailPattern"
      },
      "memory": 256,
      "timeout": 60
    }
  ]
}

Two-stage enrichment pipeline (Contact scraper → CRM push):

{
  "stages": [
    {
      "actorId": "ryanclinton/website-contact-scraper",
      "memory": 512,
      "timeout": 120
    },
    {
      "actorId": "ryanclinton/hubspot-lead-pusher",
      "fieldMapping": {
        "email": "email",
        "name": "contactName",
        "company": "companyName"
      },
      "memory": 256,
      "timeout": 60
    }
  ]
}

Minimal two-stage pipeline (no field mappings — validation only):

{
  "stages": [
    {
      "actorId": "ryanclinton/website-tech-stack-detector"
    },
    {
      "actorId": "ryanclinton/b2b-lead-qualifier"
    }
  ]
}

Input tips

• Define field mappings for every non-first stage — omitting fieldMapping triggers a warning and means the generated code passes raw dataset items rather than mapped fields.
• Use the full actor identifier — always use username/actor-name format (e.g., ryanclinton/website-contact-scraper), not just the actor name. The actor lookup will fail without the username prefix.
• Check field names against actor schemas first — use Schema Registry or Schema Diff to confirm the exact field names before building the pipeline.
• Start with 2 stages — validate the core connection first, then extend to 3 or 4 stages once the first pair validates cleanly.
• Set realistic memory values — the generated code uses the memory value you specify. Check each actor's recommended memory in the Apify Store before setting these values.

Output example

{
  "stages": 3,
  "valid": true,
  "errors": [],
  "warnings": [
    "Stage 3: mapped field 'emailPattern' not in ryanclinton/email-pattern-finder's output schema"
  ],
  "stageDetails": [
    {
      "stage": 1,
      "actor": "ryanclinton/google-maps-email-extractor",
      "actorId": "ryanclinton/google-maps-email-extractor",
      "ppePrice": 0.15,
      "memory": 1024,
      "timeout": 300,
      "outputFields": ["businessName", "website", "email", "phone", "address", "rating", "reviewCount"],
      "inputFields": ["searchQuery", "maxResults", "country", "language", "proxyConfig"]
    },
    {
      "stage": 2,
      "actor": "ryanclinton/email-pattern-finder",
      "actorId": "ryanclinton/email-pattern-finder",
      "ppePrice": 0.10,
      "memory": 512,
      "timeout": 120,
      "outputFields": ["domain", "emailPattern", "confidence", "examples"],
      "inputFields": ["urls", "maxResults", "timeout"]
    },
    {
      "stage": 3,
      "actor": "ryanclinton/bulk-email-verifier",
      "actorId": "ryanclinton/bulk-email-verifier",
      "ppePrice": 0.005,
      "memory": 256,
      "timeout": 60,
      "outputFields": ["email", "valid", "mxCheck", "smtpCheck", "score"],
      "inputFields": ["emails", "verifySmtp", "timeout"]
    }
  ],
  "generatedCode": "import { Actor } from 'apify';\n\nActor.main(async () => {\n    // Stage 1: ryanclinton/google-maps-email-extractor\n    const input = await Actor.getInput();\n    const run1 = await Actor.call('ryanclinton/google-maps-email-extractor', input, { memory: 1024, timeout: 300 });\n\n    // Stage 2: ryanclinton/email-pattern-finder\n    const ds1 = await Actor.apifyClient.dataset(run1.defaultDatasetId).listItems();\n    const run2 = await Actor.call('ryanclinton/email-pattern-finder', { urls: ds1.items.map(i => i.website) }, { memory: 512, timeout: 120 });\n\n    // Stage 3: ryanclinton/bulk-email-verifier\n    const ds2 = await Actor.apifyClient.dataset(run2.defaultDatasetId).listItems();\n    const run3 = await Actor.call('ryanclinton/bulk-email-verifier', { emails: ds2.items.map(i => i.emailPattern) }, { memory: 256, timeout: 60 });\n\n    // Collect final output\n    const finalDs = await Actor.apifyClient.dataset(run3.defaultDatasetId).listItems();\n    await Actor.pushData(finalDs.items);\n});",
  "costEstimate": {
    "perRun": 0.26,
    "monthly100": 26.00,
    "monthly1000": 260.00
  },
  "builtAt": "2026-03-20T14:32:11.000Z"
}

Output fields

How much does it cost to build an actor pipeline?

Actor Pipeline Builder uses pay-per-event pricing — you pay $0.40 per pipeline build. Platform compute costs are included.

You can set a maximum spending limit per run to control costs. The actor stops when your budget is reached.

Comparable pipeline design tools like Zapier ($19–$69/month) and Make ($9–$29/month) charge monthly subscriptions and do not generate TypeScript code or validate Apify actor schemas. With Actor Pipeline Builder, most teams spend $2–$10/month validating pipelines on demand, with no subscription.

Build an actor pipeline using the API

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("ryanclinton/actor-pipeline-builder").call(run_input={
    "stages": [
        {
            "actorId": "ryanclinton/google-maps-email-extractor",
            "memory": 1024,
            "timeout": 300
        },
        {
            "actorId": "ryanclinton/email-pattern-finder",
            "fieldMapping": {"urls": "website"},
            "memory": 512,
            "timeout": 120
        },
        {
            "actorId": "ryanclinton/bulk-email-verifier",
            "fieldMapping": {"emails": "emailPattern"},
            "memory": 256,
            "timeout": 60
        }
    ]
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"Valid: {item['valid']} | Stages: {item['stages']} | Cost/run: ${item['costEstimate']['perRun']}")
    if item.get("warnings"):
        for w in item["warnings"]:
            print(f"  Warning: {w}")
    print("\n--- Generated Code ---")
    print(item["generatedCode"])

JavaScript

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "YOUR_API_TOKEN" });

const run = await client.actor("ryanclinton/actor-pipeline-builder").call({
    stages: [
        {
            actorId: "ryanclinton/google-maps-email-extractor",
            memory: 1024,
            timeout: 300
        },
        {
            actorId: "ryanclinton/email-pattern-finder",
            fieldMapping: { urls: "website" },
            memory: 512,
            timeout: 120
        },
        {
            actorId: "ryanclinton/bulk-email-verifier",
            fieldMapping: { emails: "emailPattern" },
            memory: 256,
            timeout: 60
        }
    ]
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) {
    console.log(`Valid: ${item.valid} | Stages: ${item.stages} | Cost/run: $${item.costEstimate.perRun}`);
    item.warnings?.forEach(w => console.log(`  Warning: ${w}`));
    console.log("\n--- Generated Code ---\n" + item.generatedCode);
}

cURL

# Start the actor run
curl -X POST "https://api.apify.com/v2/acts/ryanclinton~actor-pipeline-builder/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "stages": [
      {
        "actorId": "ryanclinton/google-maps-email-extractor",
        "memory": 1024,
        "timeout": 300
      },
      {
        "actorId": "ryanclinton/email-pattern-finder",
        "fieldMapping": {"urls": "website"},
        "memory": 512,
        "timeout": 120
      }
    ]
  }'

# Fetch results (replace DATASET_ID from the run response)
curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_API_TOKEN&format=json"

How Actor Pipeline Builder works

Stage 1: Actor schema retrieval

The actor fetches metadata for all pipeline stages concurrently using Promise.all(). For each stage, it calls the Apify API at GET /v2/acts/{actorId} with a 30-second timeout. It resolves the actor's latest build ID from taggedBuilds.latest.buildId, then fetches the build record at GET /v2/acts/{actorId}/builds/{buildId} to extract the full input schema (parsed from the JSON string in buildData.inputSchema) and the dataset storage schema from buildData.actorDefinition.storages.dataset.fields. PPE pricing is read from pricingInfos[last].pricingPerEvent.actorChargeEvents and converted to a per-event USD price. Any stage whose actor cannot be resolved is recorded as a hard error, which sets valid: false in the output.

Stage 2: Cross-stage field mapping validation

After all actor schemas are fetched, the actor iterates through each stage transition (stage N to stage N+1). For every fieldMapping entry { inputField: outputField }, it checks that outputField exists in stage N's outputFields record and that inputField exists in stage N+1's inputFields array. Failed checks are recorded as warnings rather than errors, since the Apify build API does not always expose a complete dataset schema — some actors define output fields at runtime. A separate check flags any stage transition where fieldMapping is entirely absent.

Stage 3: TypeScript code generation

The orchestration code is generated by iterating through the stages array and emitting Actor.call() invocations. Stage 1 receives the top-level actor input via Actor.getInput(). Each subsequent stage retrieves the previous stage's dataset with Actor.apifyClient.dataset(prevRun.defaultDatasetId).listItems(), then applies the fieldMapping to construct a typed input object. If no field mapping is defined, the full items array is passed as a data property. Memory and timeout values from the stage config are embedded in each Actor.call() options object. The final stage's dataset items are pushed to the actor's own dataset via Actor.pushData().

Stage 4: Cost estimation and report assembly

The PPE price for each stage is summed to produce costEstimate.perRun. This is multiplied by 100 and 1,000 for monthly projections. The final report object combines the validity flag, error and warning arrays, per-stage details, the generated code string, and the cost estimate — all pushed as a single item to the output dataset. The actor then charges the pipeline-build event ($0.40) via Actor.charge(), respecting eventChargeLimitReached if a spending cap has been set.

Tips for best results

1. Validate the first stage pair before adding more stages. Start with 2 stages and confirm the field mapping is clean. Add a third stage only after the first connection validates without warnings.

2. Use Schema Registry to discover field names. Before writing field mappings, use Schema Registry to look up an actor's exact input and output field names. Typos in field names are the most common source of mapping warnings.

3. Treat warnings as research tasks, not blockers. A warning means a field reference could not be confirmed against the schema — it does not mean the mapping is wrong. Some actors define output fields dynamically and do not expose them in the build schema. Verify manually for those cases.

4. Combine with Schema Diff when swapping actors. If you're choosing between two actors for the same pipeline stage, run Schema Diff first to compare their output fields. This tells you which actor requires fewer field mapping changes.

5. Copy the generated code as a starting point, not a final implementation. The generated code handles the core Actor.call() chain, but you will likely need to add error handling, retries, and data filtering for production use.

6. Schedule validation runs after actor updates. Use Apify's scheduler to run Pipeline Builder weekly or after deployments. If an upstream actor changes its schema, the validation will catch it before your production pipeline fails silently.

7. Set explicit memory values for accurate cost modeling. Memory allocation affects Apify compute costs. If you know each actor's recommended memory, set it in the stage config so the generated code reflects your actual production configuration.

8. Use the cost estimate to choose between pipeline architectures. If two pipeline designs solve the same problem, run both through Pipeline Builder and compare the costEstimate.perRun output to pick the cheaper option before committing.

Combine with other Apify actors

Limitations

• Output schema coverage depends on actor build quality. If an actor does not declare its dataset schema in actorDefinition.storages.dataset.fields, Pipeline Builder cannot validate output field names. Warnings are issued, but the actual fields cannot be confirmed.
• No runtime validation. This actor validates schemas at design time. An actor can declare a field in its schema but return different field names at runtime. The generated code may still fail if the live actor output differs from its declared schema.
• Requires publicly accessible actors. The APIFY_TOKEN environment variable must have permission to read all actor build metadata. Private actors owned by other users cannot be fetched.
• Minimum 2 stages required. Single-actor validation is not supported. For single-actor schema inspection, use Schema Registry or Schema Diff.
• Generated code requires adaptation for production. The output is a functional starting point, not production-ready code. Error handling, pagination, logging, and retry logic must be added before deployment.
• No nested field mapping support. Field mappings are flat: { inputField: outputField }. Mapping from nested fields (e.g., contact.email) or transforming data types between stages is not supported in code generation.
• Cost estimates reflect PPE prices only. The costEstimate sums PPE event prices. Apify compute costs (memory-hours) are not included in the estimate since they depend on actual run duration.
• 30-second timeout per actor lookup. If the Apify API is slow to respond for a given actor, the fetch may time out and that stage will be recorded as an error.

Integrations

• Zapier — trigger a pipeline validation run from a Zapier workflow when an upstream actor is updated in your CI process
• Make — use Pipeline Builder as a step in a Make scenario to validate and regenerate orchestration code on a schedule
• Google Sheets — export the structured validation report (errors, warnings, cost estimate) to a shared Google Sheet for team review
• Apify API — integrate pipeline validation into deployment pipelines using the REST API; fail the build if valid: false is returned
• Webhooks — configure a webhook to POST the validation report to a Slack channel or GitHub Action when a run completes
• LangChain / LlamaIndex — feed the generated TypeScript code and validation report into an LLM-based code review or documentation generation workflow

Troubleshooting

Stage returns "Actor not found" error despite the actor existing. Confirm that the actorId uses the full username/actor-name format (e.g., ryanclinton/website-contact-scraper, not just website-contact-scraper). Also verify that your Apify token has read access to the actor — private actors belonging to other users cannot be fetched.

All output fields appear empty for a stage. The actor's latest build does not declare a dataset schema in actorDefinition.storages.dataset.fields. This is common for older actors. Pipeline Builder will issue a warning but still generate code. Use the Apify Console to inspect the actor's actual output dataset and confirm field names manually before relying on the mapping.

Generated code runs but no data appears in the final dataset. This typically means a field mapping references a field name that does not match the actual runtime output. Check the warnings array in the validation report for mapping issues. For actors where the schema is not declared, run the upstream actor manually and inspect its output dataset to get the actual field names.

Validation warnings on every stage transition. If all stages produce warnings about missing schemas, the actors in your pipeline are likely older and do not expose build-time dataset schemas. The validation will still succeed (valid: true) and the generated code will still run — the warnings indicate reduced validation confidence, not a broken pipeline.

Run completes instantly with valid: false and no stageDetails. At least one actor ID was not found. Check each actorId in the errors array, correct the identifier, and re-run.

Responsible use

• This actor only accesses actor metadata and schema information via the Apify API.
• Only actors that your API token has permission to read will be processed.
• Do not use this actor to harvest pricing or schema data from competitor actors at scale.
• Generated code uses Actor.call() which triggers billable runs on the target actors — review cost estimates before deploying generated pipelines.

FAQ

How many stages can an actor pipeline have? There is no hard maximum imposed by Pipeline Builder — pipelines with 2 to 6+ stages have been validated successfully. Performance scales linearly since all actor schemas are fetched in parallel. Practical limits come from the Apify API rate limits and the 512 MB default memory allocation of the builder itself.

Does Actor Pipeline Builder run any of the actors in my pipeline? No. Pipeline Builder only reads actor metadata and schemas from the Apify API. It never calls Actor.call() on your pipeline actors during validation. No credits are consumed by the target actors during a build.

How accurate is the field mapping validation? Accuracy depends on whether each actor publishes its dataset schema in its build metadata. Actors that declare actorDefinition.storages.dataset.fields are validated fully. Actors that define output fields at runtime receive warnings instead of confirmed passes. For best results, combine with Schema Registry to inspect field names before building.

Can I validate pipelines that include actors from other Apify users? Yes, as long as the actors are public (or your token has access to them). The actor ID format is always username/actor-name.

How long does a typical pipeline validation take? Most 2-4 stage pipelines complete in under 30 seconds. Each actor lookup has a 30-second timeout, and all stages are fetched concurrently, so total time is determined by the slowest individual lookup — typically 5-15 seconds for a 3-stage pipeline.

Is the generated TypeScript code production-ready? The generated code is a correct functional starting point for the described pipeline. It needs error handling, pagination for large datasets, logging, and environment-specific configuration before production deployment. Treat it as a reference implementation, not a finished product.

Can I use Actor Pipeline Builder to validate pipelines with non-Apify actors? No. Pipeline Builder reads schemas exclusively from the Apify API. All stages must reference valid Apify actor IDs.

How is Actor Pipeline Builder different from Zapier or Make? Zapier and Make are no-code runtime orchestration tools. Pipeline Builder is a design-time validation tool that generates Apify-native TypeScript code. It does not execute workflows — it validates that actor schemas connect correctly and produces the code for you to deploy yourself via the Apify platform.

Can I schedule Actor Pipeline Builder to run automatically? Yes. Use Apify's built-in scheduler to run Pipeline Builder on a cron schedule — for example, nightly or after each actor deployment. This acts as a schema regression test for your pipeline.

What happens if an actor has no PPE pricing? The stage's ppePrice will be 0 and the cost estimate will exclude that stage. This occurs for actors using compute-unit pricing rather than pay-per-event. The pipeline will still validate and generate code normally.

Can I use the output with the Apify MCP server? Yes. The structured JSON output and the generatedCode string can be passed to an LLM via the Apify MCP server for code review, documentation generation, or further pipeline design assistance.

Is it legal to read actor schemas and pricing via the Apify API? Yes. Pipeline Builder uses the official Apify API with your own API token to read publicly available metadata for actors you have permission to access. This is standard platform API usage, not scraping.

Help us improve

If you encounter issues, you can help us debug faster by enabling run sharing in your Apify account:

1. Go to Account Settings > Privacy
2. Enable Share runs with public Actor creators

This lets us see your run details when something goes wrong, so we can fix issues faster. Your data is only visible to the actor developer, not publicly.

Support

Found a bug or have a feature request? Open an issue in the Issues tab on this actor's page. For custom solutions or enterprise integrations, reach out through the Apify platform.