We Only Found This $0 Revenue Loophole Because We Monitor Every Failed Run The Short Version
A free-tier user was extracting data from our Apify actor without paying consistently.
Not by hacking it. Not by breaking it.
But by repeatedly forcing it to fail.
And the only reason we caught it early?
We see every failed run — including customer-triggered ones — in real time.
Most developers don’t.
The Problem Nobody Realises They Have
On Apify, you can see:
your runs aggregate stats dashboards
But what you don’t see clearly is:
every individual failed run triggered by your users
And that creates a dangerous blind spot.
Because:
failures look like noise patterns are invisible abuse hides inside “normal error rates” What We See That Most People Don’t
Because we run real-time monitoring across all runs, we get:
instant alerts for failures full visibility into customer-triggered executions patterns within seconds, not days
So when this started happening, it stood out immediately.
The Signal
Same user. Same input. Same configuration.
Over and over.
Every run:
→ FAILED → TIMED_OUT
That’s not random.
That’s a pattern.
Why This Matters
Most developers would never notice this.
Because without per-run monitoring:
you don’t see repetition you don’t see intent you don’t see who is failing
You just see:
“a slightly elevated failure rate”
What Monitoring Revealed
Because we had visibility, we could actually ask:
“Why would someone keep running something that fails?”
Answer:
Because it’s still giving them value.
What the User Had Figured Out
They were deliberately:
running with low memory setting short timeouts forcing early termination
And despite the failures:
they were still getting data
The Hidden Behaviour Behind Failed Runs
This is the part most people miss:
Even when a run fails:
data may already be written datasets still exist partial output is accessible
So failure does NOT mean:
“nothing was delivered”
The Real Issue (Only Visible If You Can See Failures)
Our actor used a standard pattern:
for (const item of results) { await Actor.pushData(item); await Actor.charge({ eventName: 'result', count: 1 }); }
Which means:
data is written immediately billing happens after
If a run fails mid-way:
some data is already delivered charges may not fully apply What the User Was Doing
They didn’t need successful runs.
They just needed:
partial execution partial output repeated attempts
So they:
forced failure collected what was written repeated
Result:
→ Data delivered → Charges inconsistent
Why This Is a Monitoring Problem First
This is the key point:
The bug existed in code But the risk existed in visibility
Because:
the loophole only matters if it’s exploited exploitation only matters if it’s repeated repetition is only visible if you track runs individually
Without monitoring:
This is invisible.
👉 This Is Exactly the Gap ApifyForge Monitor Solves
We built it for this exact reason.
Because Apify does not alert you when:
a user triggers repeated failed runs patterns emerge across executions something is clearly “off”
With ApifyForge Monitor, you get:
alerts for every failed run visibility into customer-triggered executions detection in under 30 seconds pattern recognition in real time
So instead of:
“we might have an issue”
You see:
“this user has triggered 14 failed runs in 10 minutes”
Fixing the Loophole (Once We Could See It)
Once we understood what was happening, the fix was straightforward.
We changed from:
deliver → charge
to:
charge → deliver
Example const batchResults = [];
for (const item of batch) { const result = await processItem(item); if (result) batchResults.push(result); }
// Charge first if (isPPE && batchResults.length > 0) { await Actor.charge({ eventName: 'result-scraped', count: batchResults.length, }); }
// Then deliver for (const result of batchResults) { await Actor.pushData(result); } But Here’s the Important Part
The fix is obvious once you see the problem.
The hard part is:
seeing the problem at all
What Changed After
After fixing both:
visibility (monitoring) logic (charge-before-deliver)
We saw:
failure rate drop dramatically runtime improve significantly memory stabilise
But more importantly:
We closed a gap we didn’t even know existed.
The Real Lesson
This wasn’t:
a clever hack a security breach a complex exploit
It was:
a visibility failure
If You’re Running PPE Actors, Ask Yourself This
Can you answer:
Who triggered your last failed run? How many times the same user failed today? Whether failed runs still produced data?
If not:
You’re operating blind.
Final Thought
Most developers assume:
“If something breaks, I’ll notice.”
You won’t.
Not when it happens in:
someone else’s run repeatedly without alerts 👉 That’s Why We Built ApifyForge Monitor
Because the biggest problems aren’t in your logs.
They’re in:
runs you never see