trace-repro finds the hidden event that really broke production, proves it with a test, and ships the real fix — automatically.
No war rooms. No guesswork. No band-aids.
Days → minsFrom "what happened?" to the root cause
0Humans needed to find, prove and fix it
1 testKept forever, so the bug never comes back
01The problem
Monitoring tools show you where it hurts. They never show you who hit you.
Modern apps do hundreds of things at once. Payments retry, carts expire, customers click "Pay" — often all on the same order. When two of them collide, the error shows up somewhere else entirely. So your team stares at the wrong place.
Anatomy of a 38-millisecond disaster
12:04:17.374Background
The bank re-sends a payment confirmation
The same message arrives twice. The order gets updated again — with nothing stopping it.
12:04:17.386Background
A cleanup timer cancels the order
It thinks the cart was abandoned. At that exact moment, it wasn't.
12:04:17.412Customer-facing
The customer's checkout fails
They see an error — or worse, get charged twice. This is the only thing that shows up as an error.
Blink of an eye~100ms
The entire bug38ms
02Today vs. tomorrow
The old way costs days. And it usually gets the fix wrong.
Without trace-repro
1–3 days
of a senior engineer's time per incident
Someone digs through logs from different systems, by hand, trying to guess what else happened.
AI coding tools patch the line that crashed — hiding the error instead of fixing it.
Bad data quietly piles up: double charges, wrong balances, angry customers.
The same bug returns next month, in a slightly different form.
With trace-repro
Minutes
fully automatic, from alarm to fix
The real cause is found by connecting every event that touched the same order.
A test recreates the exact crash before a single line of code changes.
The fix goes where the trouble started — not where it surfaced.
That test stays in your codebase forever. The bug can't sneak back.
03How it works
Find it. Prove it. Fix it.
01
Find
It connects the dots.
trace-repro maps every request, background job and database change into a living map. When something breaks, it walks backwards to every event that touched the same record.
FalkorDB · cross-trace graph traversal02
Prove
It recreates the crash on purpose.
Before touching any code, it writes a test that replays those exact events, in the exact order, milliseconds apart — and watches it fail. The bug is now proven, not guessed.
pytest · GitLab CI · red pipeline03
Fix
It repairs the root cause.
It follows the map to the code that started the chain reaction, adds the missing safeguard, and reruns the test until it passes. Then it ships the fix to production.
Row lock · idempotency key · GitLab CI
04Watch it work
One incident, start to finish. Zero humans.
01 / 06Incident open
What the engineer sees
05Why a graph
Logs are a list. Reality is a web.
Traditional tools file events by request — like sorting mail by envelope. trace-repro files them by what they touched. Suddenly three unrelated-looking events point at the same order, and the answer is obvious.
What your tools see50,214 events · no connection
GET /products/442 200
POST /login 200
POST /webhooks/payment 200
GET /cart 200
job cart-expiry ok
GET /search?q=shoes 200
POST /reviews 201
POST /checkout/capture 500
GET /orders/7710 200
Three lines matter. Nothing tells you which.
What trace-repro sees3 events · 1 shared order
Every event that touched the order, linked. The culprit is obvious.
Take the graph away and trace-repro simply can't work. That's the point — the graph does the real detective work.
06Compare
Others see the symptom. We see the cause.
Capability
Monitoring toolsDatadog, Sentry, Honeycomb
AI coding assistantsDevin, SWE-Agent, Claude Code
Replay toolsReplay.io, Speedscale, RR
trace-reproCausal self-healing
Connects separate events that collided
No
No
No
Yes
Proves the bug before changing code
No
No
Manual
Yes
Fixes the cause, not the symptom
No
Rarely
No
Yes
Pinpoints the few events that matter
No
No
Replays everything
Top 2–4 of 50,000
Keeps your app fast
Yes
Yes
20–50% slower
Yes
Runs end-to-end without a human
No
Partly
No
Yes
07End to end
From alarm to all-clear, on autopilot.
Covers every one of GitLab's nine DevSecOps stages — and runs natively in GitLab CI.
01
Detect
A failing workload triggers trace-repro instantly.
Monitor02
Investigate
The graph reveals which events collided — and opens an incident.
Plan03
Prove
A new test reproduces the crash. Pipeline goes red.
Verify04
Fix
The real cause is patched. Same test turns green.
Create · Verify05
Ship
Scanned, packaged and rolled out safely to production.
Secure · Package · Configure · Release06
Report
A before/after map is posted as proof. Incident closed.
Govern
08Under the hood
For the engineers in the room.
OpenTelemetry spans and ORM/Redis state mutations stream into FalkorDB. Independent traces are linked through shared (:StateEntity) nodes, and the GitLab Duo Agent queries them via MCP.
// From the crashing span, hop across shared StateEntity nodes to find
// every OTHER independent trace that mutated the same entity before the crash
MATCH (crash_trace:Trace)-[:HAS_SPAN]->(crash_span:Span {id: $crash_span_id})
MATCH (crash_span)-[r_read:READ_ENTITY|WROTE_ENTITY]->(entity:StateEntity)
MATCH (other_trace:Trace)-[:HAS_SPAN]->(mutator_span:Span)-[r_write:WROTE_ENTITY]->(entity)
WHERE other_trace.id <> crash_trace.id
AND r_write.ts_ns <= r_read.ts_ns
AND (r_read.ts_ns - r_write.ts_ns) < 60000000000 // 60s causal window
MATCH (mutator_span)-[:EXECUTED_FUNCTION]->(upstream_fn:ASTFunction)
MATCH (crash_span)-[:EXECUTED_FUNCTION]->(crash_fn:ASTFunction)
RETURN
entity.primary_key AS collided_entity,
other_trace.entry_endpoint AS interfering_request,
other_trace.payload_json AS interfering_payload,
(r_read.ts_ns - r_write.ts_ns) / 1000000.0 AS race_window_ms,
r_write.lock_held AS was_locked,
upstream_fn.file_path AS upstream_mutator_file,
upstream_fn.name AS upstream_mutator_function,
crash_fn.name AS downstream_crash_symptom
ORDER BY r_write.ts_ns ASC
// Exact chain of state transitions: entity creation → corrupted read.
// Produces the ordered requests, payloads and ms delays for the replay test.
MATCH (entity:StateEntity {primary_key: "8812"})
MATCH (first:StateMutation {seq_no: 0})-[:MUTATED]->(entity)
MATCH (crash:Span {id: $crash_span_id})
MATCH p = shortestPath((first)-[:HAPPENED_BEFORE|CAUSED_READ*..12]->(crash))
UNWIND nodes(p) AS step
OPTIONAL MATCH (s:Span)-[:EMITTED]->(step)
OPTIONAL MATCH (t:Trace)-[:HAS_SPAN]->(s)
RETURN t.entry_endpoint AS request, t.payload_json AS payload,
step.field AS field, step.old_val AS old_val, step.new_val AS new_val,
step.ts_ns / 1000000.0 AS ts_ms
ORDER BY ts_ms ASC
# payments/webhooks.py — upstream mutator, not the crashing line
def apply_webhook(db, event):
- order = db.query(Order).get(event.order_id)
+ order = (db.query(Order)
+ .filter_by(id=event.order_id)
+ .with_for_update() # SELECT ... FOR UPDATE
+ .one())
+ if order.last_event_id == event.id: # idempotency guard
+ return order
order.status = transition(order.status, event)
+ order.last_event_id = event.id
db.commit()
# graph diff: lock_held false → true · race_condition_reproduced → resolved
09The demo
Three minutes. One "wait, what?" moment.
Watch trace-repro autonomously capture a cross-trace race, query the FalkorDB causal context layer, synthesize a failing test, and deploy the verified fix with zero human prompts.
We break a live checkout. A standard AI reads the error and slaps on a band-aid that hides the problem.
▶ Jump to 0:000:40 — 1:30
The graph reveals the truth
In FalkorDB, Order #8812 sits in the middle — with two separate events pointing into it, 38ms apart. Anyone gets it instantly.
▶ Jump to 0:401:30 — 2:30
Red, then green
With zero prompts, the agent commits a failing test first. Thirty seconds later, the real fix lands — and the pipeline turns green.
▶ Jump to 1:302:30 — 3:00
Healed for good
Production is redeployed, and the reproduction test joins the regression suite. This bug can never return.
▶ Jump to 2:30
10Questions
Good questions. Straight answers.
What kind of bugs does it catch?
The nastiest kind: double charges, wrong wallet balances, orders stuck in impossible states. Bugs caused by two things happening at almost the same moment — which are exactly the ones humans struggle to find.
Why not just use an AI coding assistant?
They only see the line that crashed, so they patch the symptom. trace-repro proves the cause first with a failing test, then fixes it where it started.
Do I have to rebuild my app?
No. A lightweight middleware reports what each request touched. There's no heavy recording, and no need to replay hours of traffic.
Will it change code without anyone checking?
It can run fully hands-off, or stop at the merge request for approval. Either way, every fix comes with proof: a test that failed before, passes after, and a before/after map of what changed.
Stop guessing. Start proving.
Find the real cause. Prove it with a test. Fix it for good — before your customers notice.