Do AI-Built Repos Ship Better Code?
A DORA metrics comparison of 536 GitHub repositories.

1. Executive Summary
Vibe-coded repositories appear to outperform traditional projects on DORA metrics — 55% score “elite” compared to 42% for established OSS. But the numbers are misleading. AI-built repos achieve fast lead times (0.4 hours median) and rapid recovery because solo developers push directly to main with no review gates, not because the code is better.
When we weight by volume — looking at failures per push rather than per-repo medians — the picture inverts:
Established open-source projects deploy 50% more frequently (2.79/day vs 1.87/day) while breaking less often per push. Their longer lead times (4.6 hours vs 0.4 hours) reflect code review and CI gates — process that reduces the aggregate failure rate by 28% relative to AI-built code.
2. What Is DORA?
DORA stands for DevOps Research and Assessment, a research program started by Nicole Forsgren, Jez Humble, and Gene Kim and acquired by Google Cloud in 2018. Over a decade of annual State of DevOps surveys, covering tens of thousands of engineering professionals, the team looked for the measurable traits that set high-performing software teams apart. The findings were popularized in the 2018 book Accelerate.
The headline result is that four metrics are enough to describe how well a team delivers software. Two measure throughput (how fast changes reach users) and two measure stability (how often those changes break things, and how quickly the team recovers). The key insight from DORA’s research is that the best teams don’t trade one for the other. Elite performers are both faster and more stable.
Deployment Frequency
How often do you ship to production?
Counts successful releases over time. Frequent, small deployments are easier to test, review, and roll back than large batches. Elite teams deploy on demand, often several times a day.
Lead Time for Changes
How long does a change take to go from commit to production?
Captures everything between writing code and running it: review, testing, approvals, and the deploy pipeline. Shorter is better, but only if quality gates are still there.
Change Failure Rate
What share of deployments cause a failure?
The percentage of changes that need a hotfix, rollback, or patch. It’s the main check on speed. Shipping fast doesn’t count if a lot of what ships is broken.
Time to Restore Service
When something breaks, how fast do you recover?
Also called MTTR, and more recently failed deployment recovery time. Measures resilience: good monitoring, quick diagnosis, and a fast path to a fix or rollback.
Performance Tiers
DORA groups teams into performance clusters based on their survey responses. The thresholds are recalibrated each year, but the commonly cited benchmarks look like this:
| Tier | Deploy Frequency | Lead Time | Change Failure Rate | Time to Restore |
|---|---|---|---|---|
| Elite | On demand (multiple/day) | < 1 day | ~5% | < 1 hour |
| High | Daily to weekly | 1 day – 1 week | ~10% | < 1 day |
| Medium | Weekly to monthly | 1 week – 1 month | ~15% | 1 day – 1 week |
| Low | Monthly or less | 1 – 6 months | > 30% | > 1 week |
Why this matters for AI-built code: DORA was designed for teams, where lead time includes code review and deployments pass through a shared pipeline. A solo developer pushing straight to main can post elite-looking numbers on the two throughput metrics without any of the practices those numbers are supposed to reflect. That gap is what the rest of this analysis explores.
3. Sample & Methodology
Repositories are sourced from daily Reddit scraping (r/vibecoding and related subs) and classified during enrichment into three categories. DORA metrics are computed from GitHub Actions workflow run data over a 90-day window. Because we only see GitHub data, each of the four metrics uses a CI-based proxy:
- Deployment Frequency — completed runs per day on the default branch
- Lead Time for Changes — median hours from PR creation to merge
- Change Failure Rate — percentage of CI runs that fail
- Mean Time to Restore — median hours from failure to next success
Only repos producing at least 3 of 4 metric tiers are included in tier comparisons, filtering out projects with too little signal. This report covers 536 repos with sufficient data.
Analysis Funnel
| Stage | Vibe Coded | Indie | Established |
|---|---|---|---|
| Total repos | 2,283 | 412 | 251 |
| Has CI/CD | 879 (39%) | 208 (50%) | 141 (56%) |
| DORA candidates (≥10 runs) | 676 (30%) | 186 (45%) | 137 (55%) |
| Scanned | 656 | 181 | 134 |
| Produced DORA tier | 325 (14%) | 117 (28%) | 94 (37%) |
Selection bias note: Only 14% of vibe-coded repos produce a DORA tier, compared to 37% of established OSS. Vibe-coded repos that do have active CI are a self-selected minority — likely the most serious projects in the cohort. The remaining 86% have no automation at all.
4. Overall DORA Tier Distribution
Among repos with enough data for a composite tier, the distribution shifts meaningfully across categories:
Numbers at right are repo counts per category.
No repos in any category scored “low” overall. Established OSS is the only category where “high” outnumbers “elite” (53% vs 42%) — a signature of disciplined teams that trade speed for reliability. Vibe-coded and indie repos cluster at elite because solo developers push fast with short feedback loops.
5. Metric-by-Metric Comparison
5a. Deployment Frequency
Deployments per day:
| p10 | p25 | Median | Mean | p75 | p90 | |
|---|---|---|---|---|---|---|
| Vibe Coded | 0.56 | 0.93 | 1.87 | 3.02 | 3.29 | 6.17 |
| Indie | 0.31 | 0.67 | 1.35 | 3.02 | 3.16 | 5.76 |
| Established OSS | 0.35 | 0.80 | 2.79 | 12.21 | 8.46 | 25.23 |
Established OSS deploys at nearly 1.5× the median rate of vibe-coded repos. The gap widens dramatically at the top end: the p90 established project ships 25 times per day versus 6 for vibe-coded. Their mean (12.2/day) is pulled up by high-velocity projects with many contributors.
5b. Lead Time for Changes
| p10 | p25 | Median | Mean | p75 | p90 | |
|---|---|---|---|---|---|---|
| Vibe Coded | 0.03h | 0.10h | 0.37h | 2.87h | 1.34h | 6.55h |
| Indie | 0.04h | 0.17h | 0.62h | 11.76h | 6.00h | 24.03h |
| Established OSS | 0.27h | 0.84h | 4.56h | 27.30h | 22.75h | 119.5h |
The 0.37-hour median is not a feature — it’s the absence of review. A lead time of 22 minutes means code goes from written to merged with no human review. Established OSS at 4.6 hours reflects PR review, CI checks, and approval gates. The p90 of 119 hours (5 days) includes thorough multi-reviewer processes on complex changes.
5c. Change Failure Rate
The central finding of this analysis. Per-repo medians look similar; volume-weighted aggregates diverge sharply:
| Median CFR | Mean CFR | Aggregate CFR | Total Runs | Failed | |
|---|---|---|---|---|---|
| Vibe Coded | 6.1% | 13.5% | 12.7% | 34,866 | 4,429 |
| Indie | 5.4% | 12.0% | 12.6% | 15,835 | 1,995 |
| Established OSS | 6.6% | 12.0% | 9.1% | 20,906 | 1,910 |
The median masks the problem. At the median, all three categories look similar (5–7%). But the aggregate CFR — total failures divided by total runs — tells a different story: vibe-coded and indie repos fail at 12.7%, while established OSS fails at 9.1%. This 28% gap means established projects produce fewer broken builds per unit of work.
The mean-median gap in vibe-coded repos (13.5% mean vs 6.1% median) signals a long tail of high-failure projects — a subset of AI-built repos that push frequently with very high failure rates, pulling the aggregate up.
5d. Mean Time to Restore (MTTR)
| p10 | p25 | Median | Mean | p75 | p90 | |
|---|---|---|---|---|---|---|
| Vibe Coded | 0.09h | 0.26h | 0.99h | 29.0h | 5.09h | 40.7h |
| Indie | 0.13h | 0.30h | 0.88h | 57.9h | 15.9h | 143.3h |
| Established OSS | 0.25h | 0.41h | 1.37h | 26.6h | 9.66h | 40.7h |
Sub-hour median recovery across all categories reflects modern CI: a failed run is usually fixed by the next push. Indie repos have a strikingly high p90 (143 hours — nearly 6 days), suggesting abandoned failures that are eventually fixed in a batch rather than immediately addressed.
6. The Real Metric: Failures Per Push
DORA metrics were designed for large engineering organizations at Google scale. For small and solo projects, the standard tier labels don’t capture what matters. A better lens: how often does a push break the build, and how much failure does each project generate per day?
Failures per day:
| Median | Mean | p75 | p90 | Total/day | |
|---|---|---|---|---|---|
| Vibe Coded | 0.117 | 0.416 | 0.423 | 1.043 | 132.4 |
| Indie | 0.077 | 0.319 | 0.218 | 0.847 | 36.3 |
| Established OSS | 0.169 | 0.952 | 0.604 | 2.731 | 85.7 |
Established OSS has the highest median failures per day (0.169) simply because they push the most. But their failure rate per push (9.1%) is the lowest of the three categories. Vibe-coded repos push less often but break more often when they do — the mean of 0.416 failures/day is 3.5× their median, exposing a long tail of repos that fail frequently.
The mean-median ratio tells the story. Vibe-coded repos: 3.6×. Indie: 4.1×. Established OSS: 5.6×. Established OSS has the widest spread because a few very-high-velocity projects (25+ deploys/day) dominate the tail — but those projects also have the lowest failure rates. In vibe-coded repos, the tail is dominated by projects that push often and break often.
7. Conclusions
What the tiers say
By standard DORA tier labels, vibe-coded repos look elite. They push quickly, merge instantly, and recover fast. But these metrics reward the absence of process — no PR review, no multi-stage CI, no approval gates — as much as they reward engineering excellence.
What the volumes say
Established OSS ships more code with fewer failures per push. The 9.1% aggregate failure rate versus 12.7% for vibe-coded repos is a 28% improvement, achieved despite 50% higher deployment frequency. The process that makes their lead times longer also makes their code more reliable.
What the distributions say
The long tail of vibe-coded repos — high mean-to-median ratios on CFR and failures/day — suggests a subset of AI-built projects where the developer iterates through CI, using failing builds as a feedback loop rather than testing locally. This is workable for a solo developer but would not scale.
The DORA framework’s blind spot
DORA was built to measure team-based software delivery. Its four metrics assume code review (lead time), deployment pipelines (frequency), incident detection (failure rate), and incident response (MTTR). For a solo developer pushing to main, three of these metrics collapse to “time between keystrokes.” The framework can’t distinguish fast-and-disciplined from fast-and-careless.
Bottom line: AI-built repos score well on DORA because they skip the steps that DORA was designed to measure. Established OSS scores lower because it does those steps — and the lower failure rates prove the steps work.
Data source: Cronbox DORA scanner, analyzing GitHub Actions workflow runs over a 90-day window ending November 7, 2026. Repos sourced from daily Reddit scraping, classified during enrichment, and filtered to those producing at least 3 of 4 DORA metric tiers (n=536).
Definitions: “Aggregate CFR” is total failed runs ÷ total completed runs across all repos in a category, giving high-volume repos proportional weight. “Median CFR” is the median of per-repo failure rates, giving each repo equal weight regardless of volume. “Failures per day” is per-repo deployment frequency × per-repo CFR.