State of AI Software Delivery 2026

Recent developments (October 2026)

Several of the studies this volume draws on have published new editions since May 2026, and one has called its own headline result out of date. This section adds them, as the limitations section promises, without rewriting the body.

Corrections to the May 2026 edition. Several figures were wrong and are now fixed in the text and the charts:

• Peng et al. report a 95% confidence interval of 21% to 89% faster, not 30.6% to 80.8% at 90%, and their participants were professional programmers with an average of six years' experience, not a junior-to-mid cohort.
• METR's interval runs from 2% to 39% longer, so it does not cross zero, as Figure 2 used to show.
• Stack Overflow's 2024 figure of 76% is of all respondents and their development process, not of professional developers' daily work, and that survey did not report that 84% saw a productivity gain.
• DORA's "39,000" is a total across a decade of surveys, not the 2024 sample, and the elite-quartile adoption figures we cited could not be found in DORA's published 2024 summaries; the DORA paragraphs now use figures from DORA's own 2024 and 2025 announcements.
• The Octoverse 2024 figures are now the ones GitHub published, and two claims we could not source are gone: that fewer than 15% of teams measure AI's impact, and per-developer ranges inside the Peng and METR studies.

Why two volumes

The honest version of an annual AI-in-software-delivery report is harder than the marketing version. The marketing version has a chart of adoption growing left-to-right, a productivity claim in the 30–50% range, and a CTA to buy more AI. The honest version starts by reading every public study that already exists, notices that the productivity claims span from +55% faster to −19% slower depending on what you measure, and asks why.

We chose to publish in two volumes for three reasons.

First, a real survey takes time. The Stride 2026 primary study is designed to field through Prolific Academic with a pre-registered design, external statistical review, and a 14-day workspace-admin opt-out window before any telemetry numbers can be cited. That cycle does not collapse into a weekend.

Second, pre-registering hypotheses before looking at data is what keeps this a study and not a marketing claim with a methodology section stapled on. The hypotheses we register today are the ones we will test, not the ones we invent post-hoc to match whatever the data shows.

Third, the literature already says something. It is just not what either side of the AI debate wants it to say. Volume 0 is the honest survey of what the existing public evidence supports, and where it deliberately does not commit. The 2026 primary study fills a specific gap in that evidence, not the entirety of it.

Adoption is no longer the question

The most-published number in AI-in-software-delivery research is the adoption rate. It is also the least interesting.

The 2024 Stack Overflow Developer Survey put adoption at 76% of all respondents using or planning to use AI tools in their development process, from 65,437 responses. GitHub's Octoverse 2024 reported a 98% increase in the number of projects, a 59% surge in contributions to generative-AI projects and a 92% spike in Jupyter Notebook usage. McKinsey's early-2024 State of AI survey put the share of respondents whose organisations regularly use generative AI at 65%.

The DORA 2024 Accelerate State of DevOps report is the most measured of the lot. More than 75% of its respondents relied on AI for at least one daily professional responsibility, 39% reported little or no trust in AI-generated code, and as AI adoption increased, DORA estimated a 1.5% decrease in delivery throughput and a 7.2% reduction in delivery stability. The Anthropic Economic Index (released in February 2025 from anonymised Claude usage) puts "computer and mathematical" occupations, which include software engineering, as the largest occupational category in Claude usage, at about 37% of queries.

These numbers move with how you define "use AI" (any code completion? a paid Copilot seat? an agentic workflow?) and with who answers: Stack Overflow's survey is a convenience sample of its own community, McKinsey asks about whole organisations, and DORA surveys technology professionals. The newer editions under Recent developments put adoption higher still, at 84% to 90%. The exact percentage is no longer the interesting question.

The interesting question is what happens after adoption. That is where the literature splits open.

Productivity: the contested ground

Two of the most-cited studies in this area report findings in opposite directions on the same question.

The GitHub Copilot experiment (Peng et al., 2023) recruited 95 professional programmers through Upwork and gave them a standardised task: implementing an HTTP server in JavaScript. 45 had access to Copilot and 50 did not. The Copilot group completed the task 55.8% faster on average, with a 95% confidence interval of 21% to 89%. The study has been cited extensively by tool vendors as evidence of large AI productivity gains, and the underlying experimental rigor is real. What is also real, but less often cited, is the design: a single benchmark task, freelance developers with an average of six years of experience, larger gains for the less experienced, and completion time as the sole outcome.

The METR 2025 randomised study (Model Evaluation & Threat Research) is the most cited study from the other direction. METR recruited 16 experienced open-source contributors and randomised 246 of their real-world repository tasks to either AI-assisted or AI-disallowed conditions. The AI-assisted tasks took ~19% longer to complete. In the same study, the developers' own self-perception was that AI made them about 20% faster (a 39-percentage-point gap between measured and perceived productivity, in the same population, on the same tasks). Before the study ran, ML experts asked to predict the result expected a 38% speedup; economists expected 39%. In February 2026, METR labelled these results out of date; its newer data, covered under Recent developments, points to small gains.

These two studies do not contradict each other in the way it first looks. They are measuring different things, with different populations, on different time horizons.

Peng's task was standardised and self-contained: exactly the kind of well-bounded problem at which current LLMs excel. METR's tasks were embedded in mature codebases the developers already knew well: exactly the kind of context-dependent work where the cost of explaining context to AI exceeds the benefit of the AI's suggestions. Peng measured freelance developers on a fresh problem; METR measured experienced maintainers on their own code. The honest synthesis is: AI is unambiguously fast at greenfield, well-specified, standalone work, and the literature does not yet settle the question on context-rich legacy work.

The DORA reports add organisational-level evidence to this picture, measured against DORA metrics and always as correlation, not causation. In DORA's 2024 data, higher AI adoption came with an estimated 1.5% drop in delivery throughput and a 7.2% drop in delivery stability. In its 2025 data, AI adoption had a positive relationship with throughput and product performance and still a negative one with stability, and DORA summed up the pattern this way: "AI's primary role is as an amplifier, magnifying an organization's existing strengths and weaknesses."

There is one thing the studies agree on: the uncertainty is wide. Peng's 95% interval runs from 21% to 89% faster, and METR's from 2% to 39% slower, so neither headline is precise, and the effect depends on the task, the tool and how well the developer knows the code. The "AI helps / AI hurts" split is real, but it is task-, tool- and tenure-dependent, not a coin flip.

The four landmark studies, side by side

The table below collapses the contested ground into one frame. Read across the rows: each study answers a different question, with a different population, on a different time horizon. Read down the "key limitation" column: each study is honest about what its design cannot conclude. The Volume 1 design picks a fifth position (survey + telemetry on the same teams, with pre-registered hypotheses) to fill the gap none of these four cover.

Measurement: the gap nobody fills

If you read the methodology sections of every public survey, one fact stands out. The vast majority of "AI productivity" claims are self-reported, often by people who have never instrumented their own workflow to measure it.

The 2025 Stack Overflow Developer Survey asks developers whether AI tools have improved their productivity: 52% agree that AI tools or agents have had a positive effect. The same survey does not ask whether the respondent has ever measured their own cycle time, defect rate, or task completion before vs after adoption. The implication is that "yes, AI has improved my productivity" is, for most respondents, an introspective judgement made under the same conditions that produce the METR 39-percentage-point perception–measurement gap.

Across the DORA, McKinsey, Stack Overflow, and Octoverse reports, the evidence on AI's impact is overwhelmingly self-reported or observational; none of them measures the same teams before and after adoption under control conditions. DORA itself is the closest thing the industry has to a measurement standard, but its delivery metrics, now five (deployment frequency, change lead time, change fail rate, failed deployment recovery time and deployment rework rate), describe organisational outcomes, not AI-attributable contributions to those outcomes. A team that gets faster with AI on the input side and slower with AI's defects on the output side may net the same DORA score with a different underlying mechanism.

This is the single most consequential gap in the existing literature. We can tell you, with high confidence drawn from large samples, what developers feel about AI's effect on their productivity. We cannot tell you, from any single published study, what AI does to a development team's measured throughput when the same team measures itself before and after, with statistical rigor, on real work, at scale.

This is the gap Volume 1 of this study addresses.

What Volume 1 will measure

The Stride 2026 primary study is two arms, cross-referenced.

The survey arm targets N≥1,500 senior software-delivery practitioners (IC engineers, engineering managers, staff+ engineers, product managers, engineering directors and VPs). Recruitment is Prolific Academic for the panel arm with an organic top-up from the Stride newsletter, LinkedIn, an opt-in GitHub Action for OSS maintainers, and selected industry communities. Sample is balanced across role (IC 40 / EM 25 / PM 15 / Staff+ 15 / Exec 5), company size (≤50, 51–500, 501–5000, 5000+), region (NA / EU / APAC / RoW), and AI adoption stage (heavy daily / selective / exploratory / opted out: the opted-out cohort is non-negotiable; the "no" voice is the hardest finding to claim without it).

The telemetry arm analyses ~14,000 stories from the Stride product database covering Q3 2025 through Q1 2026. Every aggregation enforces k-anonymity at k≥10 (no cell published with fewer than 10 distinct workspaces) plus differential-privacy noise (Laplace, ε=1.0) on any cell-level "average" we report. Schema published, opt-out window provided to workspace admins 14 days pre-publish, third-party privacy counsel signs off the extraction before it runs.

Both arms are stratum-vs-stratum, never joined at the individual respondent or workspace level.

Pre-registered hypotheses

These are the five hypotheses we register on OSF before fielding closes. They are also the hypotheses we will test. The pre-registration is locked, time-stamped, public; if we find something more interesting than these hypotheses in the data, we will report it explicitly as exploratory rather than dressing it up as confirmatory.

1. H1: Self-vs-measured gap. Self-reported productivity gain from AI will be higher than telemetry-observed productivity gain, on comparable task cuts, by a factor of ≥1.5×. (This is the Peng-vs-METR question, asked on a representative sample.)
2. H2: Measurement-as-correlate. Teams that measure AI's impact will report higher tool satisfaction (NPS) than teams that don't, controlling for company size and AI tool spend. (Measurement may be a marker of organisational maturity that itself produces satisfaction, independent of any AI effect.)
3. H3: Cognitive-load U-curve. Composite cognitive load (NASA-TLX 5-item short form) will not decrease monotonically with AI adoption depth. We predict a U-curve, with lowest cognitive load in the middle-adoption cohort and a rise back up in heavy-adoption (the "more AI = more context-switching" effect).
4. H4: Burnout null. Burnout (Maslach Burnout Inventory short form) will not differ significantly between heavy-AI and opted-out cohorts after controlling for company size and role tenure. The null is the finding. Both "AI causes burnout" and "AI cures burnout" are commercially attractive claims; the literature does not yet support either, and our prediction is that ours won't either.
5. H5 (exploratory): ROI by team size. ROI per dollar of AI tool spend will be higher for teams ≤50 than teams 5,000+, mediated by adoption-speed differences. Reported with exploratory framing, not as a confirmed finding.

The hypotheses, the planned cross-tabs, the exclusion criteria, the weighting scheme, and the multiple-comparison correction method (Benjamini–Hochberg FDR at q=0.05 for the planned family) are all in the pre-registration document linked below.

Methodology summary

The full methodology is on the companion page. The short version:

• Survey instrument: ~62 substantive items + 4 screening + 3 attention checks + 8 firmographics. Median completion 14 minutes. Validated scales used: Maslach Burnout Inventory short form (Mind Garden licensed), NASA-TLX raw 5-item, SUS-anchored single item for tool satisfaction, NPS. Three attention checks at items #14, #32, #54; two honeypot questions invisible to humans; Cloudflare Turnstile at entry; vendor-side IP and device-fingerprint dedup.
• Recruitment: Prolific Academic panel arm (~1,200 completes; effective CPI ~$4) + organic arm (~300 completes via newsletter / LinkedIn / OSS-maintainer GitHub Action / industry communities). Organic responses carry a separate stratum flag in the dataset so analyses can run with/without them.
• Pilot wave: N=75 panel respondents + 8 think-aloud sessions ($75 honorarium, 30 min each) before the main field opens. Iteration window 14 days; any item flagged confusing by ≥3 of 8 think-alouds gets reworded.
• Telemetry: ~14,000 Stride stories Q3 2025–Q1 2026. k-anonymity ≥10 + Laplace DP noise (ε=1.0). Schema published; workspace-admin opt-out window 14 days pre-publish.
• Statistics: 95% Wilson confidence intervals on every quoted percentage; Cohen's h for proportion comparisons; Cliff's delta for ordinal Likert; Cohen's d for continuous. Multiple-comparison correction (Benjamini–Hochberg) for the planned cross-tab family. Post-stratification weights on role × company size × region against Stack Overflow Developer Survey 2024 base rates.
• Pre-registration: posted to OSF (Open Science Framework) before fielding closes. Locked, time-stamped, public.

Dataset publication

When the primary findings drop, the survey dataset publishes alongside under CC-BY-4.0: anonymised individual-response CSV with weighted-indicator column, pre-computed cross-tab tables, JSON-Schema data dictionary, and aggregated telemetry summary tables. Distribution as a single ZIP bundle plus a Zenodo DOI for permanent citability. Raw individual telemetry rows are not released (privacy + customer-contract reasons); the aggregated tables are.

Limitations and what to expect

Volume 0 limitations are real and easy to state.

• This is a synthesis, not a primary study. Every number on this page is downstream of another team's research. We have not yet measured anything ourselves on this report. The framing is "what the literature already supports," and where the literature is contested, we say so.
• The cited studies have their own limitations. Peng et al. 2023 was a single-task experiment with freelance developers. METR 2025 had n=16, and METR has since called its result out of date. DORA's surveys are observational and cannot establish causation. Stack Overflow's Developer Survey is a convenience sample of one community. We have tried not to over-claim any single one of them.
• The literature is moving. New studies on AI productivity keep appearing. Findings here are accurate to public studies as of the publish date; if a major study lands between Volume 0 and Volume 1, we append a "Recent developments" note rather than rewriting the body. Several have, and the October 2026 note is at the top of this page.

Volume 1 will be honest in different ways.

• Every quoted percentage will carry a 95% Wilson confidence interval. Comparative claims will carry effect sizes (Cohen's h, Cliff's delta, or Cohen's d), not just p-values.
• Every cross-tab in the planned family will run through Benjamini–Hochberg correction. Exploratory cuts will be labelled exploratory.
• Sensitivity analyses (organic excluded, attention-check failures excluded, under-4-minute completions excluded) appear in the methodology appendix for every headline number.
• If the pre-registered hypotheses turn out to be wrong, we will say so. If a sensitivity analysis flips a headline finding, we will pause publication and document the flip as the finding.
• The telemetry findings will report ranges across all 14,000-story-sample workspaces; no single workspace will be identifiable. The Stride product itself will not be named or ranked in any comparison table.

The vendor-neutrality posture is self-interested as much as it is principled. A study that buries an inconvenient finding dies in one news cycle. A study that publishes them is the one that gets cited five years later. The latter is the strategic move.

Frequently asked questions

Related work

Reference this Volume 0 in your own writing using the citation below. The dataset DOI and the Volume 1 primary-findings citation will be added at the same URL when Volume 1 publishes. Readers who cite Volume 0 today will automatically cite the most current version when crawlers re-fetch.

Stride Research. (2026). State of AI Software Delivery 2026, Volume 0: Landscape synthesis and pre-registered design. Newlight Solutions. https://www.stride.page/research/state-of-ai-software-delivery-2026

Newlight Solutions. (2026). State of AI Software Delivery 2026, Volume 0: Landscape synthesis + pre-registered study design [Report]. Stride. https://www.stride.page/research/state-of-ai-software-delivery-2026

Participate in Volume 1

Volume 1 of this study has not been published yet. If you want to take part when it fields, either via the panel (compensated) or the organic arm (entered in a $20 gift-card raffle for a 1-in-25 chance), email info@newlightai.com to be notified when recruitment opens. Maintainers of public open-source repositories who want to invite their contributors via the opt-in GitHub Action can email the same address.

If you want to be cited (researchers, journalists, vendors, practitioners with something useful to say about a finding here), that's the right address too. The vendor-neutrality policy means we won't quote a vendor's own report verbatim, but we will quote a practitioner observing one. The two are not the same.

The honest version of an annual report is harder than the marketing version. It is also the one that gets read in 2027.

References

Citations are inline above; this section consolidates the bibliography. All sources were publicly accessible when last checked, on October 25, 2026.

1. Stack Overflow Developer Survey 2024. Stack Overflow, July 2024. 76% of all respondents use or plan to use AI tools in their development process; 81% name increased productivity as the biggest benefit of AI tools. 65,437 responses.
2. GitHub Octoverse 2024. GitHub, October 2024. A 98% increase in projects, a 59% surge in contributions to generative-AI projects and a 92% spike in Jupyter Notebook usage.
3. The state of AI in early 2024: Gen AI adoption spikes and starts to generate value. McKinsey & Company, May 2024. 65% of respondents say their organisations regularly use generative AI.
4. 2024 Accelerate State of DevOps Report. Google DORA, October 2024. More than 75% of respondents rely on AI for at least one daily responsibility; as AI adoption increased, delivery throughput fell an estimated 1.5% and stability 7.2%.
5. The Anthropic Economic Index. Anthropic, February 2025. Distribution of Claude usage by occupational category; computer and mathematical occupations the largest category (37.2% of queries).
6. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M. (2023). Controlled experiment with 95 professional programmers on a JavaScript HTTP-server task; the Copilot group finished 55.8% faster (95% CI 21–89%).
7. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR (Model Evaluation & Threat Research), 2025. RCT with 16 experienced contributors and 246 tasks: AI-assisted tasks took 19% longer, while the developers estimated they were 20% faster. METR labelled these results out of date in February 2026.
8. State of AI-assisted Software Development 2025. Google DORA, September 2025. Nearly 5,000 respondents; 90% use AI at work; AI as an amplifier of existing strengths and weaknesses.
9. Stack Overflow Developer Survey 2025 and 2026. Stack Overflow, July 2025 and October 2026.
10. Octoverse 2025. GitHub, October 2025.
11. We are changing our developer productivity experiment design (February 2026) and Time Horizon 1.1 (January 2026). METR.
12. The state of AI in 2026: On the road to ROI. McKinsey & Company, August 2026. 1,719 participants surveyed from May 4 to June 8, 2026.
13. Anthropic Economic Index, March 2026 report. Anthropic, March 2026.