More than 80% of AI projects fail. That figure is twice the already-high failure rate for IT projects that do not involve AI.
It can be easy to look at this as a technology problem, in things such as immature tools, unreliable models, or insufficient compute. However, the research shows that the actual reasons AI projects fail are overwhelmingly strategic, organisational and foundational. In most cases, it is often not that the technology doesn’t work, it is the organisational machinery surrounding it that can break down.
Understanding these root causes is a step for any organisation that has experienced the frustration of promising pilots that never reach production – the condition increasingly referred to as pilotitis or pilot purgatory.
Root Cause 1: Starting With Technology Instead of a Problem
The single most common reason AI projects fail is that they begin with a capability rather than a clearly defined business problem. RAND's researchers found that misunderstandings and miscommunications about the intent and purpose of a project are the most frequently cited reason for AI project failure.
The pattern is common – a new capability, whether it be generative AI, computer vision, or a forecasting technique, captures attention, resulting in a proof of concept being commissioned around the technology ("let's build an LLM application") rather than a business outcome ("let's reduce complaint resolution time by 40%"). The resulting pilot succeeds on its own terms – the demo is impressive – but when deployment questions arise, they become harder to answers. Who will use it? How does it integrate? What metric does it move? The pilot has proved the technology can do something, however it has not proved that something is worth doing at scale.
McKinsey's 2025 State of AI survey underscores the point from the other direction. High-performing organisations – the roughly 6% that attribute more than 5% of EBIT to AI – are more than three times as likely as others to say they intend to use AI for transformative business change, not incremental automation. They start with the outcome they want, then work backwards to the technology. In this scenario, before any technical work begins, the organisation must articulate the specific business outcome, the KPIs for success and the operational context. If those answers are unclear, the project is not ready to start.
Root Cause 2: Lack of Executive Sponsorship and Organisational Alignment
AI projects that remain within a single team, such as data science, IT or an innovation lab, rarely see success in scale. The reasons are structural in that scaling AI requires changes to workflows, processes, data access arrangements and sometimes organisational structures. Those changes require authority and influence that a single project team can struggle to have.
Among AI high performers, 48% of respondents strongly agree that senior leaders demonstrate ownership of and commitment to AI initiatives – actively sponsoring them, protecting budgets and role-modelling usage. Among all other organisations, the figure is just 16%. High performers are three times more likely to have leadership that is genuinely engaged, not just approving budgets from a distance.
Among the organisations that BCG classifies as "laggards" – the 60% that report minimal revenue and cost gains from AI – top management often delegates AI strategy to middle or lower management, fails to articulate a clear value ambition and can spread resources too thinly across disconnected initiatives. Some leaders move slowly because they worry about adverse impacts, while others are reassured by operational management that value will come with patience. The result is the same: AI initiatives drift without strategic direction, commercial accountability or the organisational authority to drive the changes that scaling requires.
The absence of executive sponsorship can result in cross-functional alignment stalling, data access requests languishing, budgets defaulting to annual cycles, and when the pilot is complete there is no champion with the mandate to take it into production.
This isn’t to say that the CEO needs to understand transformer architectures, rather AI at scale should be viewed as a business transformation initiative, instead of a technology project, and it requires the same quality of executive sponsorship that any significant transformation demands.
Root Cause 3: Underestimating Data Readiness
If executive sponsorship is the most visible gap, data readiness is the most consistently underestimated one. Many organisations assume their data is ready for AI because it supports reporting and analytics, however this is rarely the case for successful AI projects.
Gartner's research on AI-ready data found that 63% of organisations either do not have, or are unsure whether they have, the right data management practices to support AI. This has led to the prediction that through 2026, organisations will abandon 60% of AI projects unsupported by AI-ready data.
Cisco's 2024 AI Readiness Index, which assessed nearly 8,000 organisations across six readiness pillars, found a similar outcome. The data pillar showed some of the lowest readiness scores, with 80% of companies reporting inconsistencies or shortcomings in data pre-processing and cleaning for AI projects. Overall, AI readiness actually declined from 2023 to 2024, with only 13% of organisations classified as fully prepared – down from 14% the previous year.
The gap between data that supports dashboards and data that can train, validate and operate AI models can be vast. Traditional data management focuses on structured data, accuracy and completeness for reporting purposes. AI-ready data demands additional capabilities, such as the ability to process unstructured data at scale, feature engineering and storage, data lineage and provenance tracking, real-time integration across multiple sources and governance frameworks that balance accessibility with control. Research finds that a lack of necessary data to train effective AI models is the second most common root cause of project failure.
The cost of inadequate data foundations is not just that individual AI projects fail, it is that every AI project ends up becoming a bespoke data engineering exercise. Teams can spend months wrangling, cleaning and integrating data for a single use case, with none of that work reusable for the next initiative. This means that organisations can end up paying the full cost of data preparation every time, with no compounding benefit.
Organisations that succeed with AI at scale treat data readiness as a parallel workstream that accelerates AI delivery. They invest in shared data platforms, common data quality standards and governance frameworks that make data discoverable, accessible and trustworthy – not just for one project, but for every future initiative that builds on the same foundation.
Root Cause 4: No Production Pathway
Perhaps the most revealing pattern in AI failure is the pilot that "succeeds" technically but has no plan for what happens next. The model works, the accuracy metrics are strong and the demo is convincing. However, there is no infrastructure to deploy the model into a production environment, no monitoring to track performance over time, no integration with the systems where decisions are made and no change management plan for the people whose workflows will be affected.
This is what we see as the infrastructure gap – organisations lacking adequate systems to manage their data and deploy completed AI models, which increases the likelihood of project failure. RAND’s report emphasises that investing in data engineers and ML engineers can substantially reduce development time and increase deployment success.
The prototype-to-production gap is significant. A prototype might run in a notebook on a data scientist's laptop, using a static dataset, with manual steps throughout the pipeline. A production system must handle live data in all its variability, integrate with enterprise applications, scale under load, recover from failures, be monitored for performance degradation and be maintainable by teams who didn't build it.
Gartner's finding that only 48% of AI projects make it into production – with an average of eight months from prototype to deployment for those that do – reflects the size of this gap. With Cisco's AI Readiness Index also finding that infrastructure readiness showed the largest decline year-on-year, with only 21% of organisations reporting they have the necessary compute resources to meet current and future AI demands.
The solution is to design the production pathway alongside the model from initial stages. Every AI initiative should have an explicit answer to "how will this run in production?" before development begins – involving platform engineers and operations teams from the outset, building deployment pipelines and monitoring into the project plan, and budgeting for ongoing operational costs, not just development.
Root Cause 5: Treating AI as an IT Project
The final root cause is perhaps the most systemic: organisations that govern and resource AI initiatives as though they were traditional IT projects are more likely to fail.
Traditional IT projects – building an application, migrating a system, deploying a platform – operate on relatively predictable timelines with well-defined requirements. The scope can be fixed early, the deliverables are known, and a waterfall or phase-gate approach, while imperfect, can work.
However, AI is fundamentally different. Models need to be trained, evaluated, refined and retrained. The performance of a model depends on the quality and representativeness of the data, which is often only partially understood at the outset, whilst user feedback reshapes the solution and integration reveals new challenges. The iterative, experimental nature of AI development is the nature of the work.
When AI initiatives are forced into traditional project management frameworks – with fixed requirements, milestone-based funding and success criteria defined at the start – the scope becomes rigid, timelines slip, success is measured by delivery milestones rather than business outcomes and the completed model is handed to an operations team that was never involved in its development.
McKinsey's research found that having an agile delivery organisation is strongly correlated with achieving value from AI. AI initiatives need funding models that accommodate uncertainty, success criteria tied to business outcomes, cross-functional teams from the start and governance proportionate to risk.
Why These Causes Reinforce Each Other
In most organisations experiencing “pilotitis” or “pilot purgatory”, several of these root cases are often present simultaneously and they compound one another in ways that make each individual problem harder to solve.
- A technology-first approach leads to initiatives without clear business outcomes, which makes it harder to secure sustained executive sponsorship;
- Without executive sponsorship, data access and cross-functional alignment stall;
- Without good data, prototypes take longer and produce weaker results;
- Without production infrastructure, successful prototypes have nowhere to go;
- And without an appropriate delivery model, the entire cycle defaults to a project management approach that treats each initiative as a one-off, preventing the organisation from building the repeatable capabilities that would make each successive effort easier.
BCG's research describes that lagging organisations – the 60% reporting minimal value from AI – experiment too widely, spreading resources across scores of disconnected initiatives and automating processes in isolation, rather than focusing end-to-end on a few important workflows that can generate value and demonstrate what is possible. Each failed or stalled initiative erodes confidence, making the next round of investment harder to justify.
What Distinguishes Those Succeeding
The mirror image of these failure patterns is equally well-documented. The organisations that are capturing real value from AI – BCG's "future-built" 5%, McKinsey's high performers – do not have access to better technology, they have better organisational foundations.
Starting small:
They start with fewer, more carefully selected initiatives focused on clear business outcomes. BCG's earlier research found that AI leaders pursue roughly half as many opportunities as their less mature peers, but expect and achieve more than twice the ROI. They follow the "10-20-70 rule": allocating roughly 10% of resources to algorithms, 20% to technology and data and 70% to people and processes. The technology is the smallest part of the investment.
Redesigning workflows:
They redesign workflows rather than bolting AI onto existing processes. McKinsey found that 55% of AI high performers have fundamentally redesigned individual workflows to deploy AI – nearly three times the rate of other organisations. Of 25 attributes tested, workflow redesign had the single biggest effect on an organisation's ability to see EBIT impact from AI.
Investing in shared infrastructure:
They invest in shared infrastructure and reusable capabilities, so that each new AI initiative builds on what came before rather than starting from scratch.
Treating a strategic transformation:
They treat AI as a strategic transformation, not a technology procurement exercise – with active executive sponsorship, cross-functional accountability and governance frameworks that enable rather than obstruct.
The performance gap is not closing. BCG's 2025 research found that future-built companies achieve 1.7 times the revenue growth, 3.6 times the three-year total shareholder return and 1.6 times the EBIT margin of lagging organisations.
Moving Forward
In order to overcome these root causes, organisations require:
Outcome discipline: starting every initiative with a clearly defined business problem, measurable KPIs and a realistic assessment of feasibility before any technical work begins.
Executive commitment: not just approval, but active sponsorship that provides cross-functional authority, protects resources and holds the organisation accountable for outcomes rather than activity.
Data investment: treating data readiness as a strategic capability, not a project-by-project exercise, and building shared foundations that serve multiple AI initiatives.
Production-first thinking: designing the path to production alongside the model from day one, involving engineering and operations teams from the outset and budgeting for operational sustainability.
Appropriate delivery models: governing AI initiatives with frameworks that accommodate iteration, cross-functional collaboration and outcome-based success criteria.
