Council Post: Maintaining Central Control Over AI Is Essential
Dom is an incisive business leader and creative product strategist with 30-plus years of experience in the cloud and data center industries.gettyI'm seeing a trend in both North America and Europe that is drive...
Dom is an incisive business leader and creative product strategist with 30-plus years of experience in the cloud and data center industries.
getty
I'm seeing a trend in both North America and Europe that is driven by the need to show quick results for an AI project. It starts with an edict from top management: "What are we doing to take advantage of AI, and what results do we have to show?"
That sets fire to one-off AI initiatives where teams identify a problem, secure a budget and acquire GPUs, cloud credits or specialized platforms for AI infrastructure required for the project. The intent is good. The result is not.
Significant money is being spent that central IT too often doesn’t see, can’t manage and certainly can’t optimize. What emerges is a fragmented AI technology ecosystem in the organization, and as AI infrastructure spending accelerates, this fragmentation is quickly becoming one of the largest hidden costs alongside security risk in enterprise technology.
Shadow AI: The New Shadow IT, But More Expensive
Traditional shadow IT revolved around circumventing on-premises computing resources by purchasing cloud computing time or bypassing software licensing with SaaS subscriptions. Shadow AI sacrifices governance and security for expediency.
IT platform teams can tell you exactly how many servers they manage, where those are deployed and how they are utilized. But when asked how many GPUs exist across the organization, the answer tends to be unsettlingly vague.
Too often, AI technology infrastructure is purchased at the project level, and those GPUs remain tied to a single project, even when utilization drops or the project ends. Unlike cloud resources, GPUs don’t automatically return to a shared pool. They become very expensive stranded assets.
How Bottom-Up AI Can Slow Enterprise Progress
This fragmentation creates three systemic problems.
First, utilization can suffer. AI workloads are bursty by nature. Without centralized scheduling and visibility, expensive accelerators sit idle while other teams request more capacity.
Second, operational complexity increases. Every project team must stand up its own DevOps, security and monitoring processes while adding its own IT people. Instead of scaling expertise in the central IT organization, duplicates are created across business units.
Third, and arguably most critically, organizational power shifts. As AI budgets grow, parallel AI infrastructure teams risk forming outside of central IT. Over time, the group managing the most expensive resources gains influence, often at the expense of the central IT platform teams responsible for the broader compute estate.
This is how organizations risk losing architectural coherence—not through a single decision, but through multiple well-intentioned ones.
By comparison, cloud and AI service providers don’t operate this way. They define AI strategy centrally because AI infrastructure is their revenue engine. Top-down alignment allows them to drive standardization, efficiency and rapid iteration. Enterprises, by contrast, often view AI as a cost center rather than a core platform. That mindset can delay centralization until fragmentation becomes impossible to ignore.
Interestingly, I've observed a different pattern in parts of Asia. In some organizations, I've noticed senior leadership establishing the AI strategy early, identifying shared AI platforms and coordinating infrastructure investments across business units. That doesn't eliminate experimentation, but it can reduce duplication, improve infrastructure utilization and accelerate organizational learning.
Why Central Control Is Not About Slowing Innovation
Reasserting central control over AI doesn’t mean blocking experimentation. It means enabling it responsibly. Platform teams should aim to deliver the following:
• Visibility: Knowing where AI infrastructure exists and how it’s being used
• Pooling: Allowing GPUs to be shared across projects instead of being locked to them
• Governance: Enforcing security, compliance and cost controls without friction
• Chargeback: Making consumption visible so teams understand true costs
The Real Risk
I don't believe the greatest risk is spending too much on AI. It’s letting AI infrastructure grow outside of enterprise control until consolidation becomes politically and technically painful. I've found that once a parallel AI organization forms governing technology, central IT doesn’t get invited back in. At that point, the conversation shifts from optimization to survival.
The better path is to treat AI infrastructure like any other core technology platform—visible, shared and governed—before fragmentation hardens into structure.
Organizations that take a centralized approach to AI technology from the outset are the ones that I believe will move faster and learn faster while retaining control over spending for one of the most strategic assets they will ever deploy.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?