In some companies, the inevitable rapidly became accepted as the way to do things, and both development and IT operations worked together to figure out how to collaborate on building systems that satisfied development’s desire for change, and operations desire for stability. Outsourcing infrastructure, and all it implied, gave rise to Devops – the unification of business needs, developer delivery, and operational capacity – but it also gave rise to something else, in companies where the operations teams weren’t quite as quick to move – Shadow IT.
Developers, long constrained in what they could request, or in the time it took to deliver on those requests, took to IAAS life it was lifesaving medicine. No more constraints! No more restrictions! Nobody telling them no! Development managers, who had been under pressure from the business to deliver no longer had to be trapped by people telling them they couldn’t. A balance which had been in place for many years was disrupted nearly overnight. Developers knew it immediately. Some IT departments still don’t know it. Any time the business tells developers to deliver something, and IT Operations tells them they can’t, shadow IT comes into play. Not because of technology, but because of people – there’s a dissonance caused by being caught between two opposing forces. When a way out presents itself, it will usually be taken. That’s human nature. Understanding that is essential to understanding that the change that has come to IT isn’t going away; it is, in fact, going to continue to change.
As the CIO (or IT Director, IT Operations Manager, etc), if the business and IT are in conflict, with IT Operations saying no, then you either already have Shadow IT or will have it soon. Someone, somewhere in your development team, is considering how to use cloud technology to make deployments faster, make infrastructure more stable, and reduce their own headaches.
From the perspective of the business, shadow IT is both a blessing and a curse. It’s a blessing because, as mentioned above, it removes the constraints that have stopped or inhibited developers from delivering business requirements. Removing these fetters helps increase delivery speed, and capacity. The downside is twofold: 1) Shadow IT is hard to track. If IT costs belong to IT Operations (as is frequently the case), shadow IT is spend against their budget that they can’t account for. This may result in budget overruns that are difficult to explain or account for. It may also result in IT Operations purchasing equipment that never gets used, as the team is bypassed. 2) IT Operations are the part of the organisation most familiar with risks of running equipment in a live environment. They are frequently responsible for security and scalability. While some of those risks go away with cloud infrastructure (e.g. auto-scaling), not everything does, and new problems are introduced (e.g. how to handle logs for services that might disappear at any time?). These are questions that IT Operations are used to asking, and answering. While there’s nothing inherently less capable with developers, they are used to thinking of different questions, in different domains, and may not have the domain knowledge required to think of the questions themselves. For these reasons, it’s important to address Shadow IT early, and figure out how to integrate external IT systems with internal IT and business requirements.
Use of cloud technologies isn’t going away. That genie is out of the bottle. So if there’s no way to eliminate the IT, what’s the solution? Eliminate the shadow.
Since it is only going to get easier to create customised infrastructure as companies gain experience with providing what customers want, the only feasible solution is to adapt. Bring shadow IT out of the shadows. Not by forbidding it, but by embracing it. Take time to understand what it is that shadow IT provides that internal IT doesn’t. Then make a decision – is it worth spending time and energy adapting internal IT systems to provide those same service, or would that time be better spent adapting internal IT people to the new way of working. Neither is easy, but the latter, while difficult, is far more likely to succeed – people will use what’s easy and gives them what they want, after all; once they’ve settled on external IT providers, it would be very hard to get them to go back to using internal systems.
The first step in adapting IT people to using external systems is to help them understand that their role in the organisation is changing. They are no longer gatekeepers, responsible for saying now. Instead, their role is to help enable the use of the best tool for the job, bringing their specialised knowledge of systems to bear on cloud-based systems, figuring out how to adapt inherently unstable cloud infrastructure to software and customer needs for stability (hint: automated recovery and scaling). One of the biggest challenges that comes with thinking this way is that a lot of Sys Admins are used to spending their time on the command line, and creating repeatable, redundant, scalable infrastructure requires moving toward infrastructure as code (terraform, ansible, mesos, etc).
As with most new ways of working, the difficulty isn’t with the new technology itself, but in people adjusting to a new way of working that in many cases challenges their deep assumptions about what they do, and how they provide value. This change takes time, and may take new people. Don’t delay starting, however – the longer you wait to get started, the more ground there will be to make up, and the harder those same people will find it.
*Shadow IT is IT spend that isn’t sanctioned by IT Operations, and so isn’t well tracked, or admitted to.