Microsoft describes its own DevOps journey

 This is what DevOps looks like inside one of the world's biggest software factories. The goals: more collaboration, improved worklives.

n this cloud-drenched era, software vendors have evolved from code-and-ship operations into 24x7 data center operators. So they, perhaps more than any other type of organization, need DevOps methodologies to keep development and operations efforts on track, pumping out releases with blinding frequency, while keeping teams in constant sync.

For that reason, Microsoft, perhaps the world's largest software factory, takes DevOps very seriously. In a recent post, Ori Zohar, senior product marketing manager for Microsoft Azure, describes the urgency for his companies to build and perfect its DevOps culture. "From Office, to Azure, to Xbox we also found we needed to adapt to a new way of delivering software," he explains. "The new era of the cloud unlocks tremendous potential for innovation to meet our customers' growing demand for richer and better experiences -- while our competition is not slowing down. The need to accelerate innovation and to transform how we work is real and urgent." 

Much of Microsoft's DevOps efforts have been overseen by the Microsoft One Engineering System (1ES) team, a group of about 200 people who work with engineering teams across Microsoft's sprawling product lines. The 1ES team focuses on tools, processes, program offices (such as open source contributions), accessibility, security and compliance, internal consulting, "inner source (sharing source code within the organization)," and amplifying best practices across engineering teams.

The 1ES team, first formed in 2014, has been overseeing the company's Azure DevOps initiatives and proliferating the best practices developed and learned to more than 50,000 Microsoft employees. The team achieved some impressive results, not the least of which was the ability to roll out updates to its primary application suites in a matter of hours. In its public summary of DevOps activities, Driving engineering culture change at Microsoft: An experimental journey, the team reports the following outcomes:

"Source control issues decreased, build times and build reliability improved, security and compliance efforts were standardized, and the majority of engineering teams were managing their work the same way. Even the vast Windows engineering team, with its thousands of users, millions of work items, and 300 GB of source code, moved from 40+ Source Depot servers to a single Git repo hosted under a single account. And build times for Microsoft Office went from days to hours."

Collaboration and improving worklives are the key. The 1ES team has overseen a series of initiatives to move DevOps forward across Microsoft's varied development and operations organizations. In one effort, a team of engineers and program managers were embedded "within one or two strategically chosen product teams each year, as a means of reducing technical blockers to high performance." 

Another initiative launched by the team seeks to improve the developer experience by focusing on "improving the day-to-day work experience of individual engineers, with the goal of having the tools they use fade into the background."  "Working to make internal projects more like open-source, where anyone can discover the project, ask for help, or submit potential fixes-as a means of tapping into the potential of engineering talent across Microsoft as a whole and building better products through shared ownership and purpose."

In the process 1ES learned a number of key lessons about building and supporting a DevOps culture across a huge, sprawling software organization, team members report:

Take the end-user's perspective.  "We had to think from the perspective of why our internal customers would want to change-and, just as important, believe that change is possible," according to Cindy Alvarez, principal PM manager on the 1ES team. "People tend to view such initiatives as 'I've seen this before... If I drag my feet, maybe it'll just go away.' Of course, you'll only hear this in the hallway. Nobody's going to say it to a director in an all-hands meeting." 

Start small, and aim for quick wins. "Our original mindset was that, if we got the largest teams doing what we wanted, then the other, smaller teams would follow," says Alvarez. "But those largest teams, like our Windows team, often pose unique challenges and have special needs. You can't 'jump to scale' because it's impossible to stand at one level within the organization and predict what will work for all. It's not about size... It's about focusing on quick wins and substantial impact over rolling out substantial change to as many as possible, all at once."