A Google-sponsored DevOps survey shows a widening gulf between the best and worst performing teams, and that the much-hyped AI has yet to deliver substantial benefits to development teams.

The Accelerate State of DevOps 2023 report is based on data from nearly 3,000 professionals, a 3.6x increase from last year, with 30 percent working on development or engineering, 18 percent on DevOps or SRE (Site Reliability Engineering), 16 percent managers and 8 percent operations or infrastructure, with the remainder occupying a variety of other roles. The USA provided 28 percent of the responses, with the next biggest groups being UK, India, Canada and Germany.

The aim of the research is to understand the relationship between ways of working and outcomes, with some effort to measure software delivery performance, organizational success, operational performance, team performance, and employee well-being. Software delivery, for example, is measured by deployment frequency, lead time for changes (how long between commit and deployment), how frequently a change causes a failure, and how long it takes to recover from a failed deployment.

This year’s report identified an elite tier of software teams, which last year had been hard to discern. 18 percent of respondents fall into this elite category with metrics such as a change lead time of under one day, and recovery time of less than one hour. At the other end of the scale, low performing teams that between one week and one month to deploy changes, and suffer a 64 percent change failure rate.

The key factors behind high-performing teams are not the tools, cloud providers or programming languages they use, but rather things such as building software with the end user in mind, and having a low-blame, high-trust culture. A user-centric approach does not mean shipping lots of features; in fact the reverse is true. “A relentless focus on shipping might distract the team from meeting users’ needs,” the report suggests. 

One notable insight is that rapid code reviews correlate with substantial increases in software delivery performance and operational performance. Another is that loosely coupled software architecture makes for better performing teams and more successful organizations.

Documentation is also highlighted as having a strong impact. “We found that quality documentation has a positive impact on an individual’s well-being: it reduces burnout, increases job satisfaction, and increases productivity. We found that some of this impact is because quality documentation increases knowledge sharing,” the researchers observe.

What about the effect on productivity of much-hyped AI? The researchers found “a lot of enthusiasm about the potential of AI development tools,” but it is not yet a key factor in outcomes. “Our evidence suggests that AI slightly improves individual well-being measures (such as burnout and job satisfaction) but has a neutral or perhaps negative effect on group-level outcomes (such as team performance and software delivery performance),” the report finds. The researchers think that this might be because it is at an early stage; but this measured report is at odds with claims by the likes of GitHub concerning the transformational benefits of AI-infused developer workflows.

Another important finding is that cloud adoption is not always beneficial. “We see strong indicators that public cloud leads to decreased software and operational performance unless teams make use of flexible infrastructure. This finding further promotes the idea that simply “lifting and shifting” (the act of shifting workloads from a data center to the cloud) is not beneficial and can be detrimental,” the researchers report.

In some respects the detail of the report is less important than its insight, formed over many years of this research, that organizational and team culture have the most influence on successful outcomes. “A healthy culture leads to high levels of employee well- being by reducing burnout, increasing job satisfaction, and increasing productivity,” the researchers state, adding that “employee well-being is not a nice-to-have: it’s foundational to an organization’s overall health and success.” It is easy to see these patterns in the research data; but unfortunate that organizational culture is among the hardest things to change.