A survey of 101 senior engineering leaders published this week finds while 87% believe their organization is “prepared” or “very prepared” to adopt artificial intelligence (AI). It’s also clear that several hurdles remain, most notably making sure application development teams have the quality assurance skill and expertise needed to validate AI outputs (66%).

Conducted by Uplevel, a provider of a platform for tracking engineering metrics, the survey also finds engineering leaders are starting to focus more on performance monitoring and optimization (39%) and system architecture and integration skills (34%).

Additionally, the survey identifies technical debt (27%) and lack of clear AI strategy (22%) as the two greatest strategic threats to AI potential.

Amy Carrillo Cotten, director of client transformation at Uplevel, said the survey results suggest that while there is a lot of AI optimism, engineering leaders are also becoming more cognizant of some of the inherent challenges. For example, 30% cited data security and privacy risks as a significant issue that will need to be addressed.

Overall, engineering leaders are investing in AI to drive operational efficiency (53%), accelerate innovation (40%), improve decision-making (28%) and boost competitive advantage (23%).

Engineering leaders to achieve those goals are investing in reskilling existing employees (40%), hiring AI specialists (34%) and partnering with vendors or consultants (22%) to address those issues, the survey finds.

The challenge is, as always, determining what level of productivity gains are being made, said Cotten. While two thirds of engineering leaders (66%) say they regularly measure the business outcomes of their work their teams complete, 37% also conceded they have difficulty isolating how specific teams contributed to the effort. While the survey finds 61% of respondents prioritize key performance indicators (KPIs) are team or organizational level, many are also still tracking metrics such as how much code was produced by individual developers even though that metric has little to do with an actual business outcome, noted Cotten.

Engineering leaders will also need to have a better appreciation for systemic constraints such as complex architectures and collaboration challenges that will only become further exacerbated as the volume of code being created increases exponentially, said Cotten. In effect, many application development teams will discover they are running into the same brickwalls, only faster, she added.

In general, the survey identifies improving development tools and automation (27%), followed closely by reducing technical debt (24%) as the two things that would have the biggest impact on productivity. Obviously, AI is improving tooling but it’s not fairy dust that can be sprinkled everywhere, said Cotten. Engineering leaders need to have a strategy for operationalizing AI across the software development lifecycle (SDLC) to ensure success, she noted.

Ultimately, software development remains a team sport and no amount of wishful AI thinking is going to magically eliminate existing bottlenecks and constraints, added Cotten.

It’s not clear to what extent DevOps teams are adopting AI but there is no going back. The issue now is determining how best to set the right expectations for what can be actually achieved despite the current level of hype being created by exaggerated claims about how AI is eliminating the need for application developers.