Platform engineering has become increasingly important to businesses as platforms have become more complex, including DevOps tools, APIs, and other components needed for effective software development. It’s a delicate balancing act as developers have asked for easier navigation within an organization’s platform.
According to Humanitec’s carte blanche , just five years ago, platform engineering was not something people talked about.
For the past decade, the concept of DevOps has been all people have thought about, ever since Werner Vogels remarked “you build it, you run it” at a launch of AWS in
This shift to DevOps has caused a dramatic shift in roles, where developers are now responsible for more aspects of an application’s lifecycle and delivery workflow, all while the industry has transitioned to more complex microservices such as Kubernetes, GitOps and Infrastructure as Code (IaC), the report added.
“Platform engineering emerged in response to the growing complexity of modern software architectures. Today, novice end users are often asked to manage a set of complicated arcane services,” said Paul Delory, analyst vice president at Gartner. “To help end users and reduce friction for the valuable work they do, companies Forward-thinking have begun building operating platforms that sit between the end user and the support services they rely on.”
Most successful engineering organizations have invested in building internal developer platforms (IDPs) resulting in better performance on all DORA metrics. The IDP is the sum of all the technology and tools that a team of platform engineers puts together to open one or more gold trails, according to Humanitec.
Gartner predicts that by 2026, the majority of software engineering firms (80%) will have established platform teams to provide mutually accessible services, components, and tools for application delivery. This will ultimately address the key challenge of collaboration between software developers and operators.
Lack of structural discipline is causing a lot of platform problems today
The default assumptions and behaviors around what came before platforms were basically very bureaucratic or consultative, but not in a good way, according to Charles Betz, vice president and director of research at Forrester.
“These groups have always said, well, if you need a computer or a virtual machine or some computing resources or a database, you really need to have a server engineer and a database administrator there to make sure you don’t get into trouble. We’ll be intimately involved in your technical project,” Betz explained. access to any of the resources you need until we approve. And that’s what makes developers really hate classic, traditional IT.”
However, Betz said he also sympathizes with the IT side of things because IT professionals have typically been overworked. They are often subject to unreasonable demands and there have been stories of people developing systems with horrendous architectures and then insisting that the infrastructure group take ownership of production.
“The reality is that the long lag periods imposed by the classic infrastructure group are simply unacceptable in the modern Agile world,” said Betz.
He added that people like to point to companies like Netflix to try and mimic their Agile way, but those are the companies that are highly disciplined from a platform engineering perspective.
“Netflix didn’t have autonomous product teams at a low granular level who willy-nilly choose which products to use. This is one of the myths that we often have to dispel when addressing a customer. People say, well, all product teams insist that they are self-contained and can do whatever they want. And they mean Agile and various things. I’m like, you know, that’s actually not how it happened,” Betz said.
Successful firms have imposed architectural discipline even though they may not have called it that. But the bottom line is that they haven’t tolerated a lot of variability and sprawl and that’s how they’ve managed to be successful, according to Betz.
Humanitec’s 2023 DevOps benchmark study found that assigning DevOps tasks to developers as a way to implement self-service is often performed poorly in many organizations. An IDP can enable developers to have greater self-service capabilities with the ability to create environments, deploy, roll back, and make architectural changes without relying on Ops.
It is important, however, to keep the relationship between Ops and developers close by maintaining a separation of concerns. This way, both teams can work together while having distinct roles.
The Humanitec report indicated that successful teams manage their app configurations across the organization in a standardized way. They also manage app configurations and infrastructure dependencies the same way, knowing how to differentiate between environment-specific and environment-independent configurations. These teams are more efficient at building new environments and are able to provide more self-service when it comes to deployments, infrastructure provisioning, and infrastructure assignment.
The path forward for platform engineering is not about modifying engineering
Overall, platform engineering is now undergoing a lot of experimentation and there is not yet an agreed best practice. However, according to Forrester’s Betz, the biggest challenges for platform engineering aren’t always the “engineering” part.
“When I went to the DevOps Enterprise Summit last fall, I talked to as many people as possible who were having success in platform engineering, and the one thing they all had in common was that they were looking for a way to bring the principles of product management to platform engineering,” said Betz.
The platforms are products, according to the authors of Team Topologies, which suggest that the principles of organizing, self-correcting, monitoring the environment and setting standards are applied to the team.
The challenge is that platform teams are now being built from existing and outdated IT and operations organizations. These organizations are better suited for engineering purposes, but aren’t necessarily well equipped for the other duties of a platform team.
The problems are not in feasibility, which continues to require high levels of engineering excellence, but rather in value addition, feasibility, and usability, according to Betz’s article, “ Platform Product Management Versus Platform Engineering .”
This platform model encourages the pursuit of automation where possible and queue management elsewhere, making internal offerings have benefits in terms of access to capital, reduction of transactional friction and maintaining high quality service. Organizations should also implement explicit service design thinking where the Employee Net Promoter Score (eNPS) is tracked and customer journeys are understood, according to the article.
“To build a platform that is worthwhile, you need to understand who the real people you are building this platform for and make sure you are talking to them on a regular basis, iterating with them as you would any product, product development and product backlog in the world of application development “said Daniel Betts, senior director research analyst at Gartner. “You’re treating it similarly, you have to build a platform like an agile product.”
Most of the teams that the products are for are software developers, application developers, infrastructure engineers, and somebody who’s building code, building assets for the business.
“These people want to be able to build software or applications to deploy on a platform. They don’t want to have to think about tools, technology, governance, change management, all of those things. They want it all to be made available to them and they want to focus on writing the code,” Betts explained.
Platform teams often consist of one or two software engineers, as they add experience to the benefits of writing machine-controlled code, code reviews, automated processes, and reusable components. They also help teach scripting and coding best practices. Additionally, SREs can be part of platform teams, as it is critical to a successful product, Gartner’s Betts added.
Also, according to Forrester’s Betz, organizations need to develop platform engineering talent internally because there’s a skills gap and they can’t find someone in the market who knows exactly how their platform works.
“You have to do things like take someone who is a good technical engineer and you have to rotate them a bit on issues that are a bit less technical like developer experience, product management, and the products have to be valuable, viable, usable and workable”, concluded Betz.”We need to see more platform product managers.”
