Platform Engineering and Security: A Very Short Introduction

I'm a DevOps engineer. I hope that's still a thing because, according to some, "DevOps is dead; long live platform engineering."

There is no denying that recently we started to see terms like "platform engineering," "developer portal," or even "Backstage" a lot more often than before.

This article will answer a few essential questions: is DevOps really dead? What is platform engineering? What are the differences? How does platform engineering handle DevSecOps/security?

Is DevOps Dead?

Short answer, no.

If I learned anything over the years in the tech world, it's the fact that technologies, frameworks, and methodologies never really die. They grow, they adapt, and they evolve.

For example, think agile development. What's the first time you heard of that? Is it dead now? That's the point.

But, in a way, DevOps did "die." Although it didn't die in the conventional sense, where it simply disappeared from the tech community, it did "die" in a way that many companies and teams trying to utilize DevOps failed to meet their expectations.

This is not news. Back in 2019, a data analyst from Gartner, a technological research and consulting firm, predicted that by 2022, 75% of DevOps initiatives would fail to meet expectations, and this number might increase to 90% in 2023.

It's worth pointing out that those DevOps initiatives failed not because of technological challenges but because of overlooked human factors like trust, ownership, and teamwork.

Initially, DevOps was touted as the ultimate solution to all issues within traditional operations silos, promising to revolutionize the industry. Still, many teams have unfortunately fallen short of their original expectations. Worse, it seems that, in the end, developers don't actually want to do Ops work!

Devs Don't Want to Do Ops?

To be fair, it depends on who you ask. Some developers think the DevOps "you build it, you run it" paradigm is definitely beneficial, even sometimes necessary; others don't want to touch operations at all.

Take, for example, this poll:

Devs… be real with me. Do y’all want to do Ops? 🧵

— Luca (@luca_cloud) August 24, 2022

The responses highlight the divide, with 41.8% of respondents saying yes to operations tasks, 42.1% saying no, and 16.1% being indifferent.

As it turns out, developers tend to steer clear of infrastructure and operations, which is not very positive for the success of DevOps.

The consequence is that platform engineering has become increasingly popular in response to this trend.

Platform Engineering

Platform engineering, just like DevOps, emerged in response to the increasing complexity of modern software architecture.

In today's world, non-expert end users, such as developers, are frequently tasked with operating a complex array of services that can be incredibly challenging to manage.

While DevOps tries to solve the collaboration and velocity problem of the software development lifecycle (SDLC) from a cultural standpoint—where shared responsibility, continuous growth, and learning mindset are valued, teamwork, best practices, and modern toolchains encouraged—platform engineering tries to approach the problem from a different angle: that of self-service.

Let's have a look at a concrete example:

Consider a scenario where a team of developers requires a relational database (RDS) for their new application, but they lack the necessary knowledge to create one in a specific cloud provider, such as AWS, using the appropriate automation tool, such as Terraform. Not all developers are familiar with the infrastructure and orchestration of their systems.

The DevOps approach to this problem involves creating a cross-functional team that includes an expert in AWS services and infrastructure as code who possesses the proper permissions to provision the cloud resources automagically by writing Terraform modules and deploying the RDS in AWS.

On the other hand, in the platform engineering approach, there should be an integrated product that serves as the "internal developer portal", so that the developers request an RDS in AWS through self-service, and the platform will automatically create it.

Note that this is only one example; internal developer portals can achieve way more if appropriately crafted.

In short, platform engineering involves designing and constructing an integrated product that includes toolchains and workflows to address the operational requirements of the entire SDLC. This approach enables developers to access the appropriate level of self-service and abstraction that suits their organization or team in the cloud-native era.

Platform Engineering vs DevOps

Although these self-service capabilities improve the developer experience and productivity, it's not for everyone, especially not for smallish teams/companies: even if you decide to purchase some internal developer portal-as-a-service instead of building it from scratch, you still need to maintain automation and templates behind the scene: think of the previous RDS-in-the-cloud example, where you need to have the knowledge to maintain Terraform templates to generate the module.

If a team or company is too small for platform engineering, DevOps may be a more effective approach. In smaller environments, collaboration tends to be more intimate, the pace is more agile, and there is typically less organizational friction.

It is important to note that platform engineering does not serve as a replacement for DevOps. It does not emerge to displace or replace DevOps. Although many say that DevOps is dead with the rise of platform engineering, it is not a situation where one will replace the other: the two approaches can come together.

Platform engineering can be seen as an evolution of DevOps, as it shares the same objective and can enhance the effectiveness of DevOps.

Likely, we will still see DevOps culture a lot, and it's also likely that many software engineering organizations will have established platform teams to help bring together software developers and IT operations.

Where Does Security Fit in this New Paradigm?

We can't talk about DevOps without talking about security (or DevSecOps); after all, security is job zero.

DevOps tackles security by "shifting left." In contrast to traditional software development methods, where security-related tasks were only addressed at the end of the SDLC, which could lead to project delays and was not scalable, DevOps/DevSecOps incorporates security at every stage of the SDLC as early as possible, using automation tools. This approach is known as "baking in" security.

To illustrate, when writing code, developers consider potential security concerns, such as securely storing and retrieving secrets and credentials, rather than waiting for a final audit or review before pushing to production. Additionally, when creating virtual machine or container images, vulnerability scans are automatically conducted during each build, and any vulnerabilities found are immediately addressed, rather than conducting a one-time scan before shipping to production.

This is "shift left."

All these methods of dealing with securities in a more agile, responsive, and automated "shifting-left" manner are still applicable to platform engineering; there is no contradiction here regarding DevOps and platform engineering.

With the assistance of platform engineering, internal developer portals can automate the entire process to a greater extent. Consider three concrete examples:

Firstly, imagine creating a new microservice. Instead of writing boilerplate code and manually assembling CI/CD pipelines, Dockerfiles, and Kubernetes manifests, the internal developer portal can bootstrap your entire repository using existing templates. These templates include scaffolding code for the app, as well as pre-configured pipelines that follow DevSecOps best practices. They also scan for hard-coded secrets in the source code and lint YAMLs, among other things.

Secondly, imagine needing to store secrets. Instead of searching for the secrets manager your team is using and determining whether you have permission to create secrets, you can go to the internal developer portal, which integrates with your secrets manager. Here, you can create secrets in a self-service manner without worrying about the underlying details, thanks to the appropriate level of abstraction provided by platform engineering.

Finally, imagine launching a new virtual machine in the cloud. Instead of writing automation code from scratch or copying and pasting from the existing codebase and reviewing for security groups settings before deploying, what if you can simply go to your internal developer portal, click a button, and everything is done using a template maintained and secured by the platform engineering team?

All these are the beauties of platform engineering:
- Self-service
- The right amount of abstraction
- Baked-in best practices and automation

Summary

I hope this introduction to platform engineering has sparked your interest! You should now have a better understanding of why it emerged and its core proposition value for developers, as well as the differences with DevOps and how security is addressed with this new paradigm. If you are interested in platform engineering, please subscribe, as we will be working on some tutorials to help you build your own developer portal from scratch.

Stay tuned!