“Full Stack” means more than Frontend:
Why every engineer needs to be a DevOps engineer

Artist: Grafana in collaboration with Prometheus, Year: 2019

I am a developer at Ravelin. For different companies and different people that tends to mean different things. Some folks are heavily specialised, others are more generalists and there is a full spectrum in between those points. Getting the balance of that right can be tricky for people and tricky for companies. “Full Stack Engineers” seem to be everywhere these days, for good or ill. Colloquially, this is understood to be someone who is comfortable in both frontend and backend development. But that isn’t the only bridge of specialization. Not by a long shot. I don’t even think it is the most interesting or useful one.

For me, the front lines of this live in DevOps.

DevOps is moving at a rapid pace

DevOps has evolved dramatically in recent times. It has had to. Scale, rapid pace of deployment, uptime, flexibility, cost, hiring. Expectations in all these areas have grown rapidly and outpaced more traditional practices. You can no longer say “maintenance window” with a straight face. Provisioning more compute needs to take minutes (seconds even) not days. Deployments need to happen continuously, not on scheduled monthly cycles.

But you already know this. The cloud isn’t new anymore. Infrastructure as code is a well understood idea. Even containerization is losing its excitement and just becoming the norm.

Not to say that DevOps isn’t exciting anymore. Kubernetes is still exciting. Service Meshes are still evolving. Distributed systems will always be a hot topic. Scaling anything is always interesting. My point is we are changing so fast that the forefront of this field becomes the norm within just a few years.

If you have a ZDNet Article in your Google results, you probably aren’t the most exciting technology.

The ramifications of these changes for developers, and by that I mean the wide spectrum of developers including DevOps, takes longer and can be hard to predict.

Changing Tech, Changing Roles

At Ravelin, we have few dedicated DevOps people. We have had one for the life of the company — only recently bringing on another. I think not many years ago this would be crazy for a company that operates with thousands of requests per second on tens of terabytes of data distributed over multiple data centers.

We were able to do this in the ways you would expect. Using managed services when we can, leveraging cloud technology and using tools such as Terraform, Kubernetes, Helm, Chef, Jenkins, etc.

As a result of all this tooling, managing complex infrastructure is significantly easier than it used to be. Much of the work is given off to our cloud provider. This means it is now within reach for a developer to take on more infrastructure responsibility. If they need a database, they can deploy that themselves via Terraform and a managed service. If they need more compute they can adjust the Helm charts accordingly to deploy their container into Kubernetes with more compute.

The role of a DevOps engineer is to lay the foundation such that developers are doing the right thing from a maintainability perspective and a security perspective. The job is primarily to provide the tooling (Terraform pipelines, templating, examples, documentation, etc) to empower the developers to manage their own deployment and operations.

This has many benefits:

Many more developers are familiar with the infrastructure as more have gotten their hands dirty building with it. More folks are more able to tackle production issues and are comfortable making changes to Terraform files if need be. Or digging in to the Kubernetes manifests. This is a boon for debugging operational issues as well as brainstorming larger infrastructure changes.

Errors relating to an aberrant traffic pattern from a client. They were extremely brief and could have easily been ignored but by tweaking our scaling policy via a Helm chart we were able to mitigate the issue. This operational issue was tackled by a backend developer and could have been tackled by just about any engineer on the team.

It forces developers to think more about operations and deployment. You don’t want to foster this idea that you write some code and just throw it over the fence to the DevOps engineers (or SRE)who have to struggle to keep your database running under the blatant antipattern you’ve accidentally invoked. It keeps developers thinking, from the start of a project, How am I going to deploy and run this? Instead of thinking that at the end of a project after it is too late.

It can help mitigate the issue of using the tool you have instead of the best tool for the problem. Developers are more aware of what is possible within the infrastructure so are more able to push to include a new cache or database or queue if they think it is appropriate to solve the problem. A huge category of production issues often arise from trying to put a square peg in a round hole.

Some things are still complex

Now of course this doesn’t work for everything. Larger deployments that are not on a managed service still take up too much time for a developer to work on. Things like Prometheus and ElasticSearch are examples that come to mind for us. These are still primarily run by our DevOps engineers. Even still, other engineers readily lend a hand as the tooling has become familiar to them.

In my experience, working on deployments and operations is one of the fastest ways to make you a better engineer. Understanding how software is deployed, how it is run, what are the limitations, how it can go wrong and how it can go right is critical to writing better, maintainable software. The more you can engage your development team into this side of the job the better. In the past this was too complex. It was hard to just dip a toe into the water. Now, the tooling and cloud support have gotten to a place where developers are empowered to engage more actively on this side of the DevOps fence. And we are writing better software because of it.