Tales from the Support Desk: Automating Our Own Jobs Away? By Lily She-Yin

TL;DR – No, Not Really

No Job Too Small, We’ll Do It All

There’s a joke about a young engineer at their very first job, in a milk bottling factory. The supervisor has a task for them, and explains: “You see this conveyor belt here? Every now and again it will jerk to a stop, and all the bottles on it are sent flying – it makes a huge mess.” Excited, the new engineer says “Ah, and my first job is to fix it?” The supervisor replies “No, your job is to stand here and catch the bottles before they hit the ground.”

Support: it’s a reactive, manual process. Often the same problem will crop up again and again. But here, we aren’t able to dive in and fix it at the source – we’re not involved with the code, the app, the infrastructure; we’re just here so that the devs have a little less to deal with. So… we stand there and catch the milk bottles.

DevOps is a Frame of Mind

That was perhaps an exaggeration for the purposes of entertainment; nonetheless, a portion of the alerts we are reacting to on a day-to-day basis are indeed quite mundane. There’s a system, rules to follow – log the incident, pick up the bottle, confirm the bottle is back on the belt, close the incident. No part of the process from the moment of receiving an alert is particularly built for automation. It is built so that a human with sufficient knowledge can pick up the alert, figure out what needs doing, and do it. Often the task itself does not require too much thought, and the main impact on we the engineers is the disruption of other work: the abrupt and frequent context-switching.

But the more of these routine alerts we resolve, the more we understand – there is a definitive logic we can follow, circuitous though it may be. And if that is the case, couldn’t that logic be handled by a script of some sort? Long story short: it could. We were able to create utility scripts that would make database calls, retrieve and process information, and poke various endpoints to set a bottle back on it’s proper path (…when will this analogy end?). But with this minor power, came great responsibility. It wasn’t enough to automate the tedious parts of the task: we had to remove the possibility of accidentally misusing the script or making a mistake – after all, real bottles were on the line (ok, that’s it. I’m done). And so began the task of making the scripts foolproof.

Now, was it really worth the time and effort to automate these tasks? (Relevant XKCD). Well… eventually, yes. The thing about people is, we make mistakes – especially when switching between many different tasks as we do, supporting many different services. With careful planning, and good coding practices, we’re reducing human error and improving our day-to-day experience through minor acts of automation.

This isn’t an exciting DevOps transformation project. We’re not here to evangelise about our lord and saviour, Agile. Some of the support services we take on can feel bafflingly pointless (a machine creating a chain reaction of work for four different teams in order to confirm that it was a false alarm?!). And yet there is so much room here for personal improvement, to foster good Agile practices, to problem-solve. To learn how to communicate to get the best outcomes, and importantly, to learn how to compromise. Through automation we free up more opportunities to explore and grow.

We Do What We Must Because We Can

Throughout the course of this project, I’ve had to remind myself of one question: what is our purpose here? …To do the grunt-work of other teams? Not so. The point of us as consultants is as always: to leave the project in a better state than when we arrived; to improve processes and build solutions. That doesn’t change when our tickets are to fix laptops and manage passwords. If it can be done better, our aim is to do it better – and to pass on the improvements to the next ones who fill this role.

We can’t just complain about nonsensical practices and talk about how things should be: we do what we can, because we must. In these minor ways, day by day, we are freeing our creative potential by automating our own small worlds.