The soul of automation is human after all

Monday 16 March 2015

A DevOps culture can start to flourish in an environment built upon a high level of automation

When I first heard about automation in software development, I stopped for a moment. I had always assumed that a profession so rooted in problem-solving would be immune from the leaps that have transformed our manufacturing industries during the last century.

Sure, it is possible for an army of robots to hack noodles for soup, but don’t we need real developers to hack project requirements and restraints into efficient code and beautiful software?

We have always been told that, while the machines have replaced our factory workers, and laptops and word processors have replaced the typing pool, we knowledge workers are safe. Safe solving hard problems that only a human can solve. Could we all be made redundant in this new world of automation?

Of course, we are still a long way from developing the artificial intelligence that will in turn replace the developers, but it is still a useful thought exercise. Imagining a future where we have a lot less to do can help us refine our concept of a developer as a thought worker, and (paradoxically) encourage us to seek this automation and apply it as far as possible in our working lives today.

When we look at our day-to-day work, it quickly becomes apparent that a large part of it is not filled with the kind of creative problem-solving we romanticise about when we believe ourselves immune from automation. The work day of a developer is often taken up with mundane and repetitive tasks such as setting up environments, compiling code or managing deployments.

How could we remove (or at least reduce) the impact of these tasks? This is where automation as we know it today comes in: to help us focus on what we enjoy doing and become more productive. Almost every step of the development process lends itself to some degree of automation or mechanisation.

Environment setup

Using tools like Vagrant, we can quickly and consistently build developer environments. Gone is the time when the first two days of a developer joining a project were taken up downloading and installing dependencies, or upgrading (and even downgrading) the ones you already had.

The complications of versions and downloads can be moved away from the individual developer and towards a configuration file.

Creating more environments

Once you have your environment set up as configuration (rather than a neglected wiki page), you can start to build it anywhere you like. You can create a test server or even a production server. You can create multiple instances and replace them as many times as you like, safe in the knowledge they will always be identical.

Project deployments

Next, we can start to automate the process of getting the software onto these new servers. Continuous integration servers such as Jenkins or Go can watch for changes in our version-controlled code and automatically rebuild and deploy our project.

Quality control

Now that we are continuously and automatically delivering our project without looking up from our code, we need to enforce some checks and balances. Unit tests and static analysis should be run locally as the code is being written. They should also be run by the continuous integration server, before the code is deployed to your test servers. The results of these tests, and the level of their coverage, should be plotted and displayed, ideally on a large monitor or information radiator where the whole team can see them.

If the tests fail, stop the build. If the build is broken, it is the whole team’s responsibility to get it fixed.

Automated testing

In addition to tests that check the quality of the code, you can also automate tests to check it behaves as expected. Tools such as Selenium can follow a user journey through your application and verify that the experience is as expected. Paired with something like Wraith, you can even automate visual regression testing.

Monitoring and feedback loops

Once we have the tested software finally up and running on our production server, there is still more automation we can apply. By adding monitoring tools and notification systems, we can make sure that the code keeps working. We can respond to bottlenecks by automatically scaling components or services, and in the worst case scenario, restart or even rebuild failed instances.

So once we have all of this automation in place, what does it mean for us as developers?

There are two obvious answers. The first is to turn this efficiency into the mythical four-hour working week – lie back in our hammocks, cocktail in hand, and pat ourselves on the back for a job well done.

If we ask the shareholders we might get a less pleasant scenario, as they transform this newfound efficiency into productivity by doubling the workload, or they cash in by reducing the workforce.

A far better investment, however, is to see these gains not as productivity gains, but as something far more valuable: time gains.

Once the mundane tasks are out of the way, we can use this gained time to truly add value to what we are doing. We can let the robots (as limited as they are for now) handle the things that get in our way and free ourselves up to focus on the parts of the job we love.

Each of these tools adds another time benefit too. As well as being automated, they are also invariably faster than we were. When we have faster iteration cycles is when the true value starts to shine.

We can get a new developer onto the project in hours – even minutes – rather than days. This means we can get fresh input into a project and move people between projects much more easily than we ever have before.

We can create identical environments for testing so that we can tweak things, test them, tear them down and rebuild them as many times as we like. Now the configuration is in version control, our environments become commodities rather than precious artefacts. This means we can take risks with them. We can try things out.

When the code for building and deploying environments is accessible by the whole team, developers can start to own the infrastructure and operations. A DevOps culture can start to flourish in an environment built upon a high level of automation.

Quality becomes something measurable and trackable. We can see problems almost immediately and take care of them while they’re fresh – before they are baked into our codebase.

The freedom to try things out and see them on a server in a matter of minutes means that we can get feedback from product owners more quickly, and bring design and development closer together. All of these factors will help us to innovate at a new rate – and it is this opportunity to create great experiences that will keep us excited in our work.

About The Author

David lives in Berlin, Germany where he works for AKQA as a Principal Web Developer, focusing largely on front end browser technologies.