Iteration and Agility

Since the 1950s, when computer software first became a thing, we’ve been looking for better ways to build things. In the early days everything had to be built from scratch and a design-first methodology was the best way to get things done. Projects were functionally limited so scope creep wasn’t a big issue. After all, when your goal is to automate a complex calculation using a program entered via punch cards there aren’t many roads you can go down.

Of course, over time projects became larger and more expensive and the desire to multi-purpose seemed like a good plan. Why build a specialised system when you can get more bang for your buck by making it flexible? And with that, the dreaded scope creep became a fact of life. Before you know it we’re in trouble. We’re still using design-first methodologies but we’re changing the design as we go along. I mean what could go wrong with that? Unsurprisingly, bad things happen. Many times.

Barry was right

Eventually, after many years and several fruitless attempts at fixing the problem via bloated software process improvement methodologies, somebody finally spots the emperor’s lack of clothes — design first doesn’t work. An iterative approach is the answer. Of course this was hardly a revelation; smart people had been advocating this approach for many years, but it wasn’t until a critical mass of mindshare, made possible by the internet and social media, was achieved that the powers that be woke up and smelled the coffee.

And then there was feedback

So here we are and the majority of new products are developed using an iterative approach. In fact, this iterative approach has proved so powerful that it’s been co-opted as a strategy for developing entire businesses. Why? Because short iterations reduce risk. How far off track can you realistically go in one short iteration? Iterative development makes perfect sense as long as a project is large enough or complex enough to require more than one iteration to validate. But now we’ve taken this idea further — to properly apply iterative development to the creation of a product we need to gather feedback on each iteration. How else can we tell if we’re on track? Being able to capture and respond to feedback is the whole point of the agile process. The more feedback we can capture the better.

This leads us into the next trend in the rollercoaster ride that is software product development — continuous integration. Before CI, there was a disconnect between developing a product and gathering feedback. Getting a product into a state where it could actually be used by an end user required a manual build, test and integration process that could take longer than an individual iteration, so we released periodically but usually not as often as each iteration. Nobody wants to update software every few weeks so often new products were released each year or so.

Do it for the camel

Sometimes these releases worked out. Sometimes they didn’t. To hedge their bets, software companies crammed more and more features into each release, leading to bloated products that were expensive to configure and a crap shoot to maintain. The more complex the products become the less likely people are to upgrade them. No upgrade means no feedback and that means a continued scattergun approach to adding new functionality. You can only go so far down that road before your camel needs to visit a chiropractor.

Continuous integration means that each iteration results in a fully tested, useable release of the product. But that doesn’t fix the problem that nobody wants to install a release every few weeks. Maybe if you’re building a library or some common component then downstream users will update to get access to new functionality, but not where you’re developing an end-user product. If the guys at Minecraft couldn’t make it work with a billion fervent kids, what chance do we have with a dull as ditch water business application?

It’s here that software as a service delivers its true value. When end users don’t need to install your product in order to use it, you can take continuous integration a step further and incorporate continuous delivery. Each new iteration is made directly available to users and you can gather feedback immediately. If the new feature you added isn’t being used you can change it in the next iteration or remove it completely. You can adapt to your customers’ changing requirements much faster because you’re in complete control of the process. If you want to release your product 50 times a day, and get feedback on each change, you can do that as long as you have process in place to make sure you don’t break anything.

Feedback leads to better products and better products yield competitive advantage. Since software as a service removes the technical risk to the user of upgrading, it accelerates the feedback loop in a way that isn’t possible using any other mechanism. Better products are a win-win for both software developers and users alike and it’s this fact alone that ensures that in the future all software will be delivered as a service. Anything else will seem as antiquated as punch cards and the waterfall development model.