Although the world depends on software to run nearly everything these days, most software development projects either fail completely or deliver only a fraction of the value intended. Why is this?
Broadly speaking, there are two major kinds of software project: off-the-shelf large systems implementation, and custom development. In this article we’ll be looking at the latter, and I’ll cover the former in another article in the near future.
First let’s provide a little background: I’ve been in the hi-tech arena for nearly thirty-five years and in that time I’ve co-founded several startups, worked for a large software company, implemented a wide range of operational systems for various organizations, and have worked on software development and remediation projects for a wide range of multinational clients. As a result of all this experience I’ve learned a few things about why projects fail and I’m going to do my best to impart some of this knowledge in this article in the hope that it may help a few people to avoid some obvious pitfalls in their own software development projects.
One of the most common mistakes is for people to have an idea in mind and then fail to take outside information into account. Of course, on extremely rare occasions the initial idea is indeed the most desirable one, but in general nearly every concept goes through several bouts of iteration before firming up into a minimum viable product. Sadly, the majority of would-be entrepreneurs filter out helpful external feedback and thereby set themselves up for failure. I’ve briefly consulted to more than thirty hopeful entrepreneurs over the years and the two who took my advice went on to found quite successful companies that generated real revenues; the other twenty-eight ignored my advice and every single one ended up burning money and abandoning their dreams. The reason this happens is because would-be entrepreneurs reject inputs that contradict their pre-existing beliefs. “This guy isn’t right for my project because he just doesn’t get it, he doesn’t have the vision” is a common self-deluding posture among inexperienced entrepreneurs and it guarantees eventual failure. Sure, experts are wrong from time to time. But if you’re hearing a reasoned argument from someone then it’s worth listening even if the argument seems to be pouring cold water over your cherished dream.
Another common error is to imagine that expertise in whatever industry you happen to be in or aiming at is an essential factor. I see countless job ads looking for skilled software developers that insist the candidates have prior experience in some particular industry. In reality, a good developer is industry-agnostic; the same sorts of basic problem arise whether you’re in healthcare, autonomous drone control systems, fast-food franchising, or finance. But all too often, those doing the hiring think that their industry is so unique that prior experience in that industry is essential. In fact, hiring developers who’ve had prior experience in your particular industry means that you’re more likely than not to repeat the standard mistakes people in your industry make every time they attempt to develop new software. It’s often far better to get development talent with a clean perspective and the willingness to challenge your assumptions than to get people who’ll just write software that fails in the same way all the other software is failing in your industry. Sure, you’ll get something out the other end in return for the time and money invested, but it’s unlikely to be anywhere near as effective as it would have been if you’d not had your industry blinkers firmly in place.
Yet another common mistake is to insist on certifications that are actually highly detrimental. Anyone who’s looked at some of the software industry’s certification programs knows that best-case they are years out of date and worst-case impose a mindset that is antithetical to good software development. Examples include Project Management Certification, SAFE certification, and the various Security and Cyber Security certifications out there. I’ve yet to encounter any certification of this type that isn’t either a complete waste of time or actually detrimental to getting the job done in real life.
Next comes the problem of roles. If you don’t know how to staff a project it’s likely you’ll fall back on insisting on roles that make no sense. Project Management in particular should be called out because, like Human Resources, it’s a function no sane organization would ever permit to be established. Project Management is a no-hope role: the person stuck in this position has zero authority to manage or change anything but takes all the blame when timelines aren’t met and feature-functionality fails to live up to the pre-sales hype. Experienced managers know that the correct role is Product Owner; this person has the authority to prioritize feature-functionality, make significant changes to product specifications based on user inputs and business needs, and can replace or supplement team members if necessary. Only by coupling responsibility with the ability to make and implement essential decisions can an appropriate role be created and a more successful software development program initiated. So, in brief: never, ever, have a Project Manager.
One particularly thorny issue for enterprise software development is that while obtaining end-user input is important, it’s essential not to confuse what people need versus what they think they want. We humans find it difficult (and often impossible) to imagine doing something in a way that’s different from how we’re accustomed to doing it. What this means is that end-users will generally ask for new software that simply mimics the way they do things at the present. For example, asking developers to replicate the functionality of an Excel spreadsheet model despite the fact the model is unrealistic. The key here is to ask “what do you need to accomplish?” rather than “how do you want this new software to work?”
Sometimes, of course, individuals will resist improved ways of working, but this is a management problem and not strictly speaking a software development problem. A good Product Owner will be alert to such issues and will flag them early to senior management, whose job it is to sort things out for the good of the organization as a whole.
A mistake common to first-time entrepreneurs is to think that a new product can be developed on the cheap because that’s all the budget the entrepreneur happens to have. The fact is, developing software requires some basics to be in place and if you can’t afford the basics then you will simply burn up what funds you had and be left with nothing after the money has run out. Sure, you can hire some cheap labor in Asia and tell yourself everything will be fine, but literally thousands of small entrepreneurs have gone down this path over the last twenty years and not a single one has had an adequate outcome. Although it is far easier to develop software products today than it was in earlier times thanks to cloud computing, a boatload of reusable open-source code, and various open-source tools that reduce a developer’s workload, the hard truth is that developing software costs time and money and attempts to deny this reality are doomed to failure. You may have a shiny dream, but that won’t help when your low-cost developer leaves you with rubbish code.
Another common mistake, related to doing things as cheaply as possible, is to fail to understand basic software development hygiene. Perhaps you can get away with having a single developer on your project if you’ve been astute enough to pay top dollar and hire an extremely good and extremely experienced senior software engineer. Otherwise, without having a second engineer in place so that essential things like DevOps can be shared and code-reviews performed, you’ll end up with a single developer who’s out of their depth in some areas and so the end product will have glaring gaps that ultimately will cost time and money to fix or which if not fixed will simply render your product non-viable.
There are of course a great many more mistakes that are frequently made when it comes to developing software, but I’ve covered some of the biggest that are also among the easiest to avoid.