Momentum in software development is never a straight line

Development does not move forward with purpose like a marching band. It weaves, stumbles and staggers like a drunk heading towards the kebab shop #HoskWisdom

Projects don’t move in a straight line and project plans are rarely accurate. Software development is full of surprises, new requirements and changes no one ever predicts. It’s only the simplest projects with a clear scope and less than 5 people involved that go to plan. They are as a common as a 5 leaf clover.

Why are they wrong?

Recipe for success: under-promise and over-deliver. Kevin kelly

Projects do the opposite, they over promise and under deliver because the project/work is underestimated and the team’s ability to deliver is overestimated

Everyone wants to hear a lower numbers in days to deliver and cost, so that’s what they hear.

The lowest bid wins and the curse of the winner is to deliver the project to an impossible deadline. How does the project scope grow so much from the original requirements?

Requirements are not understood, with many missing requirements and lots which will change as the details become clearer. It’s impossible to identify all the low level requirements from high-level requirements e.g. you discover more as you go into detail and use what’s been built.

The perfect project plan is POSSIBLE if one first documents a list of ALL the UNKNOWNS. Bill Langley

Software development uses the inside view to estimate work and create project plans. These estimates have little margin of safety and don’t assume bugs, problems or change of requirements..

Most software development projects are late. Everyone who has worked on software projects knows that rarely go to plan, yet all plans are made using the optimistic estimates that assume no problems, no changes in mind and no change in requirements.

Why will this new project be different? It won’t, it’s just what everyone wants to hear and wishes for.

It’s only in fairy tales that emperor’s get told they are naked and the project will take longer than estimated.

If you are doubting this point, then think

• Were the requirements different by the end of the project, then the start?
• Did the requirements grow in scope and size?
• Did the project have unexpected problems (technical, people, political)?
• Did the project change its mind or priorities?
• Were there any design mistakes?

More people, the longer the delivery time, the wider the scope, all increase the probability of unforeseen delays. The bigger the project, the bigger the impact they can have.

What really happens

Everyone forgets how hard it is to create software from requirements. Creative processes do not run smoothly, there are missteps, stumbles, mistakes and wrong turns.

Every old system replaced was moulded into its current state via many bugs, feedback and discussions until the team worked out how the system needed to work. Therefore most rewrites take longer than estimated because they remove lots of code they think they don’t need but was added to fix a bug.

Chesterton’s fence says never remove a fence until you know what it was there for, was it keeping things out or keeping things in?

Chesterton’s Fence: A Lesson in Second Order Thinking

Code is full of Chesterton’s fence but it removed by developers making the code efficient without understanding its purpose. Few people understand all parts of a system, which is why changing it often leads to fixing one bug only to create another in a different part of the system.

Creating a complex system is done by focusing on the details when you get to that part and finding there are complexities you didn’t see when you looked at it from a high level.

This is how it works, this is the process. Like the drunk stumbling to the kebab shop, it’s not a direct line, it’s a wibbly wobbly works of discovery, full of surprises.

Momentum on projects should be constant, steady progress. The problems, questions and new requirements become the way they are the method you create the system needed, which is different from the system we thought was needed at the start.

As Ryan Holiday says — The obstacle is the way

What’s the problem

The problem is plans give a project a false sense of control, plans give an incorrect expectation on delivery timelines and cost.

Plans are underestimated and extending then causes pain. Re-planning multiple times in small increments wastes time and results in iterations of incorrect project plans. The common response to tight deadlines is to go faster, reduce quality but this speeds creation of code but takes longer to get to production.

• Why IT projects are underestimated
• How we try to speed up IT projects and why it doesn’t work — Reducing quality slows down codes journey to production
• Creating software is the start not the end — explains that creating software is a misleading because the cost of software is in the maintenance.
• Software development is a loser’s game — Software development should focus on reducing bugs and finding them in dev.
• Why adding more people to a project doesn’t make it go faster — every person increases communication and makes collaboration harder.

Quality code gets into production faster and it reduces complexity, which reduces time spent on maintenance.

Simple example
A company wants case management and a portal, integrated with their phone system.

Dataverse and Powerapps portal is a sensible choice

The requirements quickly grow

• Call popping grows into web chat, the possibility of chatbots is mentioned
• Case management is for 4 teams
• 4 different SLAs
• There are experts for different areas
• An escalation process
• Some cases need approval
• We need to send letters out and they need an integration
• Documents need to be stored
• Documents need to be uploaded from the portal
• Some documents can only be viewed by certain users

What seems like a straightforward piece of work keeps growing as more details are uncovered. At a high level the out of the box solutions met the requirements but high level requirements are not the real requirements.

Conclusion

Projects don’t move in a straight line to their goal, the scope expands as high-level requirements are broken into low-level requirements and everyone understands how the system needs to work.

Blood, sweat and tears are put into creating software and any large project. I look back on projects and marvel at the painful process, but am proud of what we achieved and how everyone worked as a team.

Creating software is one of the most frustrating and rewarding activities you can attempt, and they rarely go in a straight line or as planned.