In the Beginning, It Was Just Code

I started building software for a living some time ago. To say “things have changed” would be possibly the biggest understatement ever.

First the editors got a little better, then there were text-oriented debuggers, then graphical user interfaces, and better languages, and then there was AWS S3, and then cloud services, and on and on until now we write code with AI assistants.

The slow drip of progress, over 40 years, becomes a flood of changes. And the rate of change is accelerating.

People and Process Haven't Changed

In spite of all the technological advances, the people writing the code haven’t changed, with their quirks and flaws and foibles. And the processes used to plan and coordinate the creation of code, designed to control and constrain our human shortcomings, have barely changed.

First there was “Waterfall” and then there was “Agile”. And then…that’s pretty much it.

Sure, the tools have gotten better — JIRA, Trello, Open Project and probably a few hundred others. Maybe you prefer Scrum vs. Kanban, or maybe a mixture of both. But really, you’re just getting a database that stores information about work assignments and their current state — To Do, In Progress, In QA, Done, Released, and so on. Each tool provides some cool graphs and metrics — burn down rates, and so on, but are they really helpful? Mostly they serve to make it appear as if software development is more engineering and less art and craft. The reality is, it’s still mostly art and craft.

Story Points + Tee Shirt Sizes = When Will It Be Done?

Remarkably, the original Agile Manifesto said nothing about story points and tee shirt sizes. That came a few years later, and somehow got a lot of traction. There’s nothing inherently wrong with using story points and tee shirt sizes. They are often used because the timeline for most significant software development can’t be accurately predicted using duration as the primary measure — even though that’s the one everyone really wants.

And that’s the rub. Many companies demand that software engineering leaders produce timelines for feature development that include fixed-in-time milestones for customer “commitments”. Meaning, whatever dates you produce will become lines in the sand that can’t be crossed. And, if you try to account for all the things that can go wrong: unknown scope, scope creep, developers leaving, and on and on, then it usually appears you are “sandbagging”. It makes for a difficult situation, where basically no one is happy.

It Doesn't Have to Be This Way

Not all companies are still time-oriented, but in my experience, many still are. In order to succeed in software development (and by “succeed” I mean not drive each other crazy), companies need to move away from customer “commitments” and focus more on customer “roadmaps”.

A customer roadmap is simply this: An ordered list of priorities. Priorities can have dates, but they are specified in quarters, not written-in-stone dates. And each “date” has an associated confidence — high, medium, low. High confidence means the engineers have designed it and are working on it, and are in the home stretch. Medium means the engineers have talked about it and have a fairly good idea of how to do it and how long it will take. Low means it’s on the wish list. And dates are updated weekly. Meaning, they will change, so deal with it.

Why does this work? Let’s go back to a key phrase in the Agile Manifesto:

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

In this approach software is released stepwise, according to the Agile Manifesto. Functionality is produced step by step - maybe behind a feature flag if necessary - but there is no "Ta Da!" moment, at least not for the engineers. If the business needs that "Ta Da!" moment, it can be a facade, meaning, the features are all there, we're just telling you about it now.

Bonus Tracks

The approach above can't always be used. Software leaders aren't typically the deciders. So if you are stuck giving timelines, I've written a tool that might help you.

I've been interested in the field of Resource Constrained Project Scheduling for a long time. Basically, it's a field of study that attempts to find optimal resource allocations for complex projects. It's actually a very difficult problem to solve, and there are no known algorithms that are guaranteed to solve it optimally. So, mostly heuristics are used to find a "good" schedule.

I've taken a few cracks at this problem over the years to aid in my own work. My latest attempt is available here. Basically, you upload a spreadsheet with a list of tasks and the skills required to complete them, as well as a list of resources each with one or more skills, and the app will find the "optimal" schedule for you by assigning resources to tasks and computing the associated schedule. And by "optimal" I mean probably better than you can do by hand. It's free, no ads, just something I want to share.

Happy coding!