A lone engineer stands at the city gate. She built a subway station here three years ago on a sunny day. On this dark, foggy night, she wishes her flashlight shone brighter, her map was up to date and that the neon signs of Ginza weren’t powered by a frozen yogurt stand ten miles away.
If you’re in production, product or some other non-tech discipline, you’ve probably found yourself face-to-face with a supposedly smart and experienced engineer saying they’re unable to tell you how long it’ll take to build your next feature. Maybe they asked for a day or five to think about it. Perhaps they even said they wouldn’t be able to provide a responsible estimate until work is already underway, essentially leaving you with a dice roll and hoping for a triple natural 20.
How can a skilled professional appear to know so little about their craft that they can’t tell you the effort involved? Why, when you go to their manager, do you get the same response (presuming they’ve staffed correctly and trust their engineers)?
It always feels contrived coming up with software development analogies based on the real world. In software, we pull invisible things out of thin air and build something real enough for a computer screen. The only resources required are coffee, pizza and the resulting brain power. The output is a system with inherent entanglements not generally present in the physical environment of regular life. No earthly bricks, wood or glass with obvious physics properties; instead you might look to space and see software as a universe filled with virtual worm holes, nuclear pasta and inexplicable gravitational effects.
But that gets weird and probably not any easier to comprehend, so let’s expand on a commonly used analogy instead.
The usual example when educating non-engineers on the sequencing of some piece of work is that of building a house. You’ve heard it many times – you can’t install a roof before placing the walls, and you probably shouldn’t place the walls before you’ve poured a foundation. (As a side note, depending on the specifics, you may be able to create the wall and roof components, or at least get started on them, while you’re working on the foundation.) This makes sense to anyone who has built any type of even remotely complex structure. However, it only addresses why we might prefer to construct things in a certain order, not so much why it can be difficult to provide a time estimate for an effort on the spot.
Let’s zoom out from our house and look at an entire city. A very big one like metro Tokyo with a positively massive number of buildings, streets, railways, bridges, tunnels, waterways, powerlines, cellular towers, lights, parks, vehicles and people. This list isn’t exhaustive but I’m running out of commas. Different parts of the city were built by various people and robots constrained by whatever budget, timeline, technological and process parameters present at the time.
This is your product codebase if it’s anything but trivial in scope. It gets more exciting.
Now imagine that for any addition or modification, in addition to that work, the entire city needs to be rebuilt from a blueprint which changes over time. This blueprint describes how resources are mined, tools and components are crafted, buildings are constructed, roads are paved, tracks are placed and power stations are built. A change to your mining operation and resulting materials may affect any number of downstream processes and outcomes. Swapping out your tooling could change the way you construct components and buildings, and even alter their appearance. Modifying the specifications of a single section of railway track might affect the entire traffic system and anything connected to it. The rebuild happens via automated means; it’s the ripple effects that are important here.
It’s not over yet. Because we’re talking about the world of software, we need to introduce a Twilight Zone factor to our city – only in support of this analogy, not because behaviors can’t ultimately be explained. (I’m sure any 4 AM coders out there will disagree with the latter.)
Changing the behavior of the elevator control system in one skyscraper may magically force all the elevators in the city to the top floor. Or not. Adding a floor to a house might be impossible because no existing home in town is taller than the one you want to modify. Or not. Erecting a new high-rise could make all the bridges in the city flip upside down. Or not. The list is endless and Godzilla is definitely in there somewhere.
Where does this nonsensical behavior originate? You’ve probably heard engineers talk about architecture, patterns, coupling, dependencies, monoliths, modularity, tech debt and test coverage. If so, they were either basking in the glow of perfection, or, more likely, giving you the reasons why boats will stop floating if you add a terminal to the airport. Everything depends on the original intentions behind the system, how those intentions were translated and understood, and how the entirety was designed and implemented. And, of course, how it’s all been maintained – or not – by a changing, veritable army of people since the beginning of time.
Literal armies are aided by intelligence units in clearing the fog of actual war. Expecting a human to have total situational awareness of our mystical, misty version of the Japanese capital is probably unreasonable. Engineers live in ever-shifting, ever-replenishing fog and its existence is one reason your request for an estimate is sometimes met with only a frustrating question mark.
This is where you ask me how to solve this whole thing, how we can lift the fog. The answer is that there is no silver bullet, but as is pretty commonly known, you can start by aiming for the smallest, clearest user stories possible. Remember that “minimum viable” means reduced scope, not reduced clarity. All coding ultimately boils down to making a hard left or a hard right. If decisions and acceptance criteria aren’t clear at that point, you have a blocker or a potentially misbehaving feature no matter how minimally viable it is.
Also, there are some things you can do when embarking on a new project to make estimating a bit easier in the future. If you provide space for setting up the architecture appropriately from the beginning, keep your codebase healthy and not riddled with tech debt, invest in automated test coverage and allow time for continuous refactoring, you may find that engineers will have an easier time providing estimates. Those estimates may also prove more accurate. A significant added bonus will be a higher-quality product and more bulletproof release process. If you’re already dealing with an untestable codebase reminiscent of a bowl of spaghetti, you’ll have a tougher time.
This, like many things, ultimately turns into more of a people thing. Being good partners becomes important in order to reduce frustrations. You have the right to ask for an estimate; it’s important for your roadmap and overall business planning. A good engineer will understand this and won’t be bothered by your asking. If they don’t understand, educate them. In return, you can show willingness to allow investigation time as part of estimating, and that it’s safe for engineers to revise their initial estimate as they go – the closer they get to the target, the more accurate they’ll be.
Also, understand that many engineers are fearful of sharing their actual estimate simply because the bigger it is, the worse it sounds. They may have become scared of appearing to be padding even when they know they’re not, so they underestimate. They may feel that whatever quasi-realistic timeline they come up with won’t be fast enough because reactions to estimates are often “what, really?” or “yikes” or “I thought you were faster than that” whether through words or body language. They don’t want to disappoint in the moment, so they kick the can of Surge down the road. In unhealthy environments, engineers may even want to seem faster or braver than their teammates.
These are things you can have conversations about and grow trust around. You probably live in a fair amount of uncertainty yourself, and bonding in the fog is as good as anywhere.
Back at the gate, our engineer installs fresh batteries in her flashlight and passes through, determined to reveal more of her map, bump into some friends and upgrade a frozen yogurt stand together. The fog remains, but it seems friendlier somehow.