Engineering Project Estimation
5 tricks to be less wrong when estimating engineering projects
Do you agree that predicting the length of an engineering project is difficult?
Is there a more challenging task than predicting the future?
Yet, engineering teams go through cycles of planning and estimating the length of projects.
In pretty much every place I've worked, we've set OKRs to plan for the upcoming quarter.
Unless the project is trivial, you’re dealing with unknowns.
The result? Lots of surprises towards the end of the quarter.
How often do you push OKRs to the next quarter?
Today, I’m going to share 5 tricks from my experience that will make you a better predictor. The post is divided into 3 parts as follows,
🤯 Why we struggle with estimates
⚙️ Top-down and bottom-up estimation
🛠️ 5 tricks to be less inaccurate
You can apply these to understand the limitations of planning and estimate like a pro. This post is all about the balance between targeted accuracy and time invested.
Let’s dive in!
Project estimation is the process of making a calculated guess about how long it will take to finish. This helps in,
Better resource planning
Right prioritization
Making business decisions
There’s no avoiding it, even if we are not good at it.
🤯 Why do we suck?
Broadly, we can divide engineering work into two categories.
🐜 1. Trivial Projects
Projects with widely accepted solutions across the industry. These types of projects are concrete and the only variable is organizational integration. Complexity can still vary, but proper design can make them manageable.
⛰️ 2. Ambitious Projects
Working on the cutting edge of technology, problems don’t have answers on stack overflow anymore. There are more unknowns and less concrete milestones. Most of the time you’ll find yourself here. Planning becomes particularly challenging.
Irrespective of type, a variety of other factors influence estimation. When you’re dealing with complex systems, no one knows anything with 100% certainty. I’ve worked on systems with more than millions lines of codebases. I felt clueless every time it came to planning how long a project should take.
⚙️ Top-Down vs Bottom-Up Estimates
We will not get into setting SMART goals, that’s for another time. Our focus will be on estimating the timeline of a project. Like Dynamic Programming, estimates of a project can be two ways.
⬇️ Top-Down Approach
You use experience from past, similar projects. Starting from the top, here we’re not worried about specifics of individual tasks. This is fast and helpful in estimating a rough timeline. Due to lack of visibility into line items, this can be inaccurate.
⬆️ Bottom-Up Approach
Here, you divide a larger project into smaller chunks. By estimating timelines for individual tasks, you calculate the overall timeline. This can be more accurate but takes more time.
Obsessing over either of them can be problematic. One time, I planned my team’s OKRs and put in multiple work streams, only to realize half of them were dependent on others. This is a result of not digging deep enough. On the other end, the next quarter I spent way too much time breaking down. An unexpected surprise in production integration made it all a waste.
In short, there’s a trade off. Depending on what you’re planning to get from the estimate, you need a hybrid approach.
Next, let’s dive deeper into some mistakes I made and how you can learn from them to be less wrong on estimates.
🛠️ 5 Tricks to Be Less Wrong
1/5 ⚔️ Divide and Conquer
In my most unproductive quarter at Google, I had four medium-sized tasks on my plan. Grouping these items made complete sense. They were closely related. However, as soon as I started on one of them, I got blocked by another team. While waiting, I couldn’t pick up any of the other three tasks! Because, they depended on the completion of the first one.
Another time I had too many design tasks in a single quarter. External dependencies slowed me down. Design discussions took longer than I anticipated.
Here’s what you can do to avoid making the same mistakes,
🧩✂️ Divide the project into sizable, independent tasks
1️⃣2️⃣ Understand the order at which they need to be delivered
🧠🤝 Borrow expertise from the team, let people estimate the time and adjust accordingly
How we break down a large project matters. Break it down into logical chunks and understand the order. This will help you estimate better.
2/5 🪢 Untangle Dependencies
When I picked up too many designs for a quarter, I forgot to consider a few things,
I will have to work with multiple teams
Each discussion may drag for days
I might end up working on multiple designs at a time
Too much cognitive load, a lot of context switches
Additionally, waiting on discussion threads quickly became painful. Seemingly independent work streams can be highly dependent on external entities. Failure to identify these early is a major cause for off estimates.
Here’s how you should avoid them,
🏷️ Label work items properly, e.g. design, experiment, develop, testing, launch etc.
👥 Identify stakeholders and dependencies
🎯 Reserve time with external teams, put it on their OKR
🔀 Lookout for potential parallelism, choose items that can be concurrent
Better estimate relies on your understanding of the problem. Your goal is not to eliminate all unknowns. Instead aim to reduce ambiguity when planning.
3/5 🛋️ Leverage Cushions for the Unknowns
No matter how hard I tried above two, there’d always be some misses. After overlooking the inevitability of bad estimates for consecutive quarters, I learned to embrace it.
We strive to set ambitious goals, it’s fair to push the limits. At the same time, never finishing what we promised can be seen as a performance dip. Balancing these two is crucial to sustain productivity in the longer term.
What helped me prepare better for unpredictable future,
Accepting that things will go wrong, at every opportunity
Incorporating “slack” time into planning
A common way to estimate team bandwidth is to collect availability. Typically this is 13 weeks in a quarter. We started reserving one extra week for slack. This would account for sick hours, prolonged timelines and other surprises.
The psychological impact on a team after missing delivery is huge. Preemptive actions to create a supportive environment goes a long way.
4/5 🔢 Use Fibonacci or T-Shirt Sizing
I’ve worked with some leaders who want to account for every hour spent. We already suck at estimating quarters, how would hours work? Well, it doesn’t. We’re often estimating on the best case side. Thanks to how human minds work (optimism bias).
Planning and discipline are very important for a team. My goal has always been to respect the process and do our best to keep promises. Rounding up is the remedy that worked for me to balance the inherent bias.
Fibonacci works really good for tasks,
Tasks are well scoped, measurable key results
But once it grows beyond 5w, break it down
T-Shirt sizing works well for larger projects,
Projects often have moving pieces leading to ambiguity
Here you get a less accurate, estimated range of weeks
Based on what you’re estimating, you can use a combination of these. For high level estimates, start with T-Shirt sizes. Once planning OKRs and distributing tasks, use a fibonacci estimate in the number of weeks.
5/5 📈 Continuous Tracking
Finally, the biggest mistake I’ve made with project estimation was not valuing it. As an engineer, it's easy to view planning as unnecessary overhead. And I was no different. I began my leadership journey believing that planning wasn't important. And that it was acceptable to do things without a plan because that was the reality in engineering.
Downsides were obvious, the team regularly missing on organizational priorities. When we were collaborating with other teams, missing timeline hurt our reputation. This cost lost confidence from the leadership as well.
You don’t have to make the same mistakes:
Continuously monitor progress against plan, keep the team focused
Re-prioritizing is okay, but communicating early is key to building trust externally
End of Quarter OKR evaluation can demonstrate a team's consistency externally. It can also boost the team’s morale and foster motivation
Benefits of investing in planning may not be obvious, but it’s necessary for sustainable performance. Establishing a process and respecting it will help you build a high performing team.
🌟 🔍 Parting Thoughts
Remember, the goal is not to get it right, but to get it less wrong. The hidden complexity in engineering projects is not going anywhere. This will always cause drifts from planned paths. Accepting this reality and continuously refining strategy is the key.
Today, I shared mistakes I made repeatedly before getting better. Hope you could resonate and apply some of these techniques into your planning.
What’s your one strategy that significantly increased accuracy in project estimation?
Share in the comments!
🐦🔥This Week’s Favorites
Skills to grow beyond Senior from
byRate limiting using Redis from
byReading whitepapers for growth from
by4 steps for making technical decisions from
by
👋 💬 Get In Touch
Want to chat? Find me on LinkedIn.
If you want me to cover a particular area of leadership, you can reach out directly on akash@chromium.org.
If you enjoyed this content, please 🔁 share it with friends and consider 🔔 subscribing if you haven’t already. Your 💙 response really motivates me to keep going.
Estimates are just hypotheses. As a hypothesis, if you have more data, it will be great. If not, we try to come up with 3 types of estimates - optimistic, realistic, and pessimistic. All these are communicated with the stakeholders, so it's clear from the beginning. If we see that something is not going in the right direction, immediate communication is needed.
Great article, Akash, and thank you for including my latest article in your top picks!
Great article, estimating project is inaccurate, enjoyed reading the post. Highly recommend it.