Engineering Project Estimation

5 tricks to be less wrong when estimating engineering projects

Do you agree that predicting the length of an engineering project is difficult?

Is there a more challenging task than predicting the future?

Yet, engineering teams go through cycles of planning and estimating the length of projects.

In pretty much every place I've worked, we've set OKRs to plan for the upcoming quarter.

Unless the project is trivial, you’re dealing with unknowns.

The result? Lots of surprises towards the end of the quarter.

How often do you push OKRs to the next quarter?

---

Today, I’m going to share 5 tricks from my experience that will make you a better predictor. The post is divided into 3 parts as follows,

• 🤯 Why we struggle with estimates

• ⚙️ Top-down and bottom-up estimation

• 🛠️ 5 tricks to be less inaccurate

You can apply these to understand the limitations of planning and estimate like a pro. This post is all about the balance between targeted accuracy and time invested.

Let’s dive in!

---

Project estimation is the process of making a calculated guess about how long it will take to finish. This helps in,

• Better resource planning

• Right prioritization

• Making business decisions

There’s no avoiding it, even if we are not good at it.

🤯 Why do we suck?

Broadly, we can divide engineering work into two categories.

🐜 1. Trivial Projects

Projects with widely accepted solutions across the industry. These types of projects are concrete and the only variable is organizational integration. Complexity can still vary, but proper design can make them manageable.

⛰️ 2. Ambitious Projects

Working on the cutting edge of technology, problems don’t have answers on stack overflow anymore. There are more unknowns and less concrete milestones. Most of the time you’ll find yourself here. Planning becomes particularly challenging.

Irrespective of type, a variety of other factors influence estimation. When you’re dealing with complex systems, no one knows anything with 100% certainty. I’ve worked on systems with more than millions lines of codebases. I felt clueless every time it came to planning how long a project should take.

⚙️ Top-Down vs Bottom-Up Estimates

We will not get into setting SMART goals, that’s for another time. Our focus will be on estimating the timeline of a project. Like Dynamic Programming, estimates of a project can be two ways.

⬇️ Top-Down Approach

You use experience from past, similar projects. Starting from the top, here we’re not worried about specifics of individual tasks. This is fast and helpful in estimating a rough timeline. Due to lack of visibility into line items, this can be inaccurate.

⬆️ Bottom-Up Approach

Here, you divide a larger project into smaller chunks. By estimating timelines for individual tasks, you calculate the overall timeline. This can be more accurate but takes more time.

Obsessing over either of them can be problematic. One time, I planned my team’s OKRs and put in multiple work streams, only to realize half of them were dependent on others. This is a result of not digging deep enough. On the other end, the next quarter I spent way too much time breaking down. An unexpected surprise in production integration made it all a waste.

In short, there’s a trade off. Depending on what you’re planning to get from the estimate, you need a hybrid approach.

Next, let’s dive deeper into some mistakes I made and how you can learn from them to be less wrong on estimates.

🛠️ 5 Tricks to Be Less Wrong

1/5 ⚔️ Divide and Conquer

In my most unproductive quarter at Google, I had four medium-sized tasks on my plan. Grouping these items made complete sense. They were closely related. However, as soon as I started on one of them, I got blocked by another team. While waiting, I couldn’t pick up any of the other three tasks! Because, they depended on the completion of the first one.

Another time I had too many design tasks in a single quarter. External dependencies slowed me down. Design discussions took longer than I anticipated.

Here’s what you can do to avoid making the same mistakes,

• 🧩✂️ Divide the project into sizable, independent tasks

• 1️⃣2️⃣ Understand the order at which they need to be delivered

• 🧠🤝 Borrow expertise from the team, let people estimate the time and adjust accordingly

How we break down a large project matters. Break it down into logical chunks and understand the order. This will help you estimate better.

2/5 🪢 Untangle Dependencies

When I picked up too many designs for a quarter, I forgot to consider a few things,

• I will have to work with multiple teams

• Each discussion may drag for days

• I might end up working on multiple designs at a time

• Too much cognitive load, a lot of context switches

Additionally, waiting on discussion threads quickly became painful. Seemingly independent work streams can be highly dependent on external entities. Failure to identify these early is a major cause for off estimates.

Here’s how you should avoid them,

• 🏷️ Label work items properly, e.g. design, experiment, develop, testing, launch etc.

• 👥 Identify stakeholders and dependencies

• 🎯 Reserve time with external teams, put it on their OKR

• 🔀 Lookout for potential parallelism, choose items that can be concurrent

Better estimate relies on your understanding of the problem. Your goal is not to eliminate all unknowns. Instead aim to reduce ambiguity when planning.

3/5 🛋️ Leverage Cushions for the Unknowns

No matter how hard I tried above two, there’d always be some misses. After overlooking the inevitability of bad estimates for consecutive quarters, I learned to embrace it.

We strive to set ambitious goals, it’s fair to push the limits. At the same time, never finishing what we promised can be seen as a performance dip. Balancing these two is crucial to sustain productivity in the longer term.

What helped me prepare better for unpredictable future,

• Accepting that things will go wrong, at every opportunity

• Incorporating “slack” time into planning

A common way to estimate team bandwidth is to collect availability. Typically this is 13 weeks in a quarter. We started reserving one extra week for slack. This would account for sick hours, prolonged timelines and other surprises.

The psychological impact on a team after missing delivery is huge. Preemptive actions to create a supportive environment goes a long way.

4/5 🔢 Use Fibonacci or T-Shirt Sizing

I’ve worked with some leaders who want to account for every hour spent. We already suck at estimating quarters, how would hours work? Well, it doesn’t. We’re often estimating on the best case side. Thanks to how human minds work (optimism bias).

Planning and discipline are very important for a team. My goal has always been to respect the process and do our best to keep promises. Rounding up is the remedy that worked for me to balance the inherent bias.

Fibonacci works really good for tasks,

• Tasks are well scoped, measurable key results

• But once it grows beyond 5w, break it down

T-Shirt sizing works well for larger projects,

• Projects often have moving pieces leading to ambiguity

• Here you get a less accurate, estimated range of weeks

Based on what you’re estimating, you can use a combination of these. For high level estimates, start with T-Shirt sizes. Once planning OKRs and distributing tasks, use a fibonacci estimate in the number of weeks.

5/5 📈 Continuous Tracking

Finally, the biggest mistake I’ve made with project estimation was not valuing it. As an engineer, it's easy to view planning as unnecessary overhead. And I was no different. I began my leadership journey believing that planning wasn't important. And that it was acceptable to do things without a plan because that was the reality in engineering.

Downsides were obvious, the team regularly missing on organizational priorities. When we were collaborating with other teams, missing timeline hurt our reputation. This cost lost confidence from the leadership as well.

You don’t have to make the same mistakes:

• Continuously monitor progress against plan, keep the team focused

• Re-prioritizing is okay, but communicating early is key to building trust externally

• End of Quarter OKR evaluation can demonstrate a team's consistency externally. It can also boost the team’s morale and foster motivation

Benefits of investing in planning may not be obvious, but it’s necessary for sustainable performance. Establishing a process and respecting it will help you build a high performing team.

🌟 🔍 Parting Thoughts

Remember, the goal is not to get it right, but to get it less wrong. The hidden complexity in engineering projects is not going anywhere. This will always cause drifts from planned paths. Accepting this reality and continuously refining strategy is the key.

Today, I shared mistakes I made repeatedly before getting better. Hope you could resonate and apply some of these techniques into your planning.

What’s your one strategy that significantly increased accuracy in project estimation?

Share in the comments!

---

🐦‍🔥This Week’s Favorites

• Skills to grow beyond Senior from

  The Caring Techie Newsletter
  by
  Irina Stanescu

• Rate limiting using Redis from

  Implementing
  by
  Marco Moauro

• Reading whitepapers for growth from

  High Growth Engineer
  by
  Jordan Cutler

• 4 steps for making technical decisions from

  The T-Shaped Dev
  by
  Petar Ivanov

  ---

👋 💬 Get In Touch

Want to chat? Find me on LinkedIn.

If you want me to cover a particular area of leadership, you can reach out directly on akash@chromium.org. 

If you enjoyed this content, please 🔁 share it with friends and consider 🔔 subscribing if you haven’t already. Your 💙 response really motivates me to keep going.