How to run a tech debt assessment

06 Feb 2023

Running assessments of your team’s current tech debts and turning them into actionable solutions should be a regular part of your work. There’s plenty of content out there to convince you why tech debt should be addressed regularly, so I will assume you’re reading this because you already agree. This post will describe one method I’ve used for conducting these assessments and review which parts worked well and which still need improvement.

The Process

1. Prep
2. Kickoff
3. Submit
4. Review
5. Prioritize
6. Budget
7. Commit
8. Plan
9. Implement

This process took about 6 weeks start to finish. It could be done much faster, but at some point engagement and quality will start to suffer. If you need to get this done faster, reserve as much time as possible for Steps 4-6 - the submission, review and prioritization of solutions.

Prep

Preparation involves two main deliverables: choosing a collaboration tool and designing a solution template. Since you’ll want to preserve your team’s engagement for submitting and prioritizing tech debt solutions I suggest completing the preparation step yourself.

Your collaboration tool should support simultaneous collaborative editing and have a mechanism for capturing discussion threads as engineers refine their solutions. We used a Google Doc since it was already part of our organizational toolkit.

Your solution template should capture basic information about each piece of tech debt and how they’re proposing to address it:

Title

For easy reference during discussion.

Description

What problem are we solving?
Why is it a problem?
How are we solving it?

Estimated Level of Effort w/ Bucket

How long will the solution take to implement?
Use whatever estimation tactic your team is familiar with. We used estimations that looked like "1 5 point Spike and 2-3 1 or 2 point Tasks" because that's what we were using for sprint work. Include a list of buckets engineers will use to categorize the effort (Small: < 10 points, Medium: 11-20 points, etc)

Estimated Impact

Why solve this problem?
Why now?
For example, if this solution automates a time consuming and error-prone step in a common process your estimated impact might be "5 hours of engineer time every 2 weeks; 95% reduction in user errors introduced during process (each user error takes 2 hours on average to resolve)"

Sponsor(s)

Who proposed this solution?
The Sponsor(s) will be responsible for leading the project, including tasking out the work involved, if this solution is prioritized.

If this is your team’s first time going through this process, consider creating an example tech debt solution using your template.

Share your completed prep with your team a few days before you get together to have a kickoff. Give them a chance to review the template and any other materials you’ve gathered.

Kickoff

Schedule a kickoff meeting with your team. Get your reports excited about the prospect of having measurable impact on their and their peer’s day to day. Clearly communicate your documented timeline for this process, highlighting the due date for the next phrase of asynchronous solution ideas. Give them time to ask questions and suggest refinements to the solution template.

Submit

Give your team about a week to asynchronously add their proposed solutions to the shared document. Periodically remind them of the due date. Keep an eye on the submitted solutions for questions.

Review

Faciliate a session where you walk through the list of proposed solutions. Give each sponsor a few minutes to pitch all of their solutions to their peers. Give their teammates a few minutes to ask questions. Try to keep things moving and avoid letting technical conversations go too deep. You can always pivot later if a solution proves unworkable.

Look for opportunities to merge similar solutions.

Set the duration of this session relative to the number of solutions submitted. Aim for about 5 minutes average per solution.

We proceeded to the next step (prioritization) as soon as we were done reviewing solutions, which is preferable. That said, if your folks need a break, give them one and reconvene at another time.

Prioritize

Give the team time to vote for their preferred solutions. Come up with a voting system that results in a measured allocation of votes per engineer based on the total number of solutions. Consider handing out extra votes to those that put in extra effort to submit more high quality solutions. They shouldn’t have so many votes they hand them out unthinkingly but they also shouldn’t have so few that they get stuck in decision paralysis.

These are the voting rules I came up with.

• Everyone starts with 3 votes.
• You get one additional vote if you contributed/sponsored any ideas.
• You get an additional vote per 3 items you have contributed/sponsored.
  • If you submit 7 items you get 3 + 1 + 1 + 1 votes.
  • If you submit 1-2 items you get 3 + 1 votes.
• You cannot vote for an item more than once.
• You don’t have to use all of your votes.

For context, we had 19 solutions to vote on.

I chose to abstain from voting as I wanted the engineers to have maximum agency here.

One thing I will change - the engineers recorded their votes in real time in the shared document. My gut tells me this led to a bit of hivemind developing that influenced some votes. It also gave an unequal amount of weight to those to voted last and chose to break ties. Next time, they’ll record their votes privately and paste their results into the shared doc simultaneously.

Budget

Now that you have a prioritized list of projects with estimated levels of efforts, talk with your product partner and figure out how much team bandwidth you can allocate to these efforts over the next quarter or half year, given product commitments.

Commit

Bring the bandwidth budget back to the team and commit to however many top projects you’re confident you’ll get done. Identify one or two stretch projects.

If the top priorities from the team are too large to commit to in the given time period, work with the Sponsor(s) to split thr project into multiple phases, each with their own deliverable impacts.

Plan

For the commited projects, have the Sponsor(s) task out the work so it is available for scheduling. Give them enough time to get this done without added stress.

Implement

Do the work! Stay true to your commitments. Celebrate the accomplishments.