Blog

“How bad is it?”

When asked to assess or take over management of a project in full swing, you need to be able to quickly figure out whether a project is on track, or how far off track it is if not. A problematic project will be weak in one or more of the three pillars of a good project:

• People (does the project have effective leadership, necessary internal resources, good external expertise?)
• Process (is the plan properly organized and at the right scale? Are proper techniques being used?)
• Technology (is the project using the right package / right tools?)

But before you can diagnose the problem, you need to determine if there is a problem to be diagnosed. There are a few basic steps to making that first assessment:

• Understand the business case and plan: what are the objectives and the expected financial return; what is the proposed scope, team, timeline and cost
• Understand the status: what is the project reporting against progress to plan and budget
• Understand the issues: what are the challenges facing the project that have delayed it (or may delay it going forward)

A project team that is reporting a ‘green’ status may still be going off the rails. One of the quickest ways to identify an out of control project or development team is to dive into the open issues.

Assessing a Project Through Issue Review

In an earlier post, I described what successful issue management for a large ERP project looks like.  Read that first – you can learn from failure, but you learn a lot more if you are comparing that failure to a successful model. If issues are being managed well, the team should be able to answer the following questions with trivial effort:

1. What are the most important open issues?
2. When will each be resolved, and what issues are expected to be resolved next?
3. What is the trend in total count, issues opened, and issues closed?
4. Is the full team in agreement on issue priority and closure status?
5. Are resolutions well documented?

1. What are the issues?

Every team needs a clear picture of the most important issues. In a high performing team, this is trivial, and anyone can easily run a report ranked by priority to see current status. The first step in assessing a team’s handle on issue management is to ask for a current view of the most important issues. What to evaluate:

• The team should be able to run a report from the issue management system ranked by priority (with a meaningful sub-sort such as major module or expected resolution date for large projects)
• Assigned priorities should make sense. Critical open issues should all be real show-stoppers; high open issues should be a significant pain but have a live-able workaround; medium issues should be noticeable but either affect only a small number of users or have smaller impact on all users; low issues should be minor annoyances. There should be at least four distinct priorities, but not many more than that
• Count of issues should make sense given size of team; a five person development team with hundreds of open issues is in trouble by definition
• Operational requests (new users; tweaks to configuration settings) should be clearly separated from software and process issues

In an out of control project, issue priorities will not be tightly managed, and the most important issues may not be documented at all. An overwhelmed team will use other attributes (client affected; issue type; subsystem affected) as proxies for priority as they struggle to meet this week’s emergency; or they might begin using email lists to schedule work without looking at the issue list at all. Users, QA or Product Management staff will label too many issues critical as they struggle to get development to focus on their issues.

2. What is the plan to close issues?

The team needs an achievable plan in order to make predictable progress. Issues should have assigned resolution dates, and the team should be meeting a large portion of the date commitments. The next level of issue assessment is to understand the plan for resolution and whether the team is meeting that plan. What to evaluate:

• Do all critical, high and perhaps medium issues have an expected resolution date? The team should be working on and resolving issues in rough priority order. There are always exceptions – for example, it may be more efficient for a developer to fix all issues in a module rather than only the critical issues; a lower priority issue may be particularly visible to an important stakeholder. However, in general issues should be worked in order of importance.
• Are the dates accurately re-forecasted when missed? A team that has given up meeting commitments may just never update expected resolution dates. A team that misunderstands ‘Agile’ will roll all critical and high issues into the next sprint without doing the work to understand what can really be accomplished in that sprint. In the first case you will see many past due issues; in the second you will see many high issues, all of which are forecast to close shortly.

A high performing team sets goals and meets commitments. The ability to set and meet issue resolution dates gives good insight into whether a team is likely to be on track with all its development tasks.

3. What is the trend?

Looking at current open issues only tells part of the story. To understand how the picture is likely to change in the weeks ahead, team leaders (and anyone assessing team leaders) should be actively monitoring issue metrics: # of open issues of each priority; count of issues opened; count of issues closed. What to evaluate:

• Does the development &/or QA lead publish these figures weekly? If not, you have no reason to believe delivery dates the team is forecasting. They are hoping that the current known issues represent the bulk of issue resolution work outstanding, but don’t have the data to confirm or refute it.
• Does the team have a defined standard to meet for release? The team should have a hurdle (agreed to before release date approaches) for what constitutes a releasable product.
• Does the trend match the top line status that the leads are sharing? If a team is in the middle of a build phase, it is fine if the count of issues is stable or rising slowly. Lots of issues opened and closed is a sign that the team is tracking progress diligently. If a team is approaching release, the average severity and total count of open issues should be predictably approaching the defined hurdle. If metrics are not trending this way, there is no way to know how many issues lie behind the blocking issues that have been reported to date.

4. Is the team working well on issue resolution?

When all members of the team are pulling in the same direction, issue management should reduce the workload on the team. Issues are reported fully and most do not require a meeting; an issue reporter knows when the issue will be resolved and can track the interim status; status alerts inform users when issues are ready to verify or sign off on.

By participating in or observing an issue meeting you can quickly identify a team not pulling in the same direction.  Evaluate:

• Is development overriding end user priorities? It is possible that an end user just does not understand the levels, but as a rule the person closest to the customer should understand priority best.
• Are issues entered once? Other than closing issues as duplicates, in general issues should be entered and tracked to resolution under the issue the discoverer enters.
• Are closed issues staying closed? In general, issues should be closed with a small number of states, such as: resolved, duplicate, could not reproduce and not an issue. Most issues should be getting closed as resolved. A small number of issues may be closed in the other states. ‘Re-opened’ issues should be tracked and if many issues are being ‘re-opened’ some process change &/or training should take place to eliminate the waste in the process

5. Are resolutions well documented?

A good issue management process serves both as a control and a communication system. By including well-documented resolution comments, the team ensures that the person opening an issue can see (and agree) that an issue was satisfactorily resolved. It also increases transparency within the team, and allows QA or end users to give feedback if they see patterns in root causes.

To be honest, most teams document resolution well. However, this bit me on a recent project; I am including it so that I don’t forget to train the next team at the start that resolution comments are important.

Summary

When called in to manage or review a project midstream, you first need to do the work to understand the objectives, the plan and current status on meeting that plan.  Diving deep into the issue management data and process can give you a good sense on whether the team is on track to meet their current plan or is likely headed for trouble.