Visibility and Insight

It is important that we are transparent with our metrics and we do frequent introspections at certain intervals. A2F recommends introspections at the end of iterations, major releases, and milestones.

At the Team level, the following metrics and reports should be generated for Scrum and Scrumban Teams. The below Metric examples were generated from an ALM tool – JIRA Agile. In JIRA Agile, “Issues” are considered work items like User Stories, Technical Stories in a backlog. “Sub-Tasks” are considered Tasks below a Work Item. A unit of work done by a single person on the team and measured in hours. “Resolved” is the equivalent of “Accepted” for Work Items in the Sprint.

Sprint Health gadget

Pie Chart: Sub-Tasks

This gadget is setup to show the workload by assignee. This is only for the current Sprint and displays the number of Issues assigned to each Team Member.

Issue Statistics: Issue Count (Issue Type)

This gadget is setup to show User Stories, Spikes, Defects, and Tech Story counts by these Issue types. This metric displays the distribution spread of main issue types below the Epic level for the current active Sprint.

Sprint Burndown Gadget

This gadget is setup to show a Guideline, Time Spent (hours logged on issues in the Sprint), Remaining Values (Estimate Hours Remaining), and Time in calendar days. The Burndown Chart shows the actual and estimated amount of work to be done in a sprint. The horizontal x-axis in a Burndown Chart indicates time, and the vertical y-axis indicates hours.

Note: This gadget will require a separate Scrum Dashboard for the Project (i.e. ProjectABC) that should only be visible to the Scrum Master, so that it is does not cause confusion at the team level. This hidden Scrum Board is setup to track work in Hours. The metric will only display information when the Sub-Tasks for the Sprint have been estimated, and actual hours are logged to the Sub-Tasks.

Issue Statistics: Blockers & Dependencies (Issue Type)

This gadget is configured to display only those Issues that have the “Display on Impediment Board” custom field set to “YES”. This metric is intended to display only those issues that have a blocker or dependent type problem by Issue Type (i.e., User Story, Spike or Tech Story).

Issue Statistics: Sub-Task Status (Status)

This gadget is setup to display only those Sub-Tasks in the current sprint that are either in the “To Do” or “In Progress” status. The intent of this metric is to show the distribution and work load of Sub-Tasks that have not started to be worked on and those that are being worked on.

Created vs. Resolved Chart: Open Sprint Tracking

This gadget displays a chart showing the number of issues created vs number of issues resolved over a given period of time. The chart is based on the current active Sprint. There is also the blue trend line displaying either a positive upward trend if more issues are created than closed or the opposite if more Issues are closed than created.

Velocity

Velocity is determined by the number of user story points the Implementation Team completes and gets accepted in each sprint, but may also be estimated through calculation. At the end of each sprint, the stories completed and accepted by the Product Owner are added up to calculate the Implementation Team’s velocity for that Sprint. Over time, the average velocity is calculated by adding up all total velocities and dividing by the number of Sprints completed. The x-axis of the velocity per sprint chart shows the number of sprints and the y-axis shows the story points for the accepted stories in a given sprint.

At the Team level, the following metrics and reports should be generated for Kanban and XP Teams. The below Metric examples were generated from an ALM tool – JIRA Agile. In JIRA Agile, “Issues” are considered work items like User Stories, Technical Stories in a backlog. “Sub-Tasks” are considered Tasks below a Work Item. A unit of work done by a single person on the team and measured in hours. “Resolved” is the equivalent of “Accepted” for Work Items.

A cumulative flow diagram (CFD) is a tool used in queuing theory. It is an area graph that depicts the quantity of work in a given state, showing arrivals, time in queue, quantity in queue, and departure. Cumulative flow diagrams are seen in the literature of agile software development and lean product development. Some people consider a cumulative flow diagram to be a more sophisticated version of a "burn up chart", which is the opposite of a burn down chart. A burn down chart tracks work remaining over time while burn up charts like the CFD track the growth (or shrinkage) of work in certain states over time. With its focus on tracking changes in queue size per state, the CFD has a stronger focus on identifying and rooting out the causes of dramatic changes in throughput.

A Control Chart can show the cycle time or lead time for your product, version or sprint. The horizontal x-axis in a Control Chart indicates time, and the vertical y-axis indicates the number of days issues have spent in those statuses. A Control Chart helps you identify whether data from the current sprint can be used to determine future performance. The less variance in the cycle time of an issue, the higher the confidence in using the mean (or median) as an indication of future performance.

At the Program Portfolio level, all of these metrics would be rolled up across all Lean | Agile Teams.

It is recommended that you do introspections at the end of every Iteration, milestone and quarter. A part from reviewing the metrics, you want to capture what went well, what did not go well, and how to improve.  At the team and program portfolio level, Speed Boat and dot voting are popular techniques to identify issues. In either technique, you would then want to do root cause analysis on the top issues. Fishbone analysis (5 Whys) is a popular approach for root cause analysis.

At the Program Portfolio level, you will need multiple facilitators like an APM and Servant Leaders, since the audience will be much larger. A typical time-box of approximately 90 minutes should be allocated. Solicited feedback and discussions in the Program Portfolio Retrospective should be at the Program  Portfolio level, and not so much at the team level. Team level issues are addressed at the team level retros. The Attendees will be Program and Leadership stakeholders and representatives from the Lean | Agile Teams.

Capture and discussion of what went well and what did not go well in the Release (at the Program level). With multiple Lean | AgileTeams and a tight time-box, its suggested to solicit the feedback as much as possible prior to the Program Portfolio Retro. This way, the time can be spent on categorizing and ranking the top issues based on dot voting by the teams. Since there will be a tight timebox, it will be important for the Agile Program Manager to facilitate the dot voting with stakeholders and the Lean | Agile Teams. This is done by giving each person 3 dots to mark on the issue cards. The person can spread their dots across 3 issue cards, or put all dots on one issue card, or one dot on one issue card and two dots on another issue card.

The 5 Whys (Fishbone Analysis) will help Teams to understand how to improve for the next Release/Quarter. Suggestion is to have each Lean | Agile Team do the analysis for the same problem statement and then compare and discuss to get consensus:

Proposed resolutions to the issues discussed in these introspection sessions can translate into work items for the teams to implement in upcoming iterations and following weeks based on prioritization of the backlog. It is also important to give demos at certain intervals like at the end of an iteration, quarter or major release. At the Program Portfolio level, we call these demos Showcases because its across all teams and the audience is typically much larger. Feedback at these session should be filtered and prioritized by the Product Managers and Backlog Owners.