| # |
Metric |
Metrics Used |
Metric Description |
Metric Definition |
Currently Reporting? |
Data Source |
Scoring |
Computation Level |
Metric Definition 2 |
| 1 |
Roadmaps |
L1 Roadmaps Up-to-date |
# of L1 roadmaps updated in last 30 days from the total count of L1 Roadmaps expected |
# of L1 roadmaps updated in last 30 days from the total count of L1 Roadmaps expected |
TBD |
TBD |
TBD |
TBD |
Is L1 roadmap updated within reporting month for PL |
| % L2 Roadmaps Up-to-date |
# of L2 roadmaps updated in last 30 days from the total count of L2 Roadmaps expected. |
# of L2 roadmaps updated in last 30 days from the total count of L2 Roadmaps expected. |
TBD |
TBD |
TBD |
TBD |
Are L2 roadmaps updated within reporting month for each SPL; |
| % Highest Level User Stories to L2 Traceability |
# Highest level user stories attached to an L2 Roadmap expected for each PL/SPL. This excludes highest level user stories for backlog items |
# Highest level user stories attached to a L2 Roadmap expected for each PL/SPL. Highest level user story is defined as user story with children linked to a L2 roadmap item or not linked or a user story with no children but linked to a L2 roadmap item, or user story with no children and no parent but slotted to either an iteration or release. Since highest level user stories are not DT specific the values are calculated and rolled up at an SPL or PL level. |
TBD |
TBD |
TBD |
TBD |
Highest level user story is:
(1) a story with a parent that is an L2 Roadmap item
(2) a story with children and no identifiable parent
(3) an orphan user story with no children associated with a release or iteration. |
| 2 |
Team Commitments |
Iteration Say / Do Ratio |
Total story points Accepted for the latest completed Sprint / Total Story points committed for the latest completed Sprint |
Total story points Accepted for the latest completed Sprint / Total Story points committed for the latest completed Sprint. The calculation is applied at the DT level. The Say/Do percentage at the DT level will be averaged to obtain values at the SPL and PL levels. |
TBD |
TBD |
TBD |
TBD |
(Total Story points Accepted / Total Story points Committed) for last iteration within reporting month; Averaged to SPL and PL |
| QTD % of Sub Product Lines meeting technology goal of 20%+ |
QTD Total ‘Technology stories’ Story points Accepted / QTD Total story points Accepted. If the sub-product lines meet technology stories story point criteria of >20% full credit is given to them. If not, no credit is given. |
QTD Total ‘Technology stories’ Story points Accepted / QTD Total story points Accepted at the end of the sprint. If the sub-product line meet technology stories story point criteria of >=20%, full credit is given to them (value of 1). If not, no credit (value of zero) is given. The technology user story points criteria of >=20% is a quarterly at a SPL level. The values at PL level are obtained by averaging the percentages of the SPL. |
TBD |
TBD |
TBD |
TBD |
(Technology Story points Accepted / Total Story points Accepted) for all iterations within calendar Quarter; SPLs that meet Technology goal of >=20%, then 1, else 0; % of SPL that meet goal at PL |
| 3 |
Scrum Team Composition |
% of Teams Co-Located in Sub-Product Line |
Count of delivery teams where all members are in the same city / Count of the total number of delivery teams |
Count of team members in a DT where maximum members are in the same city / Count of total team members in the DT. For e.g. If a delivery team has 10 members and 5 of them are in the same city then that team is 50% co-located. Team members include PO,SM and delivery team members |
TBD |
TBD |
TBD |
TBD |
(Highest # Team members in same city / Total # Team members); Team includes PO & SM; Averaged at SPL and PL |
| Team Stability in Sub-Product Line |
During last Quarter, average of scrum team stability per sub-product line (the # of Scrum team members - PO, SM, team contributors, who stayed on their team / total number on team) * (% confirmed scrum team members by product line). It is denoted in %tages. |
Average of scrum team stability per sub-product line (the # of Scrum team members PO, SM, team contributors, who stayed on their team / total number on team + additions to the team) * (% confirmed scrum team members by delivery team). It is denoted in %tages. The values are calculated at DT level and averaged to get values at product line and sub-product line level |
TBD |
TBD |
TBD |
TBD |
(# Team members remaining in DT / (Total # team members + Additions) * % Team members confirmed in DTM for last iteration within reporting month); Cumulative monthly average by DT; Averaged at SPL and PL |
| 4 |
Agile Maturity Score |
Agile Maturity Score |
Agile Maturity Score (TBD) |
|
TBD |
TBD |
TBD |
TBD |
AMS based on team's self assessed response to monthly survey |
| 5 |
Training & Coaching |
% of people trained in CDI |
# individuals trained in CDI based on the total # of people involved in the Sub-Product Line. |
# individuals trained in CDI based on the total # of population eligible for CDI training (UED, product owners and regional solutions managers) |
TBD |
TBD |
TBD |
TBD |
(# Team members trained in CDI / # CDI training population); Averaged at SPL and PL |
| % of people Trained in Basic Agile |
# individuals trained in Basic AGILE based on the total # of people involved in the Sub-Product Line. |
# individuals trained in Basic AGILE based on the total # of people involved in the Sub-Product Line. It is calculated at a delivery team level and averaged at a sub-product line and product line level |
TBD |
TBD |
TBD |
TBD |
(# Team members trained in Basic Agile / # Total team members); Averaged at SPL and PL |
| of Scrum Masters trained |
# of Scrum masters trained in Scrum Master Skills based on the total number of Scrum Masters in the Sub-Product Line. |
# of Scrum masters trained in Scrum Master Skills based on the total number of Scrum Masters in the Sub-Product Line. It is calculated at a delivery team level and averaged at a sub-product line and product line level |
TBD |
TBD |
TBD |
TBD |
(# Named Scrum Masters trained in Scrum Master Skills / # Total Scrum Masters in SPL); Aggregated at SPL and PL |
| % of Product Owners trained |
# of Product Owners trained in Product Owner Skills based on the total number of Product Owners in the Sub-Product Line. |
# of Product Owners trained in Product Owner Skills based on the total number of Product Owners in the Sub-Product Line. It is calculated at a delivery team level and averaged at a sub-product line and product line level |
TBD |
TBD |
TBD |
TBD |
(# Named Product Owners trained in Product Owner Skills / # Total Scrum Masters in SPL); Aggregated at SPL and PL |
| Coaching Support |
Degree of Coaching support provided to each SPL (High, Med, Light, None) |
Degree of Coaching support provided to each SPL (High, Med, Light, None) |
TBD |
TBD |
TBD |
TBD |
Coaching coverage across SPL; Not part of scoring |
| 6 |
Engineering Practices and Tools |
Continuous Integration Maturity Score |
# of teams with Level-4 CI Maturity Score |
% of teams within SPL and PL who achieved level 4 for continuous integration. A grading system that reflects the maturity of a delivery team’s continuous integration solution against a technology standard pattern. Technology has a goal to attain level 4 maturity across all teams. |
TBD |
TBD |
TBD |
TBD |
% Teams within SPL that attained Level 4 CI Maturity |
| 7 |
Quality |
% of code coverage using unit tests |
# of lines (functions / conditions) of code is covered when unit tests are run |
# of lines (functions / conditions) of code is covered when unit tests are run. This percentage is calculated at the SPL level and averaged to obtain values at PL levels. |
TBD |
TBD |
TBD |
TBD |
(# of lines (functions / conditions) of code covered when unit tests are run / total # of lines of code); Averaged at PL |
| % of regression testing automated, |
# of regression test cases automated vs manual |
# of regression test cases automated vs manual. This percentage is calculated at the SPL level and averaged to obtain values at PL levels. |
TBD |
TBD |
TBD |
TBD |
(# of automated regression test cases / total # regression test cases); Averaged at PL |
| # of open bugs on live |
# of live bugs that are open and not closed |
# of live bugs that are open and not closed. This percentage is calculated at the SPL level and averaged to obtain values at PL levels. |
TBD |
TBD |
TBD |
TBD |
# live bugs that are open and not closed at SPL and PL. This metric reduces the quality score achieved by unit test/regression tests. Monthly target set by CQ.
* At or below target = no reduction.
* within 10% of target = 50% reduction.
* above 10% of target = 100% reduction. |
| BIR |
This is 8 weeks average of bugs introduced on live. It excludes invalid, dups, won't fix. |
This is 8 weeks average of bugs introduced on live. It excludes invalid, dups, won't fix.The BIR percentage is calculated at the SPL level and averaged to obtain values at PL levels. |
TBD |
TBD |
TBD |
TBD |
8 week rolling average of bugs introduced on live; excludes invalid, dups, won't fix bug types. This metric reduces the quality score achieved by unit test/regression tests. Monthly target set by CQ.
* At or below target = no reduction.
* within 10% of target = 50% reduction.
* above 10% of target = 100% reduction. |
| 8 |
KPI |
KPI |
TBD |
|
TBD |
TBD |
TBD |
TBD |
SPL achievement of goals for the month compared to stated goals for the month |