Pull Request Completion Status¶
(Metric) for Tier: Product
Organizational Goal¶
Increase the efficiency and effectiveness of the process for performing code reviews.
Quantitative Goal¶
Analyze pull requests to ensure the efficient use of pull requests, proper use of the Azure DevOps tooling, and required team/role participation.
Visual Display of Measure¶
Metric Description¶
Pull request completion status is a metric to analyze pull requests to trigger root cause analysis and develop improvement opportunities with respect to team code reviews. It is meant to support consistent use of code reviews throughout the sprint with participation from all required and optional roles to increase the efficiency and effectiveness of team code reviews. If any of the following thresholds are triggered the Development Manager will preform a root cause analysis with support from the team and present the analysis.
- The total closed pull requests per day is larger than the number of developers on the team.
- Pull requests have a duration of over 48 hours excepting sprint events.
- The pull request running average duration trend is increasing.
- The pull request running average duration trend has spikes.
- Team/Role participation is below 95% for required reviewers and below 50% for optional reviewers.
- The number of “Did Not Vote” is greater than 10% of the total votes for a Team/Group.
Collection Method, Frequency and Storage¶
- Azure DevOps will automatically collect pull request completion data and use an extension to display the data for projects per Git repository.
- Developers are responsible for using the Azure DevOps pull request tools according to team norms and Azure DevOps documentation.
- Project Managers are responsible for taking a screen shot of the PR completion status page for each Git repository with changes for the release and the end of the last sprint of a release for a time period that includes all the development sprints in the release.
- Screen shots of the PR completion status should be stored in the projects SharePoint document library in a location defined by team procedures and included in the Monthly Status Review data.
- Any root cause analysis will be documented in the projects SharePoint document library in a location defined by team procedures.
- The data will be analyzed by Development Manager and the results included for the monthly status review.
Data Integrity¶
- Data will be collected by Azure DevOps automatically when executing the Integrate Changes activity.
- The Development Manager will evaluate the data integrity during data analysis, missing data, out of bound values, and unusual patters require root cause analysis.
Analysis Method¶
For the duration of the last release not to exceed the last 90 days take a screen shot of the PR completion status page for git repositories with changes for the release:
- Active pull requests should have a duration of less than the sprint length and ideally have a duration less than 48 hours, larger durations should be investigated for root cause.
- Pull request duration trend should stay stable or decreasing slightly, an increasing trend or spikes in the trend should be investigated for root cause.
- The number of closed pull requests should not spike and should average between 1-3 a day for each developer on the team, other values should be investigated for root cause.
- The approval by team/group should be above 95% for required reviewers and above 50% for optional reviewers.
- The “Did Not Vote” count should be less than 10% of the “Approve Votes” for Team/Groups.
Note
It is important to avoid pressure on individuals and only analyze the Team/Group participation and approval data.
Reporting Distribution and Frequency¶
See InnovaSystem’s Process Guidance RA for the following activities to determine which roles are included in the communication of the measurement data: