Resilience Coverage

What it is

A measure of overall protection a pull request has against bugs that can impact large portions of your user base and/or expose observability gaps.

How it works

Resilience Coverage is calculated on subsequent PR updates by tracking the completion of a customizable set of mitigations mapped to Risk Score thresholds. Generally speaking, the riskier a change is, the more mitigations are needed.

Shepherdly allows teams to scale mitigations with risk in a way they deem appropriate. For example, requiring integration tests (among others) to be added if the Risk Score is above 70 / 100.

Example of how mitigations can be mapped to Risk Score thresholds at the repository level.

Why it’s important

At its core, Resilience Coverage is an easy to understand metric that gives engineers clear guidelines, credit for doing the right thing, and objective data to justify time spent increasing the resilience of the codebase.

High Resilience Coverage encourages a proactive approach to software development that leverages best practices while using the precision of a predictive model trained on your unique codebase to apply them. In simple terms, Shepherdly is reflecting back what has and has not worked in the past and distills that knowledge down to a Risk Score + coverage goal.

By tracking this metric over time, teams can measure their progress in improving the resiliency of their codebase. Conversely, it can also highlight if a team is moving too fast and cutting corners unnecessarily.

Completing Mitigations & Auto Detection

Similar to checklists in PR templates/descriptions, any mitigation can be marked as completed. Completing required items will increase the coverage percentage.

Through AST parsing, Shepherdly can detect many of the common best practice mitigations like automated tests, feature flags, and observability, among other to alleviate manual effort.