7 Vital Metrics DevSecOps Need to Be Monitoring

For many software development teams, the move from DevOps processes to more secure DevSecOps operations may involve a significant shift in the way they operate.

But while there are clear operational benefits to going down this route, such as ensuring that the push for greater speed and efficiency in the development process doesn't leave security testing behind, how can you be sure that the time and investment you're putting into this is paying off? The answer is with metrics.

Why you need to be measuring DevSecOps

Keeping a close eye on key metrics doesn't just tell you whether or not you're meeting your software development goals for a specific project. It can also guide your future planning by telling you which elements of your strategy are working well and where you need to refocus your efforts.

Some of the basic questions that an effective metric measurement plan can answer include:

• Is the software operating properly?
• Is the organization achieving its business goals?
• Is the application secure?
• Is the infrastructure able to support the demands of the application?
• Can it be effectively scaled up to meet future requirements?

However, it can also give you insight that can be applied to future projects. For example, positive, consistent metrics can indicate that a process is reliable and repeatable - or not, if your metrics show unexpected variations - or identify the best points in a product's lifecycle to make changes.

7 key DevSecOps metrics to put at the heart of your monitoring

You won't be able to gain any of this insight unless you're monitoring and measuring the right DevSecOps KPIs. With this in mind, here are seven you shouldn’t ignore.

1. Availability

This measures the uptime or downtime of the application, either as a percentage or as a time value. This is the most critical indicator of the reliability of the solution, taking into account both planned and unplanned maintenance. Ideally, this needs to be as close to 100% (or zero minutes) as possible. As well as informing you of any issues with the resilience of the application, it relates directly to service level agreements for customers, which will usually specify a minimum performance for availability.

2. Issue resolution time

If you do have downtime to fix a problem or make an update, the time taken to resolve any issue and get back online is vital. This shows how effective your team is at identifying a problem, coming up with a solution and then deploying it. Long resolution times could indicate there are too many complexities or variables to consider, but fast fixes are essential in remedying any security vulnerabilities.

3. Application deployment frequency

This looks at how often you make deployments into production within a given time frame. However, this isn't a metric you can take in isolation - what is regarded as a success depends on context. For example, if you already have a proven and tested product, a low frequency may be desirable as it indicates you have a stable platform that needs less downtime. However, for new applications, you may want a high frequency to make changes or additions on a more regular basis.

4. Application change time

Used in conjunction with deployment frequency, change time monitors how long it takes between a code being committed and the changes appearing in production. This gives you a clear insight into the efficiency of the software development pipeline - including the time taken to build, test and release an update. Generally speaking, shorter time frames indicate a more efficient environment, but you'll need to look at this alongside other metrics to ensure you aren't sacrificing quality for speed.

5. Change failure rate

One way to tell if your DevSecOps processes are functioning well is to look at the overall failure rate for changes. How often do deployments to production have to be recalled or changed because they don’t function as intended, or introduce new vulnerabilities into an application? If you do have a higher than normal failure rate, this could be a sign that something isn't working within your team, whether this is communication, confusion over what everyone's responsibilities are or unclear operational goals.

6. Time to patch

This metric covers the time taken between identifying a vulnerability and deploying a fix to production. While it is similar to issue resolution time, it’s especially relevant to DevSecOps teams as it offers a more granular insight into your team's ability to troubleshoot security problems as opposed to other bugs and defects. If you're making security a priority, this can tell you whether or not your team has the skills and resources needed to succeed.

7. Time to value

Finally, time to value reflects the actual business outcome of the DevSecOps process - namely, how long does it take for a feature of an application to go from the planning stage to actually delivering a benefit for end-users in production? This can be hard to define, which is why it's important to set out clear, relevant business goals for every project, such as generating additional revenue, adding new functionality or making the business more competitive.