Save time with unbiased, independent feedback on vendor solutions.
Watch weekly bite-sized webinars hosted by IANS Faculty.
Phishing resilience is hard to measure, and most platforms focus on measurement of a campaign in terms of employee success or failure at a single point in time. To accurately gauge an organization’s ability to handle phishing attacks, metrics are required that demonstrate changing trends over time, identify problem areas and discern between different sophistication levels of phishing.
This piece explains how to build an organizational phishing metrics matrix to better gauge organizational resilience and identify areas of concern to strengthen your security awareness program.
Tracking phishing metrics on a team/group level enables both the security team, as well as managers across the organization, to identify resilience across different areas. The metrics should also take into account that different organizational functions have different risk levels, exposure to phishing and access to sensitive information.
To solve the first issue of identifying problem areas (individuals, groups, etc.), consider creating a “serial clickers” metric. This will track anyone who’s clicked on a link in more than three consecutive phishing campaigns. This set of metrics applies across all other sets and will help direct assistance (in the form of training, awareness, intervention, additional controls, policies, etc.) toward the problem areas as the increased risk is recognized.
You should also look to capture the other end of the spectrum: the “serial reporters” who are able to successfully identify and report more than three consecutive campaigns. This metric shows increased awareness and resilience and can be used to highlight the individuals or teams that exhibit such behavior.
The next step is to create at least three different levels of phishing templates:
Creating these levels allows you to measure phishing resilience across additional dimensions beyond success/failure and adjust the sophistication of the phishing campaigns used for different types of internal groups based on their sensitivity or progress in the awareness training. The level changes can be applied to individuals, as well as to whole teams.
Next, you should measure the following key indicators across the campaigns:
You may also wish to measure “engaged,” which is when a person clicks on a link in the email and engages with the phishing website or downloads an attachment.
The goal, of course, would be to see mostly ignored, opened and reported metrics (with a bias toward reported as an indicator of a more proactive level of security posture that is likely elevating others in the department). In more advanced settings, you can also measure reporting across out-of-band channels, such as Slack or email to non-security addresses that warn about the suspicious emails.
Once you collect the metrics, you can report them in a matrix format that clearly identifies areas of investment or reward (see Figure 1).
Figure 1: Example of a Phishing Metrics Matrix
Source: IANS, 2022
The matrix makes it easy to identify the different levels of phishing sophistication targeting each department, as well as each department’s ability to identify and respond to those attacks. Tracking those metrics over time enables security teams to take specific actions (educational, disciplinary, communications, etc.) toward the right departments and track the efficacy of those actions.
Organizations can then treat each class of action as a detractor or a promoter of phishing resiliency, where “ignore” and “open” are neutral, “report” is a promoter (i.e., better resiliency), and “clicked” is a detractor. They can also define the weights of promotion/detraction based on the type of organization and its risk posture.
READ: 10 Ways to Identify a Phishing Email
Tracking which groups are susceptible to which types of phishing campaigns over time can be difficult, and tracking the results of various interventions (education, punishment, etc.) makes the process even more complex. Creating a metrics matrix can help clarify the process. To get started:
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in
connection with such information, opinions, or advice.
September 19, 2023
By IANS Faculty
Discover the diversity of IANS Faculty's real-world expertise. Learn how our faculty members can help you solve your most challenging security issues.
September 14, 2023
Learn how to use a three-step approach to defending and managing public and private APIs while avoiding common mistakes.
September 12, 2023
Understand the main differences between first- and second-gen SAST tools and learn how to determine which will work best for your environment.