In our last blog, we explained how aligning the stages of the maturity model with your stakeholders' CTI requirements positions you to adopt a series of metrics to measure your effectiveness across each EclecticIQ CTI Capability Maturity Model pillar:
This blog defines an approach that maps metrics to your stakeholders’ requirements. We call this approach the Pyramid of Gain (PoG) (see Figure 1). And yes, this intentionally resembles the well-known Pyramid of Pain. While the Pyramid of Pain helps you shift your focus from IOCs to TTPs to better detect and respond to threats, the Pyramid of Gain enables you to progress from responsive to pre-emptive threat operations to better meet your stakeholders' needs.
Figure 1 - The EclecticIQ Pyramid of Gain (PoG) (click the image to zoom in)
The PoG depicts moving from lower to higher maturity through the five stages of the CTI-CMM. The five stages are grouped into tiers on the pyramid. The bottom tier is responsive CTI maturity, equating with Stages 1 to 3. The middle tier is predictive CTI maturity, equating with Stage 4. The top tier is pre-emptive CTI maturity, equating with Stage 5.
At each tier, your CTI team supports a different set of stakeholders:
Moving "up" the pyramid incrementally expands your CTI capabilities from responsive support of operational stakeholders to predictive support of tactical stakeholders to pre-emptive support of strategic stakeholders. At each tier, we recommend a series of effectiveness and performance metrics. Generally, the higher the level, the more your team focuses on effectiveness metrics. However, both types of metrics are essential for successful CTI operations.
EclecticIQ is taking a novel approach by precisely aligning metrics with your stakeholders' needs. We recommend picking a subset of metrics for each tier that best aligns with their goals. Many of these metrics draw on excellent work by several practicing threat analysts and researchers. [1]
Align – Your threat reality alignment is quite limited at this tier. You may have defined PIRs, but many of your actions are entirely reactive, ad hoc, and operational (e.g., firewall block lists). Key metrics include:
Understand – At this tier, your CTI team supports your operational CTI stakeholders. Most of your efforts are focused on providing IOCs to your SecOps team and updated vulnerability data to your vulnerability and fraud operations teams. Key metrics include:
Act – Here, your emphasis is on reactive controls that attempt to block ongoing attacks, for example, feeding lightly filtered IOCs to your firewalls and endpoint protection controls. Your SOC tracks MTTD as a primary performance metric. Key CTI metrics that support your ability to act (and bring down MTTD) include:
Number of intelligence packages delivered
Percentage of packages with context/confidence
Rate of disseminated intelligence targeting specific stakeholders
Frequency of integrating multiple IOCs
Number of sightings
Number of detected vulnerabilities
Percentage of new incidents discovered from sightings (e.g., false positives)
Align - When you rise to this tier, you have procedures in place to briefly catch your breath by shifting from a reactive to a more proactive stance. At this point, you have well-formed PIRs, and your CTI team is receiving regular feedback to adjust the PIRs to reflect the shifting threat landscape. Further, you are including CTI in ongoing business and mission decisions. Key metrics at this level include:
Understand – Your CTI team provides more actionable data based on more-robust feeds, IOCs, and case management practices. This CTI maturity tier makes it easier to support your tactical stakeholders with intelligence for conducting incident operations and looking for "unknown unknowns" via threat hunting. Critical metrics at this level include:
Act ¬– At this tier, your organization is shifting to a more proactive stance by enabling incident response, incident management, and threat hunting. You are leveraging context and insights from the CTI team to provide your response and hunting teams with actionable data. You are paying close attention to MTTR as a crucial operational metric. Other key metrics that support your ability to act (and reduce MTTR) include:
Align – At this level of the pyramid, your business and mission are so closely aligned with your threat reality that you can become pre-emptive. For example, leveraging CTI that includes TTPs and projected threat actor targets, you may decide not to shift your operations from on premises to a recently breached cloud provider. Metrics at this level include:
Understand – Your team has raised the maturity bar to a level that provides pre-emptive threat management, giving high-level stakeholders insights that help achieve comprehensive situational awareness and risk management. Effectiveness metrics for understanding at this level include:
Act – Here, you are disseminating actionable CTI to your partners and supply chain vendors, and integrating CTI into your controls and workflows with context, priority, specific courses of action, and intelligence that is meaningful to your highest-level stakeholders. For example, at this level, your threat modeling (i.e., red/blue and purple teams) includes MITRE ATT&CK TTPs from targeted threat actors to pre-empt future attacks. Metrics that support your ability to act pre-emptively include:
Yes, we just provided nearly 50 metrics to implement to track your CTI maturity progress! We recognize that implementing and tracking metrics is a significant undertaking. On the other hand, so is evolving your CTI maturity. The important point is that this is a long journey that starts with the commitment to take the journey. Most organizations will only implement a subset of these metrics. However, any metric adoption is a move in the right direction. In our next blog, we map six steps to begin your CTI maturity adventure.
[1] Gert-Jan Bruggink, Freddy Murstad, and “How to Get Promoted: Developing Metrics to Show How Threat Intel Works” - SANS CTI Summit 2019, https://www.youtube.com/watch?v=-d38C3992aQ