1. Introduction

Assessment is a process that allows one to determine whether the implemented measures provide the expected impact and therefore contributes to establishing cause-and-effect relationships between actions and results. One of the fundamental issues in the field of cybersecurity is the assessment of the effectiveness (the degree of completeness of the realised impact) of the implemented cyber defence measures (countermeasures against cyber threats) that is conducted to check the validity and usefulness of such measures while mitigating cyber risks, as well as for the further adjustment of the organisation’s general cybersecurity strategy. In this context, the determination of the organisation’s approach to the assessment of cyber threats as well as their identification and analysis are among the main tasks of the risk management process.

Cyber threat assessment is an actual and popular area of scientific research because both the subjective and objective multivariate interpretation of the risk concept itself creates prerequisites for the absence of a uniform approach to its assessment and defining the main factors of direct influence. As of today, the organisation of the process of cyber threat assessment in conditions of limited contextual information and data (resulting in the inaccuracy of such an assessment), the determination of typical cyber threat characteristics that can be used during cyber threat assessment in conditions of such limitations, the instability of cyber threat landscape (resulting in the need for periodic risk factors (indicators) revision in order to maintain the relevance of such assessment) are among the typical problems in this field.

Common ways to solve such problems are the adaptation of popular methodologies and specific methods of cyber threat assessment (which are almost always used not separately, but in the context of risk definition as a more complex concept) and the creation of individual adapted methodologies or methods of the cyber threat score formation, that is the topic of this work.

2. Theoretical Background

2.1. Literature Review

Currently, there is a research gap related to conducting cyber threat assessments based on network traffic, as most studies focus on cyber risk assessment, which is a more complex and comprehensive topic. More than that, according to the analysis of popular and scientific publications on the topic of cyber threat assessment based on network traffic, such assessments are not conducted solely using the indicators derived from network traffic analysis in any of the reviewed works. This is primarily because network traffic can be considered one of multiple data sources for such assessments [1–4], but a cyber threat assessment is a more complex process in general. At the same time, the need for the formation of quantitative indicators, even with limited resources and data [5], is confirmed by the active implementation of such indicators by well-known cybersecurity vendors [6–9] for making managerial decisions.

An explanation of the method of conducting cyber threat assessment based on indicators determined from the network traffic analysis results in combination with the data about vulnerabilities of organisation’s assets is given in [10]. Research on the development of a methodology for forming a quantitative score representing the network security situation that is based on attack prediction algorithms is also quite common, for example, Hu et al. [11].

Publications related to conducting cyber threat assessment that is not based on network traffic (but in a related context) were also considered during the analysis [12–16]. They helped to more accurately interpret the theoretical interdependence of cybersecurity, cyber risk, cyber threat, and cyber defence indicators, the values of which are often determined or calculated based on the expression of one through the other.

In particular, the methodology [12] describes the dependence of the nature of a cyber threat on indicators of the state of society relations and confirms the relationship between the cyber threat and cybersecurity levels in such a way that the cyber threat level is a criterion for assessing the cybersecurity level. It is also specified that the criterion for assessing the cyber threat level should be mainly based on the nature of the cyber threats and requires the consideration of their scale. Taking into account that organisations’ countermeasures against cyber threats of various risk levels differ in the level of cyber attack neutralisation it can be concluded that the level of cyber attack neutralisation (cyber defence indicator) can be considered a criterion for assessing the cyber threat level.

A method for evaluating the effectiveness of measures aimed at ensuring the cybersecurity level of organisations’ critical information infrastructure objects is proposed by Pyskun et al. [13]. While evaluating the effectiveness (along with the cybersecurity, system functional capacity, and cyber resilience indicators), the cyber risk probability indicator is proposed to be taken into account, which is determined as a combination of the cyber attack probability (that, in turn, depends on the cyber defence level) and its potential impact (amount of possible damage). Also, the criteria for assessing the cyber risk probability, cyber defence, potential impact, and the cyber attack probability are proposed with generalised recommendations on how to determine the levels by calculating the scores (without specifying the method of establishing the unambiguous correspondence of the calculated scores to specific criteria). On the one hand, such an approach makes the methodology more multi-purpose due to the lack of dependence on specific methods of calculating the scores, but on the other hand, it creates grounds for doubting the correctness of the correspondence of the calculated scores to specific criteria due to the same non-determinism of the methods of scores calculation and the lack of a described verification mechanism. In addition, this non-determinism has several levels of impact – firstly, on determining the correspondence with the criteria for the cyber attack probability and evaluating the amount of damage, then on the resulting cyber risk probability score.

In summary, the analysis of recent research publications confirms:

  • the functional dependence between cyber security, cyber risk, cyber threats, and cyber defence indicators, which is relevant for understanding the applicability of the proposed approach to network cyber threat assessment in the context of determining its relationship with the other indicators. At the same time, based on the generally accepted functional dependence definition, the value of one indicator (independent or input) affects the value of another indicator (dependent or output). In our case, the definition of dependent and independent indicators is not static but varies according to the problem statement (definition of the main goals and objectives of the research, that must be completed in order to achieve these goals) and the available input data, which are the basis for further calculations.

  • the need to define an unambiguous approach for the realisation of every sequential stage of the assessment methodology, or to apply such a level of generalisation in relation to possible approaches that would not create prerequisites for doubts about the correctness of the results obtained at different stages and at the same time would allow a certain level of abstraction (i.e., with the possibility of flexible approach adaptation depending on individual factors).

2.2. Discussion of Common Cyber Risk Factors

Cyber threats, vulnerabilities, impact, likelihood, and predisposing conditions are typical cyber risk factors (according to [17–20]). Cyber risk factors can be decomposed in greater detail (e.g., cyber threats decomposed into cyber threat sources and cyber threat events) before conducting a cyber risk assessment to take into account a greater number of relevant attributes, which, in turn, contribute to increasing the objectivity of such an assessment. Therefore, cyber risk factors are characteristics used in cyber risk models as inputs to the cyber risk assessment process.

Figure 1 represents the cyber risk model based on the typical factors that are used in the work.

Figure 1

Cyber risk model.

https://www.acigjournal.com/f/fulltexts/190345/ACIG-3-013-g001_min.jpg

Taking into consideration that network cyber threat events form the only data source for the assessment, it is more appropriate to consider cyber threat (rather than cyber risk) assessment due to the lack of metrics that could define important cyber risk factors (such as vulnerabilities and predisposing conditions). The terms ‘cyber risk assessment’ and ‘cyber threat assessment’ are often used interchangeably, but in fact, they refer to distinct processes. While both assessments complement each other and are essential components of a robust cybersecurity strategy, they serve different purposes and provide different insights. A cyber risk assessment offers a comprehensive view of an organisation’s overall cyber risks, while a cyber threat assessment provides a focused analysis of the specific threats and threat actors targeting the organisation.

2.3. Terminology

The terms used in the work, that have an interpretation different from that given in NIST or ENISA glossaries, are described by the following definitions (taking into account [21, 22]):

  • organisation’s network cybersecurity domain – a set of the organisational assets and resources that are the objects of the network cybersecurity policy of the organisation;

  • network traffic – data (encapsulated in network packets) moving between individual hosts or nodes within the network;

  • network traffic monitoring and analysis tool – a software, hardware, or software-hardware solution whose functionality allows the usage of signature or anomaly analysis methods to detect network cyber threat events in network traffic;

  • log management tool – a software, hardware, or software-hardware solution whose functionality allows the transmission, storage, analysis, and deletion of logs obtained from the network traffic monitoring and analysis tool (-s);

  • network cyber threat event – an information security event detected by the network traffic monitoring and analysis tools, that means the detection of an indicator of attack or an indicator of compromise in network traffic (that is, an attempt or the fact of the network cyber threat realisation), classified according to the taxonomy of network cyber threats and characterised by criticality and the likelihood of successful realisation;

  • indicator of attack (IoA) – a proactive indicator that determines the procedure, technique, tactic (TTP), according to which a network cyber threat can be successfully realised;

  • indicator of compromise (IoC) – a reactive indicator that identifies a network-level artifact (classified according to the list of types of network-level artifacts), that indicates the fact of the successful network cyber threat realisation;

  • network cyber threat – a threat that is identified through the characteristics of a network cyber threat source and a network cyber threat event (or a set of such events), the successful implementation of which involves the occurrence of undesirable consequences (harmful impact).

2.4. Conceptual Model of the Organisation’s Network Cybersecurity Domain

Figure 2 represents a conceptual model of the organisation’s network cybersecurity domain, considering the external and internal cyber threat surfaces. Important relationships between the entities reflected in such a high-level concept are:

  • conducting cyber attacks as a way of external and internal cyber threat realisation by cyber threat sources (in the context of this work cyber threats initiated by adversaries are considered);

  • transferring of network cyber threat events to the log management tool, where they are analysed for the purpose of classification and realisation of additional calculation operations (in particular, calculation of the Network Cyber Threat Score).

Figure 2

Conceptual model of the organisation`s network cybersecurity domain.

https://www.acigjournal.com/f/fulltexts/190345/ACIG-3-013-g002_min.jpg

2.5. Organisation’s Network Cyber Threat Assessment Process

There are numerous risk assessment methods available [17, 18, 23–27] and depending on the specific one employed, a risk assessment may have a number of steps or phases, and each of these phases may have slightly different names. The assessment of network cyber threats that is considered in the article is not a full-fledged measure of the cyber risk because the methodology was developed considering the common circumstances of the deficiency of the risk context data. Since the network cyber threat events detected by network traffic monitoring and analysis tools are the only source of information considered for the assessment, and due to the lack of metrics that could define important cyber risk factors, cyber threat assessment (rather than cyber risk assessment) is reviewed in this work. Guided by the approach to risk assessment defined in [17, 19, 23, 25], the stages of the network cyber threat assessment process for this methodology can be defined (see Figure 3), namely:

  • preparation for the assessment;

  • conducting the assessment;

  • interpreting and communicating assessment results;

  • maintaining the assessment.

The aim of the stage of preparation for the assessment is to identify the context of the network cyber threat assessment, which includes:

  • identification of the purpose of the assessment;

  • identification of the assessment scope;

  • identification of assumptions and constraints associated with the assessment;

  • identification of information sources that are used as input data for conducting the assessment.

The aim of the stage of conducting the assessment is the calculation of the Network Cyber Threat Score, which includes:

  • identification of the approach for classifying network cyber threats;

  • identification of the network cyber threat characteristics, that are considered during the assessment;

  • calculation of the Network Cyber Threat Score.

The aim of the stage of interpreting and communicating assessment results is a correct interpretation and understanding of the calculated Network Cyber Threat Score as well as a discussion of the obtained results in order to make effective managerial decisions, which includes:

  • sharing the assessment results (e.g., executive briefings, assessment reports, dashboards);

  • communicating assessment results in order to potentially make managerial decisions based on them.

The aim of the stage of maintaining the assessment is to track the trend of changes, to support making managerial decisions based on assessment results, and to incorporate any changes to the network cyber threat assessment approach if it needs to be actualised and updated, which includes:

  • regular conduction of the organisation’s network cyber threat assessment;

  • regular review of the assessment approach.

Figure 3

Stages of the network cyber threat assessment process.

https://www.acigjournal.com/f/fulltexts/190345/ACIG-3-013-g003_min.jpg

3. Methods

3.1. Defining Common Network Cyber Threat Attributes

The purpose of the organisation’s network cyber threat assessment is the calculation of a quantitative indicator that reflects the level of organisation’s network cyber threats and can be used to compare the level of network cyber threats in different periods of time in order to monitor the trend of changes, as well as to support the managerial decision-making process (that means the implementation of such an indicator that would be representative both for displaying the level of network cyber threats of a particular organisation and for comparing these levels between several organisations). Network cyber threat events, that are detected by network traffic monitoring and analysis tools, are the only source of information considered for this assessment in terms of the work.

Network cyber threat events can be discovered through the implementation of signature and (or) anomaly analysis methods when writing rules for detecting indicators of attacks or indicators of compromise in network traffic, that are applied to a network traffic monitoring and analysis tool. Since the quality of the written rules, according to which the network cyber threat events are detected, directly affects the quality of the subsequent events classification, it is important to maintain and support the detection engineering process, which means developing, updating, validating, and testing the rules.

Network cyber threat events are the manifestations of cyber threats in a network environment that need to be detected, categorised, and mitigated [28, 29]. Network cyber threat attributes refer to specific characteristics or properties associated with network cyber threats that help in identifying, analysing, and understanding the nature and behaviour of the threats. As mentioned earlier, considering a greater number of relevant attributes contributes to increasing the objectivity and accuracy of the network cyber threat assessment process. Since the network cyber threat events detected by network traffic monitoring and analysis tools are the only source of information considered for the assessment in this work, it is essential to consider the key network cyber threat attributes to classify such events. Figure 4 represents the common network cyber threat attributes that are described in Table 1.

Figure 4

Network cyber threat attributes.

https://www.acigjournal.com/f/fulltexts/190345/ACIG-3-013-g004_min.jpg
Table 1

Network cyber threat attributes.

Attribute nameAttribute description
src_ipSource IP address of the network cyber threat event.
src_portSource port of the network cyber threat event.
dest_ipDestination IP address of the network cyber threat event.
dest_portDestination port of the network cyber threat event.
vendor_signatureSignature of the network cyber threat event, defined by the author of the network cyber threatevent detection rule.
taxonomy_categoryCategory of the network cyber threat event, defined after classification by the taxonomy.
taxonomy_typeType of the network cyber threat event, defined after classification by the taxonomy.
severitySeverity of the network cyber threat event (can be defined either according to vendor_severity attribute (severity ‘by default’ that is defined by the author of the network cyber threat event detection rule) or reclassified using the individual approach).

3.2 Developing the Taxonomy of Network Cyber Threats

Currently, there are different ways in which to classify threats [30, 31] and it is worth noting that the categorisation is not always clear-cut. When dealing with the topic of threat event classification it is not possible to determine which the best or correct classification is because organisations defining a taxonomy are usually driven by different needs and have different expectations. It is determined in NIST [17] that the network cyber threat event classification can be carried out at one of the levels of detail necessary for describing such an event, depending on the existing assessment requirements. Description of the network cyber threat events can be general (e.g., phishing, distributed denial-of-service attack, etc.), more specific (identification of involved tactics, techniques, and procedures), or highly specific (relating to specific information systems, technologies, organisations, roles, or locations).

It would seem that creating a unified Network Cyber Threats Taxonomy is crucial for improving the detection, classification, and response to network cyber threats. It fosters standardisation, enhances collaboration, supports automation, and, ultimately, leads to a more cohesive and effective cybersecurity posture across organisations and even industries. However, while a uniform Network Cyberthreats Taxonomy offers numerous benefits, there are many scenarios where developing or modifying different taxonomies can be advantageous. The tailored approach ensures that the diverse and evolving nature of cyber threats is adequately addressed in various contexts.

Considering [32–35], the Network Cyber Threat Taxonomy was developed (see Table 2). It allows to correlate the detected network cyber threat events with the corresponding cyber threat types and categories (i.e., to classify the detected network cyber threat events). The aim of the proposed Network Cyber Threat Taxonomy is not to enable the community to reach a consensus on a reference taxonomy, but rather to propose one of the possible implementation options and additionally emphasise the significance and criticality of a properly adopted taxonomy in the task of threat classification.

Table 2

Network cyber threat taxonomy.

Cyber threat categoryCyber threat category descriptionCyber threat typeCyber threat type description
Malware infectionDetection of network artifacts or network behaviour that indicate a malware infection. Malware, also referred to as malicious code and malicious logic, is an overarching term used to describe any software or firmware intended to perform an unauthorised process that will have an adverse impact on the confidentiality, integrity or availability of a system.stealerDetection of network activity that indicates known stealer infection.
spywareDetection of network activity that indicates known spyware infection.
RATDetection of network activity that indicates known RAT infection.
trojanDetection of network activity that indicates known trojan infection.
wormDetection of network activity that indicates known worm infection.
browser malwareDetection of network activity that indicates known browser malware infection.
cryptomining malwareDetection of network activity that indicates known cryptomining malware infection.
post-exploitation toolDetection of network activity that indicates known post-exploitation tool infection.
loader (dropper)Detection of network activity that indicates known loader infection.
as-a-service malware toolDetection of network activity that indicates known as-a-service malware tool infection.
Example: detection of malware-as-a-service tool, phishing-as-a-service tool, ransomware-as-a-service tool infection.
proxy malwareDetection of network activity that indicates known proxy malware infection.
rootkitDetection of network activity that indicates known rootkit infection.
ransomwareDetection of network activity that indicates known ransomware infection.
misused legitimate toolDetection of network activity that indicates s known legitimate tool that is often misused.
malware (unclassified)Detection of network activity that cannot be directly attributed to known malware type but still indicates malware infection.
Example: detection of anomalous network behaviour, related to malware infection.
Threat Actors activityDetection of network artifacts, related to targeted activity.
These are artifacts of sophisticated, long-term cyber attack campaigns (usually involve a series of coordinated and targeted attacks) that are typically carried out by a well-resourced and highly skilled threat actors and focus on specific organisations/entities or whole geographic regions.Categories of cybersecurity Threat Actors, that are considered:
• State-sponsored actors
• Cybercrime actors
• Hacker-for-hire actors
• Hacktivists
malicious network connectionDetection of network connections to the malicious infrastructure that can be attributed to the known Threat Actor.
Suspicious network activityDetection of network artifacts or anomalous behaviour that indicates suspicious network activity.Suspicious network activity means a potentially unwanted activity that cannot be clearly identified as a malicious one but can cause undesirable impact. When observed in conjunction with other artifacts or behaviour, they can help identify and investigate true positive security incidents or intrusions.anomalous network traffic behaviourDetection of network anomalies (spikes, unexpected or unusual communication patterns and so on).
Example: detection of anomalous network behaviour, that indicates data hoarding or network misconfiguration.
accessing configuration fileDetection of network activity that indicates access to a configuration file.
suspicious network connectionDetection of network activity that indicates suspicious (potentially malicious) connections.
Example: detection of connections to a free web hosting service/a non-existent page, the usage of anonymous services, detection of suspicious user-agent string or content type.
scanningDetection of network activity that indicates scanning.
Example: detection of web scanning, port/ping scanning.
Malicious network activityDetection of network artifacts or behaviour, that indicates malicious network activity. Malicious network activity means unwanted activity that causes undesirable impact (disruption or exploiting systems, data, or network resources).malware distributionDetection of network activity that indicates malware distribution.
disrupting availabilityDetection of network activity that indicates availability disruption. Availability disruption means making relevant data, services, or other resources unavailable for access by users of a system or service. This can be accomplished by exhausting the service and its resources or overloading the components of the network infrastructure.
Example: detection of dos, ddos attempts.
unauthorised loginDetection of network activity that indicates unauthorised login attempts (includes one try or multiple tries).
Example: detection of default credentials login, brute force attempts.
file download/uploadDetection of network activity that indicates file upload or download attempt.
threats against dataDetection of network activity that indicates threats against data.
Example: detection of data leak, data exfiltration (breach) attempts.
directory/path traversalDetection of network activity that indicates directory/path traversal attempt.
injectionDetection of network activity that indicates injection attempt.
Example: detection of command, code, sql, xss, php injection attempts.
webshellDetection of network activity that indicates webshell upload or download attempt.
remote code executionDetection of network activity that indicates remote code execution attempt.
malicious network connectionDetection of network activity blacklisted by the reputation.

3.3. Calculating, Normalisation, and Interpretation of the Network Cyber Threat Score

During the selection of the method for calculating the Network Cyber Threat Score, a comparative analysis was conducted between the qualitative and quantitative approaches [36–39].

The qualitative approach relies on non-numerical descriptive data and subjective analysis [40], and involves expert opinions, insights, and experiences to evaluate cyber threats. The main advantage of adopting the qualitative approach is that it can be applied in situations where quantitative data are limited or unavailable. Conversely, the quantitative approach relies on measurable data and statistical techniques, utilises metrics, scores, and other numerical values derived from data analysis to assess threats. The main advantage of adopting the quantitative approach lies in reducing biases [41] by relying on numerical data and statistical methods.

It is of the belief that there is no way to completely eliminate subjectivity in risk scoring [42] even with a fully quantitative methodology. In practice, the combination of both approaches is often used for a more comprehensive and balanced assessment of network cyber threats. However, in this work, the quantitative approach was preferred because it offers clear, quantitatively defined results that facilitate comparison and prioritisation.

To achieve the assessment goal, two values of the Network Cyber Threat Score (maximum and average) are proposed to be calculated, with each being more representative of specific cases.

The maximum value of the organisation’s Network Cyber Threat Score (Sthreat(max)_normalized) is proposed to be used as a quantitative indicator that reflects the level of network cyber threats of a specific organisation. It takes the value of the maximum score among all the calculated normalised Network Cyber Threat Scores Sthreat(i)_normalized. In this case, Sthreat(max)_normalized score value provides insight into the most critical network cyber threat that has been detected in the organisation’s network traffic during the defined time period.

The average value of the organisation’s Network Cyber Threat Score (Sthreat(avg)_normalized) is proposed to be used as a quantitative indicator that can be implemented to compare the network cyber threat levels of several organisations. It takes the average value among all the calculated normalised Network Cyber Threat Scores (Sthreat(i)_normalized). In this case, (Sthreat(avg)_normalized) score value provides a general understanding of the organisation’s network cyber threat landscape.

The Network Cyber Threat ScoreSthreat(i) is proposed to be calculated using the mixed method, considering the network cyber threat characteristics (that are defined by network cyber threat event characteristics, namely severity and likelihood of successful realisation [43, 44]):

Sthreat(i) = Sdetection(i) × (Sseverity(i) + Slikelihood(i) + Sfrequency(i)) (1),

where: i = 1,2,…, n, n – the total number of network cyber threat types that are detected and taken into account during the assessment time period;

Sdetection(i) – detection factor, which is represented by the quantitative detection score value of the network cyber threat (see Table 3);

Table 3

Categories of the Network Cyber Threat Detection Score values (Sdetection(i)).

Qualitative valueQuantitative valueCategory description
Detected1Cyber threat is considered detected if some alert (from any security monitoring or analysis hardware/software tool operating within the organisational network) that indicates the cyber network threat type presence during the assessment period exists, i.e., the number of detections is not equal to zero.
Not Detected0Cyber threat is considered not detected if any alert (from any security monitoring or analysis hardware/software tool operating within the organisational network) that indicates the cyber network threat type presence during the assessment period doesn`t exists, i.e. the number of detections is equal to zero.

Sseverity(i) – severity factor, which is represented by the quantitative severity score value of the network cyber threat (see Table 4);

Table 4

Categories of the Network Cyber Threat Severity Score values (Sseverity(i)).

Qualitative valueQuantitative valueCategory description
Low1Cyber threat is within the low severity level if it has no impact at all or potentially minor impacton the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems of the organisation.
Medium2Cyber threat is within the medium severity level if it has a potentially moderate impacton the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems of the organisation.
High3Cyber threat is within the high severity level if it has a potentially severe impacton the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems of the organisation.

Slikelihood(i) – likelihood factor, which is represented by the quantitative likelihood score value of the network cyber threat (see Table 5);

Table 5

Categories of the Network Cyber Threat Likelihood Score values (Slikelihood(i)).

Qualitative valueQuantitative valueCategory description
Low1Cyber threat is within the low likelihood level if it is detected in the organisation`s inbound network trafficthat gives grounds to characterise the successful implementation of its potential impacton the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems of the organisation with a low level of confidence.
High2Cyber threat is within the high likelihood level if it is detected in the organisation`s outbound network trafficthat gives grounds to characterise the successful implementation of its potential impacton the stable, reliable and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems of the organisation with a high level of confidence.

Sfrequency(i) – frequency factor, which is represented by the quantitative frequency score value of the network cyber threat (see Table 6).

Table 6

Categories of the Network Cyber Threat Frequency Score values (Sfrequency(i)).

Qualitative valueThe method of normalisation of the absolute value of detectionsQuantitative valueCategory description
LowS frequency(i)= log10(x+1)0 < Sfrequency(i)≤ 1The frequency of detections is low if the absolute value of detections of this network cyber threat type (x) meets the condition:
1 ≤ x≤ 10
Medium1 < Sfrequency(i)< 2The frequency of detections is medium if the absolute value of detections of this network cyber threat type (x) meets the condition:
10 < x< 100
HighSfrequency(i)≥ 2, Sfrequency(max)=3
For Sfrequency(i)≥ 3:
S frequency(i) = Sfrequency(max)		The frequency of detections is high if the absolute value of detections of this network cyber threat type (x) meets the condition:
x≥ 100

In this formula, the multiplicative and additive approaches are combined [45, 46]. The multiplicative component Sdetection(i) represents the detection confidence. The additive component represents a balanced combined effect of the severity (Sseverity(i)), likelihood (Slikelihood(i)), and frequency (Sfrequency(i)) factors, where each factor is added to reflect their contribution to the overall Network Cyber Threat Score value.

Taking into account the difference in the impact of severity, likelihood, and frequency factors on the resulting score, weighting coefficients wseverity, wlikelihood, and wfrequency were determined [47] by the method of individual expert assessment. A subject matter expert (SME) assessment approach is often criticised because of potential biases [48] based on experiences or affiliations, which can influence the assessment results, as well as because of the need to consider and assess the level of expertise related to a specific narrow research topic. However, the competent management of these considerations helps to maximise the benefits of using the SME assessment approach [49]: credibility, reliability (despite a certain degree of subjectivity, involving experts adds authority and trustworthiness to the findings), and insight (SMEs can provide precise and credible evaluations based on their experience and a thorough understanding of nuanced complex topics).

In the scoring method, xij – is the weighting coefficient of the i-th factor that is defined by the j-th expert, i=1,n¯,j=1,m¯. Herewith, n – is the total number of the factors, that are compared, m- is the total number of experts (in our case, n = 3, m = 5).

Thus, a group of five SMEs was selected, whose task was to determine the weighting coefficients wseverity, wlikelihood, and wfrequency (by the method of direct assessment expressed in points), considering the condition that the sum of these weighting factors should be 10 points.

Using the coefficient of variation (V) we can analyse the extent of variability of determined expert scores wseverity, wlikelihood, and wfrequency and therefore check their reliability (the relative dispersion of data points in a data series around the mean). It is calculated according to the formula:

V=σx¯×100% 2,

where: V – coefficient of variation;

σ – mean squared deviation (MSD) of expert scores that is calculated according to (3);

x¯ – arithmetic mean of expert scores that is calculated according to (4).

σ=∑j=1mxi,j−x¯2m−1 3,

where: σ – mean squared deviation (MSD) of expert scores;

xi,j – score of the i-th factor that is defined by the j-th expert;

x¯ – arithmetic mean of expert scores;

m – the total number of experts.

x¯=∑Xi,jn 4

where: x¯ – arithmetic mean of expert scores;

xi,j – score of the i-th factor that is defined by the j-th expert;

n – the total number of factors that are evaluated.

The calculated values of variation coefficients V (see Table 5) indicate low values of variation for wseverity, wlikelihood (that means the high homogeneity of the respective data sets (low variability) and that the arithmetic mean value is a reliable characteristic for them), as well as a moderate value of variation for wfrequency (that means moderate homogeneity of the corresponding data set and the fact that instead of the arithmetic mean value, it is more appropriate to choose the mode or median as a characteristic of the distribution centre).

Therefore, the resulting weighting coefficients for the i-th factors, pre-assessed according to the experts’ scores (wi), are determined by the modes (by the values that are most often found in the sets of weights (xi,j) for the i-th factors, assessed by the scores of the m number of experts, i.e., have the highest frequency f(wi,j).

Taking into account the determined weights from Table 7 equation (1) takes the form:

Sthreat(i) = Sdetection(i) × ((wseverity × Sseverity(i)) + (wlikelihood × Slikelihood(i)) + (wfrequency × Sfrequency(i))) (5)

Table 7

The defined values of the weighting coefficients for the Network Cyber Threat Score factors and the values of variation coefficients.

Weight score of the i-th factorScore of the j-th expertx̄σVFrequency
of the weight
score (f(wij))		Resulting
weight score
(wi)
j= 1
wi1		j= 2
wi2		j= 3
wi3		j= 4
wi4		j= 5
wi5
i= 1, w1j (wseverity)x1,1
5		x1,2
6		x1,3
6		x1,4
5		x1,5
6		5.60.559,82%f(w1j= 5) = 2
f(w1j= 6) = 3		6
i= 2, w2j (wlikelihood)x2,1
4		x2,2
3		x2,3
3		x2,4
3		x2,5
3		3.20.4514,06%f(w2j= 3) = 4
f(w2j= 4) = 1		3
i= 3, w3j (wfrequency)x3,1
1		x3,2
1		x3,3
1		x3,4
2		x3,5
1		1.20.4537,5%f(w3j= 1) = 4
f(w3j= 1) = 1		1

For convenient interpretation of the Network Cyber Threat Score value, normalisation (converting the calculated values to the required scale) is applied by using the linear scaling formula [50]:

Sthreati_normalized=Sthreati−SthreatminSthreatmax−Sthreatmin×Sthreatmax_normalized−Sthreatmin_normalized+Sthreatmin_normalized 6,

where: Sthreat(min) = 1 × ((6 × 1) + (3 × 1) + (1 × 0.3)) = 9.3 (the minimal value of not normalised range);

Sthreat(max) = 1 × ((6 × 3) + (3 × 2) + (1 × 3)) = 27 (the maximum value of not normalised range);

Sthreat(min)_normalized = 1 (the minimal value of normalised range);

Sthreat(max)_normalized = 100 (the maximum value of normalised range).

Considering that Sthreat(i)_normalized values for not detected network cyber threats correspond to the same Sthreat(i) values and are equal to zero, we get normalised (see Table 8) interpretable (see Table 9) ranges of the Network Cyber Threat Score [0,100].

Table 8

Normalised ranges of the Network Cyber Threat Score values.

Detection categoriesSeverity categoriesLikelihood categoriesFrequency categoriesResulting category
(not normalised values)		Resulting category
(normalised values)
Not Detected
(0)		***Undefined
(0)
Detected
(1)		Low
(6)		Low
(3)		Low
(1)		Informational
(9.3, 10]		Informational
(1, 4.9]
Medium
(2)		Informational
(10, 11)		Informational
(4.9, 10.5)
High
(3)		Informational
[11, 12]		Informational
[10.5, 16.1]
Low
(6)		High
(6)		Low
(1)		Low
(12, 13]		Low
(16.1, 21.7]
Medium
(2)		Low
(13, 14)		Low
(21.7, 27.3)
High
(3)		Low
[14, 15]		Low
[27.3, 32.9]
Medium
(12)		Low
(3)		Low
(1)		Medium
(15, 16]		Medium
(32.9, 38.5]
Medium
(2)		Medium
(16, 17)		Medium
(38.5, 44.1)
High
(3)		Medium
[17, 18]		Medium
[44.1, 49.7]
Medium
(12)		High
(6)		Low
(1)		Medium
(18, 19]		Medium
(49.7, 55.3]
Medium
(2)		Medium
(19, 20)		Medium
(55.3, 60.8)
High
(3)		Medium
[20, 21]		Medium
[60.8, 66.4]
High
(18)		Low
(3)		Low
(1)		High
(21, 22]		High
(66.4, 72]
Medium
(2)		High
(22, 23)		High
(72, 77.6)
High
(3)		High
[23, 24]		High
[77.6, 83.2]
High
(18)		High
(6)		Low
(1)		Critical
(24, 25]		Critical
(83.2, 88.8]
Medium
(2)		Critical
(25, 26)		Critical
(88.8, 94.4)
High
(3)		Critical
[26, 27]		Critical
[94.4, 100]
Table 9

Categories of Network Cyber Threat Score values (interpretation).

Qualitative valueQuantitative valueDescription
Undefined levelSthreat(i)_normalized = 0If the calculated Network Cyber Threat Score value is within the undefined level, this indicates that there were no network cyber threat type detections in the organisation`s inbound or outbound network traffic during the evaluated time period.
Informational level1 < Sthreat(i)_normalized ≤ 16.1If the calculated Network Cyber Threat Score value is within the informational level, this indicates that a low criticality network cyber threat type with a low likelihood level of successful realization was detected in the organisation`s inbound network traffic during the evaluated time period.
The information level category doesn`t require the organisation’s response to take measures related to the detected cyber threat type, as it potentially doesn`t cause a significant impact on the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems. It is recommended to familiarise with the results of the Network Cyber Threat Score calculation to mitigate the potential cyber risk.
Low level16.1 < Sthreat(i)_normalized ≤ 32.9If the calculated Network Cyber Threat Score value is within the low level, this indicates that a low criticality network cyber threat type with a high likelihood level of successful realization was detected in the organisation`s outbound network traffic during the evaluated time period.
The low level category doesn`t require the organisation’s response to take measures related to the detected cyber threat type, as it potentially doesn`t cause a significant impact on the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems. It is recommended to familiarise with the results of the Network Cyber Threat Score calculation to mitigate the potential cyber risk.
Medium level32.9 < Sthreat(i)_normalized ≤ 66.4If the calculated Network Cyber Threat Score value is within the medium level, this indicates that a medium criticality network cyber threat type was detected in the organisation`s inbound or outbound network traffic during the evaluated time period.
The medium-level category requires the organisation’s response to take measures related to the detected cyber threat type, as it can potentially cause a significant impact on the stable, reliable and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems. It is recommended to familiarise with the results of the Network Cyber Threat Score calculation to mitigate the potential cyber risk.
High level66.4 < Sthreat(i)_normalized ≤ 83.2If the calculated Network Cyber Threat Score value is within the high level, this indicates that a high criticality network cyber threat type with a low likelihood level of successful realisation was detected in the organisation`s inbound network traffic during the evaluated time period.
The high-level category requires the immediate organisation’s response to take measures related to the detected cyber threat type (localising and eliminating the potential consequences), as it can potentially cause a significant impact on the stable, reliable, and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems. It is recommended to familiarise with the results of the Network Cyber Threat Score calculation to mitigate the potential cyber risk.
Critical level83.2 < Sthreat(i)_normalized ≤ 100If the calculated Network Cyber Threat Score value is within the critical level, this indicates that a high criticality network cyber threat type with a high likelihood level of successful realization was detected in the organisation`s outbound network traffic during the evaluated time period.
The critical level category requires the immediate organisation’s response to take measures related to the detected cyber threat type (localising and eliminating the consequences), as it can have a significant impact on the stable, reliable and regular functioning of the organisation’s informational, electronic communicational, information and communication systems, and technological systems. It is recommended to familiarise with the results of the Network Cyber Threat Score calculation to mitigate the cyber risk.

The boundary values in Tables 8 and 9 are preliminary and almost evenly distributed, but in practice, they should be chosen in accordance with the determined level of risk tolerability [51–55] and revised regularly as the risk landscape evolves [56]. Setting boundaries helps in categorising and prioritising risks accurately [57, 58]. That’s why setting the tolerability level should be tailored to the unique context [59] and be established periodically by decision makers at a strategic level in accordance with the external risk environment of the organisation and relevant justification, that in some cases becomes a contractual objective.

The average value of the organisation’s Network Cyber Threat Score (Sthreat(avg)_normalized), as a normalised average score of all detected network cyber threats is proposed to be calculated using the formula of the arithmetic mean, since the individual values of the averaged feature (normalised Network Cyber Threat Scores) and their number in the aggregate are known:

Sthreatavg_normalized=1k×∑i=1kSthreati_normalized 7,

where: i = 1, 2, …, k, k – the number of network cyber threat types, the classification of network cyber threat events according to which is taken into account during the assessment and for which the absolute number of detected cyber threat events is a non-zero value, meaning x ≠ 0; ∑i=1kSthreati_normalized – the sum of the detected normalised Network Cyber Threat Scores.

The arithmetic mean is commonly used in various risk assessment and scoring methodologies as it provides an intuitive and easily interpretable measure of the central tendency. Since the individual Network Cyber Threat Scores are normalised, they are on a comparable scale, making the arithmetic mean an appropriate measure. By averaging all normalised Network Cyber Threat Scores, the arithmetic mean accounts for the cumulative impact of all the detected threats and appears to be a consistent metric, meaning that changes in individual normalised Network Cyber Threat Score values will proportionately affect the overall average and contribute equally, avoiding bias from extreme values. Therefore, it can serve as a baseline metric for comparing changes in the organisation’s network cyber threat landscape over time as well as for comparing network security postures of different organisations.

Table 10 represents categories, according to which the calculated average value of the organisation’s Network Cyber Threat Score is proposed to be interpreted.

Table 10

Categories of the average value of the organisation’s Network Cyber Threat Score (Sthreat(avg)_normalized).

Qualitative valueQuantitative valueDescription
Undefined levelSthreat(avg)_normalized= 0The calculated value of the average value of the organisation’s Network Cyber Threat Score is undefined.
Low level1 < Sthreat(avg)_normalized ≤ 32.9The calculated value of the average value of the organisation’s Network Cyber Threat Score is within the low-level range.
Medium level32.9 < Sthreat(avg)_normalized ≤ 66.4The calculated value of the average value of the organisation’s Network Cyber Threat Score is within the medium-level range.
High level66.4 < Sthreat(avg)_normalized ≤ 100The calculated value of the average value of the organisation’s Network Cyber Threat Score is within the high-level range.

4. Results

According to the methodology, presented in the work, a scheme of the algorithm was developed (see Figure 5) for the automated calculation of the Network Cyber Threat Score, where: j – the overall number of detected network cyber threat events during the assessment period; n – the number of network cyber threat types, the classification of network cyber threat events according to which is taken into account during the assessment (according to the taxonomy, proposed to use in this work, n = 30); k – the number of network cyber threat types, the classification of network cyber threat events according to which is taken into account during the assessment and for which the absolute number of detected cyber threat events is a non-zero value, meaning x ≠ 0; k – the absolute number of detected network cyber threat events, that are classified by network cyber threat types according to the taxonomy, proposed to use in this work.

Figure 5

A scheme of the algorithm.

https://www.acigjournal.com/f/fulltexts/190345/ACIG-3-013-g005_min.jpg

The algorithm’s scheme formalises the inputs, processes, and outputs needed to grasp and implement the steps involved in calculating the maximum (Sthreat(max)_normalized) and average (Sthreat(avg)_normalized) values of the Network Cyber Threat Score. By following these steps, the algorithm can be applied and automated for the purpose of conducting the organisation’s network cyber threat assessment process more effectively, delivering real-time insights into the network’s security posture and allowing for timely responses.

Taking into consideration the conceptual model of the organisation’s network cybersecurity domain (presented in Figure 2), the algorithm scheme (presented in Figure 5) was validated in practice by its implementation in the log management tool of a specific organisation, allowing for the automated calculation of the Network Cyber Threat Score.

The dashboard was also developed for the log management tool, used within the organisation (see Figure 6). It visualises the results of the custom correlation searches that classify network cyber threat events with regard to the categories and types outlined in the Network Cyber Threat Taxonomy and contains the detailed results of the Network Cyber Threat Score calculation with all the related metrics. Grouping panels together and arranging them in a logical and visually appealing layout makes the dashboard easy to interpret. Therefore, the presented visualisation example can be used as one of the options for displaying the results of the algorithm implementation and for monitoring the Network Cyber Threat Score value (continuously or at scheduled intervals) to check for exceeding certain thresholds. It can be applied for sharing information developed in the execution of the cyber threat assessment during the stage of communicating and sharing assessment information. In particular, the dashboard panel contains:

  1. the results of calculating the maximum and average values of the Network Cyber Threat Score (single value visualisation);

  2. distribution of the number of detected cyber threat events by cyber threat categories (pie chart visualisation);

  3. timechart of the number of detected cyber threat events by cyber threat categories (single value visualisation with trend indicator);

  4. distribution of the number of detected cyber threat events by cyber threat types (histogram visualisation);

  5. timechart of the number of detected cyber threat events by cyber threat types (single value visualisation with trend indicator).

Figure 6

A dashboard panel.

https://www.acigjournal.com/f/fulltexts/190345/ACIG-3-013-g006_min.jpg

5. Discussion

A uniform approach to calculating the organisation’s Network Cyber Threat Score that involves a fixed set of factors, an assessment scale for each factor, and an algorithm for combining these factors cannot simultaneously satisfy the needs of different organisations. Therefore, the creation of an adapted methodology is a necessary step in order to take into account additional factors, determine the required level of their decomposition and select a convenient combining algorithm for conducting such an assessment.

The automated calculation of the maximum and average values of the Network Cyber Threat Score according to the methodology presented in the work allows determining the quantitative indicators that partially reflect the overall level of the organisation’s cyber risk (because network traffic analysis can detect only a certain range of cyber threats and cannot replace a complex approach to conducting a cyber risk assessment). It can be implemented for comparing the level of network cyber threats during different time periods to monitor the trend of changes, as well as for supporting the process of making managerial decisions regarding the organisation’s cybersecurity strategy (namely, planning new and improving existing preventive protection measures). The methodology of calculating the Network Cyber Threat Score is also flexible enough to be adopted by various organisations by adjusting it to their own Network Cyber Threat Taxonomy. According to their requirements, the scoring of some cyber threat types and categories (the Network Cyber Threat Severity Score values) can be adjusted to produce the most appropriate results.

In terms of limitations, it is important to take into consideration the factors that directly affect the objectivity of the calculated scores:

  • the technical component, namely the functional capabilities (methods of analysis) of the available network traffic monitoring and analysis tools that are in use;

  • the quality of the detection rules applied directly to the existing network traffic monitoring and analysis tools for detecting network events, classified as cyber threats.

The greater the number of methods or their combinations used by the available network traffic monitoring and analysis tools, as well as the better the quality of implemented detection rules, the greater the number of network events, classified as cyber threats, can be detected and the more accurate these detections will be (in terms of increasing the number of True Positive alerts).

Currently, some simplifications of the risk-based approach are being applied to conduct the network cyber threat assessment process within the discussed methodology. Future research directions include decomposing the current procedure to define categories of Network Cyber Threat Severity and Likelihood Scores, as well as considering the other possible characteristics of network cyber threats to quantify and account for them in the calculation of the Network Cyber Threat Score.