john.r.goodall

In an increasingly networked world, information security is an increasingly important domain, but one that is not well understood. Yet, an understanding of how this work is accomplished is crucial to designing tools and management policies to better support it. The work practice of intrusion detection analysts is a complex fusion of individual and collaborative resource monitoring and problem solving. This paper details the practice of intrusion detection work, specifically highlighting the tasks that make up the work, and it concludes with a discussion of the implications that this work understanding has on future design of tools and organizational policies to make intrusion detection work more efficient.

Computer network defense (CND) requires analysts to detect both known and novel forms of attacks in massive volumes of network data. Visualization tools can potentially assist in the discovery of suspicious patterns of network activity and relationships between seemingly disparate security events, but few CND analysts are leveraging visualization technologies in their current practice. To address this, we created a new visualization framework, VIAssist, based on a comprehensive cognitive task analysis of CND analysts. We designed VIAssist to fit the work practices and operational environments of those analysts. This article describes the major visual analytic features of VIAssist that address the needs of CND analysts, including its coordinated visualizations and interactive report building capabilities. A scenario illustrates how it can be used to discover the unexpected in network flow data.

Intrusion detection analysis requires understanding the context of an event, usually discovered by examining packet-level detail. When analysts attempt to construct the big picture of a security event, they must move between high-level representations and these low-level details. This continual shifting places a substantial cognitive burden on the analyst, who must mentally store and transfer information between these levels of analysis. This article presents an information visualization tool, the time-based network traffic visualizer (TNV), which reduces this burden. TNV augments the available support for discovering and analyzing anomalous or malicious network activity. The system is grounded in an understanding of the work practices of intrusion detection analysts, particularly foregrounding the overarching importance in the analysis task of integrating contextual information into an understanding of the event under investigation. TNV provides low-level, textual data and multiple, linked visualizations that enable analysts to simultaneously examine packet-level detail within the larger context of activity.

User testing is an integral component of user-centered design, but has only rarely been applied to visualization for cyber security applications. This paper describes a comparative evaluation of a visualization application and a traditional interface for analyzing network packet captures, that was conducted as part of the usercentered design process. Structured, well-defined tasks and exploratory, open-ended tasks were completed with both tools. Accuracy and efficiency were measured for the well-defined tasks, number of insights was measured for exploratory tasks and user perceptions were recorded for each tool. The results of this evaluation demonstrated that users performed significantly more accurately in the well-defined tasks, discovered a higher number of insights and demonstrated a clear preference for the visualization tool. The study presented here may be useful for future visualization for network security visualization evaluation designers. Some of the challenges and lessons learned are described.

This research advances Cyber Situation Management by proposing methods for automated mapping of Cyber Assets to Missions and Users (Camus). To enable accurate and efficient cyber incident mission impact assessment, a Camus ontology that defines entities, relationships and attributes (ERAs) associated with them has been drafted. Methods for fusing data from multiple data sources have been developed alongside an ontology-based system to populate the model using existing network data sources. The Camus system demonstrates how commonly available data sources can be rapidly collected, correlated, and fused to automatically map cyber assets to the users who depend on them, to the missions they support, and to the services they provide. Also discussed are the technical architecture and challenges to such an approach.

Analysis of voluminous computer network data has become a common practice for cyber defense, but few tools provide adequate support for cyber-infrastructure defenders' workflow, visual exploration, IP geo-location, scalability, collaboration, or reporting. The state-of-the-art in visual analysis tools for cyber defense is typically no more than spreadsheets and primitive charting. While familiar to users, this approach ignores the human perceptual ability to identify novel patterns and anomalies when data is presented graphically. This paper reports on a visual analytics systems, VIAssist, being developed for cyber-infrastructure protection that helps cyber defenders better understand the massive, multi-dimensional datasets to protect our nation's critical infrastructure.

Large corporations and government agencies are continually bombarded by malicious network attacks through the cyber infrastructure. One common method to identify and assess the impacts of these malicious activities is through the monitoring and analysis of network flow data. While already somewhat aggregated, the data can quickly become overwhelming ? a billion flow records a day for large organizations is not abnormal. We have integrated our visual analytics toolkit with network flow data to provide a seamless workflow for computer network defense analysts. This integration can facilitate the learning process of novice analysts and make expert analysts more productive.

As wireless networking has become near ubiquitous, the ability to discover, identify, and locate mobile cyber assets over time is becoming increasingly important to information security auditors, penetration testers, and network administrators. We describe a new prototype called MeerCAT (Mobile Cyber Asset Tracks) for visualizing wireless assets, including their location, security attributes, and relationships. This paper highlights our latest iteration of our prototype for visual analysis of wireless asset data, including user requirements and the various coordinated visualizations.

Designing a visualization system capable of processing, managing, and presenting massive data sets while maximizing the users situational awareness (SA) is a challenging, but important, research question in visual analytics. Traditional data management and interactive retrieval approaches have often focused on solving the data overload problem at the expense of the users SA. This paper discusses various data management strategies and the strengths and limitations of each approach in providing the user with SA. A new data management strategy, coined Smart Aggregation, is presented as a powerful approach to overcome the challenges of both massive data sets and maintaining SA. By combining automatic data aggregation with user-defined controls on what, how, and when data should be aggregated, we present a visualization system that can handle massive amounts of data while affording the user with the best possible SA. This approach ensures that a system is always usable in terms of both system resources and human perceptual resources. We have implemented our Smart Aggregation approach in a visual analytics system called VIAssist (Visual Assistant for Information Assurance Analysis) to facilitate exploration, discovery, and SA in the domain of Information Assurance.

When performing packet-level analysis in intrusion detection, analysts often lose sight of the big picture while examining these low-level details. In order to prevent this loss of context and augment the available tools for intrusion detection analysis tasks, we developed an information visualization tool, the Time-based Network traffic Visualizer (TNV). TNV is grounded in an understanding of the work practices of intrusion detection analysts, particularly foregrounding the overarching importance of context and time in the process of intrusion detection analysis. The main visual component of TNV is a matrix showing network activity of hosts over time, with connections between hosts superimposed on the matrix, complemented by multiple, linked views showing port activity and the details of the raw packets. Providing low-level textual data in the context of a high-level, aggregated graphical display enables analysts to examine packet-level details within the larger context of activity. This combination has the potential to facilitate the intrusion detection analysis tasks and help novice analysts learn what constitutes normal on a particular network.

This paper presents the Intrusion Detection toolkit (IDtk), an information visualization tool for intrusion detection (ID). IDtk was developed through a user-centered design process, in which we identified design guidelines to support ID users. ID analysts protect their networks by searching for evidence of attacks in ID system output, firewall and system logs, and other complex, textual data sources. Monitoring and analyzing these sources incurs a heavy cognitive load for analysts.
The use of information visualization techniques offers a valuable addition to the toolkit of the ID analyst. Several visualization techniques for ID have been developed, but few usability or field studies have been completed to assess the needs of ID analysts and the usability and usefulness of these tools.
We intended to fill this gap by applying a user-centered design process in the development and evaluation of IDtk, a 3D, glyph-based visualization tool that gives the user maximum flexibility in setting up how the visualization display represents ID data. The user can also customize whether the display is a simple, high-level overview to support monitoring, or a more complex 3D view allowing for viewing the data from multiple angles and thus supporting analysis and diagnosis. This flexibility was found crucial in our usability evaluation. In addition to describing the tool, we report the findings of our user evaluation and propose new guidelines for the design of information visualization tools for ID.

This paper reports on the user requirements gathering activities and design of an information visualization tool for analyzing network data for intrusion detection (ID). User-centered design methods have been widely used for many years. However, innovative visualization displays are often developed with limited consideration of user needs in the context of real-life problems. While it can be argued that this is required to generate creative new solutions, the resulting tools may not fully support actual users in their daily work. We studied ID analysts' activities in order to understand their work practices. This resulted in a simple task model of ID work and guidelines for visualization support. Noting the lack of current visualization support for the analysis ID task and grounded in the actual needs of ID analysts, we designed a visualization prototype for investigating network traffic.

Intrusion detection (ID) analysts are charged with ensuring the safety and integrity of today's high-speed computer networks. Their work includes the complex task of searching for indications of attacks and misuse in vast amounts of network data. Although there are several information visualization tools to support ID, few are grounded in a thorough understanding of the work ID analysts perform or include any empirical evaluation. We present a user-centered visualization based on our understanding of the work of ID and the needs of analysts derived from the first significant user study of ID. The tool presents analysts with both 'at a glance' understanding of network activity, and low-level network link details. Results from preliminary usability testing show that users performed better and found easier those tasks dealing with network state in comparison to network link tasks.

The work of intrusion detection (ID) in accomplishing network security is complex, requiring highly sought-after expertise. While limited automation exists, the role of human ID analysts remains crucial. This paper presents the results of an exploratory field study examining the role of expertise and collaboration in ID work. Through an analysis of the common and situated expertise required in ID work, our results counter basic assumptions about its individualistic character, revealing significant distributed collaboration. Current ID support tools provide no support for this collaborative problem solving. The results of this research highlight ID as an engaging CSCW work domain, one rich with organizational insights, design challenges, and practical import.

Intrusion detection (ID) systems have become increasingly accepted as an essential layer in the information security infrastructure. However, there has been little research into understanding the human component of ID work. Currently, security analysts face an increasing workload as their environments expand and attacks become more frequent. We conducted contextual interviews with security analysts to gain an understanding of the people and work of ID. Our findings reveal that organizational changes must be combined with improved technical tools for effective, long-term solutions to the difficulties of scaling ID work. We propose a three-phase task model in which tasks could be decoupled according to requisite expertise. In particular, monitoring tasks can be separated and staffed by less experienced ID analysts with corresponding tool support. Thus, security analysts will be better able to cope with increasing security threats in their expanding networks. Additionally, organizations will be afforded more flexibility in hiring and training new analysts.

This paper reports a framework for designing information visualization (IV) tools for monitoring and analysis activities. In this user study, the domain for these activities is network intrusion detection (ID). User-centered design methods have been widely used for many years, however, innovative IV displays are often developed with limited consideration of user needs in the context of real-life problems. While it can be argued that this is required to generate creative new solutions, the resulting tools often do not support actual users in their daily work. Several IV tools have been developed to support ID, but there is little evidence that these solutions address the needs of the users. We studied ID analysts' daily activities in order to understand their routine work practices and the need for designing IV tools. We developed a three-phase process model that frames corresponding requirements for IV tools. This model significantly extends the scope of contemporary IV for ID tools in novel ways.

Networked computers are ubiquitous, and are subject to attack, misuse, and abuse. Automated systems to combat this threat are one potential solution, but most automated systems require vigilant human oversight. This automated approach undervalues the strong analytic capabilities of humans. While automation affords opportunities for increased scalability, humans provide the ability to handle exceptions and novel patterns. One method to counteracting the ever increasing cyber threat is to provide the human security analysts with better tools to discover patterns, detect anomalies, identify correlations, and communicate their findings. This is what visualization for computer security (VizSec) researchers and developers are doing. VizSec is about putting robust information visualization tools into the hands of humans to take advantage of the power of the human perceptual and cognitive processes in solving computer security problems. This chapter is an introduction to the VizSec research community and the papers in this volume.

This paper explores the relationship between physical and cyber infrastructures, focusing on how threats and disruptions in physical infrastructures can cascade into failures in the cyber infrastructure. It also examines the challenges involved in organizing and managing massive amounts of critical infrastructure data that are geographically and logically disparate. To address these challenges, we have designed Cascade, a system for visualizing the cascading effects of physical infrastructure failures into the cyber infrastructure. Cascade provides situational awareness and shows how threats to physical infrastructures such as power, transportation and communications can affect the networked enterprises comprising the cyber infrastructure. Our approach applies the concept of punctualization from Actor-Network Theory as an organizing principle for disparate infrastructure data. In particular, the approach exposes the critical relationships between physical and cyber infrastructures, and enables infrastructure data to be depicted visually to maximize comprehension during disaster planning and crisis response activities.

Intrusion detection, the process of using computer network and system data to identify potential cyber attacks, has become an increasingly essential component of information security infrastructure. Due to the dynamic and complex nature of computer networks and the potential for inappropriate or self-damaging responses to potential attacks, intrusion detection systems are only effective when complemented by a human analyst. Human analysts utilize vast amounts of multi-dimensional data from disparate sources to make timely decisions about potential attacks. Yet, there is limited understanding of this critical human component. This research consisted of two interrelated components: a field study examining the work practices of these human analysts, and the user-centered design and evaluation of an information visualization tool for intrusion detection analysis grounded in the realities of analysts work.
The field study consisting of interviews and a survey resulted in a rich understanding of the practice of intrusion detection. This understanding informed the design of a new tool that takes advantage of humans perceptual and analytic capabilities through an interactive, graphical data presentation. This visualization tool was iteratively developed and evaluated to support a specific, complex intrusion detection task: network traffic analysis. This tool, called Time-based Network Traffic Visualizer (TNV), graphically displays network traffic patterns between networked computers. The finding from the field study that analysts rely on situated knowledge they must know their network to allow them to differentiate normal from abnormal behavior resulted in a system design that facilitates learning this behavior. This design objective was furthered as a result of a formative usability evaluation, which resulted in a design change to emphasize analysts home network. Another key finding was the disconnect in current tools between high-level overviews and low-level details, which required analysts to lose context when changing levels of analysis. This resulted in the design of TNV to underscore the importance of context by presenting high- and low-level details simultaneously. A summative evaluation demonstrated that users could use TNV to examine the low-level details while preserving context to enable better performance than the currently used tools in overview and comparison tasks.

Intrusion detection, the process of using network data to identify potential attacks, has become an essential component of information security. Human analysts doing intrusion detection work utilize vast amounts of data from disparate sources to make decisions about potential attacks. Yet, there is limited understanding of this critical human component. This research seeks to understand the work practices of these human analysts to inform the design of a task-appropriate information visualization tool to support network intrusion detection analysis tasks. System design will follow a user-centered, spiral methodology. System evaluation will include both a field-based qualitative evaluation, uncommon in information visualization, and a lab-based benchmarking evaluation.