usenix conference policies
You are here
Results and Lessons Learned from a User Study of Display Effectiveness with Experienced Cyber Security Network Analysts
Christopher J. Garneau, Robert F. Erbacher, Renée E. Etoty, and Steve E. Hutchinson, U.S. Army Research Laboratory
Background. Visualization tools have been developed for various network analysis tasks for Computer Network Defense (CND) analysts, yet there are few empirical studies in the domain of cyber security that validate the efficacy of various graphical constructions with respect to enhancing analysts’ situation awareness.
Aim. The aim of this study is to empirically evaluate the utility of graphical tools for enhancing analysts’ situation awareness of network alert data compared with traditional tabular/textual tools. This paper focuses on results of the study and lessons learned for future similar studies.
Method. A tabular display was presented along with two alternative graphical displays in a web-based environment to 24 experienced network analysts. Participants were asked to use the displays sequentially to identify intrusion attempts as quickly and accurately as possible. Data were fabricated by an experienced analyst and do not rely on alert data from a real network.
Results. Analysts performed well on the tabular (baseline) display and also preferred this display to others. However, they were slightly faster and similarly accurate using one of the graphical alternatives (node-link). Subjective feedback shows that many analysts are receptive to new tools while some are skeptical.
Conclusions. Graphical analysis tools have the capability of enhancing situation awareness by preprocessing and graphically arranging data for analysis. Real-world analysts bring a wealth of experience and insight to this sort of research, and the large number of expert responses included in this study is unique. Tempering analyst expectations for the study by clearly explaining the study environment and tasks to be completed would likely lead to more accurate results.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Christopher J. Garneau and Robert F. Erbacher and Ren{\'e}e E. Etoty and Steve E. Hutchinson},
title = {Results and Lessons Learned from a User Study of Display Effectiveness with Experienced Cyber Security Network Analysts},
booktitle = {The LASER Workshop: Learning from Authoritative Security Experiment Results (LASER 2016)},
year = {2016},
isbn = {978-1-931971-35-5},
address = {San Jose, CA},
pages = {33--42},
url = {https://www.usenix.org/conference/laser2016/program/presentation/garneau},
publisher = {USENIX Association},
month = may
}
connect with us