CSET '20 Workshop Program

Datasets are paramount to the development of AI-based technologies. However, the available cyber-physical system (CPS) datasets are insufficient. In this paper, we introduce HAI dataset 1.0 (https://github.com/icsdataset/hai), the first CPS dataset collected using the HAI testbed. The HAI testbed comprises three physical control systems, namely a GE turbine, Emerson boiler, and FESTO water treatment systems, combined through a dSPACE hardware-in-the-loop (HIL) simulator. We built an environment to remotely and automatically manipulate all components of a feedback control loop. Using this environment, we collected HAI dataset 1.0 while repeatedly running a large number of benign and malicious scenarios for a long period with minimal human effort. We will continue to improve the HAI testbed and release new versions of the HAI dataset.

Cyber attacks on Critical National Infrastructure (CNI) can be hugely detrimental to society, notably via compromising Industrial Control Systems (ICS) that underpin core CNI functions. In order to explore in-depth ICS Cyber Security challenges, testbeds are an essential tool, avoiding the need to experiment exclusively on live systems. However, ICS testbed creation is a complex multidisciplinary challenge, with a plethora of conflicting requirements. This paper, based on over six years of ICS testbed research and development that spans multiple diverse applications, proposes a flexible high-level model that can be adopted to support ICS testbed development. This is complemented by a baseline set of practical implementation guidance incorporating related and emerging technologies. As a collective, the model and implementation guidance offers a go-to guide for a wide range of end-users. Furthermore, it provides a coherent foundational structure towards establishing an online "living" resource, which can be expanded over time through broader community engagement.

A variety of tools are used to support software vulnerability analysis processes. However, when analysts want to select the optimal tool for a particular use case, or attempt to compare a new tool against others, no standard method is available to do so. In addition, we have determined that although vulnerabilities can be categorized into vulnerability types, these types are often disproportionately represented in current datasets. Hence, when comparative analyses of tools based upon these datasets are conducted, the results are distorted and unrealistic. To address this problem, we are building a Benchmark for Vulnerability Analysis Tools (B-VAT).

Representativeness is a key element of a good benchmark. In this paper, we use stratified sampling to identify a representative set of vulnerabilities from over 800 CWE’s and 75,000 CVE’s. This set becomes the foundation upon which we will build a purpose-based benchmark for vulnerability analysis tools. By using the correlation between current CWE and CVE data, the proposed B-VAT will assess tools using vulnerabilities in the proportions their types occur in the wild.

Over the recent decades, numerous evaluations of automated methods for detecting phishing attacks have been reporting stellar detection performances based on empirical evidence. These performances often neglect the adaptive behavior of an adversary seeking to evade detection, yielding uncertainty about their adversarial robustness. This work explores the adversarial robustness of highly influential and recent detection solutions, by assessing their common detection strategies. Following discussions of potential evasion techniques of these strategies, we present examples of techniques that enable evasion through imperceptible perturbations. In order to enable and improve future evaluations for adversarial robustness, a set of design guidelines is proposed.

Experimentation is an essential tool for developing networked and distributed systems. However, it is inherently complex due to the concurrent, asynchronous, heterogeneous, and prototype-based systems that must be integrated into representative scenarios to conduct valid evaluations. This paper offers a retrospective on the development and use of MAGI, an orchestration tool, that translates an experiment specification into an execution on an emulation-based testbed with high-level directives for message passing, remote process execution, and failure tracking, for conducting large and complex experiments. The MAGI tool has been used for more than seven years in a variety of experiments, including undergraduate education, anonymous communication, cyber-physical systems, and attacker-defender games on the DETER testbed. We hope the insights and takeaways learned from using our tool will aid in developing the next-generation experiment management tools.

While distributed denial-of-service (DDoS) attacks become stealthier and more disruptive, real-world network operators often ignore academic DDoS defense research and instead rely on basic defense techniques that cannot adequately defend them. In fact, prior to the deployment of a DDoS defense solution, a network operator must understand its impact specifically on their network. However, without a sound empirical analysis of the solution, which is often the case even for the most cited academic work, the network operator may fear its poor defense efficacy or its adverse effects on legitimate traffic. In this work, we elaborate on the critical missing gaps in DDoS defense evaluation and propose a new evaluation platform to help produce the missing defense analytics. To identify the impact of a defense solution in realistic network settings, our platform offers to emulate a mini Internet topology with realistic IP address space allocation and generate representational, closed-loop background traffic specific to particular networks. As such, our platform fulfills the prominent missing gaps in current DDoS research. In the end, we conduct some experiments to demonstrate the correctness and efficiency of our platform.

Sharing repeatable, reproducible, and reusable artifacts and data in cybersecurity experimentation can greatly enhance one’s ability to build upon the work of others and compare solutions. Good data, in particular, can be difficult to acquire and use for a number of reasons. Panelists discuss two NSF-funded efforts to help fill this need in the research community.