Workshop Program

Work on automating vulnerability discovery has long been hampered by a shortage of ground-truth corpora with which to evaluate tools and techniques. This lack of ground truth prevents authors and users of tools alike from being able to measure such fundamental quantities as miss and false alarm rates. In this talk, we detail LAVA, a novel dynamic taint analysis-based technique for producing ground-truth corpora by quickly and automatically injecting large numbers of realistic bugs into program source code. Every LAVA bug is accompanied by an input that triggers it whereas normal inputs are extremely unlikely to do so. These vulnerabilities are synthetic but, we argue, still realistic, in the sense that they are embedded deep within programs and are triggered by real inputs. LAVA has already been used to inject thousands of bugs into programs of between 10K and 2M LOC, and we have begun to use the resulting corpora to evaluate bug finding tools. Our vision is to scale up the LAVA infrastructure to enable frequent online self-evaluation. Developers and evaluators of bug finding tools and techniques will be able to obtain fresh corpora seeded with unknown vulnerabilities on demand, submit their results to be graded automatically, and receive feedback in a tight iterative loop. It is our hope that this will encourage lively and healthy competition that is informed by meaningful performance measures.

Software vulnerabilities originating from design decisions are hard to find early and time consuming to fix later. We investigated whether the problematic design decisions themselves might be relatively easier to find, based on the concept of “technical debt,” i.e., design or implementation constructs that are expedient in the short term but make future changes and fixes more costly. If so, can knowing which components contain technical debt help developers identify and manage certain classes of vulnerabilities? This paper provides our approach for using knowledge of technical debt to identify software vulnerabilities that are difficult to find using only static analysis of the code. We present initial findings from a study of the Chromium open source project that motivates the need to examine a combination of evidence: quantitative static analysis of anomalies in code, qualitative classification of design consequences in issue trackers, and software development indicators in the commit history.

Cloud-based documents are inherently valuable, due to the volume and nature of sensitive personal and business content stored in them. Despite the importance of such documents to Internet users, there are still large gaps in the understanding of what cybercriminals do when they illicitly get access to them by for example compromising the account credentials they are associated with. In this paper, we present a system able to monitor user activity on Google spreadsheets. We populated 5 Google spreadsheets with fake bank account details and fake funds transfer links. Each spreadsheet was configured to report details of accesses and clicks on links back to us. To study how people interact with these spreadsheets in case they are leaked, we posted unique links pointing to the spreadsheets on a popular paste site. We then monitored activity in the accounts for 72 days, and observed 165 accesses in total. We were able to observe interesting modifications to these spreadsheets performed by illicit accesses. For instance, we observed deletion of some fake bank account information, in addition to insults and warnings that some visitors entered in some of the spreadsheets. Our preliminary results show that our system can be used to shed light on cybercriminal behavior with regards to leaked online documents.

All Android markets are confronted with malicious apps, but they differ in how effective they deal with them. In this study, we evaluate the mitigation efforts of Google Play and four third-party markets. We define three metrics and measure how sensitive they are to different detection results from anti-virus vendors. Malware presence in three third-party markets – Liqucn, eoeMarket and Mumayi – is around ten times higher than in Google Play and Freeware Lovers. Searching for certain keywords in Google Play leads leads to a fifty times higher malware rate than those for popular apps. Finally, we measure malware survival times and find that Google Play seems to be the only market that effectively removes malware, though it contains a cluster of apps flagged as adware and malware over long time periods. This points to different incentives for app markets, anti-virus vendors and users.

High profile attacks such as Stuxnet and the cyber at-tack on the Ukrainian power grid have increased re-search in Industrial Control System (ICS) and Supervi-sory Control and Data Acquisition (SCADA) network security. However, due to the sensitive nature of these networks, there is little publicly available data for re-searchers to evaluate the effectiveness of the proposed solution. The lack of representative data sets makes evaluation and independent validation of emerging se-curity solutions difficult and slows down progress to-wards effective and reusable solutions.

This paper presents our work to generate representative labeled data sets for SCADA networks that security researcher can use freely. The data sets include packet captures including both malicious and non-malicious Modbus traffic and accompanying CSV files that con-tain labels to provide the ground truth for supervised machine learning.

To provide representative data at the network level, the data sets were generated in a SCADA sandbox, where electrical network simulators were used to introduce realism in the physical component. Also, real attack tools, some of them custom built for Modbus networks, were used to generate the malicious traffic. Even though they do not fully replicate a production network, these data sets represent a good baseline to validate detection tools for SCADA systems.