Summit Program

Online Capture The Flag (CTF) competitions are a popular means of engaging students with the world of cybersecurity. This paper reports on the use of a virtual machine (VM) framework that has been developed as part of cybersecurity courses offered to both second-year undergraduate and master’s degree students in the School of Computer Science at the University of Birmingham; the framework features CTF-style challenges that must be solved in order to complete the courses’ formative assessment. As well as acquiring flags from the framework, students must also provide traditional written answers to questions and sit an examination. We analyse how well students’ performance on the CTF-style challenges correlates with their achievement in the remaining formative assessment and examination, thus providing evidence to show whether CTFs are effective as an assessment tool in academic cybersecurity courses.

Cybersecurity competitions are popular tools for attracting students to cybersecurity field. Yet, many competitions require extensive preparation, strong coding skills and solid background knowledge, not just in security, but also in system administration, networking and operating systems. As such, competitions may discourage novices that lack in one of these required areas. In this paper we discuss our experience in using Class Capture-the-Flag Exercises (CCTFs) to bridge this gap in classes, and in 2015 ACM Richard Tapia Security workshop. We recount lessons learned and map a way forward, towards collaborative, more structured cybersecurity competitions that better support and engage novices, and offer a positive learning experience to all.

Over the last five years, the United States Air Force Academy (USAFA) has participated in numerous Capture the Flag (CTF) and other cyber competitions. At first, this was simply an extracurricular club activity; however, as we have seen the impact on student motivation and learning, we have greatly increased student and faculty participation. Additionally, we have started to base entire for-credit courses on a CTF framework. In this paper we discuss our rationale for utilizing CTFs as part of our formal curriculum, as well as key lessons learned relating to student engagement and avoiding cribbing. 

Hands-on Capture-the-Flag (CTF) challenges tap into and cultivate the intrinsic motivation within people to solve puzzles, much in the same way Sudoku and crossword puzzles do. While the format has been successful in security competitions, there have been a limited number of attempts to integrate them into a classroom environment. This paper describes MetaCTF, a metamorphic set of Jeopardy-style, CTF challenges for teaching reverse code engineering. MetaCTF is 1) scaffolded in a way that allows students to make incremental progress, 2) integrated with the course material so that students can immediately apply knowledge gained in class, 3) polymorphic and metamorphic so that individual students within a class and between multiple offerings of a class are given unique challenges, and 4) extensible in order to allow students to design their own CTF challenges that can be later integrated into future offerings of the course.

Instructors today, whether academic, government or corporate, are leaping on to the bandwagon of gamification with hopes high and eyes closed. Joining them onboard are those who use the techniques of last century’s corporate videos with branching choices and overt exams instead of the organic, embedded assessments of games, then rename their simulations as games. Badges and other extrinsic rewards, simply the gold stars of the digital era, are added indiscriminately to learning experiences, but do not magically transform them into good games. Game-based learning also does not mean blind faith in technology such as video games or social media clumsily shoehorned into a curriculum like a stepsister’s foot vainly squeezed into Cinderella’s slipper.

We will begin our quest with a brief history of games that purport to teach; then move to the Multiplayer Classroom, an entire class designed as a real-time game played in the real world. Elements include grading by accretion (XP and leveling up); learning by failing (allowing learners to redo assignments); intrinsic rewards (such as dividing students into guilds and rewarding an entire guild for the achievement of one member); peer teaching and more. Initial anecdotal evidence of success is now supported by recent studies involving hundreds of teachers and thousands of students: good game design is a powerful tool for learning. 

Next, we will fuse game design with storytelling on a variety of game platforms from the computer to the real world, engaging the heart as well as the mind, to create “collateral learning” where true games using competition, collaboration, human emotion, mystery and surprise and more are as exciting to play as games designed solely for entertainment. Whatever the subject being taught, students, no matter their age, learn in spite of themselves, because the well-designed game inspires their curiosity and imagination even as it challenges and sharpens their knowledge and critical thinking.

Lee Sheldon is a professional game writer and designer currently working on his 40th game. He recently joined the Interactive Media and Game Development (IMGD) program at Worcester Polytechnic Institute. For five years previously he was an Associate Professor in the Games and Simulation Arts and Sciences program at Rensselaer Polytechnic Institute. There he was co-director of the GSAS program for three years and created the first full writing for games program in the United States. 

Lee wrote the bestselling book The Multiplayer Classroom: Designing Coursework as a Game (2011); his book Character Development and Storytelling for Games (Second Edition, 2013) is the standard text in the field. The Facebook page for his method of teaching classes as multiplayer games is now followed by more than 1200 people in more than 40 countries.

His recent applied game projects include two games at Rensselaer: The Lost Manuscript 2: The Summer Palace Cipher, a virtual reality game teaching Mandarin and Chinese culture; and These Far Hills, a video game teaching engineering and science for an NSF proposal. He wrote Crimson Dilemma, a business ethics video game for Indiana University that debuted Fall 2014; and wrote and designed Secrets: A Cyberculture Mystery Game, an online class designed as a game teaching culture and identity on the Internet for Excelsior College that goes live Fall 2015.

His most recent entertainment game is the AAA Kinect title Disney Fantasia: Music Evolved for Harmonix, released in October 2014. He is currently writing a reboot of a popular Facebook game.

Before his career in games, Lee was a television writer-producer with over 200 produced shows ranging from Charlie’s Angels to Star Trek: The Next Generation.

The history of formal methods and computer security research is long and intertwined. Program logics that were in theory capable of proving security properties of software were developed by the early 1970s. The development of the first security models gave rise to a desire to prove that the models did, in fact, enforce the properties that they claimed to, and that an actual implementation of the model was correct with respect to its specification. Optimism reached its peak in the early to mid-1980s, and the peak of formal methods for security was reached shortly before the publication of the Orange Book, where the certification of a system at class A1 required formal methods. Formal verification of software was considered the gold standard evidence that the software enforced a particular set of properties. Soon afterwards, the costs of formal methods, in both time and money, became all too apparent. Mainstream computer security research shifted focus to analysis of cryptographic protocols, policies around cryptographic key management, and clever fixes for security problems found in contemporary systems.