Science in/for/of Smartphone Authentication?

Smartphone authentication has received considerable attention of the research community for the past several years. Every year, in diverse set of top conferences, such as, CHI, MobiSyS, MobiCom, UbiComp, USENIX Security, CCS, NDSS, you can find some new alternative authentication mechanism proposal. This is not surprising given how important smartphones have become for people’s daily lives.

Smartphone authentication is also of particular interest because there have been also deployments by the industry beyond the usual PINs and passwords. Two prominent examples include the iOS's fingerprint authentication mechanism TouchID and Android’s 3x3 grid-based graphical password. De Luca and Lindqvist have recently summarized related literature and issues for these authentication methods [1].

Although one could argue that humanity is making some progress that new proposals are being published, the unfortunate situation is that a lot of the work is not comparable to each other. We either do not have or do not require using comparable metrics with papers. Proponents of so called "behavioral biometrics", for example, have opted for using Equivalent Error Rates (EER) as their metric, following the legacy of biometrics, despite many reviewers disliking and distrusting the approach. One of the obvious problems with EER is that people do not have access to same datasets [2], and no comparisons are nevertheless made. (We note that EER is not necessarily a bad metric even though some in the community want to push forward this mem.) Clark and Lindqvist have used Bonneau et al.'s [3] comparative framework for web authentication as one approach to analyze a particular subdomain: gesture recognizers. Sherman et al. [4] have proposed and implemented an information-theoretic metric based on mutual information to compute complexity and memorability of gestures. Given the lack of deployment for a lot of (all) proposals, we are long way from applying statistical approaches such as \alpha-guesswork [5] to smartphone authentication.

What can and should be done when a reasonable number of participants for a new authentication method is perhaps tens or hundreds of volunteers, and how should we evaluate new proposals?

1. A. De Luca, J. Lindqvist, "Is secure and usable smartphone authentication asking too much?," Computer , vol.48, no.5, pp.64,68, May 2015 doi: 10.1109/MC.2015.134
2. G. Clark, J. Lindqvist, "Engineering Gesture-Based Authentication Systems," Pervasive Computing, IEEE , vol.14, no.1, pp.18,25, Jan.-Mar. 2015 doi: 10.1109/MPRV.2015.6
3. J. Bonneau, C. Herley, P. C. van Oorschot and F. Stajano. "The Quest to Replace Passwords: A Framework for Comparative Evaluation of Web Authentication Schemes," IEEE Security & Privacy (Oakland) 2012
4. M. Sherman, G. Clark, Y. Yang, S. Sugrim, A. Modig, J. Lindqvist, A. Oulasvirta and T. Roos, "User-Generated Free-Form Gestures for Authentication: Security and Memorability," in Proceedings of MobiSys'14, 2014
5. J. Bonneau. The science of guessing: analyzing an anonymized corpus of 70 million passwords. IEEE Security & Privacy (Oakland) 2012.