A Usable Security Audit Security is a report to help a software team make its tool more usable and useful for its target users, while at a minimum helping preserve – and ideally enhancing – those users’ privacy and security. Simply Secure's work focuses on the user-experience/usability aspects, and we do not conduct security audits of code. Instead, we will analyze what users see and make targeted recommendations for improvements. A Usable Security Audit is a complement to and not a replacement for a comprehensive security audit of code.

What a Usable Security Audit Includes

  • Understanding who are the tool’s desired, likely, and/or actual users (or “target” users)
  • Understanding relevant threats (threats users are likely to face, threats that the tool seeks to mitigate, and threats that the tool cannot mitigate against, but that users might expect it to)
  • Analyzing the tool’s user experience heuristically (i.e. through an expert review)
  • Analyzing the tool’s user experience empirically (e.g. through user studies)
  • Making actionable recommendations to the software team to help them improve the tool’s design in response to review and study results

Phases and Outputs

Phase 1 - Understanding the Background

The first phase is to understand the background goals of the tool. This includes identifying the tool’s target users and its threat model. It also includes understanding the tool’s position in its competitive landscape, and the development team’s priorities for UX improvement.

  • Identifying target users
  • Scoping threats
  • Benchmarking and competitive analysis
  • Evaluation priorities

Phase 2 - Heuristic Evaluation (expert review)

An expert review can be highly structured according to formal heuristics, such as Jakob Nielsen's 10 General Principles for Interaction Design below), or can be more informal and based on the reviewer’s instinctive understanding of UX principles that they have developed through years of work in the field. Our evaluators have additional expertise with both the relevant security or privacy design patterns and common pitfalls. This expertise, and the ability to review the app in the context of background information about the teams’ and target users’ threat models, is what distinguishes a usable security review from a traditional UX review.

Jakob Nielsen's 10 General Principles for Interaction Design

  • Visibility of system status
  • Match between system and the real world
  • User control and freedom
  • Consistency and standards
  • Error prevention
  • Recognition rather than recall
  • Flexibility and efficiency of use
  • Aesthetic and minimalist design
  • Help users recognize, diagnose, and recover from errors
  • Help and documentation

Phase 3 - Empirical Evaluation

After one or more expert reviews have been completed, the next step in evaluating and improving a user experience is to put it in front of real users. Based on the questions the software development team wants to answer, Simply Secure selects appropriate activities. For evaluating the usable security of a project, there must be at least some focus on whether users accurately understand the threat model and whether they can perform key security-related tasks with comprehension. For example, it’s not enough for a user to be able to successfully generate a key if she does not understand what that key is for or how she should manage it in the future.

User study activities include
  • Cognitive walkthrough
  • Semi-structured or group interview
  • Task analysis
  • Diary / prompted in situ study
  • Survey or questionnaire

Additional sources of data

  • Instrumentation
  • Support channels and social media

We work closely with technical teams to craft an individual research plan optimizied for their needs.

Learn more in a report about our Usable Security Audit Methodology.