Designing for a Global Audience in the Age of Remote Usability Testing

different devices scattered on a world map connecting in/out of a single server

  • Riseup offers VPN service to anyone online, but focuses primarily on human rights activists, journalists, and social justice workers due to the tool’s location anonymization feature and lack of sign-in, payment, or IP address logging requirements. 
  • Riseup asked us to redesign their entire user experience on desktop and manual selection (a feature that allows users to choose the exit point of their VPN) to both mobile and desktop.   
  • The ultimate goal of the project was not only to design and test the usability of manual selection, but also to position Riseup’s development team to be able to incorporate sustainable, user-centered design into their culture.

Though many VPN services charge a user fee or collect at least some sort of potentially identifying information (such as IP address logging or a sign-in), Riseup does not – making the VPN a favorite amongst activists and journalists. Riseup was selected to participate in the UXFund, a targeted pool of funding dedicated to improving the usability of open source security and privacy tools used by at-risk communities around the globe. The majority of the UXFund supports work addressing the needs of at-risk persons and/or vulnerable populations.

While Riseup was interested in receiving assistance to test out the beta version of a new manual selection feature (which allows users to choose their exit point), the ultimate goal in working with Simply Secure was to have us teach their development team how to self-sustain, incorporate user-centered design, and perform their own usability testing in the future. 

To begin the project, we first conducted an initial survey of other commercially available VPNs in order to obtain a sense of how others incorporate manual selection. One notable difference between Riseup and typical commercial VPNs is that Riseup has fewer overall resources and therefore needs to monitor their VPN load more closely. From a design standpoint, that means while many commercial VPNs choose to prominently feature manual selection in their user interface, Riseup is interested instead in encouraging their users to more often use the automatic selection feature.

Testing Process

1. Privacy-centered research process

Because Riseup is centered around a strong value system, we wanted to be sure to design a usability testing process that was aligned with their mission. As a collective, Riseup aims to support freedom of expression and overcome systematic oppression by providing computer resources and communication tools. Accordingly, it was important for us to maintain the privacy of all interviewees while operating in a remote setting. For our video conferencing tool, we chose Jitsi – an open source, end-to-end encrypted service that doesn’t require users to create an account. In addition, video was optional for testers as all of the testing was conducted through a clickable prototype rather than a screen-sharing service. To facilitate this process, we created a high-fidelity clickable prototype (a highly realistic and detailed version of the design that appears and functions just like the real app). This form of testing allowed the team to gather more meaningful feedback during the tests because participants were more likely to behave naturally. Throughout the testing process, we also used the observation mode feature on Figma which allowed our team to follow user actions and track interactions and movement without requiring them to share their screen. Notably, although Figma is not an open-source tool, it works best for team collaboration and real-time tracking. In the future, however, we would like to look into open source Figma alternatives.

2. Fully remote user research

Due to the anonymity-oriented nature of VPN services and travel restrictions related to the COVID-19 pandemic, all of our testing sessions had to be conducted remotely. This requirement presented a unique technical challenge because we weren’t able to simply ask users to connect to the VPN and share their feedback through a video conferencing tool due to the fact that internet connections shut off temporarily when users connect to a proxy network like Riseup VPN. To design around this obstacle, we created our hi-fidelity clickable prototype to enable network connections to remain stable throughout usability testing. Using this prototype also allowed us to reliably re-create and test various scenarios (like user redirection or slow connection) without writing a single line of code.

3. Internet access for a global audience

Finally, because Riseup has a user base all over the world, we also needed to target testing with users calling in from different time zones and with different language preferences. In addition, because VPN services are location-based and users connecting from different countries and regions often have different internet regulations and bandwidths, we worked to adjust our testing prototype to address a wide range of scenarios. The Riseup team wanted to explore how to properly balance the load between many different cities and in different scenarios.

Next Steps

Although the project is still in its user interview and testing phase, we are looking forward to hearing from different users in order to implement some of the changes to the existing prototype and eventually hand it off to Riseup’s development team.


Project Leads: Carissa Yao and Karissa McKelvey


Standing Together to Stop Online Bullying: Harassment of Women Journalists is a Community Problem

As part of the IWMF-led Coalition Against Online Violence, our team provided research and design support for the Online Violence Response Hub, a new digital resource designed to assist and inform women journalists facing threats from online bullying and harassment.