Obstacles to the Adoption of SecureCommunication Tools

Ruba Abu-SalmaUniversity College London, UK

Anastasia DanilovaUniversity of Bonn, Germany

M. Angela SasseUniversity College London, UK

Alena NaiakshinaUniversity of Bonn, Germany

Joseph BonneauStanford University & EFF, USA

Matthew SmithUniversity of Bonn, Germany

Abstract—The computer security community has advocatedwidespread adoption of secure communication tools to countermass surveillance. Several popular personal communication tools(e.g., WhatsApp, iMessage) have adopted end-to-end encryption,and many new tools (e.g., Signal, Telegram) have been launchedwith security as a key selling point. However it remains unclearif users understand what protection these tools offer, and if theyvalue that protection. In this study, we interviewed 60 partici-pants about their experience with different communication toolsand their perceptions of the tools’ security properties. We foundthat the adoption of secure communication tools is hindered byfragmented user bases and incompatible tools. Furthermore, thevast majority of participants did not understand the essentialconcept of end-to-end encryption, limiting their motivation toadopt secure tools. We identified a number of incorrect mentalmodels that underpinned participants’ beliefs.

I. INTRODUCTION

The majority of web traffic between clients and servers

is now encrypted via TLS, however, the majority of com-

munications between users are not yet end-to-end (E2E)

encrypted [1], [2]. Whenever plaintext is processed or stored

by remote servers, users are vulnerable to mass surveillance [3]

or hackers. Their personal data is also subject to commercial

analysis by service providers for advertising and enhanced

personalization [4]. As a result, security experts have long

advocated increased use of E2E encryption.Usability has long been considered a key challenge for

secure communications, especially E2E encryption. However,

the design of most communication tools (and likewise most

of the cryptographic literature on secure communication proto-

cols) has typically not involved those who are ultimately meant

to use these tools, certainly not in the early to middle stages

of design [5], [6]. Several user studies (e.g., [7]–[9]) have

examined why users fail to use existing secure communication

tools (e.g., PGP) correctly, often concluding that significant

security failures arise due to user interface (UI) design flaws.Furthermore, there has been an effort to produce educational

materials (e.g., [10]–[12]) to explain existing security tools

and extensions, such as OpenPGP [13], Tor [14], Tails [15],

off-the-record (OTR) messaging [16], and SecureDrop [17].

These guidelines provide step-by-step instructions to install

and use these tools securely. However, documentation only

helps the users who read it and are already motivated enough

to adopt a new tool.

Recent mobile phone-based secure communication tools

have often been designed to hide security from the user com-

pletely (albeit at some security cost [1]). WhatsApp famously

deployed E2E encryption to approximately a billion users

through a code update to its application for messages, voice

calls and video communications [18], with only negligible

changes to the user experience. Some other communication

tools (e.g., Signal, Threema) have launched with security

as an explicit selling point, but they also hide nearly all

cryptographic details.There are key differences in the security model of dif-

ferent E2E-encrypted tools, in addition to a large gap in

security compared to competitors (e.g., Google Hangouts,

Skype) which do not offer E2E encryption. Yet, we have little

understanding of how users perceive the threats to their com-

munications, or whether they believe secure communication

tools protect against these threats. The Electronic Frontier

Foundation (EFF) Secure Messaging Scorecard [2] is one

attempt to provide security information to non-expert users,

a kind of a “consumer guide” to secure communication tools.

However, there has been no evaluation to see if the target users

understand the scorecard, or will select more secure tools as

a result of it.We argue that to design and build communication tools that

effectively protect users, we need to understand how users

perceive secure communications, and what influences their

decision to adopt (or not adopt) secure tools. To make a

preliminary step in this direction, we used a qualitative ap-

proach [19]–[21]. We first conducted 10 unstructured face-to-

face interviews (35 minutes on average), followed by 50 semi-

structured face-to-face interviews (90 minutes on average).The key qualitative insights from our interviews are:

• Usability is not the primary obstacle to adoption.Participants reported usability issues with different tools,

but did not stop using the tools mainly because of them.

• Fragmented users bases and lack of interoperabilityare significant obstacles. The common trend of creatingnew secure communication tools and assessing the usabil-

ity of these tools is a significant obstacle to adoption due

to creating fragmented user bases. Also, to reach their

communication partners, participants needed to use tools

that are interoperable (i.e., work across different devices).

2017 IEEE Symposium on Security and Privacy

© 2017, Ruba Abu-Salma. Under license to IEEE.

DOI 10.1109/SP.2017.65

137

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

• Low Quality of Service (QoS) is an obstacle to adop-tion. Participants assessed the reliability and securityof a communication tool by the QoS of messages and

voice calls they experienced. Low QoS does not only

hinder adoption, but also creates general doubts about

how reliable and secure the tool is.

• Sensitivity of information does not drive adoption.Perceived sensitivity of information should drive the

adoption of secure communication tools, but this was

not the case with our participants. Instead, they used

voice calls (regardless of the tool) and other obfuscation

techniques to exchange sensitive information.

• Secure communications were perceived as futile. Mostparticipants did not believe secure tools could offer pro-

tection against powerful or knowledgeable adversaries.

Most participants had incorrect mental models of how

encryption works, let alone more advanced concepts

(e.g., digital signatures, verification fingerprints). If the

perception that secure communications are futile persists,

this will continue to hinder adoption.

• Participants’ security rankings of tools were inaccu-rate. We asked our participants to rank the tools they haveused in terms of how secure they are. Many participants

ranked the services (e.g., voice calls, messages) offered

by the tools, rather than ranking the tools first. They

perceived calls more secure than messages. Furthermore,

they based their rankings on how large the tool’s user

base is, QoS, social factors and other criteria, rather than

assessing the security properties a secure tool offers.

• Participants did not understand the EFF Secure Mes-saging Scorecard. The scorecard contains seven securityproperties. Four of these were misunderstood: participants

did not appreciate the difference between point-to-point

and E2E encryption, and did not comprehend forward

secrecy or verification fingerprints. The other three prop-

erties reflecting open design (documentation, open-source

code and security audits) were considered to be negativesecurity properties, with participants believing security

requires obscurity.

Our findings suggest not only a gap between users’ under-

standing of secure tools and the technical reality, but also a gap

between users’ communication priorities and what the security

research community imagines them to be.

II. RELATED WORK

A. Secure Communications

For a detailed review of the literature on secure com-

munication tools, we refer the reader to Unger et al. [1].

Secure communication tools became widely available with

the release of PGP in 1991 [22], which was followed by

the creation of a large ecosystem of PGP tools [13], [23],

[24]. PGP was designed for asynchronous, high-latency email

communications. OTR [16], originally released in 2004, was

designed for low-latency messaging environments like chat

clients, introducing additional security features (e.g., forward

secrecy, deniability). OTR has influenced many secure commu-

nication tools designed since [25]–[30], including the Signal

protocol [31], which has recently gained popularity.

The use of self-destructing messages was popularized by

Snapchat, which was released in 2011. While popular with

users who perceived this feature as an effective solution to

some of their security and privacy needs, Snapchat offers little

security against motivated attackers, and secure data deletion

in messaging has proved elusive [32]–[34]. Other tools that

appear to provide certain security properties fail to provide

these properties in the face of government requests [3].

Usability has long been considered a challenge for secure

communications, especially E2E encryption. The main UI

challenge for E2E-encrypted communication tools is believed

to be providing assurance that a user is truly communicating

with the intended party (called trust establishment by Ungeret al. [1]). This is often reduced to verifying ownership of

cryptographic keys in some fashion. In traditional PKI, this

assurance is delivered in the form of a signed certificate from

a trusted authority [35]. However, there are many issues with

PKI associated with certificate management, including key

storage, distribution and revocation, as outlined in [36]. Pop-

ular E2E-encrypted tools (e.g., iMessage, WhatsApp, Signal)

relieve users of key management; they simply query a trusted

server that vouches for the authentic public keys of other users.

Recent proposals attempt to limit the trust in these servers

using transparency logs [37], [38], but this approach has not

been deployed in practice.

The smartphone era has seen an explosion of new com-

munication tools (typically called messengers or messagingapplications). Many of these applications claim to be “secure”,but they often do not provide specific security guarantees or

documentation, and fail to draw upon the existing crypto-

graphic literature [1], [39]. This led the EFF to develop the

Secure Messaging Scorecard in 2014 – 2015 to attempt to

provide objective information about what security properties

communication tools actually offer, providing a Consumer

Reports-style guide and encouraging adoption of tools that

offer better security [2]. Yet, there was no evaluation of the

scorecard with the target community (i.e., users who are not

security specialists) to see if the scorecard was perceived as

helpful, or did influence users’ decision to adopt secure tools.

B. User Studies of Secure Communication Tools

Lack of usability has been shown to hamper both adoption

of secure communication tools and the actual level of security

in real-world use. In their seminal paper [7], Whitten and Tygar

designed a case study to assess whether PGP 5.0 could be

effectively used by non-specialist users to secure their email.

They identified some problems in the UI design relevant to

security risks (e.g., irreversible errors, lack of consistency and

feedback). They also found that only one-third of participants

were capable of using the PGP software to correctly sign

and encrypt an email. They concluded that making security

usable requires the development of domain-specific UI design

principles and techniques.

138

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

Using a similar study to [7], Garfinkel and Miller studied

CoPilot, an email prototype based on Key Continuity Man-

agement (KCM) [8]. KCM attempts to make secure commu-

nication tools more usable by making key generation, key

management, and message signing automatic. Garfinkel and

Miller concluded that KCM is a workable model for improving

email security, and that the UI of CoPilot enables users to

send protected emails easily because, for example, it visually

distinguishes encrypted emails from unencrypted ones.

Ruoti et al. conducted a user study of two mail systems:

Private Webmail (Pwm) and Message Protector (MP) [40].

They found both systems to be usable, but participants trusted

MP more than Pwm because they “could see the ciphertextafter encryption takes place”, equating this with protection.More recently, Ruoti et al. conducted a lab-based study with

pairs of novice users cooperating to send encrypted emails

with a range of email tools [41]. Again, they found that hiding

the details of how a secure system provides security reduces

trust in the system, however, participants preferred integrated

over standalone encryption solutions. They concluded that

integrated encryption solutions are a key step to increase us-

ability, but complete transparency (i.e., hiding security details)

is counterproductive. The need for visible feedback matches

the findings of Whitten and Tygar [7] as well as the “visibilityof system status” usability engineering principle encouragedby Nielsen and Molich in 1990 [42].

Bai et al. investigated whether non-expert users can evaluate

the security trade-offs between two encryption models: a

traditional key-exchange model (analogous to PGP) and a

registration model (analogous to iMessage) [43]. They asked

participants to complete a set of encryption tasks using both

models. They also described each model’s security properties

and asked participants for their opinion. They found that

participants understood both models “fairly well”. Even thoughparticipants recognized the benefits of the exchange model

for “very sensitive communications”, they preferred (and alsotrusted) the more usable, but less secure, registration model

for “everyday communications”. Bai et al. concluded thatdesigners should explain the security properties an encryption

tool offers, and that the EFF Secure Messaging Scorecard

provides an “excellent start in this direction”.Other studies (e.g., [44]–[48]) have considered PGP fur-

ther as well as contact verification in OTR [26], secure

communications in two-way radios [9], opportunistic email

encryption [49], and public-key fingerprints [50], [51]. Fur-

thermore, several studies have explored users’ perceptions of

email signatures [52], browser security indicators (e.g., [53],

[54]), and specific features of specific security tools (e.g., self-

destructing messages in Snapchat [55]).

Gaw et al. explored the social context behind users’ deci-

sions about whether and when to encrypt emails [56]. They

interviewed members of an activist organization under the

presumption that the organization’s employees would have

a strong incentive to encrypt emails. They found that the

perception of encryption behaviour by others (e.g., use of

encryption for protecting secrets is seen as “justified”, for gen-

eral communications as “paranoid”) influenced participants’

decision to adopt encrypted email.

In [57], Renaud et al. proposed seven possible explanations

for the non-adoption of E2E encryption in email, based on

the literature and researchers’ own observations. To validate

these explanations, they interviewed students and staff mem-

bers (not security experts), and surveyed computer science

students. They found that, in addition to usability issues,

incomplete threat models, misaligned incentives, and lack of

understanding of the email architecture are key drivers of the

non-adoption of E2E-encrypted email. They concluded that

security researchers should focus on building “comprehensivemental models of email security”.

Das et al. recently studied the role of social influence on

users’ decisions to adopt secure tools [58] and to use specific

security features of a specific application (Facebook) [59],

[60]. De Luca et al. also investigated how and why users use

mobile instant messengers that are advertised as being secure

(e.g., Threema) [61]. They concluded that peer influence,

not security and privacy, primarily drives users to adopt a

messenger. The objective of our study is to explore the user

experience of secure communications in more depth, identify

“other” factors that lead to the adoption and abandonment

of communication tools, and understand how users perceive

the “security” of communication tools, especially of those

advertised as being secure.

It is worth to mention that Dourish et al. studied how users

experience and practice security using a qualitative approach

(semi-structured interviews analyzed using Grounded The-

ory [20]) in 2004 [62]. Similarly, we use a qualitative approach

to understand how users manage their communications, secure

or not, as an “everyday, practical problem”. We “zoom out”to understand users’ security needs and practices, and the

background against which they decide to use or stop using

a communication tool. We also explore what users look for in

a secure communication tool.

We know that the decisions users make may not deliver

on their actual security requirements. The gaps in mental

models identified by Renaud et al. suggest that users may think

they are more secure than they are [57]. Similarly, the folk

models of home network security described by Wash led his

participants to believe that their practices were secure when

they were not [63]. Thus, we study users’ knowledge of the

threats to their communications, and their mental models of

the tools and practices they use to protect against these threats.

III. METHODOLOGY

In this section, we discuss our research questions, recruit-

ment process, interview procedure, data analysis, research

ethics, and the limitations of our work.

A. Research Questions

In this work, we explore (1) why, when and how users use

secure communications (Section III-C1), (2) what threats users

want to protect against when communicating (Section III-C2),

(3) which communication tools users perceive to be secure (or

139

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

insecure) and why (Section III-C3), and (4) how users think

secure communications can be achieved, and how they can be

breached (Section III-C4).

B. Participants

Our literature review (see Section II) shows that mainstream

users’ needs and practices of secure communications have not

been investigated. Instead of focusing on a specific at-risk

population, such as activists, whistleblowers, or journalists,

our main focus is understanding the needs and practices of

users of communication tools who do not consider themselves

to be at risk of targeted surveillance. This is because our focus

of enquiry is widespread adoption of secure communications.We recruited our participants via posting flyers around

University College London’s buildings and emailing university

staff members. We also distributed emails to staff members

in collaborating public- and private-sector organizations (e.g.,

banks, hospitals, universities). We asked interested participants

to complete an online pre-screening questionnaire, which

380 completed. The full questionnaire can be found in the

Appendix. We assessed participants’ technical knowledge and

cyber-security threat exposure via a set of simple questions.

We also provided them with a list of different communication

tools (those evaluated by the EFF Secure Messaging Score-

card), asking them to select all the tools they currently use

and the ones they stopped using. Additionally, we gave our

participants the option to specify other tools they have used,

but were not on the list.

We then divided the pool of eligible participants into sub-

groups, based on a number of variables: age, gender, education

level, study area, employment status, technical knowledge, and

previous cyber-security threat exposure. We conducted and

analyzed 10 unstructured interviews first, followed by 50 semi-

structured interviews. Tables 1 and 2 summarize the demo-

graphics of our recruited participants for both the unstructured

and semi-structured interview sessions, respectively1.

With 60 participants, our study represents the largest qual-

itative study on this topic. We interviewed 23 male and 35

female participants. Two participants preferred not to indicate

their gender. Participants’ ages ranged from 18 to 70. Two

participants did not have a formal educational qualification,

seven completed high-school education, 30 had a college

degree (e.g., BA, BSc), and 21 had a higher degree (e.g., MA,

MSc, PhD). 40 were high-school and university students, 17

were employed, and three were retired. Our participants used

a wide range of communication tools on different computing

platforms (e.g., Android, iOS, Mac OS X, Microsoft Win-

dows). None of the participants used a PGP-based tool, such

as Enigmail, GPGTools or Gpg4win. Only P23 and P57 used

an OTR-based tool; both have adopted Pidgin for some time

and then stopped using it.

We note that P2, P5 and P28 identified themselves as secu-

rity experts, so they did not necessarily represent mainstream

users of communication tools.

1 Tables 1 and 2 can be accessed from the first author’s webpage.

C. Interview Procedure

The value of conducting qualitative research lies in pro-

viding a holistic understanding of the phenomenon under

enquiry using predominantly subjective qualitative data, which

can be supplemented by observational and other quantitative

data [64]. A single trained researcher conducted all 60 in-

terview sessions in the UK in English, by first conducting

10 unstructured (open-ended) face-to-face interviews, lasting

for 35 minutes on average. The emerging themes shaped the

design of the script used for the 50 semi-structured face-to-face

interviews, lasting for 90 minutes on average. The interviewer

allowed participants to elaborate, share their thoughts, and ask

any clarification questions. The interviewer also asked follow-

up questions (or probed) where appropriate. This is a common

practice in semi-structured interviews, in which the interviewer

primarily uses a list of questions, but has discretion to ask

follow-ups or skip questions that have already been covered.

However, all interviews covered the following four areas in

the same order. Below, we describe the script we used for the

semi-structured interviews.1) Adoption of communication tools: We asked participants

to specify the communication tools they have used by giving

them the same list of tools provided during the pre-screening

stage. This allowed us to compare their answers with those in

the pre-screening questionnaire. Also, we asked them to take

out their mobile phones and check all the communication tools

they have installed.

For each tool currently used or previously used by our

participants, we asked why they decided to adopt it and why

they stopped using it (if they had). The given answers helped

us understand why specific tools were widely adopted and

others were not. The key questions were:

• Why did you decide to adopt [this communication tool]?• What computer platforms does the tool run on?• Who do you communicate with?• What is the context of use?• Do you describe yourself as a regular user of the tool?• Have you ever checked and/or changed the default set-

tings of the tool? Please elaborate.

• What kind of information do you regard as “sensitive”?• Have you ever sent sensitive information via a commu-

nication tool? If yes, why and how did you do so?

• Why did you decide to stop using [this communicationtool], if applicable?

2) How users defined secure communications: “Securing” acommunication tool is meaningless without defining a security

policy and a threat model. Many communication tools are

advertised as “secure” or “encrypted”, but a recent academic

survey suggested that many are not as secure as they claim

to be [1]. The link between users’ perceptions of secure

communications and the actual security offered by different

communication tools has not been investigated so far.

To address this gap, we asked our participants about the kind

of protection (or security properties) a secure communication

tool should provide, what they want to protect, with whom

140

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

they communicate, who the attackers (or adversaries) might

be, and what their capabilities are.

We also elicited participants’ mental models of how they

think secure communications work. Mental models are cogni-

tive representations of external reality that underpin people’s

cognition, reasoning, decision-making and behavior [65]. We

invited our participants to draw how a communication tool

works, and whether there is a distinction between calling

someone and sending them a text (or multimedia) message.

A message could be an SMS, an email or an instant message.

We provided our participants with an iPad and a stylus pen. We

also recorded and transcribed participants’ verbal commentary

while drawing, along with the rest of the interviews.

3) Security ranking of communication tools: We asked ourparticipants to rank the communication tools they have used in

terms of the security level each tool offers. We provided them

with cards with the names and logos of the tools they have

used, and asked them to sort the tools from the most to the

least secure. We used this card sorting exercise to compare our

participants’ rankings with those on the EFF Secure Messaging

Scorecard [2] and to elicit the rationale behind their rankings.

We also wanted to assess the effectiveness of the EFF

Scorecard in communicating which communication tool is

secure and why. After our participants ranked the tools and

described their reasoning, we showed them the scorecard

(printed on a sheet of paper) and gave them 10 minutes to

explore it, compare their rankings, and ask any clarification

questions they had.

4) Security properties and mechanisms: In the last part ofthe study, we wanted to probe our participants’ understanding

of how a security property can be achieved and how it can

be violated. We also asked participants about several spe-

cific security mechanisms: encryption, digital signatures and

cryptographic fingerprints. We wanted to check their broader

understanding to see whether they can interpret the criteria on

the EFF Scorecard correctly or not.

Finally, we debriefed our participants and gave them the

time to ask any clarification questions about the study.

D. Pilot Study

We conducted a pilot study of five semi-structured inter-

views to check that the questions could be understood and

identify any potential problems in the script (e.g., cost, time,

adverse events) in advance, so that the methodology could be

fine-tuned before launching into the main study. We used the

common practice of convenience sampling [66] by selecting

five colleagues for the pilot study. In addition to the five

sessions, we asked six researchers to review the study.

E. Data Analysis

To develop depth in our exploratory research, we conducted

multiple rounds of interviews, punctuated with periods of

analysis and tentative conclusions [19]. In total, we conducted,

transcribed (using an external transcription service) and ana-

lyzed all 10 unstructured and 50 semi-structured interviews.

We observed data saturation [67] between the 40th and 45th

interview; i.e., no new themes emerged in interviews 46–50,

and, hence, we stopped recruiting. Data saturation provides

a high degree of confidence that we observed the range of

reasons for adoption (or non-adoption) of secure communi-

cations. The audio-recordings of the interview sessions were

transcribed, and then independently coded by three researchers

using Grounded Theory analysis [20], [21], an inductive/open-

ended method to discover explanations, grounded in empirical

data, about how things work. After coding all interviews and

creating the final code-book, we tested for the inter-coder

agreement (or inter-rater reliability). The average Cohen’s

Kappa coefficient (κ) for all themes in the paper is 0.83 [68]. Aκ value above 0.75 is considered an excellent agreement [69].

F. Ethics

The Research Ethics Board at University College London

reviewed and approved our research project (project ID no.:

6517/002). Before each interview, we asked our participants

to read an information sheet and sign a consent form that

explained the purpose of the study, and emphasized that

all data collected was treated as strictly confidential and

handled in accordance with the provisions of the UK Data

Protection Act 1998 (registration no.: Z6364106/2015/08/61).

Participants had the option to withdraw at any point during

the study without providing any reason. We explained to them

that in such a case, none of their data would be used in the

analysis, and they would still receive the full reward of £10.No participant withdrew.

G. Limitations

Our study has some limitations. Although our sample size is

large for a qualitative study, we did not cover a wide range of

cultural backgrounds. One can argue that this limits the gen-

eralizability of our results. However, we have documented the

study protocol step-by-step, meaning that it can be replicated

with participants in different cultural contexts.

Additionally, our study has limitations common to all qual-

itative studies. Research quality depends on the researcher’s

individual skills and might be influenced by their personal

biases. A single researcher, who was trained to conduct the

interviews consistently and ask questions in an open and

neutral way in order not to influence participants, conducted all

60 interviews. We note that the length of the interviews meant

that fatigue set in during the final 20 minutes, so participants’

answers tended to be less detailed. However, the interviewer

prompted participants to give full answers to all questions.

Furthermore, some participants could have been concerned

about the interviewer’s perception of them and, therefore,

could have changed their answers in line with how they like

to be perceived.

IV. RESULTS

In this section, we present the key emerging and recur-

ring themes we observed across our interviews. We report

participants’ statements by labeling them from P1 to P60.

We additionally report how many participants mentioned each

141

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

theme to give an indication of the frequency and distribution

of themes. However, the main purpose of qualitative research

is to explore a phenomenon in depth, and not to generate

quantitative results. We identified several misconceptions of

secure communications among participants that underpinned

their reasoning and decision-making. We report those in their

respective sections: IV-A – IV-H.

A. Adoption Criteria of Communication Tools

We found nine main criteria influencing our participants’

decision to adopt a communication tool, namely (1) large user

bases and interoperability, (2) context of use, (3) services

offered by the tool, (4) QoS, (5) cost of use, (6) type of

communications (spontaneous or planned), (7) integration with

email, (8) registration (telephone numbers vs. usernames), and

(9) social influence.

Large user bases and interoperability. The ability toreach their intended communication partners is the primary

communication goal of our participants. If most of their regular

communication partners do not use the tool, it has little utility.

As P5 put it, “there is no point of using a chat service thatnot many people use”. 50 out of 60 participants explicitlymentioned that the tools they use most frequently are those that

most of their contacts use. Thus, the small and fragmented user

bases of current secure communication tools hinder adoption

of secure tools. For example, P23 and P57 who used Pidgin

(an OTR-based tool) in the past deserted it because of lack of

utility, whereas almost all participants use WhatsApp.

Even iMessage, which is available on any device running

iOS (or Mac OS X), is not used as frequently as WhatsApp

because not all of our participants’ contacts own such a device,

and iMessage is not interoperable (i.e., does not work with

non-iOS devices). The same applies to FaceTime. Because

WhatsApp works across different platforms, it is the tool of

choice; many participants who have an iOS device use What-

sApp to communicate with contacts who also have an iOS

device, instead of using iMessage (or FaceTime). Although

they perceive iMessage as more secure (see Section IV-G),

they see the overhead of using two communication tools as

not worth the better security offered by iMessage.

Context of use. Participants use communication tools in avariety of contexts: socializing, organizing events or creating

study groups. They perceive some tools as “more suitable” for

some types of communications: they use SMS and email for

formal conversations, whereas they prefer IM to communicate

informally with family members, friends and colleagues. Voice

calls using the mobile phone network (whether the call is

local or international) are preferred if the communication is

urgent, or, as P2 described his parents and grandparents, the

communication partner is “old-school”. Participants perceive

calling a contact as more convenient and “faster” than sending

a message via IM because they do not have to check if the

recipient is online. Also, our participants prefer SMS and IM

to email if they want the recipient to be notified quickly.

Services offered. Our participants choose specific toolsbased on the services the tools offer. 55 out of 60 participants

explicitly mentioned that they use email, instead of SMS,

to send large volumes of data (e.g., media messages, files)

although many of these participants (32 out of 55) perceive

sending a message via SMS as “more secure” than sending

an email (see Section IV-F). Furthermore, 20 participants who

perceive Telegram as more secure than WhatsApp (see Section

IV-G) explicitly mentioned that Telegram does not support

calls, causing them to use the “less secure” option: WhatsApp.

Lack of utility fosters insecure behaviour: Telegram sup-

ports two chat modes: (1) default chat mode (messages are

encrypted in transit), and (2) Secret Chat mode (messagesare E2E-encrypted). However, the Secret Chat mode does notcurrently support group conversations. All participants who useTelegram do not use Secret Chat when communicating withindividuals either because the overhead of switching between

the two modes is high, or because they just forget to use

Secret Chat, especially for participants who frequently use thedefault mode to send group messages. This can be conceived

as a usability problem (i.e., mode error: a type of slip where a

user performs an action appropriate to one situation in another

situation, which is common in software with multiple modes),

but is also caused by lack of utility (the secret mode does not

support group conversations).

QoS. 47 out of 60 participants assess the reliability of acommunication tool based on the QoS of voice calls and

messages they experienced. For example, P9 and P12 pre-

fer Google Hangouts because its audio has “high-quality”,

whereas P31 and P45 stopped using Google Hangouts because

they experienced “bad-quality” audio in the past. This not only

influences adoption, but also users’ perceptions of how secure

a tool is (see Section IV-G): 40 out of 60 participants said that

a tool that offers high-quality services can also be assumed to

be more secure. Thus, the perceived competence developers of

tools demonstrate by delivering high QoS makes participants

assume that they will also do a good job on security.

Cost of use. The financial cost of using a tool is anothermain factor influencing participants’ adoption decision (47 out

of 60). Participants mainly use IM when they are not in the

same country as the recipient. P2, P30 and P41 mentioned that

IM tools are not at “no cost” because they have to pay for the

Internet service most of the time. P2 reported that the cost of

the Internet service in developing countries is high.

Battery consumption is another cost our participants men-

tioned. 36 out of 60 participants said they never log out of most

of their accounts, but they do log out of their Skype accounts

because they see Skype as a “heavy” application that drains

the device battery. This in turn means it takes time and effort

to start Skype again and sign into the account. As a result, our

participants rarely use Skype for spontaneous communications.

Type of communications: spontaneous vs. planned. Par-ticipants clearly distinguish between spontaneous and planned

communications. Many participants who use Skype (30 out of

60) use it mainly for international calls and videoconferencing.

These communications are usually pre-arranged, rather than

spontaneous. P7, for instance, said she does not use Skype

for communicating with others on a regular basis because

142

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

communication partners will not notice her messages unless

they are logged in. However, the majority of our participants

always log out of their Skype accounts (see the previous point

on battery consumption).

Integration with email. Most participants have used Ya-hoo! Messenger for some time, but they stopped using it after

moving away from Yahoo! mail. For example, P46 and P56

mentioned that they had to specifically log in to their Yahoo!

mail account to access the chat service. 15 participants, on the

other hand, use Google Hangouts because they frequently use

Gmail (on their PC/laptop, not phone).

Registration: telephone numbers vs. usernames. Com-munication tools that require knowledge of a contact’s phone

number also have reduced utility. WhatsApp and Facebook

Messenger are the most frequently used tools among our

participants (45 out of 60) for sending messages. However,

WhatsApp is only convenient to use when participants have

the phone number of the person they want to communicate

with, whereas in Facebook Messenger, they can search for a

particular person by name, adding to the tool’s utility.

Social influence. A social system is a combination ofexternal influences (e.g., mass media) and internal influences

(e.g., social relationships) that affects participants decision

to adopt or stop using a particular tool (54 out of 60). A

newspaper article or a friend can influence adoption decisions.

Das et al. [58]–[60] have studied the role of social influence

on users’ decisions to adopt secure tools and to use specific

security features; we found some evidence in the reasons

our participants gave for adoption. For example, P56 said

she adopted Telegram because her father recommended it as

secure against eavesdropping by service providers. However,

we found she does not use the Secret Chat mode and, asa result, her communications are not protected. She was

motivated to adopt a secure tool, but was foiled by a usability

issue (mode error).

B. Sensitive Information: Perceptions and Practices

Perceived sensitivity of information should drive the adop-

tion of secure communication tools, but this is not the case

with our participants. When we asked participants if they send

sensitive information via communication tools, they started to

use the terms “security”, “privacy”, “safety”, and “protection”,

interchangeably. However, they do not select a secure tool to

do so. Instead, they use different practices and obfuscation

techniques. In this section, we explain how our participants

define sensitive information, which practices they use to send

this information, and the information’s level of sensitivity.

How participants define sensitive information. Our par-ticipants said they want to protect all data they transmit, and

all data stored on their personal devices. However, they regard

some information as sensitive, such as personally identifi-

able information (PII), bank account details, authentication

credentials (e.g., PINs, passwords), health data, their photos,

and political views. Only P37 mentioned that any piece of

information is potentially personal and sensitive.

Protection practices. The majority of participants (53 outof 60) believe that the best protection for sensitive information

is to speak to the recipient directly, instead of using a com-

munication tool. If they trust a communication partner with

the information and need to send the information urgently,

they regard voice calling or videoconferencing as most secure,

regardless of the tool used. Voice calling and videoconfer-

encing are seen as the “closest thing” to telling the recipient

face-to-face because there is “no record” of calls, as opposed

to messages (see Section IV-F for the reasons). Only seven

out of 60 participants (P2, P5, P37, P42, P45, P47 and P51)

mentioned that voice calls have the same security properties

as messages giving the reason that the same communication

tool and channel are used.

Other practices our participants perceive as secure include

sending information by post (P46), sending a voice message in

a foreign language (P17 and P48), or cutting the message into

“chunks” and sending these via different communication tools

(P20 and P43). P56 also reported sending different chunks

of information using the different modes of Telegram: when

sending a 4-digit PIN, she sends two digits via the SecretChat mode and the other two digits via the default chatmode, believing the two modes of Telegram use “two differentchannels”, which cannot be associated with each other.

P8 told us about using an encryption tool to encrypt a

document, sending the “encrypted document” via one com-

munication tool and the “encryption key” via another. The

encryption tool turned out to be Microsoft Word’s password-

based document encryption feature, with the password serving

as the encryption key. 10 participants have their own “code” to

exchange sensitive information via any communication tool.They share the code (effectively a substitution cipher) with

trusted parties in advance before sending any message. They

said that the “design” of these codes or schemes must be kept

secret, so that only the parties who know the schemes can

decode the scrambled message. P13 also mentioned using the

practice of sending her password to a trusted recipient as a text

message via any tool and then changing her password later.

Level of sensitivity. 54 out of 60 participants said theyshare sensitive bank account details with trusted recipients

via a phone call, but discuss political views only face-to-

face. They believe that (1) neither the government nor service

providers are interested in users’ PINs and passwords, and (2)

a government agency (especially with repressive regimes) can

target a particular person and record their calls, as portrayed

so memorably in the following movie: “The Lives of Others”.

None of our participants mentioned meta-data (e.g., identity

of sender and recipient) as worth protecting. Even when we

hinted at the potential sensitivity of meta-data, they (except for

P2 and P5) described them as “less sensitive”. Clearly, they

are not aware of the highly publicizing and debated “we killpeople based on meta-data” comment [70]. Our participants’mental models of both the technology they are using and the

threats to their communications seem very much influenced

by traditional telephony, rather than digital communications.

143

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

C. Security Properties

Our participants used the terms “secure communications”

and “security” in previous discussions. In this section, we

analyze what security properties they expect from secure com-

munication tools. Their discussion of security properties falls

into three main categories: (1) secrecy of message content, (2)

message integrity, and (3) “no impersonation”.

Secrecy of message content. When our participants de-scribed this property, they did not use the terms “confi-

dentiality” or “encrypted communications”. Instead, they ex-

plained that exchanged messages via a secure communication

tool should only be accessed by the sender and intended

recipient(s). Third parties, including government intelligence

agencies and service providers, should not be able to read

the messages, or listen to voice calls. P5 mentioned that

information exchanged via a communication tool should not

be “re-routed to unintended recipients”.Message integrity. No participant mentioned unprompted

that a message should not be modified in transit (for several

reasons discussed later in Section IV-D.II). However, when we

explained the threat to them, all agreed that integrity is an im-

portant property a secure communication tool must offer. Only

three participants (P2, P5 and P28), who identified themselves

as security experts, discussed man-in-the-middle attacks and

digital signatures, the essential cryptographic mechanisms for

assuring integrity.

“No impersonation”. All participants believe a user willbe impersonated if their username and password are used to

log in to their account. They, therefore, want their passwords

stored in a secure place (the service provider’s server) where

they cannot be compromised. Many participants used the

term “hacking” in connection with this security property. Six

participants (P15, 17, 32, 43, 49, 56) expect to be notified,

and to be asked for consent, before the government or service

provider accesses their accounts. This is an expectation of

conduct by snoopers that in reality is unlikely to be met.

Our participants did not mention or describe plausible

deniability (or repudiation), forgeability, forward or backward

secrecy, recipient authenticity, or confidentiality of usernames.

When we started discussing anonymous communications, all

participants mentioned that anonymity is an unimportant secu-

rity property. From our participants’ perspective, anonymous

communications mean sender-anonymity [71] and/or third-

party anonymity [71] (expressed in their own words). P2,

P6, P32, P39, P45 and P50 also mentioned that only people

who engage in political discussions need sender anonymity.

P2 incorrectly stated that Telegram and Signal (formerly

known as TextSecure) offer sender-anonymity and third-party

anonymity. He stated (also incorrectly) that Skype, Snapchat

and Telegram’s Secret Chat mode provide deniability becausethey do not offer “evidence preservation”; i.e., a sender candelete a message they have already sent.

P8, P11, P22, P27, P32, P43 and P60 suggested that

anonymous communications can be achieved by using a public

PC, creating a fake account, sending the data, and then logging

out. However, they believe this only works for communication

tools that do not require a phone number at registration time

(e.g., Facebook Messenger).

Availability is hugely important to our participants, referring

to it as “reliable connection”. However, they regard it as a

utility feature (see Section IV-A), not a security property.

D. Threat Models

Our participants described different types of adversaries

that can violate the security of communications. We describe

these adversaries and their capabilities in Section IV-D.I. In

Section IV-D.II, we explain how participants think the security

properties of secure communication tools (discussed in Section

IV-C) can be breached.

D.I. Adversaries

All participants, except for P2 and P5, believe that the

security of any communication tool can be breached by threetypes of adversaries: (1) intelligence agencies, (2) application

service providers, and (3) technically-skilled attackers.

Intelligence agencies. 58 out of 60 participants believe gov-ernment agencies (e.g., NSA, GCHQ) have the resources and

capabilities required to monitor any citizen. They also believe

that governments can coerce or compel service providers to

hand over all the data related to a particular user. 21 par-

ticipants believe governments do this to protect their national

security; e.g., to prevent terrorism. P51 mentioned a “universaldecryption key” that allows governments to decrypt and readany encrypted communication.

Application service providers. 54 out of 60 participantsthink that all messages pass through the service provider

who “knows how the communication tool works” (P10) and,therefore, is able to access all messages. They also believe

that service providers can access any account stored on their

servers either because passwords are not encrypted, or en-

crypted in a way that can be “reverse-engineered” (P9). Eightparticipants mentioned that companies access the content of

messages not for malicious, but commercial reasons (e.g.,

targeted advertisements, removing inappropriate content). P1,

P12, P13, P35 and P42 reported that when they download

an application to their device, the application asks for their

permission to access PII, geo-location data, photo albums, and

contact lists. To them, this means that providers have ways of

circumventing the security properties of communication tools.

55 participants mentioned that they have to accept a

provider’s Terms and Conditions (T&Cs), which they do not

read because they are “too long” and “intentionally vague”,

and contain “a lot of jargon” (like Data Privacy Policies and

End-user Licence Agreements). 15 participants mentioned that

these terms are regularly updated without users being notified.

Our participants suspected they have agreed, because of a

clause somewhere, that the provider can access their data.

Hence, “having my data anyway” means trying to protect it ispointless (P47).

Technically-skilled attackers. All participants (except forP2 and P5) believe that the use of a secure communication

144

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

tool cannot protect against attackers with technical expertise,

described as hackers, computer science students, or competing

companies (e.g., Apple vs. Google).

Only P2 and P5 said that a secure communication tool is

as secure as the device they install it on, provided that the

security protocols are proved to be secure and implemented

correctly. Reasons for the device not being secure that P2 and

P5 are aware of include software and hardware bugs, malware

(e.g., viruses) and backdoors.

D.II. Violating the Security of Communications

Below, we explain how participants believe the security

properties of secure communication tools (discussed in Section

IV-C) can be violated.

Secrecy of message content. Almost all participants (exceptfor P2, P4, P5, P6, P9 and P28) believe that information

exchanged via any tool can be accessed and read by (1)physically accessing the user’s mobile phone or PC, and

reading messages from the chat history, (2) a communication

partner colluding with a third party and sending them the chat

history, (3) accessing the microphone and speaker to listen to

phone calls using some “sophisticated techniques”, (4) using

CCTV cameras to capture exchanged messages on a users’

device screen, or (5) falling for a social engineering attack.

Some participants also believe that confidentiality (i.e.,

secrecy of message content) can be easily breached by the

service provider because when users download an application,

it asks for their permission to access the device’s contact list,

camera, microphone and photo gallery. According to P1, if

the user decides not to agree to such a request, they will not

be able to exchange photos with others. This finding is in

line with the threat model explained earlier in Section IV-D.I.

P8 also reported that providers access log files to perform

quality monitoring of the service, hence, they can read the

information exchanged if they want to. She also mentioned that

a law enforcement agency that has a subpoena can “obviously”

access users’ information.

Only P2, P4, P5, P6, P9 and P28 mentioned eavesdrop-

ping, wiretapping or decrypting cipher-texts. No participant

explicitly talked about man-in-the-middle attacks (although we

cannot rule out that these attacks could have been part of the

“sophisticated techniques” mentioned above). P6 believes that

confidentiality can be breached by wiretapping the commu-

nications between one point and another, though he believes

that as long as “basic encryption, which is signing in to anapplication” is used, this attack can be avoided. He thinks thepassword used to log in to an account is a form of encryption

to protect the data in transit against unsophisticated attackers

(other members of the public).

P9 also mentioned that if many people use a communication

tool (whether secure or not), there will be “billions of messagesbeing exchanged via the network”. This, he believes, makesit hard to identify a message sent by a particular person. He

thinks that as long as a tool has a large user base, attackers

cannot associate exchanged messages with specific parties,

even if messages are sent in cleartext.

P2, P4 and P5 believe that confidentiality can be breached

through social engineering attacks, exploiting vulnerabilities,

using weak cryptographic schemes, or inserting backdoors.

Only P2, P4, P5 and P6 mentioned the terms “encryption” or

“decryption”, albeit with simplistic mental models. We discuss

participants’ mental models of encrypted communications in

detail later in Section IV-E.

Message integrity. As discussed in Section IV-C, thissecurity property was not mentioned by any participant. When

we hinted at it, all participants said that messages should

be protected from modification, but many did not think that

messages can be modified in transit (50 out of 60). P3 believes

her messages have never been modified because her phone has

never been stolen, and her account “has never been hacked”.Thus, no one can send modified messages from her account.

She believes that integrity is assured as long as authentication

takes place. 21 other participants share P3’s belief. Many

believe that their messages cannot be tampered with, which

is in stark contrast to their other belief that confidentiality

cannot be achieved.

P4 does not worry about integrity being breached because

“any message modification can be detected even after somepoint in time” by the recipient (a belief shared by P11, P25,P49 and P60). P4 believes that if someone sends a message

encrypted and then it gets modified in transit by an attacker, the

recipient will receive “nonsense”, and resending the message

will resolve the problem. 30 participants said they have never

thought of the possibility that messages can be tampered with

because, as P11 put it, “the chat history does not change whensending a message”.

P6, P12 and P18 believe that integrity does not get breached

unless people live under a repressive regime. Hence, govern-

ments can modify or censor communications. 40 participants

believe that service providers can tamper with messages,

however, P12 thinks it is not worth the effort: “this wouldrequire someone to have access to the intermediate serverbetween me and the recipient, so it could probably only bedone by someone within the company, who has access to thecentral server. But, this is unlikely, and I don’t know whythey would do it either, so I think it’s a very small concern”.P13 reported that message integrity can be violated if the

application software has a “bug”.

None of the participants knows how integrity can be

achieved, except for P2 and P5 who correctly explained

hashing and digital signatures. We discuss participants’ mental

models of digital signatures in Section IV-E.

“No impersonation”. All participants believe that as longas passwords are hard to guess or steal, authentication is

achieved. Passwords can be stolen by hacking, social engi-

neering, or brute forcing.

According to our participants (41 out of 60), hacking means

(1) stealing the username and password by mounting a social

engineering attack, guessing the password, intercepting the

password when logging into the application, or stealing the

password from the company’s server, (2) logging into the

account on behalf of the legitimate user, and then (3) reading

145

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

messages from the victim’s chat history and accessing PII.

Many participants (32 out of 60) believe that hacking generally

happens over the “Internet”; the traditional network (3G) is

more secure and, as a result, hacking is impossible.

All participants think social engineering attacks are possible,

and that they need to be aware of these attacks. They believe

security can be increased by not writing passwords down and

by changing them regularly, but doing so is onerous.

43 out of 60 participants mentioned that passwords can

be brute-forced. Furthermore, 21 out of 60 stated that an

attacker can create fake accounts to impersonate others, but

“the company providing the service should be aware of thisand ensure this does not happen” (P4). 25 participants alsobelieve that providers store passwords encrypted on their

servers: “they [service providers] are immune to brute-forcingattacks because encryption is used to protect credentials” (P9).

E. Mental Models of (Secure) Communications

During the interview, we asked our participants how a

communication tool works, and who the actors in a commu-

nication system are. We also asked about different security

mechanisms, such as encryption, digital signatures and cryp-

tographic fingerprints. We provided participants with an iPad

and a stylus pen, so they would draw if they wished to explain

a specific concept (e.g., encryption). This helped us identify

whether our participants know the mechanisms used to achieve

a particular security property, such as associating encryption

with confidentiality, and how this relates to their threat models

in Section IV-D. We also found a misconception about deleting

accounts shared by most participants.

Actors in a communication system. All participants,except for P1 and P11, believe the actors in a communication

tool are the sender, the recipient(s) and a single service

provider, referred to as the “company providing the service”.

This architecture is the same, irrespective of whether the

information exchanged is via telephony, SMS, email or IM.

P12 mentioned that the topology of a 3G network is different

from that of the Internet (or Wi-Fi). She incorrectly believes

there are only the sender and the recipient(s) in a 3G network

without a provider.

P1 has never thought of how a communication tool works.

She said the process is “too complicated” for her to think

about. As long as the message is “sent”, “delivered” and

“read”, she will be satisfied. Also, P11 does not know how

communications work.

An important finding of our study is that unlike experts’

network centric view, our participants’ mental models are

somewhat “ego-centric”: they see themselves as the centre

of their personal communications universe and being able

to choose across different tools, which they see as separate

channels. For example, 18 participants think that segmenting

information and sending different “bits” via different tools

means segments cannot be intercepted by the same attacker.

Participants assume that attackers can hack one tool or listen to

one channel. Participants who have more technical expertise

(P2, P4, P5, P16 and P28) showed the same basic mental

models (i.e., ego-centric models).Encrypted communications. When we asked our partici-

pants how secrecy of message content can be achieved, P2, P4,

P5 and P6 mentioned the terms “encryption” or “decryption”

(albeit with simplistic mental models). The remaining partic-

ipants did not. Hence, we probed and asked what encryption

is, why it is used, and how it works (including client-server

and E2E encryption, as distinguished by the EFF Scorecard).Ten participants confused encryption with authentication.

Nine mentioned “multiple encryption”: using a username and

multiple passwords to log in to an account. P12 mentioned

“double encryption” to describe two-factor authentication. In

other words, “encryption would be something like what banksuse. I have a mobile banking app, but they send me a codein the post, so only I have it, so protection means only I canaccess it in a way with the unique code” (P12). P19 statedthat when encryption is used, “it will be harder to get to thedata because of the passcode and password used to log in tothe account”. He believes that encryption is used to protectthe company providing the service from other companies and

“hackers”. P17 also described encryption as using the account

password in a way to protect the data in transit; the more

passwords the account has, the stronger the encryption is.P1 and P59 conflated encryption with data encoding. P1

explained encryption as sending messages in “computer lan-guage: 01010011110100” (i.e., binary representation) and said“these messages can only be understood by computer scien-tists, hackers, service providers and governments. Lay peoplecannot”. P59 explicitly described encryption as sending textin “binary language: 122121122”.

Other participants explained encryption as follows:

1) Turning a message into random text that people cannot

understand (27 out of 60).

2) Using a special language, such that if someone (like a

computer scientist) knows the language, they can decrypt

the message (P26, P27, P32 and P35).

3) Using a special code (P14 and P27).

4) Making conversations “invisible” (P14 and P60).

5) Slowing down the process of understanding the data;

“encryption is (no encryption + adding some time tosend the data packets)” (P23).

6) Using proxies when accessing websites to protect

against attackers (P29).

Seven participants said they have not heard of encryption

and, hence, did not provide any definition.All participants, except for P2, P4 and P5, believe that

encryption protects against the unsophisticated attackers “whodo not know how to hack” (P32). They believe that serviceproviders should not be able to read exchanged messages

in theory, but “this sort of encryption” (P9) is not offeredby existing communication tools. They think that encrypted

communications are futile because the designers who create

the encryption scheme know how to decrypt messages. As

P15 put it, “even the ultimate encryption can be broken, likethe ENIGMA machine in WWII”.

146

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

Only P2, P4 and P5 distinguished between client-server

encryption and E2E encryption; they provided a good (al-

though simplistic) understanding of both types of encryption

and discussed private-key and public-key cryptography. They

also stated that E2E encryption could protect against all types

of attackers.

The 57 remaining participants either did not know the

difference between both types of encryption or gave wrong

answers. For example, P13 equated client-server encryption

to SSL, and described E2E encryption as a special encryption

program (or software) used to manually encrypt messages. P16

equated keys to passwords, describing client-server encryption

as using one key (one password) for encryption and decryp-

tion, whereas E2E encryption as using two different keys (two

passwords): one for encryption and one for decryption.

Passcodes, digital signatures and fingerprints. Sometools, such as Telegram, allow users to set up a passcode

to lock their accounts. However, 45 participants said they do

not set up a passcode because it is time-consuming to unlock

accounts. They see the phone lock of their handset as sufficient

(i.e., Apple’s touch ID or passcode, Android’s pattern/PIN

lock). Others (P4, P11, P14, P15, P39, P40, P56) explicitly

said that locking the application has the undesirable effect of

being notified that a message has been received without thesender’s name and text. This is another example of a security

feature reducing the utility users are looking for.

57 participants (excluding P2, P4 and P5) provided various

incorrect explanations of digital signatures: (1) inserting a

USB stick into the PC to sign a document using a unique

code, (2) scanning a hand-written signature and then adding

the signature electronically to a document, or (3) signing a

digital document using a stylus pen. P29 described a digital

signature as a specific font type in Microsoft Word used to

type names. Only P2 and P5 correctly explained what digital

signatures are.

We also asked about verification fingerprints, and only P2

was able to explain them. All participants who use Telegram,

for example, believe that the fingerprint in the Secret Chatmode is the encryption key shared between the sender and the

recipient to encrypt and decrypt messages in transit, or the

encrypted message itself.

Account Deletion. At the beginning of the study, weasked our participants to take out their mobile phones and

check all the communication tools they have downloaded. All

participants (except for P2, P4, P5 and P28) uninstalled a

communication tool when they decided to stop using it, be-

lieving their accounts and chat history have been removed. We

can attribute this misconception to misleading feedback from

devices: both iPhone and Nexus warn their users that their

data will be deleted if they “delete” a particular application.

The warning message does not specify whether “all” the data

deleted is the application-related data stored on the phone, or

the data associated with the account on the provider’s servers.

F. Security Ranking of Communication Services: Calling vs.Messaging

We asked our participants to rank the communication tools

they have used in terms of how secure they are. Many partici-

pants ranked the services offered by the tools first, rather than

ranking the tools. Our participants exhibited high agreement

on the relative ranking of services (calling and messaging).

All, but seven participants, agreed on the following ranking,

ordered from the most to least secure:

1) Voice calls via the mobile network.

2) Voice calls via the Internet (e.g., Wi-Fi).

3) SMS messages (mobile network).

4) Emails (Internet).

5) Instant messages (Internet).

Seven participants (P2, P5, P37, P42, P45, P47 and P51)

disagreed with the ranking above, noting that voice calls

have the same security level as messages because several

communication tools (e.g., WhatsApp, Google Hangouts) offer

both services.

Calls are more secure than messages. Below, we discussthe reasons given by our participants for why calls are more

secure than messages:

1) According to most participants (53 out of 60), there

is no mass surveillance of phone calls. They are aware that

phone calls can be intercepted, but think it is unlikely unless a

government agency is monitoring a specific person. According

to P17, the calling parties “need to be targeted during theirconversation. This requires special wiretapping equipment”.

2) Nine participants believe that routine recording of phone

calls requires many resources, such as disk space. Hence, they

do not consider phone calls being recorded and stored on the

provider’s servers a threat. P17 also mentioned that text and

multimedia messages are “discarded from the servers as longas they were not suspicious”. In fact, providers store messagesfor long periods of time [72].

3) Nine participants mentioned that a phone call requires

a lot of time and effort to process and analyze, compared to

a text message. They stated that a human has to listen to a

phone call and extract the sensitive information (as portrayed

in movies, perhaps most memorably “The Lives of Others”).It is onerous to convert audio to text for analysis, whereas

text messages can be easily searched for specific keywords.

We speculate this is because participants are used to word

processors that scan text for words, but have never seen this

technology for scanning audio.

4) Seven participants mentioned that there is a record of

text messages stored on the user’s device. They said that if

the user’s device gets compromised, the adversary can access

all previously sent messages, unless the user deletes their

chat history regularly (something none of our participants

regularly does). P12 also mentioned that it should be common

practice not to write sensitive information down on a piece

of paper or as a text message, regardless of whether the

tool is secure or not. Sensitive information should be shared

in person, or via a phone call (if the situation is urgent)

147

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

“because there is no chat history of calls”. 16 participantsmentioned that it is possible to capture a sensitive exchange

by taking a screen-shot of a message, not something attackers

can do with a phone call. This finding suggests users have a

rudimentary understanding of forward secrecy, unconnected to

the cryptographic definition.

SMS is the most secure messaging service. We havediscussed why users regard voice calls more secure than

messages above. We here provide the rationale behind why

SMS messages are perceived as the most secure, while emails

the second most secure, and instant messages the least secure.

According to our participants:

1) Telephone service providers, as opposed to email (and

IM) service providers, are regulated by the government. Hence,

the mobile phone network can protect against competing

companies seeking intelligence, as opposed to the Internet (33

out of 60).

2) Many banks send banking details and notifications (re-

garded as sensitive information by our participants) via SMS

messages, so SMS must be secure (32 out of 60).

3) SMS is accessible only through the “Messages” applica-

tion on the phone, whereas email systems and IM tools can

be accessed through the PC as well, increasing the scope of

vulnerability (P21, P26, P29, P39 and P50).

4) Emails and instant messages (text and multimedia mes-

sages) are less secure than SMS messages because email

systems and IM tools are “free” (30 out of 60), and the Internet

is less secure than other networks (e.g., 3G) (see point 1

above). According to P12, “privacy is a general problem ofthe Internet”. In contrast, P2 and P5 believe it is possible tocommunicate over the Internet securely if vulnerabilities do

not exist.

5) Email was designed to send formal messages and not

to socialize, as opposed to IM tools (28 out of 60). As far as

our participants are concerned, formality of messages indicates

better security. In contrast, P12 believes that Gmail (an email

service) and Google Hangouts (an IM tool) are one entity,

hence, they have the same level of security. Also, P17 and P24

mentioned that their Yahoo! email account has been hacked,

hence, Yahoo! Messenger is perceived as insecure because

Yahoo! email and Yahoo! Messenger are one entity. We discuss

this theme in more detail in Section IV-G.

Some participants (29 out of 60) believe that “professional”

email (e.g., Outlook, P11’s university email) is more secure

than “commercial” email services (e.g., Gmail), provided that

the sender and the recipient have professional email accounts.

According to P11, there is no clear evidence that Outlook is

more secure than Gmail. However, since she receives more

spam emails in her Gmail’s spam folder, she believes that

Gmail is less secure. Also, P11’s university sends regular

warnings about spam emails, which is interpreted as a sign

that the university cares about protecting Outlook, as opposed

to Gmail that “only has a folder for spams”. Here, we havean example of effortful but visible security that makes the

participant believe that Outlook is secure, whereas security

being done automatically (i.e., the filtering done by Gmail)

makes her perceive Gmail as insecure due to invisible security.

Other participants (15 out of 60) feel secure as long as

they use their university email account, even if the recipient

does not use the same email system. P14 and P18 believe

that the university email account is more secure than Gmail

because the university (an educational, non-profit organization)

owns the service and is responsible for protecting it. This

misconception can be attributed to the ego-centric models

explained earlier in Section IV-E.

G. Security Ranking Criteria of Communication Tools

We here discuss the reasons for our participants’ rankings

of the communication tools they have used, not the services

offered by the tools. We provided participants with cards with

the names and logos of the tools, and then asked them to

rank them from the most to the least secure. Our aim was

not to analyze the rankings, but to elicit the rationale behind

our participants’ choices. We found that our participants base

their security rankings of communication tools on several

adoption criteria discussed earlier in Section IV-A, namely

(1) users bases, (2) QoS, (3) cost of use, (4) registration:

telephone numbers vs. usernames, and (5) social influence,

rather than on the security properties they expect from a

secure tool. Below, we discuss the different reasons given by

our participants to justify their rankings of the tools (without

necessarily mentioning the most recurrent reasons first).

User bases. 20 participants believe that popular communi-cation tools (e.g., Facebook Messenger, WhatsApp) have large

user bases and, hence, they are more likely to be targeted. 10

participants, on the other hand, believe that Facebook Mes-

senger is more secure than Yahoo! Messenger because more

people use the former and, hence, there is more investment to

secure it.

QoS. The QoS our participants experience while using atool influences their perceptions of how secure the tool is

(40 out of 60). For example, P7 and P17 said that Viber

has low audio/video quality: “the signal is bad, and thereare continuous disconnections” (P7), which means it is alsoless secure compared to other tools. P12 believes that Google

Hangouts is secure because its audio/video quality is better

than that of, for example, Skype.

Cost of use. 40 participants mentioned that “cheap” toolsshould not be trusted. For example, P59 thinks that Blackberry

Messenger Protected offers better security compared to “otherfree tools” because its subscription cost is high. 22 participantsalso said that tools with advertisements are insecure.

Registration: telephone numbers vs. usernames. 27 par-ticipants perceive WhatsApp as more secure than other tools

because it requires a phone number when creating an account.

They said that using the phone number is a guarantee the

account can only be accessed from the users’ phone. The

phone is seen as strongly linked to the communication partner,

whereas other IM tools that require a username and a password

can be “easily hacked”. P2, P5 and P48 see no difference

between both methods.

148

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

Integration with other tools. 25 participants distrust toolsused in combination with other less secure tools. For instance,

10 participants said that if a user imports their personal details

from Facebook to WhatsApp, WhatsApp’s security will drop

to that of Facebook.

Tools integrated with SMS. Many participants believe thatSMS is more secure than IM for several reasons previously

discussed in Section IV-F. However, 12 participants who use

iMessage and Google Hangouts on their phone have the mis-

conception that these two IM tools are equivalent to SMS and,

hence, have the same security level. For instance, P6 stated

that “iMessage is designed as part of Apple’s SMS service”.He sends banking details via iMessage for this reason.

Attractive UIs. 22 participants stated that if the tool creatorscare enough to make the tool usable, they will also care

about its security. A “bad” (unattractive) UI is a sign that the

developer “does not care” or is not competent, so the security

of the tool is also likely to be shoddy. P17 and P23 cited Kik

and Ebuddy XMS as examples. This finding shows that a good

user experience on one aspect of the tool increases trust in the

competence and motivation of the developers.

Visible security. Visible security indicates “there must bea threat”. 21 participants believe that the mobile version of a

communication tool is more secure than other tools accessed

via browsers because users do not have to deal with HTTPS

locks and certificates. Hence, they prefer to have a stand-

alone desktop application similar to that on the mobile phone.

According to P27, “the information is just on your device, itis not easy to access data on a personal device, as opposedto the web browser”.

An emerging theme is that our participants’ experience of

warning messages and need for security indicators lead them

to perceive the services they access via web browsers as

insecure. Applications on mobile phones have comparatively

fewer indicators and warnings and are, thus, perceived to be

more secure, despite this being technically incorrect [73], [74].

30 participants also think that the probability of a mobile

phone getting infected by a “virus” is lower than that of a

PC because (1) they have never experienced any issue with

their phones, unlike PCs, and have never installed a mobile

phone version of an anti-virus program, and (2) the sender of

an instant message is known, unlike SMS and email: “thereare spam emails, but not spam instant messages” (P18).

Social influence. Social factors largely influence partici-pants’ perceptions of the security offered by a communication

tool (54 out of 60). Some tools are deemed more secure

and trustworthy than others because a friend, colleague, or

newspaper article said so.

Geopolitical context. The local laws and practices thata service provider is subject to influence perception. P12

believes Facebook Messenger is less secure than other tools

because Facebook is US-based. She believes that US gov-

ernment agencies, the NSA in particular, are able to read

transmitted data. Hence, she does not share sensitive infor-

mation via Facebook Messenger. Five participants mentioned

that Threema is the most secure tool because Germans “who

are more privacy-concerned” use it extensively, showing the

“crowd follower” characteristics described in [75].

Self-destructing messages. P15 and P43 believe Telegram’sSecret Chat mode deceives participants into thinking thatmessages are deleted from the recipient side, when they are

actually stored on the server. They compare Telegram to

Snapchat and believe both are insecure.

Open-source vs. proprietary tools. Kerckhoffs’ principleof avoiding security-by-obscurity is well-established in the

cryptographic literature. However, 51 out of 60 participants

largely believe obscurity is necessary for security. P6, P12,

P13, P18, P26, P36 and P59 explicitly stated that Apple

products are secure because they are closed-source. However,

Garman et al. found significant vulnerabilities in iMessage

that can be exploited [76]. Our participants are not aware of

the long line of cases where proprietary encryption schemes

have been broken, despite recent high-profile cases, such as

the Volkswagen key [77].

Finally, seven participants (P3, P4, P8, P11, P19, P22 and

P26) did not rank the communication tools, perceiving them

to have the same level of security for several reasons:

No clear understanding of security. P3, P4, P8, P11and P26 did not compare the tools. They said they do not

understand what makes a communication tool secure. P8 said

that companies do not provide a clear definition of security

because “things are always changing”, and what is securetoday will not be secure tomorrow. Legal liability is seen as

another reason: P26 believes companies want to be able to

change the definition of security in privacy policies in response

to developments.

Security is expensive. P3, P19, P22 and P26 believe noneof the tools are secure because security is expensive, and the

companies who own these tools put profit first. They said that

PII and conversations are not protected because most tools

are free. Without data collection, advertisements cannot be

generated and, hence, there will be no profits.

Past experiences. P19 and P22 believe that all messengersare secure because they have never experienced a breach.

P24 and P46, in contrast, experienced a security breach with

Yahoo! Messenger: “But, talking about this Yahoo! thing, myYahoo! email account is probably one of the least securebecause actually, you know, it has got hacked again recently”(P46). Hence, they believe all tools are insecure.

Security is not possible. P8 believes that “completelysecure” tools exist only in theory. Due to bugs, software can beattacked and communications traced. P2 and P12 were the only

participants to mention that one can evaluate the security of a

tool based on how well the program is written, and that source

code should be audited. P12, however, believes that audits need

to be confidential because the designs of secure tools should

not be published (see Section IV-D on threat models).

H. EFF Secure Messaging Scorecard

We provided our participants with the first-generation EFF

Secure Messaging Scorecard [2] (printed on a sheet of pa-

per), and invited them to compare their rankings with those

149

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

of the scorecard. Not a single participant gave a ranking

that reflected the scorecard. The scorecard contains seven

security criteria. Four criteria are completely misunderstood:

participants do not appreciate the difference between point-

to-point and E2E encryption, and do not comprehend forward

secrecy and fingerprint verification. The other three criteria

reflecting open design (documentation, open-source code and

security audits) are considered to be negative, with participantsbelieving security requires obscurity. We describe below how

participants perceive the importance of the scorecard’s criteria.

Encrypted in transit vs. encrypted so the providercan’t read it. 57 participants (except for P2, P4 and P5) donot differentiate between point-to-point encryption and E2E

encryption. Recent literature [41] suggests that users develop

more trust in an encrypted communication system that makes

the cipher-texts visible. However, whether the cipher-text is

visible or not, our participants do not know what security

properties each tool offers, and they (incorrectly) believe that

encryption can be broken anyway (see Section IV-D).

Can you verify contact’s identity? Recent studies [50],[51] have assessed the usability and security of various repre-

sentations of verification fingerprints. However, no participant

(except for P2) appreciates why some communication tools

can verify a contact’s identity (i.e., the role of fingerprints).

Are past communications secure if your keys are stolen?All participants (except for P2 and P5) do not recognize the

importance of forward secrecy.

Open design. The EFF Scorecard has three explicit criteriato ensure the design and code have undergone independent

reviews. Our participants, in contrast, said proprietary tools

are more secure. This belief in “security by obscurity”, an

anathema to security researchers, stems from the fact that users

perceive security properties to be akin to trade secrets: if a

skilled attacker learns how a tool works, they can compromise

it. This fundamental misconception feeds the perception of

futility. Only P2, P5 and P28 appreciate open design.

V. DISCUSSION

Most user studies of secure communication tools, in particu-

lar encrypted email, have been lab studies conducted following

the same pattern (see Section II): assessing the usability of

specific tools in an artificial setting, where participants are

given a series of security tasks associated with those tools

(e.g., managing keys, sharing keys, encrypting a message)

with fictional communication partners (study coordinators) to

accomplish a particular security goal (e.g., confidentiality)

without errors, and then measuring success, or failure, based

on the goals and tasks imposed on participants, rather than

being their own.

Indeed, users will not adopt a communication tool if they

cannot use it effectively and efficiently. Our study identified

some usability problems (e.g., participants who used Telegram

were not able to recognize the Secret Chat mode). However,our results also show that to be adopted, secure tools have

to offer their intended users utility; i.e., the ability to reach

their communication partners. Security may be part of users’

primary communication goals, but given a choice between a

usable and secure tool that does not offer utility and a usable

but insecure tool that does, users choose the latter. Our results

suggest it is unrealistic to expect that users will switch to

secure tools and only communicate with those who do the

same. Also, they will not expend the effort associated with

maintaining two communication tools (one secure and one

insecure) depending on whom they are talking to. For example,

our participants with iOS devices used WhatsApp and Skype,

instead of iMessage and FaceTime, even when communicating

with other Apple users. Although they perceived the Apple

services as more secure (see Section IV-G), they did not live

in an Apple-only universe; using different tools was perceived

as an overhead they were not willing to carry for security.

When a new tool is usable and attractive enough, users

may accept the initial switching cost and adopt it. However,

creating a new tool that will be adopted by a critical mass of

users requires resources and a set of skills (e.g., user research,

user experience design, communication, affective interaction,

marketing) the creators of secure communication tools do

not have at their disposal. If we want users to adopt secure

communications in the near future, security engineers should

consider putting their skills to securing tools that have a large

use base. WhatsApp’s implementation of E2E encryption for

text, voice calls and video communications is an example of

this more pragmatic approach [18].

In [61], De Luca et al. found that security and privacy are

not a primary factor that drives users to adopt a particular

messenger. We argue that this is not because users do not

care about security at all. Users are aware of some threats and

willing to make some effort to manage them (e.g., by chopping

up credentials into segments and sending these via different

tools). Our participants preferred these quite cumbersome

processes, instead of using a secure tool, because they did not

believe the tools available are actually secure. This impression

was fed by several misconceptions (e.g., they believed service

providers can read E2E-encrypted messages). Besides the lack

of usability and utility, such misconceptions undermined the

case for adoption in their eyes.

There are some users who want to be secure and are

“shopping” for tools that offer specific security properties.

The EFF Secure Messaging Scorecard [2] aims to tell users

about what security properties various communication tools

actually offer. Our findings show that the scorecard is not

supporting typical users effectively because our participants

did not understand these fine-grained security properties. In-

deed, participants believed these properties are either impos-

sible to achieve or detrimental to security (like open design).

These misunderstandings cannot be fixed by just changing the

wording on the scorecard, as our results show that participants

had very inaccurate understanding of fundamental security

properties, such as confidentiality (see Section IV-E).

The key takeaway from mental models research is that

non-experts do not understand abstract security properties.

They can only understand why a property matters in the

context of a specific threat model that matters to them. For

150

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

example, if users do not want their service providers to be

able to read their messages, we need to explain how E2E

encryption protects against this threat. Based on our results,

our participants’ existing models were the “toxic root” of their

belief that ultimately using any form of a secure tool is futile

because they believed even the best encryption scheme can

be broken by the resources and skills of governments and

service providers. We need to make users understand that it is

in their power to protect themselves because several security

mechanisms have been developed based on the best available

knowledge from security research, and are open to audits by

security researchers and practitioners.

Based in part on our feedback, the EFF is redesigning

the scorecard to group tools into general tiers from “most

secure” to “insecure”. Instead of check marks for specific

properties, textual descriptions will be provided for what

security properties each tool provides. The goal is to help

casual readers correctly understand which tools are considered

secure (e.g., E2E-encrypted) without needing to understand

security mechanisms specifically, while also providing text to

help readers acquire accurate mental models of confidentiality,

integrity and authentication. The scorecard will also attempt to

provide more non-security information that users desire: Does

the tool have a large user base? What devices/platforms is it

available on? Can it be used over 3G and Wi-Fi? Does it offer

audio or video chats? Is the tool free? While not necessarily

related to security and privacy, these items drive adoption and

would be recommended to include them in the scorecard.

A final interesting high-level observation is that while efforts

to secure email systems with PGP that were interoperable

across email providers failed on the usability front, current

approaches (e.g., iMessage) succeed on the usability front

at the expense of interoperability with different devices. We

believe examining whether some of the lessons learnt from

securing these communication tools can be transferred to

interoperable secure tools without sacrificing usability is an

interesting open research question for the security community.

VI. CONCLUDING REMARKS

Our research, based on 10 unstructured and 50 semi-

structured interviews, provides the broadest study of user

perceptions of secure communications to date. Although our

participants have experienced usability issues with different

communication tools, these are not the primary obstacles

to adopting secure tools. Low motivation to adopt secure

communications is due to several factors (e.g., small user

bases, lack of interoperability, incorrect mental models of

how secure communications work). Based on our findings,

we conclude with three concrete recommendations:

Secure tools with proved utility. We encourage the securitycommunity to prioritize securing the communication tools

that have already been adopted by mainstream users over

improving the usability of different secure tools. Users’ goal to

communicate with others overrides everything else, including

security. Growing a user base for a new tool is difficult and

unpredictable. Therefore, we encourage security researchers to

work with today’s existing popular tools.

Understand the target population. In the long run, ifsecurity developers want to develop new paradigms and secure

communication tools using a user-centered design process,

they need to understand users’ goals and preferences. The

technical security community must develop a deeper under-

standing of what is important (and not important) to users.

Security properties and threats should be framed in terms that

users can understand.

Improve QoS. Secure communication tools must feel pro-fessional. Security itself is difficult for users to evaluate

directly; they often use proxy signals. This suggests that

engineering effort spent on improving the performance of

cryptographic tools still matters to the extent that it can reduce

latency and dropped packets.

VII. ACKNOWLEDGMENTS

We thank the reviewers for their helpful comments and

suggestions. This work is supported by a gift from Google.

Joseph Bonneau is supported by a Secure Usability Fellowship

from the Open Technology Fund and Simply Secure.

REFERENCES

[1] N. Unger, S. Dechand, J. Bonneau, S. Fahl, H. Perl, I. Goldberg, andM. Smith, “SoK: Secure Messaging,” in IEEE Symposium on Securityand Privacy, 2015, pp. 232–249.

[2] Electronic Frontier Foundation (EFF), “Secure Messaging Scorecard,”https://www.eff.org/secure-messaging-scorecard, accessed on:09.07.2016.

[3] D. Yadron, “Apple Transparency Report: Over 1,000 GovernmentRequests for User Data,” The Guardian, 2016.

[4] S. Gibbs, “Gmail Does Scan All Emails, New Google Terms Clarify,”The Guardian, 2014.

[5] R. Anderson, “Why Cryptosystems Fail,” in ACM Conference onComputer and Communications Security, 1993, pp. 215–227.

[6] S. Fahl, M. Harbach, H. Perl, M. Koetter, and M. Smith, “RethinkingSSL Development in an Appified World,” in ACM Conference onComputer and Communications Security, 2013, pp. 49–60.

[7] A. Whitten and J. D. Tygar, “Why Johnny Can’t Encrypt: A UsabilityEvaluation of PGP 5.0,” in USENIX Security Symposium, 1999.

[8] S. L. Garfinkel and R. C. Miller, “Johnny 2: A User Test of KeyContinuity Management with S/MIME and Outlook Express,” in ACMSymposium on Usable Privacy and Security, 2005, pp. 13–24.

[9] S. Clark, T. Goodspeed, P. Metzger, Z. Wasserman, K. Xu, andM. Blaze, “Why (Special Agent) Johnny (Still) Can’t Encrypt: ASecurity Analysis of the APCO Project 25 Two-Way Radio System,”in USENIX Security Symposium, 2011, pp. 8–12.

[10] M. Lee, “Encryption Works: How to Protect Your Privacy in the Ageof NSA Surveillance,” Freedom of the Press Foundation, 2013.

[11] “Tips, Tools and How-tos for Safer Online Communications,”https://ssd.eff.org/en, accessed on: 19.08.2016.

[12] McGregor, Susan E, “Digital Security and Source Protection forJournalists,” http://towcenter.org/digital-security-and-source-protection-for-journalists-research-by-susan-mcgregor/, accessed on: 20.08.2016.

[13] “The OpenPGP Alliance Home Page,”http://www.openpgp.org/resources/downloads.shtml, accessed on:20.08.2016.

[14] “Tor,” https://www.torproject.org/projects/torbrowser.html.en, accessedon: 20.08.2016.

[15] “Tails: The Amnesic Incognito Live System,” https://tails.boum.org/,accessed on: 20.08.2016.

[16] “Off-the-Record Messaging,” https://otr.cypherpunks.ca/, accessed on:20.08.2016.

[17] “SecureDrop: The Open-source Whistleblower Submission System,”https://securedrop.org/, accessed on: 20.08.2016.

151

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

[18] Natasha Lomas, “WhatsApp Completes End-to-End EncryptionRollout,” https://techcrunch.com/2016/04/05/whatsapp-completes-end-to-end-encryption-rollout, accessed on:09.09.2016.

[19] A. J. Onwuegbuzie and N. L. Leech, “Validity and QualitativeResearch: An Oxymoron?” Quality & Quantity, vol. 41, no. 2, pp.233–249, 2007.

[20] A. Strauss and J. Corbin, “Grounded Theory Methodology,” Handbookof Qualitative Research, pp. 273–285, 1994.

[21] B. Harry, K. M. Sturges, and J. K. Klingner, “Mapping the Process:An Exemplar of Process and Challenge in Grounded TheoryAnalysis,” Educational Researcher, vol. 34, no. 2, pp. 3–13, 2005.

[22] P. R. Zimmermann, The Official PGP User’s Guide, 1995.[23] “GPGTools,” https://gpgtools.org/, accessed on: 11.07.2016.

[24] “GPG4WiN,” https://www.gpg4win.org/, accessed on: 11.07.2016.

[25] “Off-the-Record Communication, or, Why Not To Use PGP,” in ACMWorkshop on Privacy in the Electronic Society, 2004, pp. 77–84.

[26] C. Alexander and I. Goldberg, “Improved User Authentication inOff-the-Record Messaging,” in ACM Workshop on Privacy in theElectronic Society, 2007, pp. 41–47.

[27] J. Bian, R. Seker, and U. Topaloglu, “Off-the-Record InstantMessaging for Group Conversation,” in IEEE International Conferenceon Information Reuse and Integration, 2007, pp. 79–84.

[28] R. Stedman, K. Yoshida, and I. Goldberg, “A User Study ofOff-the-Record Messaging,” in ACM Symposium on Usable Privacyand Security, 2008, pp. 95–104.

[29] I. Goldberg, B. Ustaoğlu, M. D. Van Gundy, and H. Chen,“Multi-party Off-the-Record Messaging,” in ACM Conference onComputer and Communications Security, 2009, pp. 358–368.

[30] H. Liu, E. Y. Vasserman, and N. Hopper, “Improved GroupOff-the-Record Messaging,” in ACM Workshop on Privacy in theElectronic Society, 2013, pp. 249–254.

[31] “Open Whisper Systems: Signal,”https://whispersystems.org/blog/signal/, accessed on: 11.07.2016.

[32] R. Perlman, “The Ephemerizer: Making Data Disappear,” SunMicrosystems, Inc., 2005.

[33] R. Geambasu, T. Kohno, A. A. Levy, and H. M. Levy, “Vanish:Increasing Data Privacy with Self-Destructing Data,” in USENIXSecurity Symposium, 2009, pp. 299–316.

[34] J. Reardon, D. Basin, and S. Capkun, “SoK: Secure Data Deletion,” inIEEE Symposium on Security and Privacy, 2013, pp. 301–315.

[35] R. Housley, W. Polk, W. Ford, and D. Solo, “Internet X. 509Public-key Infrastructure Certificate and Certificate Revocation List(CRL) Profile,” Tech. Rep., 2002.

[36] P. Gutmann, “PKI: It’s Not Dead, Just Resting,” Computer, vol. 35,no. 8, pp. 41–49, 2002.

[37] M. D. Ryan, “Enhanced Certificate Transparency and End-to-EndEncrypted Mail,” in Network and Distributed System SecuritySymposium, 2014.

[38] M. Melara, A. Blankstein, J. Bonneau, M. Freedman, and E. Felten,“CONIKS: Bringing Key Transparency to End Users,” in USENIXSecurity Symposium, 2015.

[39] G. Cluley, “WhatsApp Doesn’t Properly Erase Your Deleted Messages,Researcher Reveals,” https://www.hotforsecurity.com/blog/whatsapp-doesnt-properly-erase-your-deleted-messages-researcher-reveals-16169.html, accessed on: 02.08.2016.

[40] S. Ruoti, N. Kim, B. Burgon, T. Van Der Horst, and K. Seamons,“Confused Johnny: When Automatic Encryption Leads to Confusionand Mistakes,” in ACM Symposium on Usable Privacy and Security,2013, p. 5.

[41] S. Ruoti, J. Andersen, S. Heidbrink, M. O’Neill, E. Vaziripour, J. Wu,D. Zappala, and K. Seamons, ““We’re on the Same Page”: A UsabilityStudy of Secure Email Using Pairs of Novice Users,” in ACMConference on Human Factors and Computing Systems, 2016.

[42] J. Nielsen and R. Molich, “Heuristic Evaluation of User Interfaces,” inACM Conference on Human Factors and Computing Systems, 1990,pp. 249–256.

[43] W. Bai, D. Kim, M. Namara, Y. Qian, P. G. Kelley, and M. L.Mazurek, “An Inconvenient Trust: User Attitudes toward Security andUsability Tradeoffs for Key-Directory Encryption Systems,” in ACMSymposium on Usable Privacy and Security, 2016, pp. 113–130.

[44] S. L. Garfinkel, D. Margrave, J. I. Schiller, E. Nordlander, and R. C.Miller, “How to Make Secure Email Easier to Use,” in ACM

Conference on Human Factors and Computing Systems, 2005, pp.701–710.

[45] J. F. Ryan and B. L. Reid, “Usable Encryption Enabled by AJAX,” inIEEE International Conference on Networking and Services, 2006, pp.116–116.

[46] S. Sheng, L. Broderick, C. A. Koranda, and J. J. Hyland, “WhyJohnny Still Can’t Encrypt: Evaluating the Usability of EmailEncryption Software,” in ACM Symposium on Usable Privacy andSecurity, 2006, pp. 3–4.

[47] E. Atwater, C. Bocovich, U. Hengartner, E. Lank, and I. Goldberg,“Leading Johnny to Water: Designing for Usability and Trust,” inACM Symposium on Usable Privacy and Security, 2015, pp. 69–88.

[48] S. Ruoti, J. Andersen, D. Zappala, and K. Seamons, “Why JohnnyStill, Still Can’t Encrypt: Evaluating the Usability of a Modern PGPClient,” arXiv preprint arXiv:1510.08555, 2015.

[49] S. L. Garfinkel, “Enabling Email Confidentiality through the Use ofOpportunistic Encryption,” in Annual National Conference on DigitalGovernment Research, 2003, pp. 1–4.

[50] S. Dechand, D. Schürmann, T. IBR, K. Busse, Y. Acar, S. Fahl, andM. Smith, “An Empirical Study of Textual Key-FingerprintRepresentations,” in USENIX Security Symposium, 2016.

[51] J. Tan, L. Bauer, J. Bonneau, L. Cranor, J. Thomas, and B. Ur, “CanUnicorns Help Users Compare Crypto Key Fingerprints?” in ACMConference on Human Factors and Computing Systems, 2017.

[52] S. L. Garfinkel, J. I. Schiller, E. Nordlander, D. Margrave, and R. C.Miller, “Views, Reactions and Impact of Digitally-Signed Mail inE-commerce,” in Financial Cryptography and Data Security, 2005, pp.188–202.

[53] J. Sobey, R. Biddle, P. C. Van Oorschot, and A. S. Patrick, “ExploringUser Reactions to New Browser Cues for Extended ValidationCertificates,” in European Symposium on Research in ComputerSecurity, 2008, pp. 411–427.

[54] A. P. Felt, R. W. Reeder, A. Ainslie, H. Harris, M. Walker,C. Thompson, M. E. Acer, E. Morant, and S. Consolvo, “RethinkingConnection Security Indicators,” in ACM Symposium on UsablePrivacy and Security, 2016, pp. 1–14.

[55] F. Roesner, B. T. Gill, and T. Kohno, “Sex, Lies, or Kittens?Investigating the Use of Snapchat’s Self-destructing Messages,” inFinancial Cryptography and Data Security, 2014, pp. 64–76.

[56] S. Gaw, E. W. Felten, and P. Fernandez-Kelly, “Secrecy, Flagging, andParanoia: Adoption Criteria in Encrypted E-mail,” in ACM Conferenceon Human Factors in Computing Systems, 2006, pp. 591–600.

[57] K. Renaud, M. Volkamer, and A. Renkema-Padmos, “Why Doesn’tJane Protect Her Privacy?” in Privacy Enhancing TechnologiesSymposium, 2014, pp. 244–262.

[58] S. Das, T. H.-J. Kim, L. A. Dabbish, and J. I. Hong, “The Effect ofSocial Influence on Security Sensitivity,” in ACM Symposium onUsable Privacy and Security, vol. 14, 2014.

[59] S. Das, A. D. Kramer, L. A. Dabbish, and J. I. Hong, “IncreasingSecurity Sensitivity with Social Proof: A Large-scale ExperimentalConfirmation,” in ACM Conference on Computer and CommunicationsSecurity, 2014, pp. 739–749.

[60] ——, “The Role of Social Influence in Security Feature Adoption,” inACM Conference on Computer Supported Cooperative Work andSocial Computing, 2015, pp. 1416–1426.

[61] A. De Luca, S. Das, M. Ortlieb, I. Ion, and B. Laurie, “Expert andNon-Expert Attitudes towards (Secure) Instant Messaging,” in ACMSymposium on Usable Privacy and Security, 2016.

[62] P. Dourish, R. E. Grinter, J. D. De La Flor, and M. Joseph, “Securityin the Wild: User Strategies for Managing Security as an Everyday,Practical Problem,” Personal and Ubiquitous Computing, vol. 8, no. 6,pp. 391–401, 2004.

[63] R. Wash, “Folk Models of Home Computer Security,” in ACMSymposium on Usable Privacy and Security, 2010, p. 11.

[64] H. Sharp, Y. Rogers, and J. Preece, Interaction Design: BeyondHuman-Computer Interaction, 2007, vol. 11, no. 4.

[65] P. N. Johnson-Laird, Mental Models: Towards a Cognitive Science ofLanguage, Inference, and Consciousness. Harvard University Press,1983, no. 6.

[66] R. H. Bernard, Non-probability Sampling: Social Research Methods:Qualitative and Quantitative Approaches. SAGE, 2006.

[67] C. Seale, “Quality in Qualitative Research,” Qualitative Inquiry, vol. 5,no. 4, pp. 465–478, 1999.

152

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.

[68] J. Cohen, “A Coefficient of Agreement for Nominal Scales,”Educational and Psychosocial Measurement, vol. 20, no. 1, pp. 37–46,1960.

[69] J. L. Fleiss, B. Levin, and M. C. Paik, Statistical Methods for Ratesand Proportions. John Wiley & Sons, 2013.

[70] D. Cole, “‘We Kill People Based on Metadata’,” http://www.nybooks.com/daily/2014/05/10/we-kill-people-based-metadata/,accessed on: 09.07.2016.

[71] G. Danezis and C. Diaz, “A Survey of Anonymous CommunicationChannels,” Microsoft Research, Tech. Rep., 2008.

[72] “Snapchat Law Enforcement Guide,” http://www.documentcloud.org/documents/717257-snapchat-law-enforcement-guide-12112-1.html,accessed on: 11.06.2016.

[73] S. Fahl, M. Harbach, T. Muders, L. Baumgärtner, B. Freisleben, andM. Smith, “Why Eve and Mallory Love Android: An Analysis ofAndroid SSL (in)Security,” in ACM Conference on Computer andCommunications Security, 2012, pp. 50–61.

[74] M. Georgiev, S. Iyengar, S. Jana, R. Anubhai, D. Boneh, andV. Shmatikov, “The Most Dangerous Code in the World: ValidatingSSL Certificates in Non-Browser Software,” in ACM Conference onComputer and Communications Security, 2012, pp. 38–49.

[75] A. Morton and M. A. Sasse, “Desperately Seeking Assurances:Segmenting Users by Their Information-Seeking Preferences,” in IEEEAnnual International Conference on Privacy, Security and Trust, 2014,pp. 102–111.

[76] C. Garman, M. Green, G. Kaptchuk, I. Miers, and M. Rushanan,“Dancing on the Lip of the Volcano: Chosen Ciphertext Attacks onApple iMessage,” in USENIX Security Symposium, 2016.

[77] R. Verdult, F. D. Garcia, and B. Ege, “Dismantling Megamos Crypto:Wirelessly Lockpicking a Vehicle Immobilizer,” in USENIX SecuritySymposium, 2015, pp. 703–718.

APPENDIX

PRE-SCREENING QUESTIONNAIRE

• Please indicate which of the following ranges your age fallswithin.

Under 1818 – 2021 – 3031 – 4041 – 5051 – 6061 – 7070+

• Please indicate your gender.MaleFemalePrefer not to say

• What is your highest level of education? If you are currentlyenrolled, please specify the highest level/degree completed.

Some high-school educationHigh-school education or equivalentSome college education (incomplete degree)College degree (e.g., BSc, BA)Graduate degree (e.g., MSc, MA, MBA, PhD)Vocational training (e.g., NVQ, HNC, HND)Other

• If you have (or are currently pursuing) a BSc or BA degree,what is your area of study?

• If you have (or are currently pursuing) an MSc, MA or MBAdegree, what is your area of study?

• If you have (or are currently pursuing) a PhD degree, what isyour area of study?

• What is your current employment status?StudentEmployedSelf-employedUnemployedRetired

• If employed, what is your current occupation?• Do you own a desktop computer and/or a laptop?

Yes No• Do you own a smartphone?

Yes No• What communication tools have you ever used? Please select

all that apply.• What computing platforms do you use to communicate with

your contacts via communication tools? Please select all thatapply.

Android (e.g., Google Nexus, Galaxy Samsung)iOS (e.g., iPhone)Microsoft WindowsMac OS XOther

The following questions assessed participants’ general technicalexpertise.

• Do you have an engineering or computer science background?Yes No

• Have you ever configured a network firewall?Yes No Do not know

• Have you ever written a computer program?Yes No Do not know

• Have you ever changed your web browser’s search engine(e.g., Google, Yahoo! Search, Bing, Ask.com)?

Yes No Do not know• Have you ever changed your web browser’s homepage?

Yes No Do not know• Have you ever registered a domain name?

Yes No Do not know• Have you ever designed a website?

Yes No Do not know• Have you ever unscrewed anything on your PC or laptop?

Yes No Do not know

The following questions assessed participants’ cyber-security threatexposure.

• Have you ever lost data because of an infected computer (e.g.,Trojan horse, virus or worm infection)?

Yes No Do not know• Have you ever been impersonated (or have your account

credentials been stolen)?Yes No Do not know

• Have you ever fallen for a phishing e-mail?Yes No Do not know

• Has your personal data ever been misused?Yes No Do not know

• Have you ever received an unsolicited e-mail (i.e., spam)?Yes No Do not know

153

Authorized licensed use limited to: IEEE Xplore. Downloaded on August 30,2020 at 23:30:14 UTC from IEEE Xplore. Restrictions apply.