Is Real-time Phishing Eliminated with FIDO?

Social Engineering Downgrade Attacks against

FIDO Protocols
Enis Ulqinaku, ETH Zürich; Hala Assal, AbdelRahman Abdou, and
Sonia Chiasson, Carleton University; Srdjan Capkun, ETH Zürich
https://www.usenix.org/conference/usenixsecurity21/presentation/ulqinaku

This paper is included in the Proceedings of the

30th USENIX Security Symposium.
August 11–13, 2021
978-1-939133-24-3

Open access to the Proceedings of the

30th USENIX Security Symposium
is sponsored by USENIX.
 Is Real-time Phishing Eliminated with FIDO?
Social Engineering Downgrade Attacks against FIDO Protocols ∗

Enis Ulqinaku∗ , Hala Assal† , AbdelRahman Abdou‡ , Sonia Chiasson‡ , Srdjan Capkun∗

∗ Department of Computer Science, ETH Zürich, Switzerland

† Department of Systems and Computer Engineering, Carleton University, Canada
‡ School of Computer Science, Carleton University, Canada

Abstract gies [3]. The challenge-response computations are performed

FIDO’s U2F is a web-authentication mechanism designed on the dongle itself, and the private key never leaves the don-
to mitigate real-time phishing—an attack that undermines gle. U2F thus enjoys relatively high resistance to the common
multi-factor authentication by allowing an attacker to relay cases of malware that run on the user’s machine. Physical
second-factor one-time tokens from the victim user to the theft of the dongle compromises its defence, however, such
legitimate website in real-time. A U2F dongle is simple to attacks are not scalable and cannot be performed remotely.
use, and is designed to restrain users from using it incorrectly. In U2F, the domain (string) in the browser’s address bar is
We show that social engineering attacks allow an adversary to a function of the challenge-response protocol. The browser1
downgrade FIDO’s U2F to alternative authentication mech- sends that string to the dongle. In case of phishing [85, p.269],
anisms. Websites allow such alternatives to handle dongle the domain will be that of the attacker’s website. Thus, an
malfunction or loss. All FIDO-supporting websites in Alexa’s attacker relaying the result of the challenge-response from the
top 100 allow choosing alternatives to FIDO, and are thus po- browser to the legitimate website does not gain access; the re-
tentially vulnerable to real-time phishing attacks. We crafted sponse will not match the website’s expectation. U2F is there-
a phishing website that mimics Google login’s page and im- fore a strong defender against phishing attacks [60], including
plements a FIDO-downgrade attack. We then ran a carefully- the devastating real-time phishing attacks that undermine var-
designed user study to test the effect on users. We found that, ious Two-factor Authentication (2FA) alternatives [50, 57]. In
when using FIDO as their second authentication factor, 55% real-time phishing, attackers relay the One-Time Password
of participants fell for real-time phishing, and another 35% (OTP) (generated on the user’s phone or sent over SMS) on
would potentially be susceptible to the attack in practice. the fly to the legitimate website. The FIDO alliance highlights
the abilities of its suite of technologies in handling phish-
ing [4]: “This security model eliminates the risks of phishing,
1 Introduction all forms of password theft and replay attacks”, “[the] built-
in phishing resistance and ease-of-use give it the potential to
Fast Identity Online (FIDO) is driven by an industry alliance drive widespread adoption”.
with the goal of reinforcing web authentication by “reducing
We show that FIDO could nonetheless be downgraded to
the world’s over-reliance on passwords” [5]. The alliance
weaker options, enabled by websites that allow users to setup
now comprises 42 members including Amazon, Apple, Arm,
second-factor alternatives to FIDO. These are typically con-
Google, Microsoft, PayPal, as well as financial corporations
figured to account for, e.g., dongle loss, malfunction, or other
like American Express, Mastercard, Visa, and Wells Fargo.
reasons where a user simply wants to avoid using the dongle
FIDO’s U2F standard defines cryptographic challenge-
(e.g., grant access to a remote spouse). Despite extensive de-
response protocols where a dongle with a private key can
sign efforts to empower users with a complete mental model,
prove its identity to a pre-registered website. The dongle inter-
and previous literature showing the high usability and like-
acts with a user’s device through a Universal Serial Bus (USB)
ability of FIDO [30], we show how clever social engineering
port, or wirelessly using Near-Field Communication (NFC)
tactics enable a real-time phishing attacker to impersonate
or Bluetooth (BLE). Such dongles are now manufactured by
FIDO users, requiring neither malware nor dongle theft.
many companies, including Yubico and Feitian Technologies.
We construct a real-time phishing attack which targets
The technology resists exposing the secret key, comparable
FIDO users and works as follows. When the legitimate web-
to some Physically Unclonable Function (PUF) technolo-
∗ An extended version of this work is available in [83]. 1 FIDO assumes a trusted browser.

USENIX Association 30th USENIX Security Symposium 3811

site prompts the adversary to insert the U2F dongle, the adver- 2 Background
sary likewise prompts the user on their phishing website. As
the user inserts their dongle, the adversary asks the legitimate Two-Factor Authentication. 2FA is a widely deployed
website to use an alternative method, and prompts the user strategy to strengthen password authentication. It usually
to submit the OTP of that method on its phishing website. requires users to enroll a second factor (e.g., smartphone
We posit that users can perceive this as an additional, third or special hardware) to their accounts during registration.
authentication factor, on top of the dongle they just inserted, Afterwards, upon submitting the correct password for lo-
thus interpreting more steps as higher security [51,91]. On the gin, the user is asked to prove possession of the second
phishing website, the adversary simply ignores the dongle’s factor. To do so, most 2FA schemes require the user to
response, and relays the user-submitted OTP of the alternative submit an OTP displayed, or confirm a prompt, on their
method to the legitimate website, hence gaining access. phone [11, 48, 50, 57, 58, 82]. To enhance user experience
(reduce inconvenience of a method) and availability (access
We inspected Alexa’s Top 100 websites to verify if they
to the user’s account), online services typically allow users to
allow choosing alternatives to FIDO during login. We found
enroll more than one 2FA alternative per account.
that all websites that support FIDO (23 out of 100) allow
choosing weaker alternatives. Hence, their users remain po-
tentially vulnerable to real-time phishing despite using FIDO. Threat Model: Real-time phishing. Existing 2FAs protect
Ironically, most of these websites force users to first register an users from password compromise but they largely remain vul-
alternative 2FA method before being able to register FIDO as nerable to real-time phishing. In real-time phishing, the user
a second factor. Google’s Advanced Protection [32] program interacts with the malicious page posing as the genuine web-
accepts logins only with security keys, however recovery at site, while the adversary authenticates simultaneously on the
scale remains challenging for such accounts (Sec. 8.1). real website by relaying victim’s credentials. The attack is
relatively easy from a technical perspective, and very effec-
In this paper we approach two research questions. (RQ1)
tive in practice [19]. However, it is very challenging to be
How susceptible are users to phishing attacks when using
prevented because it mostly exploits human mistakes. Prompt
FIDO? (RQ2) How do users detect phishing attacks when
notifications enhance user experience, however, they put the
using FIDO? By implementing a website that mimics real-
burden onto users to detect ongoing attacks and risk user
time phishing of Google’s login form, and through a carefully-
habituation [7]. Automated tools, e.g., Evilginx [34], make
designed user study of 51 participants, we found that only 10%
real-time phishing easy to deploy and largely scalable.
of participants are unlikely to fall for (general) phishing in
practice. They detected our phishing attempts early in the
study, e.g., from the phishing email or the phishing URL, FIDO Specification. FIDO alliance aims to reduce the re-
before reaching our downgrading FIDO part. Had they missed liance on passwords, while preserving usability. FIDO as-
the regular phishing indicators, it is unclear whether these sumes three trusted and cooperating components: i) relying
participants would detect our downgrade attack in practice. party, which is the server where the user authenticates; ii)
client, which typically is the browser; and iii) authenticator,
Contributions. This paper contributes new social engineer- which is the device the user possesses. The key advantage of
ing attacks that allow an adversary to downgrade FIDO to FIDO compared to other 2FA schemes is that the browser pro-
weaker 2FA alternatives. Such alternatives are vulnerable to vides the authenticator with the domain of the visited website.
real-time phishing, which is the primary attack that FIDO is Therefore, if the user falls for phishing, the browser communi-
designed to exhaust. By allowing such downgrade, FIDO’s cates the malicious domain to the authenticator, which signs
defence against real-time phishing is only partial. The pre- a message that is invalid to the honest server.
sented methodology of evaluating the effectiveness of the new The alliance published three specifications [5]: (1) U2F
attacks can be of independent interest to future researchers. covers use cases where the authenticator is used as a second
None of our attacks exploit weaknesses in the FIDO stan- factor; (2) UAF, which is known as “passwordless authentica-
dards, APIs, or cryptographic protocols themselves. The core tion”; (3) FIDO2, which is the latest specifications covering
enabler is rather the availability of authentication alternatives. use cases of both U2F and UAF. Unless specified, “FIDO”
So long as users are allowed to login using weaker alterna- herein refers to all three specifications described above.
tives, attackers also can always leverage them. In general, FIDO2 includes WebAuthn API and the Client to Authenti-
it is necessary to either allow alternative login methods to cator Protocol (CTAP2). CTAP2 triggers browsers to display
hardware tokens, or implement non-weaker account-recovery a prompt window, which includes the domain name, when
mechanisms to account for token losses/malfunctions. Manual a website tries to communicate with the dongle. CTAP2 is
recovery is costly [61]. And with adversaries now capitaliz- backward compatible and supports U2F functionalities, which
ing on an ongoing pandemic [41], and a global work-from- do not trigger the prompt. An attacker can use the latter to
home pattern, it becomes increasingly important to make sure avoid the browser prompt, or even exploit it to their favor (see
promising defences like FIDO are not undermined. Sec. 4.1).

3812 30th USENIX Security Symposium USENIX Association

Table 1: All 23 FIDO-enabled websites in Alexa’s top 100 al-
low weaker 2FA alternatives to be registered alongside FIDO.

Support FIDO
Do not
allow do not allow
support FIDO Total
alternatives alternatives
FIDO partner 14 0 15 29
Others 9 0 62 71
Total 23 0 77 100

3 Problem Statement
Different login means affect the security of users’ ac-
counts [11, 37]. Reports from Google [20] and Microsoft [62]
show that multiple 2FA schemes are widely deployed as alter-
native logins (users select the 2FA challenge in every login
attempt), or recovery mechanisms. Except FIDO, none of the
common 2FA is secure against real-time phishing. Previous
work on FIDO focused on its usability [14, 24, 30, 73]; limited
work questioned its security in real-world deployments, where
alternative 2FA and secure recovery are necessary. In prac- Figure 1: Downgrading FIDO via social engineering. Dashes
tice, account recovery is an expensive operation for service indicate longer time stretches, reflecting when the user acts.
providers [61], and remains vulnerable to social engineering
attacks [29, 76]. FIDO specifications focus on authentication,
but provide only general recommendations for recovery [31]. the browser (Step 5).2 The attacker can leverage standard
To measure the extent by which weaker 2FA alternatives API functions (e.g., u2f.register and u2f.sign for U2F),
to FIDO are used, we manually inspected Alexa’s top 100 so that the attacker is notified when such an authorization-
websites, reviewing documentation for websites’ FIDO au- for-interaction occurs. When the browser communicates the
thentication policy (when available), and creating accounts result of the challenge-response, the attacker ignores the result
on those websites that offer public access to test their policy of this interaction because all they need to know is that the
in practice. Results are shown in Table 1; 23 websites (10 user has inserted the token. The attacker then chooses, on the
organizations) allow choosing alternatives to FIDO. Users of legitimate website, to use an alternative second factor method
these sites are thus potentially vulnerable to real-time phish- from the list pre-configured by the (victim) user on the website
ing, even when using FIDO. More disturbing, most of these (Steps 6–9), and displays a page prompting the user for that
sites force users to first register an alternative 2FA before same method (Step 10). Depending on the website, this step
enrolling their FIDO dongle, and as we show this practice can simply be presented to the user without any indication
undermines the added security of FIDO. as to whether her FIDO-trial was successful. In our phishing
Google’s Advanced Protection [32] is the only known pro- implementation below (Sec. 4.1), we show how Google’s
gram where weaker 2FA alternatives are not supported. The default message to users helps our (attacker’s) cause. Upon
program is opt-in (and account recovery does not scale easily getting the token from the user (Step 11), the attacker forwards
to millions of users, see Sec. 8.1), thus not included in Table 1. it on to the legitimate website (Step 12), hence gaining access.
Timing and ordering notes. In Fig. 1, Steps 6–9 can vary
between websites; some present the user with options; others
may choose for the user. These four steps (i.e., 6–9) must
however occur quickly so that the page in Step 10 is displayed
4 Downgrading FIDO via Social Engineering to the user right after the user’s FIDO authorization in Step 5.
To speed-up displaying the OTP prompt to the user (Step 10),
Our attack starts as a typical real-time phishing (see Fig. 1), the attacker can initiate Steps 6–9 before 5, so that the OTP
with the user on the phishing website and the attacker on the prompt (Step 10) is ready immediately after the user’s autho-
legitimate website at the same time. After relaying the user’s rization. However, the delay between Steps 9 and 12 must
credentials (Step 2 in Fig. 1), the attacker is presented with also be kept small before the website’s OTP token expires. All
the FIDO-prompt page from the legitimate site (Step 3), and such steps can be automated, thus delays can be kept minimal.
in turn displays a FIDO-prompt page to the user (Step 4). Reflections on Step 10. A key element in this attack oc-
At this point, the attacker waits until the user authorizes
their FIDO token to interact with the attacker’s page through 2 Some models require a button press; others a touch.

USENIX Association 30th USENIX Security Symposium 3813

 Google’s login page. Details of the user study, including ethi-
cal considerations, are discussed in Sec. 5.
Our complete phishing pages are available in [83, Fig. 5].
We obtained the domain two-step.online as our phishing
domain, got a Let’s Encrypt certificate for the domain, and
placed our phishing pages inside a google.com directory on
our server. We intentionally opted for a domain with valid
words in a non-traditional TLD such as .online for two
reasons: (1) we could get a TLS certificate without being
flagged as suspicious [72], and (2) users that do not understand
how URLs work but might have a look at it would not be
alerted as google.com is present [81, 92]. Our index.html
page would get periodically blocklisted every few days, and
so we hid it such that it is only accessible through a randomly
generated alpha-numeric string. The page would thus only
be reachable by a link, which would be emailed to potential
victims. While on the phishing website, the browser’s URL
bar would have a green padlock icon with the URL:
(a) FIDO prompt generated by the browser. (b) Google Authenticator prompt
https://login.two-step.online/google.com/index.php?acc=8[..]b
Figure 2: Screenshots of the Google’s login page.

Corresponding PHP code at the start of index.php reads:

curs in Step 10, where the user will be prompted for another
<? php h e a d e r ( ’ Access − C o n t r o l −Allow − O r i g i n : * ’ ) ;
authentication factor after using the FIDO token. We believe i f ( h t m l s p e c i a l c h a r s ( $_GET [ " a c c " ] ) ! = " 8FkuX . . . " ) {
that seeing three login steps (password + FIDO + OTP) likely e c h o " T h i s i s I n d e x . php ! " ; e x i t ( 0 ) ; } ?>
sends a false signal to the user that this login trial is even more
secure than with two factors (password + FIDO). Google’s where acc is the variable containing the random string.
OTP page (see Fig. 2), for example, has the sentence “This When implementing our phishing pages, shown in Fig. 3,
extra step shows that it’s really you trying to sign in”. When we did not borrow content from Google’s website; we neither
an attacker displays that page after its fake FIDO-prompt, pre-downloaded content from Google to upload to our pages,
the user would interpret it as an “extra” beyond password + nor linked to Google content from our pages. The former is
FIDO, but it is intended (by the legitimate site) as extra to not quite straightforward because Google employs code ob-
only the password. In our implementation, we constructed fuscation techniques on its webpages (e.g., to thwart phishing
this (phishing) page with the statement as-is. The “extra step” attacks); the latter was avoided to evade potential phishing
here enables our attack, as it helps attackers downgrade FIDO detection through analyzing our server’s requests to Google’s
to other methods. web-content [67]. The only object we downloaded and up-
Variations to Step 10. Depending on the design of the loaded onto our server was Google’s logo (image). Note that
legitimate website, variations other than presenting a page creating our phishing page would be feasible for any attacker
with an alternative authentication (Step 10) immediately af- with moderate web programming experience. Our implemen-
ter the FIDO prompt may be more effective in tricking the tation of Google’s pages resulted in fewer than 2K lines of
user. For example, similar to approaches discussed in previ- combined PHP/JavaScript/HTML/CSS code.
ous research [77], the attacker may display: “due to technical Recall from Sec. 2, the authentic FIDO prompt is typ-
error, we are unable to process your FIDO token at this time”, ically displayed outside of the attacker-controlled area of
or “our FIDO-handling service is currently down, please use the browser to prevent attackers from replicating the prompt
another method”. The latter avoids the use of FIDO APIs al- within the content pane; note, e.g., for Chrome, the top tip
together, so alert messages familiar to the user in the browser- of the box overlapping the URL bar (see Fig. 2a). Recall
displayed FIDO-prompt box (where attackers have no control also that browsers capture the domain from the URL bar
over the message within) are avoided. and display it to the user within the FIDO-prompt box. It is
thus helpful (to the attacker) to use API functions that do
not display this box to the user, yet gets the browser to no-
4.1 Attack Implementation
tify the webserver that a dongle was inserted. For Step 10
In preparation for running a user study to test the effective- (Fig. 1), we used the u2f.register function, which does
ness of this attack, we implemented a phishing website that not display browser-generated prompts. With this function,
behaves in the manner explained above. The website targets communications with the user are left to the website devel-

3814 30th USENIX Security Symposium USENIX Association

 5.1 Design Decisions

We considered several study designs before ultimately de-

veloping our methodology, including carefully considering
legal concerns [44, 45] and ethical issues as summarized by
Finn & Jakobsson [26]. While it does add realism, we dis-
missed the idea of using participants’ real credentials in a field
study without a priori consent because previous work [44]
has shown that it can lead to participants feeling violated even
after learning that no personal data was compromised. A vari-
ation of this approach [45], where a clever study design has
participants entering their real credentials to a legitimate site
rather than a researcher-controlled site was technically not
viable for our particular attack.
(a) Attacker’s FIDO prompt. (b) Attacker’s Google Authenticator Another line of phishing studies [19, 46, 56, 71] ask par-
prompt.
ticipants to classify pages as phishing or legitimate without
Figure 3: Screenshots of the phishing page. submitting any credential, usually to measure the effective-
ness of security indicators. This approach did not align with
our intended goal of measuring the effectiveness of FIDO
protocol against our phishing attack, thus was not a viable
oper (i.e., through standard HTML and JavaScript).3 As an study design. Yet another approach [67, 86] to studying phish-
attacker, we do not control the legitimate displayed message; ing is to analyze logs from service providers to investigate
it is browser-generated. So we implemented a mimicry of the the occurrence of real phishing attacks. To the best of our
Chrome-generated FIDO prompt as a gif image that looks knowledge the downgrade attack presented in this paper is
like Chrome’s box,4 with a message identical to the authen- novel and there are no reports that it has been exploited in the
tic one: “Use your security key with google.com” (Fig. 3a). wild, thus logs would be unhelpful. Focusing on user perspec-
The gif had an animated indeterminate progress bar, almost tives instead, qualitative studies [16, 46] aim to understand
similar (visually) to Chrome’s authentic one (Fig. 2a). Since users’ attitudes and reasoning regarding phishing, but these
it was an image, it was fully contained within the browser’s subjective accounts do not provide objective measures of the
content pane, located vertically at pixel 0 (top-most point). effectiveness of particular attacks.
Finally, since our aim is only to test the effect of our attack
We next considered phishing studies based on role playing.
on participants in Sec. 5 (i.e., we do not want to actually steal
Such studies typically use fictitious scenarios to simulate
credentials), we did not implement back-end communication
the experience of a user that receives legitimate and unsafe
between our phishing website and Google.
emails. The tasks are typically easy and can be performed
by an average user, thus participants do not need the proper
experience for the role. Prior studies have used various roles: a
university worker (who receives 6 phishing emails out of 14 in
5 Evaluation Methodology total) [78], a political campaign volunteer (3 malicious emails
out of 8) [28], a company employee (5 malicious emails out
We designed a user study to test the effectiveness of the above of 19) [53], or a user doing online shopping (gets a phishing
social engineering tactics. In comparison to studies that test email after each online purchase) [22]. Such studies have
the usability of systems, designing a user study to test attack limitations because participants may behave less securely with
effectiveness is often challenging. The study must be ethi- mock credentials [77]. However, role playing experiments do
cal. It should reflect a user’s true keenness in protecting their not raise ethical concerns and they would allow us to perform
assets. Moreover, the explanation of the study tasks to partic- a realistic downgrade attack during experiments.
ipants should not (1) artificially lead participants to fall for After careful consideration of the technical requirements
the attacks in question, and (2) artificially alert participants of our attack, and the ethical and legal implications of exploit-
so they detect/avoid the attacks. ing participants’ personal credentials, we decided to design a
role playing experiment followed by a semi-structured inter-
3 Note that even if a browser-generated box was used, users may already view. We believed that the combination of the two methods
be oblivious to the messages displayed within that box. would minimize the limitations of either individual method
4 An adversary can generate similar prompts for other browsers. by providing an opportunity for cross-checking our data.

USENIX Association 30th USENIX Security Symposium 3815

5.2 Study Design 5.2.1 Study Scenario

We recruited participants using flyers, university mailing lists, Participants were asked to role play Jordan Hart, a new em-
and social media posts. Participants visited a webpage de- ployee in a technology company on her/his first day of work.
scribing the study, its duration, and the compensation before They were provided with their company gear: a laptop, smart-
scheduling the interview. The study advertisement generically phone, and a security key (the FIDO dongle). Participants
explained that the purpose of the study was to evaluate and were asked to read and sign the employee on-boarding in-
improve the usability of email clients. To eliminate any later formation sheet (Appendix A), a common practice in indus-
doubt by participants about the safety of their legitimate cre- try. This sheet outlined the company policy with respect to
dentials, participants did not use their own email accounts. safeguarding company information and avoiding scams and
We provided user accounts and credentials (with a randomly- phishing attacks, as well as explaining FIDO keys and their as-
generated strong password) created specifically for this study. sociated security benefits in language adapted from Google’s
However, to maintain ecological validity, we designed a study Security and identity products pages [33]. The sheet listed
scenario that indirectly encouraged participants to think about Microsoft Outlook as the company’s primary email provider,
the security of these accounts. included Jordan’s Outlook account credentials (username and
We ran the study in-person concurrently in two interna- password), and provided their Google services credentials.
tional cities near the end of 2019, one in North America (Ot- We created real Microsoft and Google accounts. The sheet
tawa, Canada), below suffixed with -N, and one in Europe also included the names and email addresses of Jordan’s man-
(Zürich, Switzerland), -E. To maintain consistency in both ager, IT manager, and HR person, from whom Jordan would
cities, we carefully documented the study protocol and had receive emails. We created real Microsoft email accounts for
the two researchers running study sessions follow this com- each of them. To make sure participants were comfortable
mon protocol. Participants were monetarily compensated for using the FIDO key, the researcher–acting as the IT manager–
their time, receiving CHF20 in Zürich, and $10 in Ottawa. asked participants to login to their email with the key as a
Participants first completed a demographics questionnaire second factor, and explained how to use the Google Authen-
then they went through the study scenario, during which they ticator app (pre-installed on Jordan’s smartphone) in case of
were asked to think-aloud (i.e., to describe their thought pro- technical difficulties (Appendix A).
cess out loud). We next gathered feedback from participants Jordan’s Microsoft email inbox contained 15 emails [83,
through a semi-structured interview.5 We designed the inter- Appendix F], divided into 5 folders, one for each day of the
view questions to indirectly gauge participants’ awareness week [83, Fig. 7]. Participants were asked to assume that they
of the phishing attempts. Following previously established login to their Outlook account daily, handle emails received
notions of determining participants’ thoughts [29], we asked: that day (as tagged), logout and shutdown their laptop before
going home, and come back the next day to do the same steps.
“If we told you that 50% of our participants access fake The researcher simulated shutting down the laptop when in-
websites during their study sessions, do you think you are dicated by the participant by logging-out of their email and
one of them? Why/Why not?” clearing the browser cache after finishing each day’s emails.
Participants used Google Chrome.
We did not ask participants about each email, one-by-one,
whether it was a phishing attempt to avoid making them overly 5.2.2 Emails
vigilant, and potentially biased to answer “yes”. However, we
still allowed participants to go back to the emails and check Four of the 15 emails were phishing, containing a link to our
them during the interview, should they ask to do so. phishing website (Sec. 4.1). Such emails were spearphishing
At the session’s end, the researcher provided participants (targeted). We chose to have such a high number of phishing
with a debriefing form, explaining the true purpose of the emails in the first week of employment to give participants a
study and answered any questions they had. Each session higher-than-normal chance to recognize our phishing attack.
lasted approximately an hour, throughout which the researcher In Sec. 6.2, we explain how detecting a single phishing email
took notes to provide insight into participants’ thought pro- suffices for us to count the participant amongst those who did
cesses (e.g., if participants hesitated when opening links not fall for our attack.
(phishing or legitimate), if they hovered over the links to view We used PHP’s mail function to send out the phishing
the URL, and any comments they had on the emails). Study emails using a spoofed source email address. To ensure real-
sessions were audio-recorded, and the interview portion was ism, these emails included errors like grammatical mistakes
transcribed for analysis, and the researchers referred back and typos, mimicking typical phishing emails. Non-phishing
to the audio recordings for more context when needed. The emails were sent from the authentic email accounts of the
study received IRB approval in both cities. company’s employees (Jordan’s manager, IT manager, and
HR person) through the email web client. All emails were
5 The interview script is detailed in [83, Appendix E]. sent at once before we started recruiting participants, and sim-

3816 30th USENIX Security Symposium USENIX Association

ply marked as unread before the next participant. When we in the email). We developed an analysis matrix to cover the
initially sent them, we manually moved those that were placed main topics relevant to our research questions. The matrix
into Jordan’s Spam folder (legitimate or phishing) into the In- comprised of four categories with which we coded our data:
box folder (see [83, Fig. 8] for an example of a legitimate and identifying phishing links, participants’ perception of FIDO,
a phishing email, both appearing to be from the IT manager). their perception of 2FA, and their security attitude and aware-
Note that, as in real-life, when visiting our phishing pages, ness. We then followed an inductive analysis method, and
participants will see the fake login form even when they are al- performed open coding to look for interesting themes and
ready logged-in to Jordan’s Google account. This has alerted common patterns in the data using NVivo software. Themes
vigilant participant, P2-N, to our phishing attempts. irrelevant to our research questions are not discussed herein.
Some emails, legitimate and phishing, included links to doc- As recommended by previous work [13], and similar to previ-
uments. We created actual documents for every such email, ous research (e.g., [16]), data was coded by a single researcher
and stored them on Google drive. Legitimate documents were with considerable experience in Human-Computer Interaction
only accessible through Jordan’s Google drive account. We (HCI), security, and qualitative data analysis, so that this re-
(attacker) set the other documents on Google drive as accessi- searcher would perform rigorous analysis by being immersed
ble with a link, and redirected to them after the user finished in the data. Two researchers met regularly to discuss the codes
logging-in to our phishing website. This way, the browser’s and interpret the data. To verify the reliability of our coding,
URL bar would display an authentic Google domain after the we had a second researcher code 25% of the data individually.
participant’s persona credentials were phished. We calculated Cohen’s Kappa coefficient [15] to determine
inter-rater reliability, which indicated “almost perfect agree-
ment” [54] (κ = 0.82).
5.3 Participants
We recruited 51 participants for this study: 25 in city-E and
26 in city-N. Our dataset (available in [83, Appendix D]) 6.1 How to assess phishing susceptibility?
is balanced in terms of gender: 26 participants identified as To identify participants who could be victims to our attack
female, 24 as male, and one chose “Other or prefer not to in practice, we need a mapping between their behaviour in
answer”. Our participants were between 18 and 64 years old the study and their attack susceptibility in practice. Simply
(µ = 29.9, Med = 27). The vast majority of participants had classifying those who submit their credentials to one of our
an undergraduate or a graduate degree (n = 46), and most phishing links as potential victims may not be accurate be-
have previous experience with 2FA (n = 31). cause: (1) participants may not be as keen to protect their
study credentials as they would their own, and (2) partici-
5.4 Limitations pants may think they need to process all emails regardless
of their suspicion because this is what the study is asking
We designed our study to create an atmosphere that empha- them to do. We took measures to reduce the impact of both
sized the importance of keeping the persona’s accounts secure. points, e.g., through emphasizing the importance of security
However, participants did not use their own credentials, which to the persona’s employer, and integrating actions with inter-
may have affected their motivation to protect these creden- view responses. Only two participants mentioned they were
tials. The semi-structured interview along with researchers’ not paying attention because they thought it was the study’s
notes provided deeper insights in identifying reasons why instructions, highlighting the importance of our measures.
participants’ accessed links in our phishing emails. Although Participants’ actions (during the study) and awareness of
one of FIDO’s design goals is ensuring security regardless the attacks (in the study) may or may not align. We use both
of users’ experience and anti-phishing skills [55], our results parameters, i.e., actions and awareness, to assess each partic-
may have been affected by participants’ lack of familiarity ipant’s phishing susceptibility in practice; combining both
with FIDO keys, and the lack of proper context for judging if parameters yields four possible cases, which are summarized
emails are following-up on real events. Additionally, our data in Table 2 (first 3 columns). According to these two parame-
set is skewed towards relatively young participants who may ters, we will rate each participant as susceptible, potentially
lack anti-phishing training usually received in work settings. susceptible, or not susceptible to our phishing attacks in prac-
tice (column 4 in the table). (Details on how we determine
6 Analysis Methodology a participant’s awareness of the attacks in our study can be
found in Sec. 6.2 below.)
We used the Qualitative Content Analysis Methodology [47] Normally, a participant who is unaware of our phishing
to analyze qualitative data collected throughout the study attempts would submit their credentials to our phishing web-
(e.g., post-testing interview scripts, and researchers’ notes in- site. This is Case 1 in the table. A vigilant participant would
cluding participant’s comments on the content of emails and normally refrain from submitting their credentials, and con-
whether they entered their credentials to the website linked firm their awareness of phishing attempts in the post-study

USENIX Association 30th USENIX Security Symposium 3817

interview—Case 4. Table 2: Classifying participants’ susceptibility to our phish-
Cases 1 and 4 are straightforward; we classify the former ing attack in practice, from their study behaviour.
as “susceptible to phishing”, the latter as not. We classify
Participant Results
Cases 2 and 3 as “potentially susceptible to phishing”. In Case
aware-of-phishing-attempts submitted credentials
Susceptible
# %
Case 2, although they did not submit credentials, participants 1 Unaware Yes Yes 28 55
are unaware of any phishing attempt. In Case 3, participants 2 Unaware No Potentially 1 2
are classified as aware, yet they submit credentials. 3 Aware Yes Potentially 17 33
4 Aware No No 5 10

6.2 How to determine awareness of phishing?

whether they were truly aware. Thus we provide an upper
Determining participants’ awareness from the interview is not bound on awareness. For example, a participant who named a
trivial. Participants responses’ varied greatly. For example, true phishing indicator, yet asserted seeing no phishing emails
to the above question (“If we told you that 50%...”), some is still classified as aware-of-phishing-attempts. Classified
participants answered affirmatively, but only name examples likewise is a participant who gave an example of one phishing
of non-phishing emails. Others answered affirmatively, but email, mistakenly identified two non-phishing emails, and as-
said they did not remember which ones were phishing. We serted there were no other phishing emails (i.e., missing three
also had participants who first denied being in the 50% that others). We used conservative criteria for two reasons: (1) we
accessed fake sites, then hesitated, alternating between “yes” increase certainty that participants classified as unaware-of-
and “no”, then changed their minds, and gave a few true phish- phishing-attempts would most likely be unaware of similar
ing examples. And there were participants that provided an attempts in practice, and (2) participants may have forgot-
immediate affirmative response, reconsidered, and finally de- ten which emails were truly phishing by the time they reach
cided there were no phishing emails. We thus ignored their the post-study interview (there were 15 emails in total). We
direct ‘yes/no/maybe’, and instead relied on objective portions purposefully avoided showing each of the 15 emails to partic-
of their comments to assess awareness, as described next. ipants and asking them which were phishing to avoid priming.
Our hypothesis here is that, if during the study, a participant
suspected a phishing attempt, they would recall that and indi-
cate it in a manner captured by the criteria in Fig. 4.

Conservative classification of attack susceptibility. We

determined susceptibility based on two factors that we first
assessed independently: awareness and submission of creden-
tials. “Awareness” is not per email, but per participant. So
even if a participant named one phishing email but missed all
Figure 4: Determining awareness of our phishing attempts. others (or asserted there were no others), we still classify them
as aware-of-phishing-attempts. When we check whether this
Any participant who (i) identified at least one phishing participant submitted credentials to our phishing website, we
email or (ii) named a true phishing indicator is classified as do not match the phishing email they fell for in the study with
aware-of-phishing-attempts, regardless of what else was said the email they named in the interview. For example, a partici-
during the interview. Participants classified as unaware-of- pant who noticed only one phishing email, E2, is classified as
phishing-attempts included those who: just denied being in aware-of-phishing-attempts, even if they asserted there were
the 50%; affirmed being in the 50%, but gave only examples of no others. If this participant submits credentials upon clicking
non-phishing emails; or affirmed but gave only false phishing on the link in any phishing email (E2 or another), we classify
indicators. Figure 4 shows this criteria, alongside common them as “potentially susceptible”, not as “susceptible”. One
example responses in our study that we discarded because the would argue that this is a “susceptible” participant because
awareness criteria was met. By true phishing indicators, we an email successfully phished their credentials. Being conser-
mean the website’s URL, and commonly agreed upon (though vative, we opt to use any minor indication that a participant
non-robust) signs of phishing emails [40], like typos, lack of might notice similar attacks in practice as grounds for avoid-
context, and grammatical mistakes. Unencrypted email is an ing classification as “susceptible”. By classifying only the
example of a false phishing indicator. most blatant case as “susceptible”, we provide a lower bound
for susceptibility to attacks.
Conservative classification of attack awareness. Follow-
ing the criteria in Fig. 4, we classified participants as aware- Examples of aware participants. From our analysis, the
of-phishing-attempts even in situations where it is hard to tell following are examples classified as aware-of-phishing-

3818 30th USENIX Security Symposium USENIX Association

attempts. P17-E said, “No, I think I haven’t... Ah! maybe the emails, she simply said, “There is no particular reason”.
this Sam Logan is a phishing [email]. [...] he [emailed] twice, In contrast, 43% of participants (Cases 3 + 4) were classified
it could be... I don’t know. If I got phishing, this is the only as aware of phishing attempts, but only the Case 4 participants
email I feel it could be.”. P17-N said, “Yes, [I was in the (10% of all participants) are likely to detect the discussed
50%] [...] I was taking it for granted that the emails I was phishing attempts in practice. A Fisher’s Exact Test suggests
getting from the employees at the company were legitimate. that participants who were aware of phishing attempts were
[...] So I think that Sam Logan ones were, at least the one that less likely to submit their credentials than those who were
I got from Sam Logan on the Friday was definitely a phishing unaware (p = .047, N = 51). Contrarily, Fisher’s Exact Tests
email [...] Now that I’m thinking about it, that was definitely showed that neither gender (p = .60, N = 51), generation6
a phishing email, because of how poorly worded it was.”. (p = .34, N = 51), nor having work experience (p = .23, N =
P25-E said, “I received many phishing emails here (identified 51) significantly influence susceptibility to phishing.
them correctly during the study). I think there were two types,
first the email about account change. The address looked it is
7.1.1 Takeaway
coming from the source but as the company doesn’t have any
encryption I cannot be sure. I would have gone physically to Our focus in the present paper is to determine user’s suscepti-
the person. And the others that asked for google credentials, bility to phishing, particularly while using FIDO. We noticed
for those I just checked the address.” that all participants who appear to have detected and avoided
our phishing attempts (Case 4) would have done so also with-
Examples of unaware. P10-N said, “I don’t think so. [...] out using FIDO. The phishing indicators they mentioned, and
Everything seemed legitimate enough and seemed business-y. the reasons they discussed as to why they avoided submitting
And I look[ed], everything looks like pretty work-related and credentials to our phishing site are not related to FIDO. Like-
exactly related to what the e-mail said it would be. Yeah. It wise, those whom we classified as susceptible to phishing are
wasn’t like I just clicked on a link and it really brought me to susceptible despite using FIDO. That is, using FIDO did not
some random page or something, it was related to what the protect them from our downgrade attacks. Essentially, what
e-mail was saying. So it seems legitimate to me.”. P13-N said, we were looking for in this research is cases of users who
“I just went to hotmail, the outlook website which I very often would have fallen for phishing without FIDO, but have not
go. And I logged in from there. So I think it seemed fine.” because of using FIDO. We found none.

7 Results 7.2 Phishing detection while using FIDO (RQ2)

When we asked participants if they had accessed fake web-
Through our data analysis, we looked for an effect of location sites during their session, participants were evoked to think
by comparing data from the North American and European about the emails more deeply, and discuss reasons they used
cities where the studies were conducted; we found no clear to classify emails as phishing or safe. Through our analysis
distinctions between the two groups. Our qualitative analysis of qualitative data, we found that participants relied mostly
did not reveal themes distinct to either city and we found no on general phishing indicators for determining whether the
statistically significant difference between the two groups’ emails they received were phishing. Table 3 summarizes rea-
susceptibility to phishing attacks (X 2 (2, N = 51) = 1.64, p = sons why participants classified an email as phishing (seven),
.44). We thus discuss the amalgamated results, within the and reasons for classifying an email as safe (eleven).
context of the two research questions in Sec. 1.

7.2.1 Reasons for classifying emails as phishing

7.1 Phishing Susceptibility with FIDO (RQ1)
We grouped the phishing indicators discussed by participants
Table 2 summarizes the results; 57% of participants (Cases 1 into two categories: technical, and non-technical. The two
+ 2) were classified as “unaware of phishing attempts”, and technical indicators were suspicious URL, and that the re-
all but one of these participants (P12-E) submitted credentials peated login prompts were unusual behaviour. P2-N explains,
to our phishing website. Given our conservative measures in “It wants [me] to log onto Google even though I was already
classifying susceptibility, our results suggest that at least 55% logged on to Google on just another tab[...] This was not a
(Case 1) of participants would be susceptible to our phishing thing I noticed at the beginning when I was doing the experi-
attacks in practice. The one participant in Case 2, P12-E, was ment [...]. Now that I’m thinking about it. Yeah, makes sense,
very rapid in going over the emails. She did not click on any right? Like why are they asking you to log onto Google again
phishing link, and also ignored several non-phishing links. when you’re already logged onto Google?!”
She gave very short, non-informative, responses in the post-
study interview. When asked why she did not click on links in 6 We assigned generation labels, based on participants’ ages, as in [49].

USENIX Association 30th USENIX Security Symposium 3819

 Although the identified technical indicators can alert users, and were protected against phishing because they were using
they are not ideal from a usability perspective. Users do not FIDO (using FIDO/2FA). Interestingly, requiring the use of
always check the URL bar [42], and even if checked, users Google Authenticator (part of our attack) gave some partici-
do not necessarily know the correct URL [6]. Users may also pants a false sense of security. P21-N said, “I had to put in the
lack a proper understanding of the structure of URLs to be information [code] as well and I felt secure: the company even
able to assess their legitimacy [6, 19, 92]. In addition, it is took me to verify everything [using the Google Authenticator]
unlikely that typical end users would know that a website to make sure that it was secured”. These participants either
would only require users to re-login if the session cookie viewed the authenticator an additional factor or assumed it
has expired (the three participants discussing the repeated was part of FIDO, and some even thought FIDO was more
login indicator had studied Computer Science or Computer secure because of the authenticator. P10-E explained, “If you
Security). In fact, repeated login prompts can provide users have to use the authentication app on the phone, with the
with a false sense of security, because the system appears changing number always, it is really difficult for someone to
more secure if it asks for two factors at every login. P22-N hack your system to find this kind of information.”
explained, “I think when I clicked on the one that didn’t ask Non-technical reasons also emerged for classifying emails
[...] for the two factor authentication, it just went straight to as safe, based on context and the user’s expectations (e.g., their
the [Google] Doc then I [thought] something bad happened. social and professional life). For example, users expect to re-
Because every other time [...] it would ask me to sign in again ceive emails from their institutions’ official communication
[...] but that one suddenly didn’t. And it just makes me kind channels, which gives these emails credibility. This finding
of feel like ‘uh-oh what happened?!”’ mirrors that of Conway et al. [16] where participants felt more
Participants also discussed non-technical indicators, such secure at work. We also found, similar to previous work [6],
as poor grammar and styling, and the presence of a hyperlink that participants relied on quickly inspecting the login inter-
in the email. Other non-technical indicators stem from the face or the content (e.g., a Google Sheet) to which the link
participant’s knowledge of the sender, including that the tone in the email redirects, and comparing it to their expectations.
of the email was unexpected, the language of the email did P18-N explained, “I think I did a little bit of due diligence
not match that of the sender (sender language consistency), when I signed in, so [I] should be OK. Like I checked when I
and that the email was out of context. P16-E said, “If it is was logging in [that] I was logging in to the right thing. Most
just, like my boss sending a book to download, and we talked of the things that came up were Gmail and Outlook. The only
about it, it’s fine. But if it is a random book, then it’s weird.” one document that [I] opened was a Google document which
Non-technical indicators rely exclusively on users’ judge- did ask for my authentication” (as part of our attack).
ment, are inconclusive, and are relatively easy to manipulate.
For example, while context appears to alert some users against In summary, we highlight that participants cited FIDO as
phishing (as described herein and in previous work [46]), at- one of the reasons for classifying as safe, when they have in
tackers can adjust their techniques to increase their phishing fact fell for our phishing attack. As such, FIDO did not only
emails’ credibility [84]. In fact, an email can have an expected fail to protect them, but it potentially gave them a false sense
tone, a language that appears consistent with the legitimate of security. FIDO can be relied upon as a safety indicator
sender’s, relevant context, proper grammar and styling, no when it works successfully, without having the user authorize
hyperlink, and yet belong to a phishing campaign. any other factors (beyond the initial password).
None of our participants mentioned relying on FIDO to
identify phishing emails, and rightfully so. It is not quite clear
how FIDO can be used as a complete phishing indicator—it
may benignly fail due to technical errors, e.g., broken dongle. 7.2.3 Takeaway

7.2.2 Reasons for classifying emails as safe Despite using FIDO, we noticed that none of the participants
have relied, or indicated that they would rely, on FIDO for
When examining reasons why participants identified emails detecting phishing attempts. In contrast, we had three partic-
as safe, we found that they again relied on technical and non- ipants who said they were secure because they used FIDO
technical indicators (Table 3). As many participants ended-up in all their logins, even when some of these were accompa-
misclassifying emails as safe, such reasons may have mis- nied by other authentication factors. Evidenced by our attacks,
guided participants. In these cases, participants erroneously the proper usage would be to refuse to login with alternative
interpreted the URL (linked in the email) as a safety indica- methods if a user has enabled FIDO. Seeing a FIDO-only
tor. For example, some participants concluded they were safe login is practically opposite to using FIDO alongside other
from phishing because opening the links in the emails did not factors—the former prevents downgrade attacks, the latter
lead to obviously malicious behaviour (e.g., popups). Others enables them. We found no evidence that any of our 51 par-
(n = 3) indicated they “felt more secure with 2FA” (P23-E) ticipants understood this concept.

3820 30th USENIX Security Symposium USENIX Association

 Indicator Explanation Example Quote
Reasons for classifying emails as phishing
URL The URL of the hyperlink in the email is suspicious “The URL looks really weird, I think it’s not safe, or like that’s not the normal. This is just like fanciness that looks
Technical

like Google” (P11-E)

Repeated logins The participant is required to login although they have al- “I logged in my Gmail, and then I clicked on an email again. And I had to, re-enter my login credentials. Like
ready logged in and the session is supposed to be maintained something like this ought to be kind of phishing” (P15-N)

Tone The tone of the email is unexpected (e.g., demanding, or not “ ‘We demand you’ I feel like somebody would not be using that kind of language at work.” (P1-N) “One email
professional as expected in the workplace), or the email does was not addressed to me with a name, but to the username, so it looked like a bot.” (P19-E)
not include greetings or greets the receiver by their username
rather than their name
Sender language The language in the email is not consistent with how the “[That’s] not the right person, that’s not the person I know from the way, it’s the tone of writing and the language
Non-technical

consistency sender usually writes emails and the way it’s said.” (P20-N)
Context The circumstances surrounding the email received and its “there was [an email] that [was] for a job or something, and I was thinking I already have a job, I thought it was
subject; the timing of the email in terms of events is inappro- weird” (P14-E)
priate/unexpected
Grammar and The email contains mistakes in grammar, punctuation, or “Now that I’m thinking about it, that was definitely a phishing email. Because of how poorly worded it was.”
styling capitalization (P17-N)
Hyperlink The email includes a hyperlink “Um well, most of the red flags I got were from when there is a link in it.” (P7-N)
Reasons for classifying emails as safe
URL The URL of the hyperlink in the email looks legitimate “I didn’t click any of the suspicious links. I mean, I did click links to Google Docs and things like that and they
looked legit to me” (P2-N)
Popups Clicking on the hyperlink the email did not lead to popups “I don’t know that anything is entirely compromised but maybe I clicked on a link, but I didn’t see any indicators
of that. Like I didn’t see like any pop ups or any extra spam come in or anything like that” (P25-N)

Using FIDO or Using FIDO/2FA makes it more secure “It kind of seemed to be fine, I suppose I felt more secure with with the 2FA [FIDO token] because they cannot
Technical

2FA steal all information if it is encrypted.” (P23-E)

Google authenti- Requiring Google authenticator is an added level of security “I had to put in the information [code] as well and I felt secure: the company even took me to verify everything
cator [using the Google Authenticator] to make sure that it was secured” (P21-N) “More steps [authenticator + FIDO],
more security” (P13-E)
Sender address The sender’s address is correct in the email header (The “I verified their email [address] and some like I would assume that, that is the legitimate person” (P11-N)
FROM part of the header)
Antivirus Relying on the antivirus to handle security “I am kind of a lazy person and as I said before I rely on my antivirus too much, but I guess it is what it is” (P11-E)

Communication The emails and linked content were sent through the official “I didn’t open something that looked suspicious. [...] Everything was from official channels, from work, so I think
channel company emails, by employees of the company it should be ok.” (P10-E)
Login interface The login interface looked legitimate “I was logging in to the right thing. Most of the things that came up were Gmail and Outlook.” (P18-N)
Content The hyperlink in the email redirected the user to the expected “Everything looks like pretty work related and exactly related to what the e-mail said it would be. Yeah. It wasn’t
Non-technical

content like I just clicked on a link and it brought me to some random some random page or something, it was related to
what the e-mail was saying. So it seems legitimate to me.” (P10-N)

Context The circumstances surrounding the email received and its “If it is just, like my boss sending a book to download, and we talked about it, it’s fine. But if it is a random book,
subject; the timing of the email in terms of events is appro- then it’s weird. [...] I think if [the download book email] was sent to me in real life, I would click on it, because it
priate/expected is mentioning nanotechnoloty, it has a context that makes sense” (P16-E)
Sender The receiver knows the sender, the email is not from a com- “Since this is a secure network, and all the people that were sending me emails were company, colleges, I suppose
plete stranger there were no phishing emails” (P24-E)

Table 3: Reasons noted by participants when identifying phishing and when mislabelling emails as safe.

8 Discussions and Countermeasures must register at least two security keys, one for daily use,7 and
others as backup. Google does not detail the recovery process
We provide practical insights regarding potential defenses, in case both keys are unavailable, but states that “it may take
based on our study results and our analysis of the attack itself. a few days to verify it’s you and restore your access”. This
delay poses a major trade-off for users.
Limitation: non-scalable recovery. Doefler et al. [20] re-
port that challenges requiring security keys have a lower pass
8.1 Disable Weaker Alternatives
rate than device-based ones. So, if alternatives were disabled,
A straightforward countermeasure to the downgrade attack more users would need the recovery process. On the other
presented herein is to disable alternative 2FA methods if a user hand, such recovery adds significant costs to service providers,
enables FIDO. This would have mitigated situations where and does not scale to millions of users [61]. Disabling weaker
our participants thought the extra factor was a feature rather FIDO alternatives comes at the cost of non-scalable recovery.
than an indicator of attack. Google’s advanced protection Limitation: usability impact. Previous literature [14, 24,
program [32] achieves this for critical accounts, e.g., those 7A phone running Android 7+, or iOS 10+ with the Google Smart Lock
of politicians or journalists. The program is opt-in and users app, can be used as one security key.

USENIX Association 30th USENIX Security Symposium 3821

73] reported that users have difficulties enrolling security keys Use your security key with google.com
into their accounts, and are concerned about being locked out
if keys are lost. Registering multiple keys can enhance the Mozilla Firefox includes the fully qualified domain name
user experience but may be costly for users,8 which might (e.g., accounts.google.com) in a callout panel as:
be a barrier to some users. Moreover, service providers tend
to facilitate user onboarding and enhance overall experience accounts.google.com wants to authenticate you
by offering a variety of channels to connect to its backend, using a registered security key. You can connect
e.g., browsers, native apps on different OSes, or third-party and authorize one now, or cancel.
software such as email clients. Disabling FIDO alternatives
can degrade usability because channels that do not support Since the prompt contains a short message and the website’s
FIDO should then be dropped—otherwise, the attacker con- domain, rendered in boldface in Firefox, it can potentially
nects to the server through such channels. alert visitors of a phishing website.
Limitation: users’ susceptibility to social engineering.
8.2 Risk Based Authentication Relying on users to notice the domain mismatch is not a reli-
able countermeasure for three reasons. First, FIDO promises
Risk-based Authentication (RBA) refers to a set of server-side to relieve users’ from the burden of detecting phishing, hence
techniques to assess the risk of an authentication attempt, and security should not depend on prompts or visual indicators.
block malicious ones [35,79,89]. Secure IP geolocation [1,2], Second, previous research [6, 7, 19, 81] have shown that users
device, network, user agent, and installed plugins are exam- typically do not pay attention or understand browser hints re-
ples of metadata that RBA systems analyze for deciding the lated to security. Third, the adversary can use the U2F API to
risk score of a login attempt. A low risk attempt (e.g., same interact with the device, which does not trigger such prompt
user agent and same IP address) gives confidence to the server windows (as we did in our implementation—Sec. 4.1) and
that the honest user is authenticating. For higher risk requests, although not tested in our study, previous work shows that
the server challenges the user to provide additional factors, or users easily overlook the absence of security cues [77].
restricts user’s access depending on the provider’s policy [90].
Limitation: mimicry of user’s attributes/behaviour. A
recent study [12] shows that attackers have already developed 8.4 Secure Login and Recovery Alternatives
malicious tools that can circumvent RBA defenses. Such
tools are made available as public services. Campobasso and Doefler et al. [20] discuss Google’s categorization of login,
Allodi [12] reveal that attackers collect necessary data from second factor authentication, and account recovery methods.
victims on top of their credentials, so they can bypass RBA de- Methods of comparable security are placed in the same cat-
fenses. Similarly, an adversary performing real-time phishing egory, and should be allowed depending on the account’s
can adapt such tools to bypass RBA mechanisms on-the-fly. security status. Such a status could possibly be indicated by
This adversary has a connection with the victim’s browser, the user’s security configuration (e.g., enabled 2FA, config-
and may be able to mimic attributes/behaviours to the legiti- ured robust recovery methods).
mate website [3], or execute the JavaScript code (related to Promising countermeasure. It appears that a viable coun-
RBA analysis) directly on the victim user’s browser. termeasure to the attacks discussed herein is: when FIDO
is enabled, only enable authentication (or 2FA) alternatives
that provide resilience to similar attacks that FIDO resists.
8.3 Browser Hints Suitable candidate alternatives include other FIDO protocols.
For example, a phone-based authenticator through FIDO2 can
The recent WebAuthn API [8] instructs browsers to always
serve as an alternative to physical security keys. This should
show a prompt window when a website interacts with the
be recommended/enforced by service providers (websites). In-
authenticator during both registration and authentication. The
tuitively, a user choosing to register a security key for login is
prompt is part of the user consent process, which means that
implicitly requesting resilience to advanced attacks (e.g., real-
the user agrees (by tapping the authenticator device) to com-
time phishing). To that user, a service provider should only
plete the request displayed on the prompt. The prompt itself
allow alternatives of equivalent defence capabilities.
contains a short message, and browsers display it as a native
Login and recovery are two sides of the same coin—
popup that extends slightly above the address bar. For ex-
recovery alternatives must also match the security level of
ample, Google Chrome captures the TLD and second-level
the user-configured login methods. For example, for a FIDO-
domains of the website (e.g., google.com), and displays them
enabled account, recovery through a secondary email that has
to the user within the prompt box alongside the message:
weaker security undermines the security of that account. Ham-
8 Atthe time of this writing, security keys from Yubico (a popular vendor mann et al. [37] discuss how account-access graphs could
and FIDO Alliance partner) cost around $20. help users and service providers discover vulnerable paths.

3822 30th USENIX Security Symposium USENIX Association

8.5 User Education real-world deployments. However, the necessity for alterna-
tive 2FA is already emphasized on previous studies [20, 73]
Many participants in our study relied on wrong phishing indi- because users cannot always complete the FIDO step. On the
cators. Several reported that once they click a link in an email, users side also, the possibility of being locked out is reported
they wait to see if the visited page is rendered correctly. Partic- as the main obstacle for using FIDO in daily routine [24, 30].
ipant P9-E said, “[...] if the website looks fine, I mean the front Anti-Phishing ecosystem. Service providers, browser ven-
page, I am not suspicious”. Similarly, P20-E mis-classified dors, and other entities have developed an ecosystem to detect
the phishing website as legitimate: “It’s the same because it and prevent phishing, however adversaries adapt their tools
looks the same up here [refers to logo section], and I would continuously and evade such systems [64–66, 95]. Oest et
be trusting it’s fine”. Others rely solely on the email itself; al. [67] report that a phishing campaign is detected nine hours
participant P18-E said: “I decide before whether to click or after the first victim, hence spear-phishings that target individ-
not, and once I click it, it’s opened (done)”. When asked if uals are much more difficult to be prevented by the ecosystem.
she continues checking the visited website, she added: “Not Another line of work [25, 39, 59] try to detect malicious web-
really”. This is not new; the fact that phishing and similar so- sites based on the URL analysis. Email providers have devel-
cial engineering tactics rely on users’ lack of understanding or oped frameworks to filter out phishing emails before reaching
incomplete mental models is well established [6, 19]. So long users [21, 38], however attackers still find their way to their
as authentication relies on user actions, education remains a target’s inbox [63]. To limit the consequences of password
plausible (yet somewhat ineffective) countermeasure. reuse [10, 27], prior works have proposed frameworks that al-
Limitation: impracticality. When available, education low servers to learn when a password is compromised [80,87],
can help users form reasonable mental models of phishing while [52] shows that secure implementation of critical proto-
and may help them detect some attacks. This is an incomplete cols, such as TLS is not trivial for developers.
countermeasure, however, because it assumes that users are Client side. Password managers are a possible countermea-
capable of continuously devoting all of their attention to this sure to phishing attacks because credentials are released only
task and that all attacks will have user-noticeable indicators. if the user visits the correct domain. Blanchou and Youn [9]
Security is rarely users’ primary task [88] and humans are were among the first to report vulnerabilities in password man-
incapable of remaining highly vigilant 100% of the time. As agers. Others [36, 75] describe the challenges of designing
we saw in our study, some attacks include sufficient contex- and implementing secure extensions, while [93] reported that
tual indicators, either by design (e.g., spear phishing) or by spoofing the sidebar is effective in phishing the master pass-
coincidence, to trick even an attentive user who is actively word as well. Yang et al. [94] measured the effectiveness of
evaluating for security [6]. browser indicators, while [71] show that users lose the ability
Educational campaigns or marketing efforts that promote to detect phishing some period after training.
security keys as phishing resistant can further contribute to
users forming incorrect mental models [69], and can thus have
adverse effects as users develop a false sense of security and 10 Concluding Remarks
become less attentive to attacks.
OTP-based 2FA schemes are amongst the most common
phishing defences. Being replayable [3], they fail to defend
9 Related Work against real-time phishing, where the adversary relays user-
submitted OTPs to the legitimate site in real-time. The FIDO
Phishing is a widely-studied attack vector from the social alliance has designed challenge-response mechanisms with
engineering category. It uses effective techniques to take over browser involvement, which enables the inclusion of a web-
accounts, fooling even knowledgeable users [19, 26, 43–45]. site’s URL in the challenge. Relaying the response becomes
2FA schemes. The industry and the academic community useless, and real-time phishing is thus defeated. U2F is one
has developed several 2FA schemes [48, 50, 57, 58, 68] to such standard, where the response is computed on a hard-
protect users’ accounts. However, real-time phishing is still ware token. To handle token loss/malfunction, websites often
very effective to bypass 2FA and automated tools [34] make allow/force users to register 2FA alternatives to FIDO. All
such attacks simpler, cheaper, and easy to scale. Previous FIDO-supporting websites in Alexa’s top 100 adopt the prac-
works [23,63] report that phishing is widely employed and pre- tice. We ran a user study to test whether a phishing attack
ferred by malicious actors, even at hack-for-hire services [63]. that downgrades FIDO to weaker alternatives is effective. Al-
FIDO is based on public key cryptography [17] and its though the study tested U2F tokens, findings (particularly re-
benefits are already demonstrated in a company setting [55]. garding downgrade effectiveness) can extend to other relevant
The protocol itself is considered secure and it is promoted FIDO specifications. We make the following four remarks.
by the industry as being foolproof phishing-resistant [33]. User studies that evaluate attacks must be gracefully
The research community has focused on the usability aspects executed. Evaluating attacks through user studies is challeng-
of FIDO [18, 70, 74] but have not questioned its security in ing. Participants may fall for said attacks during the study, not

USENIX Association 30th USENIX Security Symposium 3823

because of successful deception, but rather due to participants’ Acknowledgments
lack of investment in protecting assets or misinterpretation
of study requirements. If participants’ actions were the sole Abdou acknowledges funding from the Natural Sciences and
metric, we would have misidentified 88% (instead of 55%) of Engineering Research Council of Canada (NSERC) through
participants as susceptible to our attacks (Table 2). Such stud- a Discovery Grant. Chiasson acknowledges funding from the
ies followed by semi-structured interviews delicately gauging NSERC Discovery Grants and Canada Research Chairs pro-
explanations of participants’ actions, without calling attention grams. We thank the anonymous reviewers and our shepherd,
to said attacks, provide richer results. Kent Seamons, for their insightful feedback. We also thank
Even with FIDO, users remain susceptible to real-time Sebastian Navas Chaparro for his assistance in collecting our
phishing that downgrades FIDO to weaker alternatives. Canadian data sample.
Most participants failed to detect our phishing attacks. Those
who succeeded (10%) have done so without FIDO’s help. References
We found no case where a participant was close to fall for
real-time phishing, but FIDO protected them. Our social en- [1] A. Abdou, A. Matrawy, and PC van Oorschot. Location ver-
gineering involved displaying the FIDO-prompt to users (its ification on the internet: Towards enforcing location-aware
result is discarded), followed by a prompt for another 2FA access policies over internet clients. In IEEE Conference on
alternative (its result would be relayed to the legitimate server Communications and Network Security (CNS), 2014.
in practice). This amassed to what appeared to participants as [2] A. Abdou and PC van Oorschot. Server location verification
a three-factor login, giving an increased sense of (false) secu- (SLV) and server location pinning: Augmenting TLS authen-
rity rather than arousing suspicion. The effect of such attacks tication. ACM Transactions on Privacy and Security (TOPS),
in practice is exacerbated by two points: (1) users can become 21(1):1–26, 2017.
less careful seeing more factors, and (2) reassuring wording [3] F. Alaca, A. Abdou, and PC van Oorschot. Comparative Analy-
on login pages (e.g., Google’s statement on 2FA pages “This sis and Framework Evaluating Mimicry-Resistant and Invisible
extra step shows that it’s really you trying to sign in”). Web Authentication Schemes. IEEE Transactions on Depend-
able and Secure Computing (TDSC), 2019.
Despite understanding how to use FIDO [30], users
do not understand how FIDO protects them. While dis- [4] FIDO Alliance. FIDO2: WebAuthn & CTAP. https://
cussing how they detected our attacks, no participant men- fidoalliance.org/fido2/. [Accessed Oct-2020].
tioned relying on FIDO. FIDO protects users when login is [5] FIDO Alliance. Specifications overview. https://
granted after using only FIDO, not after using FIDO plus other fidoalliance.org/specifications/. [Accessed Oct-
factors. The former prevents real-time phishing and down- 2020].
grade attacks, the latter enables them. As it is counter-intuitive, [6] M. Alsharnouby, F. Alaca, and S. Chiasson. Why phishing
no participant appears to have assimilated this concept. still works: User strategies for combating phishing attacks.
Enabling only FIDO alternatives to FIDO is an effec- International Journal of Human-Computer Studies, 82:69 – 82,
tive countermeasure. To address the necessity of allowing 2015.
alternatives to FIDO’s U2F, without enabling downgrade at- [7] M. AlZomai, B. AlFayyadh, A. Josang, and A. McCullagh. An
tacks, websites should only allow alternatives of comparable Experimental Investigation of the Usability of Transaction Au-
security. Many of the other countermeasures we explored thorizationin Online Bank Security Systems. In Australasian
would either expose users to lockouts due to token losses, or Conference on Information Security, 2008.
continue to make users potentially susceptible to other social [8] D. Balfanz, A. Czeskis, J. Hodges, JC. Jones, MB. Jones, A. Ku-
engineering variations. Relevant FIDO specifications that are mar, A. Liao, R. Lindemann, and E. Lundberg. Web Authenti-
also resilient to real-time phishing (e.g., CTAP2) appear to be cation: An API for accessing Public Key Credentials Level 1.
suitable alternatives from a security perspective. https://www.w3.org/TR/webauthn. [Accessed Oct-2020].
Actionable Takeaways. We call upon the FIDO alliance, [9] M. Blanchou and P. Youn. Browser extension password man-
its industry partners, and the security research community to agers. https://isecpartners.github.io/whitepapers/
undertake and promote the following two actionable items as passwords/2013/11/05/Browser-Extension-Password-
applicable: (a) to pursue efforts to inform, educate, and design Managers.html. [Accessed Aug-2020].
technologies that persuade users who have configured security [10] J. Blocki, B. Harsha, and S. Zhou. On the economics of of-
keys to only use such keys for login; and (b) develop new fline password cracking. In IEEE Symposium on Security and
recovery schemes that are phishing-resistant and scalable to Privacy (S&P), 2018.
millions of users. For the latter, such schemes may prioritize [11] J. Bonneau, C. Herley, PC van Oorschot, and F. Stajano. The
security guarantees over usability, as recovery is normally Quest to Replace Passwords: A Framework for Comparative
performed less frequently, whereas standard 2FA schemes Evaluation of Web Authentication Schemes. In IEEE Sympo-
typically prioritize usability for everyday use. sium on Security and Privacy (S&P), 2012.

3824 30th USENIX Security Symposium USENIX Association

[12] M. Campobasso and L. Allodi. Impersonation-as-a-Service: [28] S. L. Garfinkel and R. C. Miller. Johnny 2: A user test of key
Characterizing the Emerging Criminal Infrastructure for User continuity management with s/mime and outlook express. In
Impersonation at Scale. In ACM Conference on Computer and Proceedings of the 2005 Symposium on Usable Privacy and
Communications Security (CCS), 2020. Security, SOUPS ’05, 2005.
[13] K. Charmaz. Constructing grounded theory. SAGE, 2014. [29] N. Gelernter, S. Kalma, B. Magnezi, and H. Porcilan. The
[14] S. Ciolino, S. Parkin, and P. Dunphy. Of Two Minds about Two- password reset MitM attack. In IEEE Symposium on Security
Factor: Understanding Everyday FIDO U2F Usability through and Privacy (S&P), 2017.
Device Comparison and Experience Sampling. In USENIX [30] S. Ghorbani Lyastani, M. Schilling, M. Neumayr, M. Backes,
Symposium on Usable Privacy and Security (SOUPS), 2019. and S. Bugiel. Is FIDO2 the Kingslayer of User Authentica-
[15] Jacob Cohen. A Coefficient of Agreement for Nominal Scales. tion? A Comparative Usability Study of FIDO2 Passwordless
Educational and Psychological Measurement, 20(1):37–46, Authentication. In IEEE Symposium on Security and Privacy
1960. (S&P), 2020.
[16] D. Conway, R. Taib, M. Harris, K. Yu, S. Berkovsky, and [31] H. Gomi, B. Leddy, and D. Saxe. Recommended Account
F. Chen. A Qualitative Investigation of Bank Employee Expe- Recovery Practices for FIDO Relying Parties. FIDO Alliance,
riences of Information Security and Phishing. In Symposium 2019.
on Usable Privacy and Security (SOUPS), 2017. [32] Google. Google’s strongest security helps keep your pri-
[17] A. Czeskis, M. Dietz, T. Kohno, D. Wallach, and D. Balfanz. vate information safe. https://landing.google.com/
Strengthening user authentication through opportunistic cryp- advancedprotection/. [Accessed Oct-2020].
tographic identity assertions. In ACM Conference on Computer [33] Google. Titan security key. help prevent account takeovers
and Communications Security (CCS), 2012. from phishing attacks. https://cloud.google.com/titan-
[18] Sanchari Das, Andrew Dingman, and L Jean Camp. Why security-key/. [Accessed Oct-2020].
Johnny doesn’t use two factor a two-phase usability study of [34] K. Gretzky. Standalone man-in-the-middle attack framework
the FIDO U2F security key. In Financial Cryptography and used for phishing login credentials along with session cookies,
Data Security (FC). Springer, 2018. allowing for the bypass of 2-factor authentication. https:
[19] R. Dhamija, JD. Tygar, and M. Hearst. Why Phishing Works. //github.com/kgretzky/evilginx2. [Accessed Oct-2020].
In ACM Conference on Human Factors in Computing Systems [35] E. Grosse and M. Upadhyay. Authentication at scale. IEEE
(CHI), 2006. Security Privacy, 11(1):15–22, 2013.
[20] P. Doerfler, M. Marincenko, J. Ranieri, Y. Jiang, A. Moscicki, [36] JA. Halderman, B. Waters, and EW. Felten. A Convenient
D. McCoy, and K. Thomas. Evaluating Login Challenges as a Method for Securely Managing Passwords. In ACM World
Defense Against Account Takeover. In ACM World Wide Web Wide Web (WWW), 2005.
(WWW), 2019.
[37] S. Hammann, S. Radomirovic, R. Sasse, and D. Basin. User
[21] S. Duman, K. Kalkan-Cakmakci, M. Egele, W. Robertson, and Account Access Graphs. In ACM Conference on Computer
E. Kirda. EmailProfiler: Spearphishing Filtering with Header and Communications Security (CCS), 2019.
and Stylometric Features of Emails. In IEEE Annual Computer
Software and Applications Conference (COMPSAC), 2016. [38] Y. Han and Y. Shen. Accurate Spear Phishing Campaign Attri-
bution and Early Detection. In ACM Symposium on Applied
[22] Serge Egelman, Lorrie Faith Cranor, Jason Hong, Serge Egel- Computing (SAC), 2016.
man, Lorrie Faith Cranor, and Jason Hong. You’ve been
warned: An empirical study of the effectiveness of web browser [39] S. Hao, A. Kantchelian, B. Miller, V. Paxson, and N. Feamster.
phishing warnings. In Proc. ACM CHI, ACM, 2008. PREDATOR: Proactive Recognition and Elimination of Do-
main Abuse at Time-Of-Registration. In ACM Conference on
[23] J. Esparza. Understanding the credential theft lifecycle. Com-
Computer and Communications Security (CCS), 2016.
puter Fraud and Security, 2019(2):6 – 9, 2019.
[40] United States Federal Trade Commission-Consumer In-
[24] F. Farke, L. Lorenz, T. Schnitzler, P. Markert, and M. Dürmuth.
formation. How to Recognize and Avoid Phishing
“You still use the password after all” – Exploring FIDO2 Se-
Scams. https://www.consumer.ftc.gov/articles/how-
curity Keys in a Small Company. In USENIX Symposium on
recognize-and-avoid-phishing-scams. [Accessed Oct-
Usable Privacy and Security (SOUPS), 2020.
2020].
[25] MN. Feroz and S. Mengel. Phishing URL Detection Using
[41] INTERPOL. INTERPOL report shows alarming rate of cy-
URL Ranking. In IEEE International Congress on Big Data,
berattacks during COVID-19. https://www.interpol.int/
2015.
News-and-Events/News/2020/INTERPOL-report-shows-
[26] P. Finn and M. Jakobsson. Designing and Conducting Phish- alarming-rate-of-cyberattacks-during-COVID-19.
ing Experiments. In IEEE Technology and Society Magazine, [Accessed Oct-2020].
Special Issue on Usability and Security, 2007.
[42] Iulia Ion, Rob Reeder, and Sunny Consolvo. “...no one can hack
[27] X. Gao, Y. Yang, C. Liu, C. Mitropoulos, J. Lindqvist, and my mind”: Comparing expert and non-expert security practices.
A. Oulasvirta. Forgetting of Passwords: Ecological Theory In Symposium on Usable Privacy and Security (SOUPS), 2015.
and Data. In USENIX Security Symposium, 2018.

USENIX Association 30th USENIX Security Symposium 3825

[43] C. Jackson, DR. Simon, DS. Tan, and A. Barth. An Evaluation [60] L. Mathews. Homeland Security Chief Cites Phishing As
of Extended Validation and Picture-in-Picture Phishing Attacks. Top Hacking Threat. https://www.forbes.com/sites/
In Financial Cryptography and Data Security (FC). Springer, leemathews/2016/11/29/homeland-security-says-
2007. phishing-biggest-hacking-threat/#111b1f771978.
[44] TN. Jagatic, NA. Johnson, M. Jakobsson, and F. Menczer. So- [Accessed Oct-2020].
cial Phishing. Communications of the ACM, 50(10):94–100, [61] M. Maxim and A. Cser. Best practices: Select-
2007. ing, deploying, and managing enterprise password man-
[45] M. Jakobsson and J. Ratkiewicz. Designing Ethical Phishing agers. https://www.keepersecurity.com/assets/pdf/
Experiments: A study of (ROT13) rOnl query features. In ACM Keeper-White-Paper-Forrester-Report.pdf. Forrester
World Wide Web (WWW), 2006. Research. [Accessed Oct-2020].
[46] M. Jakobsson, A. Tsow, A. Shah, E. Blevis, and Y. Lim. What [62] Microsoft. Set up a security key as your verification method
Instills Trust? A Qualitative Study of Phishing. In Financial | azure ad. https://docs.microsoft.com/en-us/azure/
Cryptography and Data Security (FC). Springer, 2007. active-directory/user-help/security-info-setup-
[47] EE. Jones. Content analysis for the social sciences and human- security-key. [Accessed Oct-2020].
ities. PsycCRITIQUES, 14(11):615–616, 1969. [63] A. Mirian, J. DeBlasio, S. Savage, GM. Voelker, and
[48] N. Karapanos, C. Marforio, C. Soriente, and S. Čapkun. Sound- K. Thomas. Hack for Hire: Exploring the Emerging Mar-
Proof: Usable Two-Factor Authentication Based on Ambient ket for Account Hijacking. In ACM World Wide Web (WWW),
Sound. In USENIX Security Symposium, August 2015. 2019.
[49] Andy Kiersz. How different age groups identify with their [64] A. Oest, Y. Safaei, A. Doupé, G. Ahn, B. Wardman, and K. Ty-
generational labels. https://www.weforum.org/agenda/ ers. PhishFarm: A Scalable Framework for Measuring the
2015/09/how-different-age-groups-identify-with- Effectiveness of Evasion Techniques against Browser Phishing
their-generational-labels/. [Accessed May-2021]. Blacklists. In IEEE Symposium on Security and Privacy (S&P),
[50] D. Kogan, N. Manohar, and D. Boneh. T/Key: Second-Factor 2019.
Authentication From Secure Hash Chains. In ACM Conference [65] A. Oest, Y. Safaei, P. Zhang, B. Wardman, K. Tyers, Y. Shoshi-
on Computer and Communications Security (CCS), 2017. taishvili, and A. Doupé. PhishTime: Continuous Longitudinal
[51] K. Krombholz, K. Busse, K. Pfeffer, M. Smith, and E. von Measurement of the Effectiveness of Anti-phishing Blacklists.
Zezschwitz. If HTTPS Were Secure, I Wouldn’t Need 2FA In USENIX Security Symposium, 2020.
- End User and Administrator Mental Models of HTTPS. In [66] A. Oest, Y. Safei, A. Doupé, G. Ahn, B. Wardman, and
IEEE Symposium on Security and Privacy (S&P), 2019. G. Warner. Inside a phisher’s mind: Understanding the anti-
[52] K. Krombholz, W. Mayer, M. Schmiedecker, and E. Weippl. phishing ecosystem through phishing kit analysis. In APWG
“I Have No Idea What I’m Doing” - On the Usability of De- Symposium on Electronic Crime Research (eCrime), 2018.
ploying HTTPS. In USENIX Security Symposium, 2017.
[67] A. Oest, P. Zhang, B. Wardman, E. Nunes, J. Burgis, A. Zand,
[53] P. Kumaraguru, S. Sheng, A. Acquisti, L. F. Cranor, and K. Thomas, A. Doupé, and G. Ahn. Sunrise to Sunset: Analyz-
J. Hong. Teaching johnny not to fall for phish. 10(2), 2010. ing the End-to-end Life Cycle and Effectiveness of Phishing
[54] J. Richard Landis and Gary G. Koch. The measurement of Attacks at Scale. In USENIX Security Symposium, 2020.
observer agreement for categorical data. Biometrics, 33(1):159–
[68] T. Petsas, G. Tsirantonakis, E. Athanasopoulos, and S. Ioanni-
174, 1977.
dis. Two-factor authentication: is the world ready?: quantifying
[55] J. Lang, A. Czeskis, D. Balfanz, and M. Schilder. Security 2FA adoption. In ACM European Workshop on Systems Secu-
Keys: Practical Cryptographic Second Factors for the Modern rity (EuroSec), 2015.
Web. In Financial Cryptography and Data Security (FC).
[69] E. Redmiles, N. Warford, A. Koneru, S. Kross, M. Morales,
Springer, 2016.
R. Stevens, and M. Mazurek. A Comprehensive Quality Eval-
[56] E. Lin, S. Greenberg, E. Trotter, D. Ma, and J. Aycock. Does uation of Security and Privacy Advice on the Web. In USENIX
domain highlighting help people identify phishing sites? CHI Security Symposium, 2020.
’11, 2011.
[70] Ken Reese, Trevor Smith, Jonathan Dutson, Jonathan
[57] Google LLC. Google authenticator. https:
Armknecht, Jacob Cameron, and Kent Seamons. A usability
//play.google.com/store/apps/details?id=
study of five two-factor authentication methods. In USENIX
com.google.android.apps.authenticator2. [Accessed
Symposium on Usable Privacy and Security (SOUPS), 2019.
Oct-2020].
[58] RSA Security LLC. Rsa securid hard token. [71] B. Reinheimer, L. Aldag, P. Mayer, M. Mossano, R. Duezguen,
https://www.rsa.com/en-us/products/rsa-securid- B. Lofthouse, T. von Landesberger, and M. Volkamer. An
suite/rsa-securid-access. [Accessed Oct-2020]. investigation of phishing awareness and education over time:
When and how to best remind users. In USENIX Symposium
[59] J. Ma, LK. Saul, S. Savage, and GM. Voelker. Beyond Black- on Usable Privacy and Security (SOUPS), 2020.
lists: Learning to Detect Malicious Web Sites from Suspicious
URLs. In ACM Conference on Knowledge Discovery and Data
Mining (KDD), 2009.

3826 30th USENIX Security Symposium USENIX Association

[72] Google Transparency Report. HTTPS encryption on the with fido? social engineering downgrade attacks against fido
web. https://transparencyreport.google.com/https/ protocols. Cryptology ePrint Archive, Report 2020/1298, 2020.
certificates. [Accessed Oct-2020]. [84] Amber van der Heijden and Luca Allodi. Cognitive Triaging
[73] J. Reynolds, T. Smith, K. Reese, L. Dickinson, S. Ruoti, and of Phishing Attacks. In USENIX Security Symposium, 2019.
K. Seamons. A Tale of Two Studies: The Best and Worst [85] PC van Oorschot. Computer Security and the Internet: Tools
of YubiKey Usability. In IEEE Symposium on Security and and Jewels. Springer Nature, 2020.
Privacy (S&P), 2018.
[86] Arun Vishwanath, Tejaswini Herath, Rui Chen, Jingguo Wang,
[74] Joshua Reynolds, Nikita Samarin, Joseph Barnes, Taylor Judd,
and H. Raghav Rao. Why do people get phished? testing
Joshua Mason, Michael Bailey, and Serge Egelman. Empirical
individual differences in phishing vulnerability within an in-
Measurement of Systemic 2FA Usability. In USENIX Security
tegrated, information processing model. Decision Support
Symposium, 2020.
Systems, 51(3):576 – 586, 2011.
[75] B. Ross, C. Jackson, N. Miyake, D. Boneh, and JC. Mitchell. [87] KC. Wang and MK. Reiter. How to end password reuse on the
Stronger Password Authentication Using Browser Extensions. web. In Network and Distributed System Security Symposium
In USENIX Security Symposium, 2005. (NDSS), 2019.
[76] S. Schechter, AJB. Brush, and S. Egelman. It’s No Secret. [88] Alma Whitten and J Doug Tygar. Why johnny can’t encrypt:
Measuring the Security and Reliability of Authentication via A usability evaluation of pgp 5.0. In USENIX Security Sympo-
“Secret” Questions. In IEEE Symposium on Security and Pri- sium, volume 348, pages 169–184, 1999.
vacy (S&P), 2009.
[89] S. Wiefling, M. Dürmuth, and L. Lo Iacono. More Than Just
[77] S. E. Schechter, R. Dhamija, A. Ozment, and I. Fischer. The Good Passwords? A Study on Usability and Security Percep-
Emperor’s New Security Indicators. In IEEE Symposium on tions of Risk-based Authentication. In Annual Computer Secu-
Security and Privacy (S&P), 2007. rity Applications Conference (ACSAC), 2020.
[78] S. Sheng, M. Holbrook, P. Kumaraguru, L. F. Cranor, and [90] S. Wiefling, L. Lo Iacono, and M. Dürmuth. Is This Really
J. Downs. Who Falls for Phish? A Demographic Analysis You? An Empirical Study on Risk-Based Authentication Ap-
of Phishing Susceptibility and Effectiveness of Interventions. plied in the Wild. In ICT Systems Security and Privacy Protec-
CHI ’10. 2010. tion. Springer, 2019.
[79] K. Thomas, F. Li, A. Zand, J. Barrett, J. Ranieri, L. Invernizzi,
[91] H. Wimberly and L. Liebrock. Using Fingerprint Authentica-
Y. Markov, O. Comanescu, V. Eranti, A. Moscicki, D. Margo-
tion to Reduce System Security: An Empirical Study. In IEEE
lis, V. Paxson, and E. Bursztein. Data Breaches, Phishing, or
Symposium on Security and Privacy (S&P), 2011.
Malware? Understanding the Risks of Stolen Credentials. In
ACM Conference on Computer and Communications Security [92] M. Wu, R. Miller, and S. Garfinkel. Do security toolbars
(CCS), 2017. actually prevent phishing attacks? In ACM Conference on
Human Factors in Computing Systems (CHI), 2006.
[80] K. Thomas, J. Pullman, K. Yeo, A. Raghunathan, PG Kelley,
L. Invernizzi, B. Benko, T. Pietraszek, S. Patel, D. Boneh, and [93] M. Wu, R. Miller, and G. Little. Web wallet: Preventing phish-
E. Bursztein. Protecting accounts from credential stuffing with ing attacks by revealing user intentions. In Symposium on
password breach alerting. In USENIX Security Symposium, Usable Privacy and Security (SOUPS), 2006.
2019. [94] W. Yang, A. Xiong, J. Chen, RW. Proctor, and N. Li. Use
[81] C. Thompson, M. Shelton, E. Stark, M. Walker, E. Schechter, of Phishing Training to Improve Security Warning Compli-
and A. Felt. The web’s identity crisis: Understanding the ef- ance: Evidence from a Field Experiment. In ACM Hot Topics
fectiveness of website identity indicators. In USENIX Security in Science of Security (HoTSoS): Symposium and Bootcamp,
Symposium, 2019. 2017.
[82] E. Ulqinaku, D. Lain, and S. Čapkun. 2FA-PP: 2nd Factor [95] P. Zhang, A. Oest, H. Cho, Z. Sun, RC. Johnson, B. Wardman,
Phishing Prevention. In ACM Conference on Security and S. Sarker, A. Kapravelos, T. Bao, R. Wang, Y. Shoshitaishvili,
Privacy in Wireless and Mobile Networks (WiSec), 2019. A. Doupé, and G. Ahn. CrawlPhish: Large-scale Analysis
of Client-side Cloaking Techniques in Phishing. In IEEE
[83] Enis Ulqinaku, Hala Assal, AbdelRahman Abdou, Sonia Chi-
Symposium on Security and Privacy (S&P), 2021.
asson, and Srdjan Čapkun. Is real-time phishing eliminated

USENIX Association 30th USENIX Security Symposium 3827

A Employee On-boarding Information

NanoTech IT department

Employee Onboarding Information

Welcome to NanoTech​! We’re excited to have you!

At NanoTech​, we’re committed to protecting the company, employees, and

customers’ resources, internal and external networks, and sensitive data.

Our goals:
- Safeguard NanoTech ​confidential information, employee information, and
our customers’ confidential information
- Ensure uninterrupted and efficient operations at NanoTech
- Protecting NanoTech​ against scammers, including phishing attacks
- Comply with industry, regulatory, and customer requirements

Your role:
- Report theft, loss, or unauthorized disclosure of NanoTech​ information
- Report attempts for stealing NanoTech information, including suspicious
phishing emails
- Adhere to copyright, trade secret, patent and IP laws
- Log off from your email account(s) at the end of your work day

As part of your onboarding process, you’ll receive your work devices and
credentials. Reach out to the IT department if you have any issues.

You will be using two-factor authentication to authenticate network resources,

computer resources, and Google services. For authentication, you will need your
password and your security key. If you lose your security key or if it was damaged,
you can login to your account using other backup mechanisms, eg, using the
Google Authenticator app already installed on your work phone. In case of loss or
damage to the security key, please reach out immediately to the IT department
to replace your key.

Why the FIDO security keys?

A FIDO security key is a phishing-resistant two-factor authentication (2FA)
device. FIDO keys use cryptography to provide two-way verification: it makes
sure that you are logging into the service you originally registered the
security key with, and the service verifies that it’s the correct security key as
well. This provides superior protection to code-based verification, like SMS
and one-time password (OTP).

We’ve already registered your security key with your accounts! Start inventing!

Please sign here to indicate that you have read this information and agree to
adhere to it.

Signat re Date

NanoTech IT department

Jordan Hart
Email: jordan.hart540 hotmail.com
Password: Ben32tart
(Email is the primary method of correspondence in the company)

Google services account

Username: jordan.hart540 gmail.com
Password: Ben32tart
(You will need to use two-factor authentication for Google services)

Manager Alex James (alex.james1231 hotmail.com)

IT manager Sam Parker (sam.parker000 hotmail.com)
HR corre pondence Sam Logan (sam.logan2019 hotmail.com)

3828 30th USENIX Security Symposium USENIX Association