Post-Quantum Authentication in TLS 1.

3:
A Performance Study

Dimitrios Sikeridis⇤ , Panos Kampanakis† , Michael Devetsikiotis⇤

⇤ Dept. of Electrical and Computer Engineering, The University of New Mexico, Albuquerque, NM, USA
† Security & Trust Organization, Cisco Systems, USA

{dsike, mdevets}@unm.edu, [email protected]

Abstract—The potential development of large-scale quantum Apart from connection integrity and confidentiality, TLS
computers is raising concerns among IT and security research provides authentication usually with the use of X.509 certifi-
professionals due to their ability to solve (elliptic curve) discrete cates [45]. Such certificates are issued by trusted third-parties
logarithm and integer factorization problems in polynomial time. called Certificate Authorities (CAs). Endpoints verify the
All currently used public key algorithms would be deemed communicating peer’s identity and public key (PK) contained
insecure in a post-quantum (PQ) setting. In response, the National
Institute of Standards and Technology (NIST) has initiated a pro-
inside his certificate by leveraging a chain of certificates that is
cess to standardize quantum-resistant crypto algorithms, focusing rooted to a pre-trusted root CA. The two most popular digital
primarily on their security guarantees. Since PQ algorithms signature algorithms used in certificates today are the Elliptic
present significant differences over classical ones, their overall Curve Digital Signature (ECDSA) and RivestShamirAdleman
evaluation should not be performed out-of-context. This work (RSA). Their security guaranties rely on the hardness of the
presents a detailed performance evaluation of the NIST signa- elliptic curve discrete logarithm (ECDL) and integer factoriza-
ture algorithm candidates and investigates the imposed latency tion (IF) problems respectively.
on TLS 1.3 connection establishment under realistic network
conditions. In addition, we investigate their impact on TLS While the security of the aforementioned schemes cannot
session throughput and analyze the trade-off between lengthy be practically challenged by conventional computer systems,
PQ signatures and computationally heavy PQ cryptographic this would not be the case in a post-quantum world where
operations. Our results demonstrate that the adoption of at least a large scale quantum computer has become a reality [62].
two PQ signature algorithms would be viable with little additional Shor’s quantum algorithm [76], [87], assuming a practical
overhead over current signature algorithms. Also, we argue that quantum computer (QC) was available, would solve ECDL and
many NIST PQ candidates can effectively be used for less time- IF problems in polynomial time which would render ECDSA
sensitive applications, and provide an in-depth discussion on the
and RSA insecure. In this scenario, a QC-equipped attacker
integration of PQ authentication in encrypted tunneling protocols,
along with the related challenges, improvements, and alternatives. would be able to impersonate signers that use these algorithms.
Finally, we propose and evaluate the combination of different PQ Thus, encrypted tunnel (e.g. TLS, IKEv2) authentication, PK
signature algorithms across the same certificate chain in TLS. infrastructure (PKI), CAs, and software signing would be
Results show a reduction of the TLS handshake time and a broken.
significant increase of a server’s TLS tunnel connection rate over
using a single PQ signature scheme.
To address the issue, the cryptographic community has
been researching quantum-resistant public key algorithms for
some time, while the US NIST started a public project to
I. I NTRODUCTION standardize quantum-resistant public key encapsulation and
signature algorithms. Similarly, ETSI has formed a Quantum-
Digital communications have completely penetrated ev- Safe Working Group [32] that aims to make assessments and
eryday life and the physical world as enablers of numerous recommendations on the various proposals from industry and
critical services including telemedicine, online banking, mas- academia regarding real-world deployments of quantum-safe
sive e-commerce, machine-to-machine automation, mobile and cryptography. Moreover, the IETF has seen multiple proposals
cloud computing. In this reality, public-key cryptography is that attempt to introduce and investigate PQ algorithms in
ubiquitous in almost all cryptographic protocols, such as TLS protocols like TLS and IKE [33], [40], [68], [91], [93].
which builds encrypted tunnels between digital entities. Studies
At the moment of this writing, NIST’s evaluation pro-
suggest that over 60% of Internet connections are implemented
cess has moved from Round 1 to Round 2 where 26 PQ
over the TLS-based secure HTTPS protocol [20], [64]. TLS
algorithms were chosen with security guarantees being the
adoption is expected to keep increasing as users and client
primary criterion, while performance was treated as a future
vendors strive for ubiquitous encryption and privacy [51].
goal [2]. Evidently, the actual integration of these algorithms
into existing protocols (e.g., TLS, IKEv2, SSH) and their
Network and Distributed Systems Security (NDSS) Symposium 2020, 23-26 coexistence with today’s Internet infrastructure present chal-
February 2020, San Diego, CA, USA lenges that pertain to (a) additional latency due to their heavy
ISBN 1-891562-61-4, https://dx.doi.org/10.14722/ndss.2020.24203
www.ndss-symposium.org operations, (b) communication overhead from the increased
Permission to freely reproduce all or part of this paper only for non- public key and signature sizes, and (c) optimal use of exist-
commercial purposes is granted provided that copies bear this notice and the ing hardware towards faster implementations. These gaps are
full citation on the first page.
actively being studied by research teams in the industry with in the NIST standardization process. We also do not perform
NXP Semiconductors, Microsoft and Queensland University an exhaustive benchmark of all the available parameter sets of
of Technology [15], Amazon [18], Cloudflare and Google the signature algorithms under consideration by NIST. Relying
focusing on the impact of key exchange mechanisms (KEMs) on preliminary findings and individual algorithm claims, we
on TLS [52], [54]–[56]. have explicitly chosen to focus on a subset of algorithms
and parameter sets that would seamlessly fit in TLS. To the
These efforts mostly focus on TLS PQ key exchange as
best of our knowledge, this is the first work that assesses
confidentiality is considered more urgent. Since a QC-equipped
the performance of PQ certificates in TLS 1.3 by considering
attacker would be able to decrypt stored communications
realistic network conditions.
retroactively, ensuring quantum-resistant encryption for critical
data is a priority. This is not the case with authentication as The rest of the paper is organized as follows: Section II
digital entity impersonation cannot happen retroactively. How- presents background on X.509 PKI and TLS 1.3. In Section III,
ever, there are numerous incentives that drive the early study the different PQ candidate signature families and algorithms
of PQ authentication in today’s protocols. First, PKI refresh are presented, while Section IV details the integration of
cycles are traditionally long and migration to new primitives PQ authentication into TLS 1.3. Section V presents the ex-
can take years. A case in point is ECDSA. Even though perimental procedure, results and their analysis. Section VI
it was standardized in 2005 [4] offering clear performance summarizes related work. Finally, Section VII discusses the
advantages over RSA [39], its adoption was still not broad general implications and potential solutions of integrating
a decade later. Finally, the computational performance of PQ new PQ signatures in encrypted tunnels, while Section VIII
algorithms, along with the fact that the sizes of the resulting PQ concludes this paper.
certificates are significantly larger, will definitely impact the
TLS handshake by worsening user experience and connection
performance. Thus, it is important to investigate and identify II. BACKGROUND
promising PQ signature candidates specifically for utilization This section presents an overview of the TLS 1.3 hand-
in TLS. shake protocol, along with a summary of the X.509 PKI
In this paper, we study the overhead introduced by PQ currently used in TLS.
certificates in the establishment of TLS 1.3 tunnels. Our
goal is to identify PQ signature candidates that could be A. X.509 Certificates and PKI
employed in TLS without major protocol updates, measure
their performance in real-world deployments and contribute to A digital entity’s (e.g., a server) identity is bound to its
the overall discussion about their use in encrypted tunnels. The public key via a digital certificate. Since X.509 is the most
key contributions of our work are summarized as follows: common PKI standard adopted by IETF protocols, X.509
certificates play an important role in digital authentication for
(i) We analyze the candidate signature algorithms from various protocols (e.g., TLS, SSH, IKEv2). X.509 certificates
NIST’s Post-Quantum Cryptography Project and compare are defined by RFCs 5280 [22] and 6818 [94], and their sizes
them in terms of performance, security level claims and usually vary between 0.5 and 1.5KB.
key/signature sizes. In addition, we integrate software imple-
mentations of these schemes in X.509 certificates for TLS 1.3 A PKI infrastructure consists of various parts. A Certificate
authentication in the OQS OpenSSL library [74]. Authority (CA) issues an entity’s certificate that assures the
entity’s identity and public key tie to that identity. The identity
(ii) We conduct large-scale measurements to investigate the is included in the Subject field of the certificate, while the
PQ authentication algorithm impact on TLS 1.3 handshake entity’s public key is stored in the Subject Public Key
establishment in realistic network conditions. Moreover, we Information along with the algorithm used by the issuer to
investigate the impact that larger PQ certificate chains or create the signature. A certificate contains a specific validity
slower sign/verify operations have on the throughput of a period and extensions included by CAs to enable additional
server performing PQ authentication in HTTPS. functionality. The certificate is signed by the CA’s private key
(iii) We demonstrate the viability of adopting two PQ can- using the specified signature algorithm and the signature is
didate signature algorithms, Dilithium, and Falcon, for time- added to the certificate’s Signature field.
sensitive applications over TLS and we argue that Falcon is
Certificates are exchanged between entities during a session
more suitable for the web if floating point hardware is available
setup in order for each party to verify the peer’s identity. At the
at the server. We argue that less time-sensitive applications can
top of the X.509 PKI there are trusted CAs that self-sign their
use a larger set of PQ candidate algorithms.
own certificates known as root CA certificates. Normally a root
(iv) We propose and study the use of different PQ signature CA issues certificates for intermediate CAs (ICAs). Following
scheme combinations in the same certificate chain to improve that, the root CA is kept offline for security purposes. An ICA
the performance of PQ authentication in TLS which, to the can further issue certificates for other ICAs that in turn sign
best of our knowledge, has not been examined before. leaf certificates in the PKI. This process results to the creation
of certificate chains of trust that usually consist of two to four
(v) We provide an in-depth discussion of PQ authentication
certificates but can be arbitrarily long. A leaf certificate is
challenges for encrypted tunnelling protocols, alternatives, and
validated by an endpoint if (a) the endpoint trusts the chain’s
present insights towards optimizing future deployments.
root CA, and (b) all the signatures from the leaf to the root
Note that we do not evaluate the PQ security claims or certificate of the chain are verified by using the public key of
security proofs of the PQ algorithms. These will be assessed the issuer.

2
B. TLS 1.3 Encrypted Tunnels A. Quantum-Resistant Families of Problems
The TLS protocol design provides endpoint authentication Since currently used public-key cryptography schemes
and establishes encrypted communication tunnels between would be threatened by large-scale quantum computers, re-
them. The aim is to ensure data integrity, confidentiality, and search for alternatives —namely post-quantum schemes— able
authenticity. In August 2018, the latest version of the protocol to resist QC attacks has been surging for the last decade. The
—TLS 1.3— was published as RFC 8446 [78]. TLS 1.3 offers goal has been to identify suitable problems or subproblems
significant improvements over its predecessor TLS 1.2 that of NP-hardness that are not solvable in polynomial time by
include elimination of insecure or obsolete features, complete quantum algorithms [16].
encryption of the handshake, and reduced handshake latency
by eliminating a round-trip. Hash-based Cryptography This family of PQ signa-
ture algorithms relies on Merkle trees and few or one-time-
Below we present the full TLS 1.3 handshake, and a signature (FTS/OTS) used with secure cryptographic hash
summary of the messages exchanged between a client that functions. Important security requirements for these functions
initiates a connection with a remote server: Initially, the client include collision and preimage resistance. The first scheme
calculates an ephemeral public/private key pair for the key in the family, Merkle signature scheme (MSS), was presented
exchange and sends to the server a ClientHello message in the late 1970s [16]. The use of hash functions with Merkle
that contains a random nonce, the protocol versions and trees and FTS/OTS for generating signatures is considered ma-
cipher suites that the client supports, and possible extensions ture, well-understood, and significantly reliable. Hash-Based
which include pre-shared keys, a list of supported signature Signature (HBS) schemes generate keypairs for the FTS/OTS.
algorithms (signature_algorithms extension), or a list The FTS/OTS signs a message and the corresponding public
of supported signature algorithms specifically for certificates key becomes a leaf in the Merkle tree. The resulting Merkle
(signature_algorithms_cert extension). tree root is the public key. Currently, the most mature schemes
of this family are the stateful LMS [60] and XMSS [43], and
In turn, the server calculates its own ephemeral key stateless SPHINCS+ , one of the NIST signature candidates [5].
pairs, determines the desired cryptographic parameters and A stateful HBS relies on an OTS and a signer needs to ensure
responds with a ServerHello message that contains that the OTS private key is never reused. This state man-
the server’s nonce, a public key for the key exchange, agement requirement is considered an important disadvantage.
the preferred cipher suite and optionally key_share and While stateless SPHINCS+ alleviates this issue, it also leads
pre_shared_key extensions. At this point, by combining to an increase in signature size to ⇠40-60 KB and slower
the ClientHello and ServerHello messages the two performance.
entities generate a shared key and the connection gets en-
crypted. The server continues by sending a Server Change Lattice-based Cryptography Another family of hard
Cipher Spec message for TLS 1.2 compatibility pur- problems rely on lattices [65], [70]. A lattice is the set of
poses (”middlebox compatibility mode”). This is followed by all integer linear combinations of linearly independent vec-
the server sending ServerEncryptedExtensions ex- tors in real n-space Rn . There are many lattice-related NP-
tensions, and optionally a ServerCertificateRequest hard problems used for cryptographic purposes including the
message if client certificate authentication is required. The shortest vector problem (SVP) (i.e., find a shortest vector
encrypted extensions include the server’s certificate and certifi- in the Euclidean norm), the closest vector problem (CVP)
cate chain for authentication (ServerCertificate), and a (i.e., find a lattice vector that minimizes the distance from
ServerCertificateVerify message that contains a sig- another target lattice), and the lattice basis reduction problem.
nature over the handshake. Finally, the server ends his part of The lattice problems NIST PQ signature candidates depend
the handshake with a ServerFinish message that contains on are learning with errors (LWE) [77], ring learning with
a Message Authentication Code (MAC), namely verification errors (RLWE) [59], module learning with rounding or errors
data generated by a hash of all the messages exchanged so far. (MLWR or MLWE) and NTRU [41]. Some of these may be
reducible to these NP-hard lattice problems. The NTRU and
Following that, and again for compatibility purposes, LWE families have been studied more extensively than oth-
the client responds with a Client Change Cipher ers. Lattice-based algorithms are promising quantum-resistant
Spec message to the server. If client authentication is solutions with relatively efficient implementations and strong
requested the client then sends its public key certifi- security properties. Among the NIST PQ signature candidates
cate (a ClientCertificate message) and optionally a the list of lattice-based schemes includes Dilithium [30], qTesla
ClientCertificateVerify. The TLS 1.3 handshake [3], and Falcon [34].
finishes with the client ClientFinish message. The com-
plete TLS 1.3 handshake and its variations are detailed in RFC Multivariate Cryptography Another family of problems
8446 [78], while an illustrated byte-per-byte description can be that are used by some NIST’s signature algorithm candidates
found in [29]. is related to solving multivariate quadratic equations over finite
fields which is an NP-hard problem. The system’s hardness to
solve depends mainly on the degree, the number of variables,
III. P OST-Q UANTUM C ANDIDATE S IGNATURE S CHEMES and the underlying finite field’s size. Multivariate PQ schemes
often lead to excessive key or signature sizes. Thus, recent
In this section we present the different families of PQ sig- research has focused on reducing keys sizes for these schemes
nature algorithms, and document details regarding the specific [16]. The NIST multivariate-based signature candidates are
schemes and parameter sets used in this study. Rainbow [27], MQDSS [21], LUOV [12], and GeMSS [19].

3
 Signature Hard Public Key Private Key Signature Claimed Classical Claimed PQ
Specification
Algorithm Problem Size (Bytes) Size (Bytes) Size (Bytes) Security Level Security Level

RSA 3072 [61] Integer Factorization 387 384 384 128 bits ⇠0 bits
ECDSA 384 [4] EC Discrete Logarithm 48 48 48 192 bits ⇠0 bits

Dilithium II [30], [31] Module Learning with Errors 1184 2800 2044 100 bits 91 bits
Falcon 512 [34] NTRU 897 1281 690 114 bits 103 bits
MQDSS 48 [21] Multivariate 46 13 20854 160 bits 99 bits
Picnic L1F S [38] Zero-Knowledge Proofs 33 49 34036 128 bits 64 bits
SPHINCS+ SHA256-128f-simple [5] Hash-Based 32 64 16976 128 bits 64 bits
Rainbow Ia - Cyclic [27] Multivariate 58144 92960 64 143 bits 106 bits

Dilithium IV [30], [31] Module Learning with Errors 1760 3856 3366 174 bits 158 bits

Falcon 1024 [34] NTRU 1793 2305 1330 263 bits 230 bits

TABLE I: Conventional and Post-Quantum Signature Algorithms and Parameter Sets used in this study.

Finally, Picnic [38] is a NIST candidate signature scheme would double the handshake time in TLS. Therefore, we did
that does not rely on structure hardness (lattice, quadratic not evaluate qTesla further.
equations, Merkle trees). Picnic depends on zero-knowledge
proofs, and symmetric key primitives like hash functions and Picnic [38]: In this study, we examine the Round 1 Picnic
block ciphers. L1FS parameter set which existed in the OQS OpenSSL [74]
and corresponds to NIST’s Level 1. The Round 2 submission
preserves all Round 1 parameter sets, and adds another three
B. PQ Signature Algorithms and Parameter Sets Studied that are referred to as Picnic2. This new parameter set reduces
the size of the produced signature by making the whole scheme
In their PQ Project, NIST defined five security levels based significantly slower while the signature is still over 10KB [37,
on the bits of security offered by the algorithm parameter set. § 11.3, § 11.4]. Thus, although our evaluation of Picnic is
In our study, we focus on Level 1, 3 and 5. Level 1 corresponds slightly optimistic, it still does not perform well in TLS.
to 128-bit security, while Levels 3 and 5 offer 192 and 256
bits of security respectively. The choice of specific parameter MQDSS [21]: The MQDSS specification provides multiple
sets for each algorithm in our study was done specifically for parameter sets, and specifically recommends two of them.
use in TLS, and by taking into account their performance and MQDSSS-31-48 corresponds to NIST Levels 1-2. In our study,
signature/key size outputs. we integrated and evaluated the MQDSSS-31-48 implementa-
tion from liboqs [73], [90]. Note that, in a recent NIST
SPHINCS+ [5]: The SPHINCS+ signature algorithm mailing list thread, G. Zaverucha & D. Kales offered a new
specification defines 36 different parameter sets, that uti- attack which the MQDSS team acknowledged and will address
lize different hash functions including Haraka, SHA256, and in the future with a new parameter set that will probably
SHAKE256. For our experiments, we chose to integrate and worsen our measured performance.
evaluate the SHA256-128f-simple parameter set that cor-
responds to NIST’s Level 1 because it was the most efficient Rainbow [27]: The specification of Rainbow suggests
one integrated in PQClean [75]. three different variants, and parameter sets. By examining the
cryptographic operations’ performance, along with the public
Dilithium [30], [31]: In our study, we integrated and eval- keys of each alternative, we chose to test the Rainbow Ia -
uated Dilithium II which was in PQClean [75] and corresponds Cyclic parameter set integrated in the PQClean project [75]
to NIST’s Level 1. We also examined Dilithium IV, Dilithium’s because it was the most promising one for TLS. Rainbow Ia
highest security level (Level 3). We also tested Dilithium III offers Level 1 security.
(Level 2) as an alternative since Dilithium II offers less than
128 bits of classical security. Table I summarizes the conventional, and PQ signature
algorithms examined in this study and presents the sizes of
Falcon [34]: Regarding the Falcon signature scheme, its public keys, private keys, and signatures.
Round 2 submission describes two parameter sets, Falcon 512
that provides NIST’s Level 1 security, and Falcon 1024 that Two NIST Round 2 candidate signature schemes were not
corresponds to Level 5. In our experiments we integrated included in our study, namely GeMSS and LUOV. The GeMSS
and evaluated the version provided by the Falcon team in specification [19] presents 9 parameter sets with large public
liboqs [73], [90]. That version did not include the floating key sizes that range between 350 and 3087 KB. Those values
point hardware optimizations that improve Falcon’s signing are too large to be considered for practical use in TLS and
performance by ⇠20 times [72]. would even present functionality limitations (see discussion
about Rainbow in Section V-A). The LUOV specification [12]
qTesla [3]: Initially, the heuristic version of the qTesla presents 6 parameter sets for Round 2 with big public key
I parameter set (NIST Level 1) tested as a very promising sizes, accompanied with small signatures and relatively good
TLS candidate. However, at the time of this writing, an cryptographic operation performance. This scheme was not
update on qTesla’s Round 2 submission eliminated all Round considered for our testing for two reasons: (a) it resembles,
2 heuristic parameter sets. In their place the qTesla team kept like GeMSS, Rainbow (small signatures, big public key)
the provably-secure parameter sets that result in significantly which we examined, and (b) a new attack against LUOV and
bigger signature (2.5KB) and public key (⇠15KB) which lifted structure schemes was presented at the Second PQC

4
 Version PQ Signature Algorithm
Client Server
Serial Number Issuer PQ
X.509
Validity Not Before Not After Cert.
ClientHello
Subject Subject Public Key Info
+ signature_algorithms
PQ Public Key Algorithm
X509v3 + signature_algorithms_cert
Extensions PQ Public Key

ServerHello

TLS Handshake Time

PQ Signature

Certificate Transfer Time

{Server Change Cipher Spec}
PQ X.509 Certificate
{EncryptedExtensions}

Fig. 1: Post-Quantum X.509 Certificate {PQ ServerCertificate} One

PQ Sign
{PQ ServerCertificateVerify} Operation
{ServerFinished}
Standardization Conference [28].

IV. P OST-Q UANTUM AUTHENTICATION IN TLS 1.3 PQ Verify

Operations
{ClientFinished}
This section presents details regarding the implementation
and integration of PQ authentication into TLS 1.3 with its Application Data
implications. To integrate PQ signatures into X.509 and TLS
we utilized the OQS OpenSSL [74] library, which is a fork
of OpenSSL that introduces post-quantum algorithms from the Fig. 2: Post-Quantum TLS 1.3 Handshake
liboqs library [73], [90]. The OQS OpenSSL version we
used was based on OpenSSL version 1.1.1c with ASM operations to the received signatures before sending its own
optimizations for SHA256, SHA512, RSA and ECDSA256. Finished message to end the PQ TLS 1.3 handshake.
No ASM optimizations were included for ECDSA384 which
we used as reference/comparison in our experiments (see V. P ERFORMANCE E VALUATION
Section V).
In this section we present the performance of each PQ
Migrating to PQ authentication will require changes in signature scheme after its integration into OQS OpenSSL. In
X.509 and SSL libraries; in our case OpenSSL. Fig. 1 shows our experiments, we assume a classic web scenario where only
the format of a PQ X.509 certificate and the fields where PQ the server is authenticated using X.509 certificates. All TLS
algorithm support will need to be added. The new certificate 1.3 handshakes are full 1-RTT mode without PSK resumption.
will carry the subject’s PQ public key and the specific PQ In order to compare PQ authentication in TLS against current
signature algorithm used to create the signature. Then the algorithms, we utilize the RSA and ECDSA as baselines.
certificate will be signed by the issuer. The PQ signature For RSA authentication, we use the 3072-bit version that
will be placed in the Signature field. The addition of the provides 128 bits of security, (Table I). Regarding ECDSA,
PQ public key and PQ signature to the X.509 certificate will we utilize the secp384r1 curve which offers 192 bits of
increase the size of the certificate and the size of the related security. We note that ECDSA with secp256r1 or Ed25519
certificate chains (Table I. In TLS 1.3, the maximum default would offer 128-bit security level equivalent to RSA3072 with
size of a X.509 certificate or certificate chain is 224 -1 bytes better performance, but we chose higher security level for
and the signature size limit is 216 -1 bytes. To include new ECC signatures because it is believed that these primitives
algorithm support, the OQS OpenSSL library defines new will be broken by a QC before their equivalent security level
X.509 algorithm identifiers [25], [74]. big number (RSA) signatures. We could have tested against
RSA2048 or ECDSA256 instead, but our experiments showed
Next, we focus on the TLS 1.3 handshake pieces that slightly faster and smaller signatures will negligibly speed
affected by PQ algorithms. Fig. 2 shows an overview up handshakes that take an average ⇠110ms [63]. RSA and
of the messages exchanged by a client who attempts ECDSA/EdDSA would offer 0 bits of security in a post-
to set up a TLS 1.3 session with a server utiliz- quantum setting. Our objective is to document strictly PQ au-
ing quantum-resistant authentication. We assume a classic thentication in TLS. Thus, we utilize a classic X25519 elliptic
web scenario where the client is not authenticated. The curve Diffie-Hellman (ECDH) key exchange with Curve25519.
ClientHello message will negotiate the desired PQ sig-
nature algorithm using the signature_algorithms or The experiments involved one local machine and several
signature_algorithms_cert extensions. These are es- cloud-based instances. The local host was equipped with an
sentially lists of algorithm identifiers. The next adjustment for Intel i5-8350U processor and 16 GBs of RAM, while the
PQ authentication is the PQ X.509 certificate/chain transmis- experiments were implemented on a virtual machine that
sion by the server with the ServerCertificate message utilized four cores running at 1.7 GHz each and 8GB of RAM.
that will now include PQ certificates. In addition, the server Our cloud machines were Google Cloud Platform (GCP)
will sign the transcripts of the handshake and transmit a PQ n1-standard-2 [88] instances running on an Intel Xeon
CertificateVerify message that contains a PQ signa- processor with two cores at 2 GHz each and 8 GBs of RAM.
ture. Currently, the maximum default size of this message is All the participating machines were running Ubuntu 18.04 in
102.4 KB. Additionally, when a certificate chain exceeds 16 an x86_64 architecture. TCP Segmentation Offload (TSO)
KB, TLS utilizes Record Fragmentation [78]. Finally, the client and Large Receive Offload (LRO) were enabled on the hosts’
will will use the PQ signature algorithm to perform verification virtual network cards.

5
 Local Machine (ms) Cloud Instance (ms)
Signature
Algorithm Sign Verify Sign Verify

Mean St. Dev. Mean St. Dev. Mean St. Dev. Mean St. Dev.
RSA 3072 3.19 0.023 0.06 0.001 2.39 0.010 0.04 0.002
ECDSA 384 1.32 0.012 1.05 0.020 1.28 0.015 0.93 0.004

Dilithium II 0.82 0.021 0.16 0.005 0.41 0.018 0.12 0.008

Falcon 512 5.22 0.054 0.05 0.004 6.50 0.091 0.07 0.003
MQDSS 48 10.30 0.147 7.25 0.100 10.25 0.181 7.40 0.110
Picnic L1F S 4.09 0.050 3.25 0.049 3.17 0.051 2.39 0.044
SPHINCS+ SHA256-128f-simple 93.37 0.654 3.92 0.043 62.7 0.548 2.50 0.037
Rainbow Ia 0.34 0.015 0.83 0.036 0.25 0.020 0.48 0.044

Dilithium IV 1.25 0.021 0.30 0.012 0.46 0.019 0.23 0.010

Falcon 1024 11.37 0.102 0.11 0.005 14.20 0.156 0.14 0.005

TABLE II: Duration of Sign and Verify Operations: Mean and Standard Deviation

Readers should note that in our evaluation we do not con- A. Speed of Cryptographic Operations
sider the impact of PQ signatures employed in CRL and OCSP
revocation checks which would take place over a different First, we evaluate the performance of cryptographic opera-
connection. In addition, potential use of OCSP staples would tions for the signature algorithms used in this study. Sign and
introduce another signature generation and verification. Staples verify operations are significant differentiation points for the
can be cached which would not affect the handshake until PQ algorithms since they are performed by the server and the
the staple expired. That falls outside the scope of our work. client respectively in the TLS handshake. The measurements
Similarly, we do not consider SCT checking which requires for the classic algorithms (RSA, ECDSA) were taken using
more signature verifications and TLS data. PQ revocation and the OpenSSL speed command, while for the PQ schemes
SCTs could slow down the handshake, as they do already, but we used the testing scripts from liboqs [73], [90]. Table II
there are techniques to alleviate their impact which we discuss shows the average sign and verification times of the various
in Section VII-B. schemes run on the machines of this study. Since the duration
of these operatiosn directly affects the experiment’s TLS
Finally,we use the following metrics to evaluate the TLS handshake time, we measure the algorithms’ standalone perfor-
performance from the server and client’s perspective: mance in milliseconds for each type of machine used instead of
CPU clock cycles which is the common practice. We observe
• TLS Handshake Time: The time between the
that Dilithium, and Rainbow offer competitive performance for
ClientHello message and the end of the handshake.
128 and 192-bit security comparable to classical algorithms
This metric models the latency a client experiences
(RSA3072 and ECDSA384). Falcon offers great verification
before establishing a secure encryption tunnel excluding
times, and slower signing. We can see that only Falcon
the TCP handshake.
is slower in the cloud machine. We attribute this behavior
• Certificate Transfer Time: The time between the
to Falcon’s floating-point operations which do not exist in
ServerHello message and the Client Change
other PQ signature schemes. MQDSS and Picnic are slower
Cipher Spec message that includes the transfer of
and SPHINCS+ is significantly slower, especially for sign
certificate chains from the server to the client.
operations. Our signing and verification measured performance
• TLS Transactions per Second: This metric expresses the
is generally confirmed by the SUPERCOP benchmarks [9] and
rate of successful HTTPS connections the server is able
[83].
to serve.
NIST Round 2 submissions included optimized versions
of the algorithms that take advantage of parallelization and
Cert Chain Size (KB)
Signature CertificateVerify hardware acceleration (e.g., AVX2). These optimizations speed
Algorithm One ICA Two ICAs Size (KB) up performance of the algorithms significantly. The imple-
RSA 3072 1.63 2.44 0.38 mentations we used in our experiments (Table II) were not
ECDSA 384 1.34 2.15 0.05 optimized. Using the optimized versions would make signing
Dilithium II 6.90 10.42 2.04
and verification greatly faster. For example, AVX2 optimized
Falcon 512 3.54 5.37 0.69 Falcon signs ⇠20 times faster and AVX2 optimized Dilithium
MQDSS 48 42.24 63.42 20.85 IV signs faster than Dilithium II without AVX2. We expect
Picnic L1F S 66.20 99.57 30.03 that future hardware acceleration of the algorithms themselves
SPHINCS+ SHA256-128f-si 34.46 51.74 16.98 would improve their performance further and eventually the
Rainbow Ia 116.86 175.35 0.06 TLS cert chain and CertificateVerify size will be the
Dilithium IV 10.70 16.11 3.37 bottleneck of PQ authentication. The results of our work are
Falcon 1024 6.56 9.89 1.33
still relevant for use-cases where acceleration is not available
(potentially on constrained devices), but they could underes-
TABLE III: Sizes of certificate chains (excluding the root), and timate the performance we can get from each scheme which
CertificateVerify extension we discuss in the relevant sections in our analysis below.

6
 0.15

TLS Handshake Time (sec)

One ICA Two ICAs

0.03
0.1
0.02

0.01
0.05 II .
RSA ECD
SA Dil. Falc

0
RSA ECDSA Dilithium II Falcon 512 MQDSS Picnic SPHINCS Rainbow*
Authentication Algorithm

Fig. 3: TLS Handshake Time for NIST’s PQ Security Level 1 Signature Algorithms: Average and Standard Deviation

Certificate Transfer Time (sec)

Cumulative data transfet (bytes)
1 RSA Falcon 512 SPHINCS
1 ECDSA MQDSS 0.06
6
10 Dilithium II Picnic
0.8 0.99 0.04

5
10
0.6 0.98 0.02
CDF

0 0.1 0.2 0.3 0.4

104 0
0.4

SA

co II

2
SS

SP nic

S
SA
RSA MQDSS

51

C
m

D
R

IN
D

c
iu

Pi
Q
EC

H
th
ECDSA Picnic

M
ili

l
0 50 100 150 200

Fa
0.2

D
Dilithium II SPHINCS Number of Handshakes Authentication Algorithm
Falcon 512
0 Fig. 6: Impact of PQ Certificate Transfer NIST’s PQ Security Level 1
0 0.1 0.2 0.3 0.4 Signature Algorithms (one ICA): (a) Cumulative Transfer Overhead,
TLS Handshake Time (sec) (b) Average Transfer Time
Fig. 4: TLS Handshake Time for NIST’s PQ Security Level 1
Signature Algorithms: Empirical CDF - One ICA SCT and OCSP signatures would add significant data to the
handshake. We further discuss this point in Section VII-B.
B. PQ TLS Overhead Analysis - NIST Security Level 1 In our performance experiments, we first measure the
average TLS handshake time over 1000 handshake attempts.
At the first phase of our experiments we studied the per- Fig. 3 shows these results for the PQ algorithms in NIST
formance of PQ schemes in NIST’s Level 1. The experimental Level 1 for the two certificate chain lengths considered. During
setup consisted of our local machine operating as a client in N. our experiments, all TLS records were received successfully
Carolina and a remote machine from GCP’s N. Virginia region and no errors were observed due to the excessive size of
acting as the server. Two chain lengths were considered in our the certificates for six out of the seven algorithms exam-
experiments as shown in Fig. 5. The first involves a single ined. More specifically, the TLS client application was not
ICA and the second involves two ICAs since these account able to process the size of a Rainbow certificate yielding
for 77% of the SSL cert chains on the Internet today [86]. an excessive message size error. After examining the
The same PQ signature scheme trace, we observed that the client was issuing an SSL Alert
Root CA was used for all certificates because of the certificate public key size, however, the server
PQ Root
across each chain. Table III sum- transmitted the chain before closing the connection. Since this
Cert. marizes the exact Distinguished happened consistently for all handshake attempts, Fig. 3 shows
Encoding Rules (DER) encoded the duration of this exchange (the * denotes partial handshake
ICAa ICAb1 certificate chain sizes excluding without a ClientFinish message).
PQ Int. PQ Int. the root CA cert which is not
Cert. a Cert. b1
sent to the client, and the TLS Fig. 4 shows the empirical cumulative density function
CertificateVerify exten- of the single ICA measurements. The long tails observed
ICAb2
PQ Int. sion size. are attributed to the network transmission path. It is evi-
Cert. b2 dent from the results that long certificates and slow opera-
Current WebPKI certificates tions significantly reduce the authentication performance in
PQ Server PQ Serverinclude extensions (SCT, OCSP, TLS for algorithms like MQDSS, Picnic, and SPHINCS+ .
Cert. Cert.
Server Server
CRL, SAN) that we did not use Fig. 6(a) shows the cumulative data transfer between the
in our experiments. These fields ServerHello and the Client Change Cipher Spec
Fig. 5: X.509 certificate sometimes lead to conventional message for 200 consecutive handshakes providing a insight
chains in our experiments certificates that exceed 1.5KB. on the network traffic generated when implementing PQ certs
Our tests included less exten- in TLS. Fig. 6(b) shows the average time elapsed during
sions resulting to slightly smaller sizes than common web this period. As expected, algorithms with bigger public keys
certificates. Based on the data we collected, we do not expect and signatures lead to longer time to transfer the certificates
that one or two hundreds of bytes of certificate extensions and TLS CertificateVerify. For small signatures (i.e.,
would effectively change our results unless they push the Dilithium II, Falcon 512) that equates to less than 5ms extra
certificate chain over the TCP congestion window (cwnd). PQ whereas for bigger ones (i.e., MQDSS, Picnic, SPHINCS+ ) it

7
 Certificate Transfer Time (sec)
Cumulative Data Transfer (bytes)
1 0.02
TLS Handshake Time (sec)

One ICA 105

0.04
Two ICAs 1
0.8 0.015
0.99
0.03
0.6 0.98 0.01

CDF
0.02 0.97
0.4 0 0.02 0.04 0.06 0.08 0.1
104 0.005
0.01
0.2
RSA Dilithium IV RSA Dilithium IV
ECDSA Falcon 1024 ECDSA Falcon 1024 A A V 4
0
IV 24 RS DS mI 102
RSA ECDSA Dilithium Falcon 10 0 EC lithiu lcon
0 0.02 0.04 0.06 0.08 0.1 0.12 0 50 100 150 200 Di Fa
Authentication Algorithm TLS Handshake Time (sec) Number of Handshakes Authentication Algorithm

Fig. 7: TLS Handshake Time for Higher PQ Security Level Signature Fig. 8: Impact of PQ Certificate Transfer for Higher PQ Security
Algorithms: (a) Average and Standard Deviation, (b) Empirical CDF Level Signature Algorithms (one ICA): (a) Cumutative Transfer
- One ICA Overhead, (b) Average Transfer Time

becomes ⇠35-55ms extra. in N. Virginia. Fig. 7(a) shows these results for the examined
algorithms for the two certificate chain lengths, while Fig. 7(b)
The client and server were in close proximity in this shows the empirical cumulative density function of the single
experiment. The transfer times of lengthy signatures and chains ICA measurements. We observe that by increasing the security
could exceed 0.5s for longer client-server distances. That is level of the PQ schemes the absolute added latency to the TLS
because algorithms with excessively large certificate sizes and handshake is ⇠10-15ms over RSA3072. Fig. 8(a) shows the
signature messages are penalized by the TCP congestion win- cumulative data transfer between the ServerHello and the
dow. Certificate chains and CertificateVerify payloads Client Change Cipher Spec message for 200 consec-
of high size (i.e. MQDSS, Picnic, and SPHINCS+ from Table utive handshakes. Fig. 8(b) shows the average time elapsed
III) incur round-trip(s) which significantly slows down the during this period which is an additional ⇠5ms for Falcon
handshake due to the TCP congestion window (cwnd). Falcon 1024. The Dilithium IV cert transfer is slower because the data
512 and Dilithium II, on the other hand, do not incur extra size triggers an extra round-trip (⇠11ms). Falcon 1024 does
round-trips and the additional cert transfer time would remain not add an extra round-trip which means that the additional
⇠5-10ms even over longer distances. cert transfer time would remain ⇠5-10ms even over longer
In Fig. 3, algorithms that result to smaller certificate chains distances, but it adds ⇠14ms of signing. An additional 14ms
and CertificateVerify payloads (i.e., Dilithium, Falcon is more noticeable in a localized scenario (N. Carolina to N.
from Table III) show competitive performance against conven- Virginia) with a baseline RSA3072 handshake time of ⇠15ms.
tional signature algorithms. Falcon 512 has smaller signatures
and public key, but we see it performing worse than Dilithium D. Combining PQ Signature Schemes
II because its signing operation at the server takes longer The diverse nature of the cryptographic primitives used for
(⇠6.5ms without AVX2 optimizations). Signing duration of each PQ signature algorithm leads to solutions with different
6.5ms and chain/TLS signature transfer of ⇠3ms are more characteristics and behaviour, namely different signature/key
noticeable in a localized scenario (e.g., a server in N. Virginia) sizes and computational complexity (Tables I, II). Therefore,
with a baseline RSA3072 handshake time of ⇠15ms. Falcon by using different PQ signature schemes in the same certificate
512 would look better for the average Internet web connection chain, we can leverage each algorithm’s specific strengths and
of ⇠110ms [63] as we show in Section V-E. effectively reduce the overall TLS handshake time. To the best
of our knowledge, this is the first time where using different
C. PQ TLS Overhead Analysis - NIST Security Levels 3, 5 algorithms along the certificate chain has been proposed to
Next, we extend our experiments to measure the perfor- improve total handshake performance. To illustrate this intu
mance of algorithms that belong to higher security levels in tion, we examine a proof-of-concept combination of the Falcon
NIST’s level scale. For the majority of the examined PQ 1024, and Dilithium IV algorithms across a chain with one
algorithms, the parameter sets that provide higher quantum- ICA.
resistant authentication result to significantly larger public keys The root CA and ICA certs include a Falcon 1024 public
and signatures [3], [5], [21], [27], [31], [34], [38] and worse key and signature from the root CA, while the server cert
performance. Due to these sizes, the certificate chain and utilizes a Dilithium IV public key and a Falcon 1024 signature
CertificateVerify message were shown in Section V-B from the ICA. By doing so, we reduce the duration of the
to exceed the TCP initcwnd which causes significant slow- server’s signing operation by approximately 30 times. In
downs because of the extra round-trip(s). addition, the overall certificate chain size is equal to 6.52
KB, which is smaller than both the pure Falcon 1024 chain
Thus, for Levels 3 and 5, we limit our attention to the
(6.56 KB), and the pure Dilithium IV chain (10.70 KB)
two PQ algorithms that presented the most promising results
(Table III). Table IV shows a comparison yielded following the
at Level 1, namely Dilithium II and Falcon 512. We examine
experimental procedures of Section V-C (client-server round-
Dilithium IV at NIST security Level 3, and Falcon 1024 at
trip ⇠11ms). The combined case reduces (on average) the
Level 5. Currently, Dilithium does not offer a parameter set
overall handshake by 25.16% compared to the single Dilithium
at Level 5, while the Falcon’s Level 3 parameter set (Falcon
IV, and by 33.27% when put against the single Falcon 1024
768) was removed in Round 2 [34].
case. The combination can work with other signature schemes
Again, we measure the average TLS handshake time over too as discussed in Section VII-B. It is especially suitable
1000 handshakes for our client in N. Carolina and a server for the cases where (a) the client and server are in close

8
 (b)
N. Virginia Oregon 1

TLS Handshake
Server Server RSA ECDSA

Time (sec)
Dilithium II Falcon 512
+ 11 ms + 69 ms
MQDSS Picnic
19 hops 22 hops
Zurich 0.5 SPHINCS
Server
+ 103 ms
24 hops
Client 0
N. Virginia Oregon Zurich Sao Paulo Sydney Singapore
+ 139 ms
+ 225 ms 25 hops (c)

TLS Handshake
28 hops
Sao Paulo 0.4 RSA ECDSA

Time (sec)
Singapore Server Dilithium IV Falcon 1024
Fal. 1024 & Dil. IV
Server + 205 ms
26 hops 0.2

Sydney
Server 0
N. Virginia Oregon Zurich Sao Paulo Sydney Singapore
(a) Round-trip time and hops be- Server Location
tween client and servers
(b) NIST Level 1, (c) NIST Levels 3, 5
Fig. 9: TLS Handshake Time at a global scale

proximity, (b) sufficiently fast implementations of the signature introduced to the TLS handshake by the PQ signature algo-
scheme are not available at the signer or verifier, or (c) when rithms when compared against RSA3072 at the 50th and 95th
leaf certificates are renewed relatively often and lower secu- percentile. The observed latency overhead of Level 1 Dilithium
rity levels with better performing parameters are acceptable. II was notably small making it the most pertinent for integra-
The drawback is that tion into TLS. Level 1 Falcon 512 also showed small overhead
TLS Handshake (ms) the verifier is required over RSA3072. Regarding higher security levels, Dilithium
Signature to support both IV nearly doubled the handshake duration because the data
Scheme Mean St. Dev.
signature algorithms. size triggered an extra round-trip, while Falcon performed
RSA 3072 15.13 6.03 The performance better. Although the % increase seems high for Dilithium IV,
Dilithium IV 24.20 2.62 improvement of the absolute time increase is ⇠145ms at the 50th percentile
Falcon 1024 27.14 3.30 this method will be and ⇠229ms at the 95th percentile. The Dilithium+Falcon
Fal. 1024 & Dil. IV 18.11 1.58 marginal for long combination (Section V-D) reduced the handshake latency
client-server distances. by ⇠8-50% in comparison to single PQ algorithm certificate
TABLE IV: Average Handshake The client and server chains.
Time and of the Dilithium and Section V-E and V-F
Falcon combination (single ICA). Our experiments so far have measured the time between
further examine this
the ClientHello message and the end of the handshake.
proposed alternative of combining schemes.
Extending the period under examination to include the TCP
handshake (start at the TCP SYN message) increases the mean
E. Global Scale Performance Analysis TLS handshake with X25519 key exchange and RSA3072
Next, we extend our evaluation by adding more remote certificates duration to 269.5ms. The respective median of the
servers across different continents. Fig. 9(a) shows the exper- handshake measured in Firefox client-server connections is
imental setup, the average number of hops, and the measured 111ms considering 9.91 billion data points from their nightly
average round-trip latency between our local client and each versions 69, 70, and 71 (excluding 32-bit platforms which
remote server. Again, the handshake duration was averaged could deteriorate the performance unrealistically) [63]. By
over 1000 handshakes for each PQ signature algorithm and
parameter set. A certificate chain with a single ICA was
Handshake (ms) Latency (%)
utilized. Fig. 9(b) shows the mean and standard deviation of Signature
the TLS handshake time when algorithms with NIST’s security Algorithm 50 th
95 th
50th 95th
Level 1 were tested, while Fig. 9(c) does the same for Level RSA3072 131.54 227.26 0 0
3 and 5. The performance was similar across all regions, Dilithium II 140.20 232.51 6.58 2.31
and no handshake failures were reported due to middlebox Falcon 512 142.22 235.46 8.12 3.49
misbehaviour. Consistently, Dilithium and Falcon show the MQDSS 48 598.61 726.20 355.05 219.53
most promise for replacing RSA3072 with minimum additional Picnic L1F S 634.90 985.88 382.63 333.79
delay. SPHINCS+ SHA256-128f-simple 553.15 904.98 320.49 298.19

Using the same setup, in a second experiment, our client Dilithium IV 276.55 449.88 110.22 97.95

uniformly performed 3000 handshakes with the remote servers Falcon 1024 152.96 240.74 16.28 5.93
in the course of a whole day for each examined algorithm. Fal. 1024 & Dil. IV 140.74 228.42 6.98 0.50
The goal was to account for Internet’s unpredictability by
measuring TLS handshake latency in diverse topologies across TABLE V: TLS Handshake Times and Additional Latency
longer time-frames. Table V summarizes the additional latency over RSA at the 50th and 95th Percentile.

9
 Transactions/sec
RSA Dilithium II Falcon 512 Picnic Ts/sec Fail Rate
10 availability by measuring the number of TLS handshake fail-
(a)
ures during the load testing period. Fig. 11 shows these results

TLS Handshake
800

Failure (%)
600
5
for promising signature algorithms of NIST’s Level 1 as the
400
total number of clients increases from 20 to 1000. Both cases
200
of one and two ICAs are examined. Evidently, while still at
low load RSA3072 outperforms Dilithium II and Falcon 512
0 0
0 100 200 300 400 500 600 700 800 900 1000

10
as expected from Fig. 3. However, as the server’s saturation
point is reached we observe that Dilithium II allows the server
Transactions/sec

(b)
800

TLS Handshke
Failure (%)
600
5
to handle ⇠25% more connections per second compared to
400
RSA3072 because Dilithium signing is over five times faster
200
compared to RSA3072 at the server (Table II). Regarding
0
0 100 200 300 400 500 600 700 800 900
0
1000 Falcon 512, the server’s saturation point is reached early at
Number of Clients a much lower transaction rate than RSA and Dilithium due to
Fig. 11: Server Performance for NIST’s PQ Security Level 1 slow signing at the server (Table II).
Signature Algorithms: (a) One ICA, (b) Two ICAs
Fig. 3 showed that the Dilithium bigger cert chain and
CertificateVerify size do not affect the server perfor-
extrapolation we estimate that the average PQ TLS handshake mance other than slightly slowing down the handshake. To
measured from the TCP SYN to Ready for HTTP will further reinforce that point, the server’s transaction rate is
amount on average to ⇠122-135ms for the best algorithm cases similar for both cert chain lengths (Fig. 11(a) vs. Fig. 11(b))
of each security level, namely Dilithium II (level 1), and Falcon for the same algorithms which shows that just a few additional
1024 (level 5). KB of certificate payload do not affect the server performance.

The PQ signing and verification times were relatively To further demonstrate the higher impact of signing
small (<15ms) in our analysis so far for all tested algorithms over cert chain size on a busy server we tested Picnic
except SPHINCS+ . Such times have relatively little impact on L1FS which presents much bigger certificate chain and
an average handshake time of ⇠110ms [63] or 269.5ms in CertificateVerify message (Table III) while maintain-
our tests. Arguably using optimized or hardware accelerated ing a better than Falcon 512 sign operation (Table II). Fig. 11
versions of these algorithms would speed up the handshake, shows that the server signing with Picnic L1FS outperforms
but not by a lot because the certificate and signature data will signing with Falcon 512 by ⇥1.7 higher transaction rate. That
still incur a slowdown. can be attributed to the sign operation performance. Moreover,
and in accordance with our earlier experiments, Picnic still
leads to a much lower maximum transaction rate compared
F. Server-Side Performance of TLS PQ Authentication to RSA3072 and Dilithium II due to its excessively big cert
Up to this point, we focused on measuring the im- chains that slow down the handshake significantly (Fig. 3).
pact of PQ authentication in TLS from the client’s Another interesting observation is that before reaching the
perspective. Next, we measure the performance of a server saturation point, Picnic’s transaction rate is half than
server that utilizes PQ certificates in TLS 1.3 to ser- RSA3072. The reason is that up to this point the server has not
vice simultaneous secure connections with multiple clients. gotten overloaded and the transaction slowdown solely comes
To do so, an Nginx web server from the TLS performance slowdown seen in Fig. 3.
[66] was set up on one of the
N. Virginia Oregon The same behaviour is observed for parameter sets that
Clients Clients aforementioned cloud instances, and
yield higher NIST security levels as seen in Fig. 12. The
+ 11 ms was configured to utilize the OQS
4 hops + 69 ms server’s saturation point is reached significantly early with
7 hops OpenSSL library. Thus, our server
Falcon 1024 that also presents the highest handshake failure
S. Carolina was able to sign handshakes and
rate among alternatives because of its signing performance.
Server send PQ certificates using the signa-
+ 65 ms + 33 ms
4 hops
ture schemes under study. We used
10 hops
the Siege tool [35], configured with RSA Dilithium IV Falcon 1024 Fal. 1024 & Dil. IV Ts/sec Fail Rate
California Iowa the OQS OpenSSL library, to sim-
Transactions/sec

800 (a)
TLS Handshake

15
Clients Clients
ulate multiple clients and set up
Failure (%)

600
10
Fig. 10: Round-trip PQ authenticated TLS connections. 400

latency and number of Siege clients were running in four 200

5

hops between clients remote cloud instances. For this ex- 0 0

0 100 200 300 400 500 600 700 800 900 1000
and server periment, all client instances were
placed in relatively close proximity
Transactions/sec

(b) 15
800
TLS Handshke

to each other. The exact topology details are shown in Fig. 10.
Failure (%)

600
10

The clients that were uniformly distributed among the four 400
5
locations, attempted simultaneous TLS connections with the 200

server for 60 seconds. During that time we measured the

0 0
0 100 200 300 400 500 600 700 800 900 1000

request rate that the server was able to handle, i.e. the number Number of Clients

of successful transactions per second. The requested web page Fig. 12: Server Performance for NIST’s Higher PQ Security Level
was only 0.6KB. In addition, we captured the server’s overall Signature Algorithms: (a) One ICA, (b) Two ICAs

10
Also, under high load, the use of Dilithium IV results to evaluation, Google and Cloudflare resumed the experiments
higher transaction rates in comparison to RSA3072 and Falcon on KEM integration into TLS. The new study [56] defined
1024 because of its better signing performance (Table II). three KEM families based on the primitive and the key size
Moreover, we examined the proposed PQ combined certificate and tested them with dummy extensions of proportional sizes
chain scenario (Dilithium and Falcon in Section V-D). Results in the TLS ClientHello message. The goal was to emulate
demonstrate that by using PQ combined certificate chains the the overhead that these key sizes introduce and explore issues
server is able to handle more TLS connection requests than all or failures related to such larger handshakes. Recently, Google
alternatives including the conventional RSA3072. More specif- and Cloudflare have concluded work on a new experiments
ically, the Dilithium+Falcon combination slightly increases the (CECPQ2 [55]) that integrated a hybrid KEM based on the
average post-saturation transaction rate by ⇠10% and ⇠4% HRSS NIST PQ candidate and X25519 into TLS 1.3 for
compared to RSA3072 and pure Dilithium IV respectively. further performance measurements on Chrome. Additionally,
The improvement over RSA3072 is because Dilithium IV they tested the SIKE NIST PQ candidate with X25519 in TLS
operations are still faster. However, the difference is smaller 1.3 as part of their CECPQ2b experiment [52]. The results of
than in Fig. 11 because the Dilithium IV chain slows down the this work are included in [53]. Moreover, Campagna recently
handshake more (Fig. 7). The server’s performance increase discussed hybrid key exchange and double tunnelling for TLS
with Dilithium+Falcon over Falcon 1024 is due to the com- and presented the slowdown of TLS 1.2 introduced by using
putationally lighter signing of Dilithium IV (Table II). hybrid key exchange with ECDHE and SIKE or BIKE NIST
PQ candidates [18]. Campagna et al. showed that SIKE’s
In summary, we saw that signing at the server is an impact on the handshake byte count is small, whereas BIKE’s
important factor for the server’s performance. Cert chain and impact is significant.
signature size affect the server transaction rate by increasing
the TLS handshake time but the effect is smaller than the More closely related to our work that focuses on signature
impact of a heavy signing operation. We expect that the server performance, Cronin et al. [26] modeled and benchmarked the
performance penalty introduced by expensive sign operations performance of forward-secure signatures against conventional
will be alleviated by optimized implementations and hardware RSA, DSA and ECDSA. Focusing more on PQ signatures,
acceleration in the future. Kampanakis et al. discussed the impact of PQ signature
schemes on protocols that utilize X.509 certificates in [46].
Finally, it is worth noting that in our tests we observed They considered the case of hybrid certificates that include
that the client closest to the server saw a higher transaction hash-based signatures, and conducted performance experi-
rate due to the lower propagation delay. Thus, servers closely ments on TLS 1.2 and IKEv2. Also, their work discussed the
located to clients will see better performance than the average use of PQ certificates by emulating their increased size through
we are showing. enlarging certificate chains with additional certificates. On the
same topic, earlier work by Bindel et al. emulated large hybrid
VI. R ELATED W ORK PQ certificates and studied their impact on TLS libraries and
browsers [13]. The authors in [17] prototyped stateful XMSS
A large body of research on QC and PQ cryptography [11], signatures in TLS and S/MIME and pointed out the challenges
[44], [50], [95] led to the NIST initiative to come up with novel with state management in live TLS connections. In [25], the
PQ algorithms for use in a post-quantum world. Recently, authors discuss the challenges of implementing NIST’s PQ
more studies are exploring NIST’s Round 2 PQ candidate key exchange and authentication algorithms in TLS, with
schemes focusing mainly on their computational performance, their focus being mainly on hybrid (combination of PQ and
while some even evaluate their energy consumption [82]. The conventional) schemes. In addition, they perform proof-of-
authors in [6], [89] conduct a hardware implementation and concept experiments on PQ TLS that involve single certificate
comparison of NIST Round 2 signature and KEM candidate exchanges, without accounting for real-world network condi-
algorithms. Their focus is on comparing specific algorithm tions. Finally, Paquin et al. [69] perform experiments with
operations and their impact on hardware design parameters, some hybrid PQ KEM and Signature schemes in TLS and
without taking into account performance as a part of a system show that lossy conditions have more negative effects on data-
or a protocol. In [48], Kannwischer et al. benchmark Round 2 heavy TLS handshakes. The loss rates they use are not very
algorithms on ARM Cortex-M4 and evaluate the more suitable common on the Internet today, but they could be experienced
ones for embedded devices without studying additional impact in constrained or congested environments, or remote cellular
on specific use-cases. Similarly, the authors in [85] enhance the networks.
DTLS protocol with the PQ key exchange algorithm NTRU
which showcases its feasibility for securing current systems.
VII. D ISCUSSION ABOUT ENCRYPTED TUNNELS
In addition, research teams from the industry are in- A. Implications of our Findings
vestigating the performance of PQ candidate algorithms in
current Internet protocols. They have mostly been focusing PQ signatures will have an impact on authenticated tun-
on studying key exchange. In [15], Bos et al. proposed the nels like (D)TLS and IKEv2/IPSec. These protocols provide
use RLWE for key exchange in TLS 1.2 and study its impact fragmentation mechanisms to allow for lengthy signatures,
that slows down performance by a factor of ⇥1.4 for the but as we showed above, larger chains and potentially slower
client and ⇥1.2 for the server. A year later, in 2016, Google algorithms will impact the tunnel establishment. Applications
tested a hybrid KEM (NewHope combined with elliptic- with lower connection rates and tunnels that stay up longer will
curve schemes) and ECDSA certs with TLS 1.2 in Google be less impacted than applications that establish short and fast
Chrome [54]. Following the conclusion of NIST’s Round 1 connections. Thus, per connection overhead is important to be

11
considered for the migration to PQ authentication algorithms. for such applications.
In addition, the more signatures used in a handshake, the
more impact PQ algorithms would have on the protocol. Dilithium II and Falcon 512’s claimed classical security
For example, SCTs in leaf certificates will add two or three levels are ⇠100 and ⇠114 bits respectively. Although these
signatures to the handshake and OCSP stapling would add one asymptotic bounds are probably higher in practice because
more. of memory costs, these parameter sets could be considered
unacceptable to use. The variants that offer more than 128
IPSec VPN tunnels usually stay on for long periods of time. bits of classical security are Dilithium III, IV and Falcon
Experimental RFC4478 [67] defines how a tunnel can be re- 1024. We already discussed that Falcon 1024 had acceptable
authenticated, but in practice, IKE_AUTH messages are not handshake time with the shortcomings of higher energy cost
exchanged unless the tunnel is torn down and re-established. and slow signing when floating point hardware is not present
Thus, for the majority of IKEv2/IPSec VPN applications, a and that Dilithium IV was introducing extra round-trips. A
connection establishment that would take up to a few seconds rerun of our experiments in Section V-E showed that the use
would not have material impact on the tunnel. Similarly, of Dilithium III certificates added 2.7KB to the handshake
WebVPN applications establish a TLS control connection and (compared to Dilithium II) with fast signing/verification times.
subsequently data DTLS connections over a duration of a That led Dililthium III to perform ⇠7% slower than RSA3072
few seconds. Onward, these tunnels stay up for long sessions which is very similar to Dilithium II. Based on the above, if
that usually last hours or more. A connection establishment Dilithium II and Falcon 512 security levels are considered low,
slowdown of up to a few seconds will not impact these Dilithium III and Falcon 1024 are the best options. Falcon 1024
connections. The caveat with such use-cases is the impact would be preferable when floating point hardware is available
on the head-end that terminates these connections. It is not at the signer and energy is not a concern.
unusual for a VPN concentrator to terminate thousands of
connections. A possible attempt of VPN clients to establish None of our tests included OCSP revocation checks and
a connection to the hub simultaneously could lead to overload SCTs returned from the server. OCSP and SCTs could account
if the signing or verification operations are heavy. Most PQ for three or more PQ signatures which could push the server
signature candidates we studied (except SPHINCS+ ) are not TLS data over the TCP initcwd and introduce round-
likely to be a significant performance concern for VPN tunnels trips for all Dilithium variants. Falcon data could fit in the
that take advantage of algorithm optimizations. initcwd at the cost of one slower OCSP signature assuming
floating point hardware is not available. Falcon would be
Web connections, on the other hand, are usually short-lived. preferable specifically for SCTs because of its relatively fast
At the time of this writing, clients perform 70 requests per page verification and small signature. Readers should note that on
to fetch 1-2KB resources per request on average [42]. If some the web, OCSP responders are often not used by browsers [58]
of these requests are performed over new TLS connections, a and staples can be cached and are not supported by most
heavier PQ authenticated TLS handshake would have a signifi- servers. Thus, we consider OCSP signatures a less pressing
cant impact on HTTP performance overall. A few KB of extra challenge for the PQ WebPKI case.
authentication data per TLS connection has low amortization
over 1-2KB of actual web content. On the other hand, the ever-
increasing adoption of HTTP/2 [7] (55% at the time of this B. Minor Adjustments to Enable PQ Signatures
writing [42]) will improve amortization as HTTP/2 multiplexes
data over a single HTTP connection. It is hard to say with There are straightforward changes needed for
certainty how much delay is excessive on the web. Reports TLS and IKEv2 to support PQ signatures. The TLS
like [1] lead us to believe that hundreds of extra milliseconds extensions signature_algorithms_cert and
per handshake are not acceptable. More importantly, extra signature_algorithms will need to use new identifiers
round-trips mean almost doubling total handshake times which to convey to the TLS peer the PQ signature algorithms
we consider excessive for time-sensitive web applications. supported [25, § 4.1.1]. Also, X.509 will need to use new
Public Key and Signature identifiers to convey the new
In summary of our testing, we found that Dilithium II, algorithms [25, § 4.1.1], [46, § 1]. In the context of IKEv2,
Falcon 512 and 1024 could be deployed in X.509 certs with- signature negotiation was not included in the protocol from
out detrimentally impacting time-sensitive TLS applications. the beginning. Recently RFC7427 [49] added support for
Dilithium is preferable over Falcon, at least when floating signature algorithm negotiation. RFC7427 is rarely supported
point hardware is not available, because its superior sign- by VPN vendors, so in a PQ authenticated IKEv2 case,
ing performance allows for higher connection rates. Floating vendors will have to support all possible algorithms and not
point hardware improves Falcon’s signing performance by negotiate with the peer, or support RFC7427 and negotiate
⇠20 times [72] which would make Falcon a better candidate the PQ signature algorithms with new identifiers.
than Dilithium in terms of performance in live protocols.
Falcon would also have more impact on energy constrained Crockett et. al. describe in [25, § 4.3.1] the challenges
devices [83]. Dilithium IV nearly doubled the handshake introduced by some algorithms with signatures and cert chains
duration over RSA3072 because the data size triggers an extra that exceed 214 B and 102.4 KB respectively. Out of the tests
round-trip. Dilithium IV could still be used if applications were we performed, only Rainbow fell in these categories and led
amenable to approximately double the TLS handshake time or to failed handshakes. We do not expect schemes with such
if some of the mechanisms in Section VII-B (e.g., [92], [79, big signatures to be used in TLS, thus TLS and protocol
§ 5.1.3]) were widely deployed. We also showed that other implementations will likely not require further updates to
NIST PQ signature candidates would not be good candidates accommodate them.

12
 On the other hand, the work in [92] proposes a new TLS signature algorithms. Given that the industry is constantly
extension to inform the server that the client does not need striving for faster handshakes and better performance [36],
the ICA certificates in the certificate chain. The authors in [79, [92], [79] it is unlikely that the impact of the PQ signature al-
§ 5.1.3] propose a similar method of omitting certificates from gorithms (excluding Dilithium and Falcon) will be acceptable.
a handshake by using a pre-established certificate dictionary. In that case, more drastic protocol changes may be necessary.
Such mechanisms could save significant amounts of data from
handshakes that approach or exceed the TCP initcwnd to One thought proposed in public fora is to follow the
prevent extra round-trips similar to Dilithium IV. They would Certificate Transparency paradigm [57] and offer a public
also alleviate the impact of lossy networks which was shown repository or service that could be used out-of-band to retrieve
to be high for data-heavy TLS handshakes [69] . The way certificates of the entities that a client would communicate
to implement the mechanism would be either for the client with. To offer the same functionality, the work in [80] uses
to keep track of ICAs, or maintain an ICA cache and a special DNS Resource Records to serve public keys used
server leaf certificate Common Name (CN) cache. Both caches for SNI encryption. Such proposals would eliminate the need
would get updated from a new TLS handshake when there for certificates to be transferred in the handshake, but it
is a cache miss. The server certificate itself would not be comes with a requirement of retrieving and verifying the
cached. It would still be returned from the server with every peer certificate out-of-band. Unless caching is enforced, this
handshake. The CN cache could save additional data overhead certificate retrieval step would need to be repeated before any
from OCSP and SCT PQ signatures as well. Valid OCSP TLS connection establishment which of course is unlikely to
staples and SCTs could be bound to a CN cache entry. As improve the overall performance. If caching the peer certificate
long as the client had an entry in the cache for the server is possible, it comes with concerns [84, § 7], [46, § 2.1]. We
leaf certificate, he would not need to receive the ICAs, OCSP consider such a mechanism a drastic paradigm shift from the
or SCT information. One challenge would then be that, at the way the Internet works today.
time of this writing, SCTs are most usually included in the leaf On the other hand, the authors in [14, Figure 2], [10, § 2.3],
certificate. To omit the SCT data, the certificate could no longer [81] propose an alternative authentication method that uses
include the SCT information which would now be carried in a PQ Key Encapsulation which could be more efficient than
signed_certificate_timestamp extension [57]. That PQ signatures. The mechanism would require for the server
is a drastic paradigm change but it would allow the client to leaf certificate to include a PQ KEM public key which can
limit the TLS handshake data if the information is in its cache. be relatively short. But it would also require new protocol
message extensions and an extra round-trip in TLS because
Section V-D showed how combining algorithms could
the signature ciphertext can only be generated after the client
improve the performance of the handshake. Using the same
has retrieved the server’s public key. It would work similarly
rationale at the root CA, the algorithm combination with
for IKEv2. Changing the TLS or IKEv2 state machine in
relatively small signature size could shrink the ICA certificate
order to prevent the extra round-trip would not be trivial as
size and speed up the handshake. Such schemes could be
it would affect current security analyses of the protocols [23],
multivariate candidates like LUOV, GeMSS, and Rainbow or,
[24]. Moreover, the public key would still need to tied to the
qTesla, or Stateful HBS [43], [60] with small tree heights
peer identity using PKI so the PQ cert chain containing PQ
used at the root CA. Certificate chains that include different
signatures would still need to be transferred which minimizes
signature algorithms impose an extra requirement for a client
the improvement this mechanism offers.
to support multiple PQ signature algorithms.
VIII. C ONCLUSION AND F UTURE W ORK
C. Drastic changes to Enable PQ Authentication
In this work, we integrated PQ signature algorithms into
CPU-intensive algorithms like Falcon signing could add TLS 1.3 and evaluated the TLS handshake latency observed
significant load to busy servers. Such cases could benefit by a client along with the throughput of a PQ authenticated
from batch signing options proposed in [8]. Batch signing server by considering realistic network conditions. We proved
would allow the server to sign batches of handshakes with one that the signature and certificate chain size impact the total
signature, but it would require broad client upgrades and could handshake time, especially with relatively fast signing and
introduce delays while the server is buffering handshakes to verification primitives. Our results show that the PQ algo-
batch-sign them. rithms with the best performance for time-sensitive applica-
Some lattice signature schemes like Falcon could offer tions are Dilithium and Falcon. When floating point hardware
message recovery as part of the signature [71]. If the message is available Falcon seems more suitable for the web. Other
is longer than a plain Falcon signature then message recovery applications (e.g. VPN, SSH) that do not require frequent
shortens the message+sig size. This mechanism could be connection establishments could use one of the other PQ
integrated in X.509 to make the PQ certificate smaller. An signature algorithms as well, assuming they do not overload
X.509 certificate could then only include the algorithm iden- a server terminating multiple client connections. We showed
tifier and the signature. The message (tbsCertificate) that although slightly slower signing does not significantly
would be recovered from the signature itself. Such methods affect the overall duration of a single handshake, it could
would require significant changes in X.509 standards and their significantly impact the total throughput of a server. However,
implementations in tunnelling protocols. as optimizations and hardware acceleration improve signing
performance, we expect signature and key size to have the
If Dilithium and Falcon are not standardized, tunnelling most impact on the handshake. We also proposed combining
protocols will be significantly affected by the rest of the PQ different PQ signature algorithms in a certificate chain and

13
we confirmed that it can improve the overall handshake speed [7] M. Belshe, R. Peon, and M. Thomson, “Hypertext Transfer Protocol
and throughput. Finally, we offered improvements to avoid Version 2 (HTTP/2),” RFC 7540, May 2015. [Online]. Available:
round-trips by leveraging ICA suppression and discussed other https://rfc-editor.org/rfc/rfc7540.txt
challenges and alternatives in real-world protocols. [8] D. Benjamin, “Batch Signing for TLS,” Internet Engineering Task
Force, Internet-Draft draft-davidben-tls-batch-signing-02, Nov. 2019,
As future work, we plan to evaluate the performance of work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/
html/draft-davidben-tls-batch-signing-02
PQ authenticated VPNs and UDP-based tunnels like QUIC
and DTLS [47]. We also want to test hybrid PQ KEMs as [9] D. J. Bernstein, “eBACS: ECRYPT Benchmarking of Cryptographic
Systems,” https://bench.cr.yp.to/primitives-sign.html, 2019, Web page.
tested by others [52], [53] along with PQ signatures to evaluate Accessed 2019-02-08.
the ”total” latency introduced to TLS by PQ algorithms. By [10] ——, “Visualizing size-security tradeoffs for lattice-based encryption,”
combining results in [53] with our work, we could extrapolate Cryptology ePrint Archive, Report 2019/655, 2019, https://eprint.iacr.
that the total slowdown of a PQ handshake would be around org/2019/655.
10-25%. Also, given that PQ key exchange could add 1- [11] D. J. Bernstein, D. Hopwood, A. Hülsing, T. Lange, R. Niederhagen,
2KB each direction, this could push the server data to the L. Papachristodoulou, M. Schneider, P. Schwabe, and Z. Wilcox-
initcwnd limit and introduce round-trips. Further experi- O’Hearn, “SPHINCS: Practical Stateless Hash-Based Signatures,” in
Advances in Cryptology - EUROCRYPT 2015 - 34th Annual Interna-
ments would be necessary to quantify the total PQ algorithm tional Conference on the Theory and Applications of Cryptographic
impact on TLS under realistic conditions that include lossy Techniques, 2015, pp. 368–397.
networks. It would also be of interest to study the impact of [12] W. Beullens, A. Szepieniec, F. Vercauteren, and B. Preneel, “Luov:
PQ SCT and OCSP signatures while using the caching and Signature scheme proposal for NIST PQC project (Round 2 version),”
ICA suppression mechanisms already discussed. Finally, it is https://github.com/WardBeullens/LUOV/blob/master/Supporting
worth investigating hybrid certificates’ [68] performance and Documentation/luov.pdf, 2018.
the improvement that Stateful HBS, qTesla, GeMSS, LUOV [13] N. Bindel, U. Herath, M. McKague, and D. Stebila, “Transitioning to a
quantum-resistant public key infrastructure,” in Proc. 8th International
or Rainbow would achieve when used only at the root CA. As Conference on Post-Quantum Cryptography (PQCrypto) 2017, ser.
a longer-term goal, in an attempt to reduce the certificate size, LNCS, T. Lange and T. Takagi, Eds. Springer, June 2017, to appear.
we would also like to study the message recovery capabilities [14] J. Bos, L. Ducas, E. Kiltz, T. Lepoint, V. Lyubashevsky, J. M.
offered by schemes like Falcon. Schanck, P. Schwabe, G. Seiler, and D. Stehl, “Crystals – kyber: a cca-
secure module-lattice-based kem,” Cryptology ePrint Archive, Report
2017/634, 2017, https://eprint.iacr.org/2017/634.
ACKNOWLEDGMENTS
[15] J. W. Bos, C. Costello, M. Naehrig, and D. Stebila, “Post-quantum
Many thanks to Richard Barnes from Cisco for his valuable key exchange for the TLS protocol from the ring learning with errors
feedback. We would also like to acknowledge Luke Valenta problem,” in 2015 IEEE Symposium on Security and Privacy. IEEE,
2015, pp. 553–570.
from Cloudflare for his comments and experimental results
[16] J. A. Buchmann, D. Butin, F. Göpfert, and A. Petzoldt, “Post-quantum
regarding longer TLS records in TLS handshakes. Hanno Böck cryptography: state of the art,” in The New Codebreakers. Springer,
also was kind to challenge us insightfully on the concerns 2016, pp. 88–108.
PQ signatures bring to PKI infrastructures. We are grateful to [17] D. Butin, J. Walde, and J. A. Buchmann, “Post-quantum authentication
Thom Wiggers and Prof. Lejla Batina from Radboud Univer- in OpenSSL with hash-based signatures,” in Tenth International
sity for their thoughtful feedback which improved this paper. Conference on Mobile Computing and Ubiquitous Network, ICMU
Finally, thank you to Douglas Stebila, Christian Paquin and 2017, Toyama, Japan, October 3-5, 2017. IEEE, 2017, pp. 1–6.
[Online]. Available: https://doi.org/10.23919/ICMU.2017.8330093
the whole OQS OpenSSL [74] team for providing a library
that made our testing possible. [18] M. Campagna, “Hybrid-key Exchanges as an Interim-to-
Permanent solution to cryptographic agility,” Jun. 2019.
[Online]. Available: https://docbox.etsi.org/Workshop/2019/
R EFERENCES 201906 ETSISECURITYWEEK/202106 DynamicNatureOfTechno/
SESSION03 CHANGINGCRYPTOGRAPHY/AWS CAMPAGNA.pdf
[1] Akamai, “Akamai online retail performance report: Milliseconds are
[19] A. Casanova, J.-C. Faugere, G. Macario-Rat, J. Patarin, L. Perret, and
critical,” https://www.akamai.com/uk/en/about/news/press/2017-press/
J. Ryckeghem, “GeMSS: A Great Multivariate Short Signature,” Ph.D.
akamai-releases-spring-2017-state-of-online-retail-performance-report.
dissertation, PhD thesis, UPMC-Paris 6 Sorbonne Universités, 2017.
jsp, 2017.
[2] G. Alagic, G. Alagic, J. Alperin-Sheriff, D. Apon, D. Cooper, Q. Dang, [20] C.-l. Chan, R. Fontugne, K. Cho, and S. Goto, “Monitoring TLS
Y.-K. Liu, C. Miller et al., Status report on the first round of the NIST adoption using backbone and edge traffic,” in IEEE INFOCOM 2018-
post-quantum cryptography standardization process. US Department IEEE Conference on Computer Communications Workshops (INFO-
of Commerce, National Institute of Standards and Technology, 2019. COM WKSHPS). IEEE, 2018, pp. 208–213.
[3] E. Alkim, P. S. L. M. Barreto, N. Bindel, P. Longa, and J. E. Ricardini, [21] M.-S. Chen, A. Hülsing, J. Rijneveld, S. Samardjiska, and P. Schwabe,
“the lattice-based digital signature scheme qtesla.” “MQDSS specifications,” http://mqdss.org/specification.html.
[4] ANSI, “ANSI X9.62, Public Key Cryptography For The Financial [22] D. Cooper, S. Santesson, S. Farrell, S. Boeyen, R. Housley, and W. Polk,
Services Industry: The Elliptic Curve Digital Signature Algorithm “RFC 5280: Internet X. 509 public key infrastructure certificate and
(ECDSA),” September 2005, american National Standards Institute, X9- certificate revocation list (CRL) profile,” IETF, May, 2008.
Financial Services. [23] C. Cremers, M. Horvat, J. Hoyland, S. Scott, and T. van der Merwe,
[5] J.-P. Aumasson, D. J. Bernstein, C. Dobraunig, M. Eichlseder, “A Comprehensive Symbolic Analysis of TLS 1.3,” in Proceedings of
S. Fluhrer, S.-L. Gazdag, A. Hülsing, P. Kampanakis, S. Kölbl, the 2017 ACM SIGSAC Conference on Computer and Communications
T. Lange et al., “SPHINCS+ - Submission to the 2nd round Security, ser. CCS ’17. New York, NY, USA: ACM, 2017, pp. 1773–
of the NIST post-quantum project,” https://sphincs.org/data/sphincs+ 1788. [Online]. Available: http://doi.acm.org/10.1145/3133956.3134063
-round2-specification.pdf, 2019, Specification document (part of the [24] C. J. F. Cremers, M. Horvat, S. Scott, and T. van der Merwe, “Au-
submission package). tomated Analysis and Verification of TLS 1.3: 0-RTT, Resumption
[6] K. Basu, D. Soni, M. Nabeel, and R. Karri, “NIST post-quantum and Delayed Authentication,” 2016 IEEE Symposium on Security and
cryptography - a hardware evaluation study,” Cryptology ePrint Archive, Privacy (SP), pp. 470–485, 2016.
Report 2019/047, 2019, https://eprint.iacr.org/2019/047. [25] E. Crockett, C. Paquin, and D. Stebila, “Prototyping post-quantum and

14
 hybrid key exchange and authentication in tls and ssh,” Cryptology attribute certificate frameworks,” https://www.itu.int/rec/T-REC-X.509/
ePrint Archive, Report 2019/858, 2019, https://eprint.iacr.org/2019/858. en.
[26] E. Cronin, S. Jamin, T. Malkin, and P. McDaniel, “On the performance, [46] P. Kampanakis, P. Panburana, E. Daw, and D. Van Geest, “The Viability
feasibility, and use of forward-secure signatures,” in Proceedings of of Post-quantum X.509 Certificates.” IACR Cryptology ePrint Archive,
the 10th ACM Conference on Computer and Communications Security, vol. 2018, p. 63, 2018.
ser. CCS ’03. New York, NY, USA: ACM, 2003, pp. 131–144. [47] P. Kampanakis and D. Sikeridis, “Two post-quantum signature use-
[Online]. Available: http://doi.acm.org/10.1145/948109.948130 cases: Non-issues, challenges and potential solutions,” Cryptology
[27] J. Ding, M.-S. Chen, A. Petzoldt, D. Schmidt, and Y. Bo-Yin, “Rainbow ePrint Archive, Report 2019/1276, 2019. https://eprint. iacr. org , Tech.
- Algorithm Specification and Documentation,” https://csrc.nist.gov/ Rep., 2019.
projects/post-quantum-cryptography/round-2-submissions, 2019, The [48] M. J. Kannwischer, J. Rijneveld, P. Schwabe, and K. Stoffelen, “pqm4:
2nd Round Proposal. Testing and Benchmarking NIST PQC on ARM Cortex-M4,” Cryptol-
[28] J. Ding, Z. Zhang, J. Deaton, K. Schmidt, and F. Vishakha, “New ogy ePrint Archive, Report 2019/844, 2019, https://eprint.iacr.org/2019/
Attacks on Lifted Unbalanced oil vinega,” https://csrc.nist.gov/CSRC/ 844.
media/Events/Second-PQC-Standardization-Conference/documents/
[49] T. Kivinen and J. Snyder, “Signature Authentication in the Internet
accepted-papers/ding-new-attacks-luov.pdf, 2019.
Key Exchange Version 2 (IKEv2),” RFC 7427, Jan. 2015. [Online].
[29] M. Driscoll, “The Illustrated TLS 1.3 Connection: Every byte ex- Available: https://rfc-editor.org/rfc/rfc7427.txt
plained,” https://tls13.ulfheim.net, 2018, Web page. Accessed 2019-21-
[50] S. Kölbl, M. M. Lauridsen, F. Mendel, and C. Rechberger, “Haraka
08.
v2-efficient short-input hashing for post-quantum applications,” IACR
[30] L. Ducas, E. Kiltz, T. Lepoint, V. Lyubashevsky, P. Schwabe, G. Seiler, Transactions on Symmetric Cryptology, pp. 1–29, 2016.
and D. Stehlé, “Crystals-dilithium: A lattice-based digital signature
[51] P. Kotzias, A. Razaghpanah, J. Amann, K. G. Paterson, N. Vallina-
scheme,” IACR Transactions on Cryptographic Hardware and Embed-
Rodriguez, and J. Caballero, “Coming of age: A longitudinal study of tls
ded Systems, pp. 238–268, 2018.
deployment,” in Proceedings of the Internet Measurement Conference
[31] ——, “CRYSTALS-Dilithium Algorithm Specifications and Support- 2018. ACM, 2018, pp. 415–428.
ing Documentation,” https://pq-crystals.org/dilithium/resources.shtml,
[52] K. Kwiatkowski, “Towards Post-Quantum Cryptography in
2018, Submission to round 2 of the NIST post-quantum project.
TLS,” Jun. 2019. [Online]. Available: https://blog.cloudflare.com/
[32] ETSI, “ETSI TC Cyber Working Group for Quantum-Safe Cryptogra- towards-post-quantum-cryptography-in-tls/
phy,” https://portal.etsi.org/TBSiteMap/CYBER/CYBERQSCToR.aspx,
2017, Web page. Accessed 2019-07-25. [53] K. Kwiatkowski and L. Valenta, “The TLS Post-Quantum Experiment,”
https://blog.cloudflare.com/the-tls-post-quantum-experiment/.
[33] S. Fluhrer, D. McGrew, P. Kampanakis, and V. Smyslov,
“Postquantum Preshared Keys for IKEv2,” Internet Engineering [54] A. Langley, “CECPQ1 results,” Nov. 2016. [Online]. Available:
Task Force, Internet-Draft draft-ietf-ipsecme-qr-ikev2-08, Mar. 2019, https://www.imperialviolet.org/2016/11/28/cecpq1.html
work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/ [55] ——, “CECPQ2,” Dec. 2018. [Online]. Available: https://www.
html/draft-ietf-ipsecme-qr-ikev2-08 imperialviolet.org/2018/12/12/cecpq2.html
[34] P.-A. Fouque, J. Hoffstein, P. Kirchner, V. Lyubashevsky, T. Pornin, [56] ——, “Post-quantum confidentiality for TLS,” Apr. 2018. [Online].
T. Prest, T. Ricosset, G. Seiler, W. Whyte, and Z. Zhang, “Falcon: Available: https://www.imperialviolet.org/2018/04/11/pqconftls.html
Fast-Fourier lattice-based compact signatures over NTRU,” https://csrc. [57] B. Laurie, A. Langley, and E. Kasper, “Certificate Transparency,” RFC
nist.gov/projects/post-quantum-cryptography/round-2-submissions, 6962, Jun. 2013. [Online]. Available: https://rfc-editor.org/rfc/rfc6962.
2018, Specification v1.1. txt
[35] J. Fulmer, “Siege HTTP regression testing and benchmarking utility,” [58] Y. Liu, W. Tome, L. Zhang, D. Choffnes, D. Levin, B. Maggs,
https://www.joedog.org/siege-home/, 2019, Web page. Accessed 2019- A. Mislove, A. Schulman, and C. Wilson, “An end-to-end measurement
02-09. of certificate revocation in the webs pki,” in Proceedings of the
[36] A. Ghedini and V. Vasiliev, “TLS Certificate Compression,” 2015 Internet Measurement Conference, ser. IMC 15. New York,
Internet Engineering Task Force, Internet-Draft draft-ietf-tls-certificate- NY, USA: Association for Computing Machinery, 2015, p. 183196.
compression-05, Apr. 2019, work in Progress. [Online]. Available: https: [Online]. Available: https://doi.org/10.1145/2815675.2815685
//datatracker.ietf.org/doc/html/draft-ietf-tls-certificate-compression-05 [59] V. Lyubashevsky, C. Peikert, and O. Regev, “On Ideal Lattices and
[37] Z. Greg et al., “The Picnic Signature Algorithm Design Docu- Learning with Errors Over Rings,” Cryptology ePrint Archive, Report
ment,” https://github.com/microsoft/Picnic/blob/master/spec/design-v2. 2012/230, 2012, https://eprint.iacr.org/2012/230.
1.pdf, 2019.
[60] D. McGrew, M. Curcio, and S. Fluhrer, “Leighton-Micali Hash-
[38] ——, “The Picnic Signature Algorithm Specification,” https://github. Based Signatures,” RFC 8554, Apr. 2019. [Online]. Available:
com/microsoft/Picnic/blob/master/spec/spec-v2.1.pdf, 2019. https://rfc-editor.org/rfc/rfc8554.txt
[39] V. Gupta, D. Stebila, S. Fung, S. C. Shantz, N. Gura, and H. Eberle, [61] K. Moriarty, B. Kaliski, J. Jonsson, and A. Rusch, “PKCS# 1: RSA
“Speeding up Secure Web Transactions Using Elliptic Curve Cryptog- cryptography specifications version 2.2,” Internet Engineering Task
raphy,” in NDSS, 2004. Force, Request for Comments, vol. 8017, 2016.
[40] P. E. Hoffman, “The Transition from Classical to Post-Quantum [62] M. Mosca, “Cybersecurity in an era with quantum computers: will we
Cryptography,” Internet Engineering Task Force, Internet-Draft draft- be ready?” IEEE Security & Privacy, vol. 16, no. 5, pp. 38–41, 2018.
hoffman-c2pq-05, May 2019, work in Progress. [Online]. Available:
https://datatracker.ietf.org/doc/html/draft-hoffman-c2pq-05 [63] Mozilla, “Mozilla Telemetry Portal - Measurement Dashboard -
HTPT PAGE TLS HANDSHAKE distribution for Firefox Desktop,”
[41] J. Hoffstein, J. Pipher, and J. H. Silverman, “Ntru: A ring-based https://telemetry.mozilla.org/new-pipeline/dist.html, 2018, Beta 68/69,
public key cryptosystem,” in International Algorithmic Number Theory any OS, any architecture, any process. Web page. Accessed 2019-21-
Symposium. Springer, 1998, pp. 267–288. 08.
[42] http archive, “Trends,” http://httparchive.org/trends.php. [64] D. Naylor, A. Finamore, I. Leontiadis, Y. Grunenberger, M. Mellia,
[43] A. Huelsing, D. Butin, S.-L. Gazdag, J. Rijneveld, and A. Mohaisen, M. Munafò, K. Papagiannaki, and P. Steenkiste, “The cost of the S in
“XMSS: eXtended Merkle Signature Scheme,” RFC 8391, May 2018. HTTPS,” in Proceedings of the 10th ACM International on Conference
[Online]. Available: https://rfc-editor.org/rfc/rfc8391.txt on emerging Networking Experiments and Technologies. ACM, 2014,
[44] A. Hülsing, J. Rijneveld, S. Samardjiska, and P. Schwabe, “From 5-pass pp. 133–140.
MQ-based identification to MQ-based signatures.” IACR Cryptology [65] H. Nejatollahi, N. Dutt, S. Ray, F. Regazzoni, I. Banerjee, and R. Cam-
ePrint Archive, vol. 2016, p. 708, 2016. marota, “Post-quantum lattice-based cryptography implementations: A
[45] International Telecommunications Union, “X.509: Information technol- survey,” ACM Computing Surveys (CSUR), vol. 51, no. 6, p. 129, 2019.
ogy - open systems interconnection - the directory: Public-key and [66] Nginx, “NGINX: High Performance Load Balancer Web Server and

15
 Reverse Proxy,” https://www.nginx.com, 2019, Web page. Accessed [88] D. Sikeridis, I. Papapanagiotou, B. P. Rimal, and M. Devetsikiotis,
2019-02-09. “A Comparative taxonomy and survey of public cloud infrastructure
[67] Y. Nir, “Repeated Authentication in Internet Key Exchange (IKEv2) vendors,” arXiv preprint arXiv:1710.01476, 2017.
Protocol,” Internet Requests for Comments, RFC Editor, RFC 4478, [89] D. Soni, K. Basu, M. Nabeel, and R. Karri1, “A Hardware Evalua-
Apr. 2006. [Online]. Available: http://www.rfc-editor.org/rfc/rfc4478.txt tion Study of NIST Post-Quantum Cryptographic Signature schemes,”
[68] M. Ounsworth and M. Pala, “Composite Keys and Signatures Second PQC Standardization Conference, Aug 2019.
For Use In Internet PKI,” Internet Engineering Task Force, [90] D. Stebila and M. Mosca, “Post-Quantum Key Exchange for the Internet
Internet-Draft draft-ounsworth-pq-composite-sigs-01, Jul. 2019, work and the Open Quantum Safe Project,” Cryptology ePrint Archive, Report
in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/ 2016/1017, 2016, https://eprint.iacr.org/2016/1017.
draft-ounsworth-pq-composite-sigs-01 [91] D. Steblia, S. Fluhrer, and S. Gueron, “Design issues for
[69] C. Paquin, D. Stebila, and G. Tamvada, “Benchmarking post-quantum hybrid key exchange in TLS 1.3,” Internet Engineering Task
cryptography in tls,” Cryptology ePrint Archive, Report 2019/1447, Force, Internet-Draft draft-stebila-tls-hybrid-design-01, Jul. 2019, work
2019, https://eprint.iacr.org/2019/1447. in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/
draft-stebila-tls-hybrid-design-01
[70] C. Peikert, “A decade of lattice cryptography,” Found. Trends Theor.
Comput. Sci., vol. 10, no. 4, pp. 283–424, Mar. 2016. [Online]. [92] M. Thomson, “Suppressing Intermediate Certificates in TLS,”
Available: http://dx.doi.org/10.1561/0400000074 Internet Engineering Task Force, Internet-Draft draft-thomson-tls-
sic-00, Mar. 2019, work in Progress. [Online]. Available: https:
[71] R. Pino, V. Lyubashevsky, and D. Pointcheval, “The Whole is Less //datatracker.ietf.org/doc/html/draft-thomson-tls-sic-00
Than the Sum of Its Parts: Constructing More Efficient Lattice-Based
AKEs,” in Proceedings of the 10th International Conference on [93] C. Tjhai, M. Tomlinson, [email protected], S. Fluhrer, D. V.
Security and Cryptography for Networks - Volume 9841. Berlin, Geest, O. Garcia-Morchon, and V. Smyslov, “Framework to Integrate
Heidelberg: Springer-Verlag, 2016, pp. 273–291. [Online]. Available: Post-quantum Key Exchanges into Internet Key Exchange Protocol
https://doi.org/10.1007/978-3-319-44618-9 15 Version 2 (IKEv2),” Internet Engineering Task Force, Internet-
Draft draft-tjhai-ipsecme-hybrid-qske-ikev2-04, Jul. 2019, work in
[72] T. Pornin, “New efficient, constant-time implementations of falcon,” Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/
Cryptology ePrint Archive, Report 2019/893, 2019, https://eprint.iacr. draft-tjhai-ipsecme-hybrid-qske-ikev2-04
org/2019/893.
[94] P. Yee, “Updates to the Internet X.509 public key infrastructure certifi-
[73] O. Project, “liboqs,” https://github.com/open-quantum-safe/liboqs, cate and Certificate Revocation List (CRL) profile,” 2013.
2019, Web page. Accessed 2019-02-08. [95] Y. Yoo, R. Azarderakhsh, A. Jalali, D. Jao, and V. Soukharev, “A Post-
[74] ——, “OQS OpenSSL,” https://github.com/open-quantum-safe/openssl, Quantum Digital Signature Scheme Based on Supersingular Isogenies,”
2019, Web page. Accessed 2019-02-08. Cryptology ePrint Archive, Report 2017/186, 2017, http://eprint.iacr.
[75] P. Project, “PQClean,” https://github.com/PQClean/PQClean, 2019, Web org/2017/186.
page. Accessed 2019-02-09.
[76] J. Proos and C. Zalka, “Shor’s discrete logarithm quantum algorithm
for elliptic curves,” Quantum Info. Comput., vol. 3, no. 4, pp.
317–344, Jul. 2003. [Online]. Available: http://dl.acm.org/citation.cfm?
id=2011528.2011531
[77] O. Regev, “On lattices, learning with errors, random linear codes, and
cryptography,” Journal of the ACM (JACM), vol. 56, no. 6, p. 34, 2009.
[78] E. Rescorla, “The transport layer security (TLS) protocol version 1.3,”
2018.
[79] E. Rescorla, R. Barnes, and H. Tschofenig, “Compact TLS
1.3,” Internet Engineering Task Force, Internet-Draft draft-rescorla-
tls-ctls-03, Nov. 2019, work in Progress. [Online]. Available:
https://datatracker.ietf.org/doc/html/draft-rescorla-tls-ctls-03
[80] E. Rescorla, K. Oku, N. Sullivan, and C. A. Wood, “Encrypted
Server Name Indication for TLS 1.3,” Internet Engineering
Task Force, Internet-Draft draft-ietf-tls-esni-05, Nov. 2019, work
in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/
draft-ietf-tls-esni-05
[81] E. Rescorla, N. Sullivan, and C. A. Wood, “Semi-Static Diffie-Hellman
Key Establishment for TLS 1.3,” Internet Engineering Task Force,
Internet-Draft draft-rescorla-tls-semistatic-dh-02, Nov. 2019, work
in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/
draft-rescorla-tls-semistatic-dh-02
[82] C. Roma, C.-E. A. Tai, and M. A. Hasan, “Energy Consumption
of Round 2 submissions for NIST PQC Standards,” Second PQC
Standardization Conference, Aug 2019.
[83] M.-J. O. Saarinen, “Mobile energy requirements of the upcoming NIST
post-quantum cryptography standards,” 2019.
[84] S. Santesson and H. Tschofenig, “Transport Layer Security (TLS)
Cached Information Extension,” RFC 7924, Jul. 2016. [Online].
Available: https://rfc-editor.org/rfc/rfc7924.txt
[85] J. Sepúlveda, S. Liu, and J. M. B. Mera, “Post-quantum enabled cyber
physical systems,” IEEE Embedded Systems Letters, 2019.
[86] SHODAN, “HTTPS (443) Overview,” Jul. 2019, https://www.shodan.
io/report/nWlAWhKG.
[87] P. W. Shor, “Polynomial-Time Algorithms for Prime Factorization and
Discrete Logarithms on a Quantum Computer,” SIAM J. on Computing,
vol. 26, no. 5, pp. 1484–1509, 1997.

16