11  Encryption in Practice

\[ \newcommand{\F}{\mathbb{F}} \newcommand{\G}{\mathbb{G}} \newcommand{\N}{\mathbb{N}} \newcommand{\R}{\mathbb{R}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\Zq}{\Z_q} \newcommand{\zo}{\{0,1\}} \newcommand{\bin}{\zo} \newcommand{\zon}{\{0,1\}^n} \newcommand{\A}{\mathcal{A}} \newcommand{\B}{\mathcal{B}} \newcommand{\calA}{\mathcal{A}} \newcommand{\calB}{\mathcal{B}} \newcommand{\calC}{\mathcal{C}} \newcommand{\calD}{\mathcal{D}} \newcommand{\calK}{\mathcal{K}} \newcommand{\calM}{\mathcal{M}} \newcommand{\calT}{\mathcal{T}} \newcommand{\calX}{\mathcal{X}} \newcommand{\calY}{\mathcal{Y}} \newcommand{\xor}{\oplus} \newcommand{\cross}{\times} \newcommand{\deq}{\mathrel{\mathop:}=} \newcommand{\abs}[1]{\left| #1 \right|} \newcommand{\set}[1]{\left\{ #1 \right\}} \renewcommand{\vec}[1]{\mathbf{#1}} \newcommand{\poly}{\operatorname{poly}} \newcommand{\negl}{\operatorname{negl}} \newcommand{\classP}{\mathsf{P}} \newcommand{\classNP}{\mathsf{NP}} \newcommand{\INDCPA}{\text{IND-CPA}} \newcommand{\Enc}{\mathsf{Enc}} \newcommand{\Dec}{\mathsf{Dec}} \newcommand{\Gen}{\mathsf{Gen}} \newcommand{\Derive}{\mathsf{Derive}} \newcommand{\Sign}{\mathsf{Sign}} \newcommand{\Ver}{\mathsf{Ver}} \newcommand{\MAC}{\mathsf{MAC}} \newcommand{\MACSign}{\mathsf{MAC.Sign}} \newcommand{\MACVerify}{\mathsf{MAC.Verify}} \newcommand{\Funs}{\mathsf{Funs}} \newcommand{\ToString}{\mathsf{ToString}} \newcommand{\sk}{\mathsf{sk}} \newcommand{\pk}{\mathsf{pk}} \newcommand{\vk}{\mathsf{vk}} \newcommand{\sig}{\sigma} \newcommand{\rgets}{\mathrel{\overset{\scriptscriptstyle\mathrm{R}}{\leftarrow}}} \newcommand{\getsr}{\rgets} \newcommand{\bits}{\{0,1\}} \newcommand{\Tag}{\operatorname{Tag}} \newcommand{\Vrfy}{\operatorname{Verify}} \newcommand{\EncAM}{\operatorname{Enc}^{\mathrm{E\&M}}} \newcommand{\DecAM}{\operatorname{Dec}^{\mathrm{E\&M}}} \newcommand{\EncME}{\operatorname{Enc}^{\mathrm{MtE}}} \newcommand{\DecME}{\operatorname{Dec}^{\mathrm{MtE}}} \newcommand{\concat}{\,\|\,} \]

So far, we have established several constructions that allow us to hide the contents of transmissions: we created chosen-plaintext- and chosen-ciphertext-secure encryption schemes that worked with and without a shared key. We will now discuss a few practical applications of transport encryption, and why it is often difficult to get right.

11.1 File Encryption

Perhaps the most straightforward use of encryption is file encryption.

Example: WhatsApp Encrypted Backup. WhatsApp allows the app’s users to back up their messages and contacts to the cloud. This way, a user can recover her messages if she loses or breaks her phone. To hide the user’s data from WhatsApp’s cloud servers, WhatsApp uses encrypted backup. To achieve this, the user’s device generates a 256-bit AES key \(k\) at the time of backup and encrypt the message data (photos, messages, etc.) likely using AES-GCM\((k, \cdot)\) before sending the ciphertext to the server. In order to allow you to restore your backup on a new phone, the app allows you to export 64 hexadecimal digits that encode the AES key used. When restoring your backup, you will enter these digits and your phone will fetch the ciphertext from the server and use these digits to reconstruct the key and decrypt your messages.

See this document for details on how WhatsApp encrypts backups. (The document also describes a more complicated backup scheme that uses passwords for encryption.)

This is a fairly simple application of file security. However, file encryption can be much more tricky: many applications require or desire features beyond simple encryption and decryption.

Case Study: PDF v1.5 Encryption

One instance of this desire for extra features that ended up going wrong was a previous version of the PDF standard, PDF v1.5.(Müller et al. 2019) This standard provided several features:

1. It is possible to password-encrypt some or all of the document. (The encryption scheme does not matter, but think of it as a secure authenticated-encryption scheme.) For example, a PDF could have an unencrypted title page and have the rest of the pages be encrypted.
2. A PDF document can contain a form that the PDF reader submits to a server via HTTP.
3. A form in a PDF document can reference other parts of a document.
4. The PDF reader may submit a form when an event happens: when the PDF is opened, closed, decrypted, etc.

Each of these features individually seems innocuous. However, when combined, there is a clever attack that allows an attacker to learn the contents of the encrypted portion of a PDF.

The attack works against a PDF document with an unecrypted (public) title page and an encrypted (secret) body. We assume that the attacker can modify the PDF file on its way to the victim.

To mount the attack, the attacker replaces the title page with an “evil” title page. The attacker needs write access to the file at some point, which they can get from an email attachment they forward along, a document on a shared drive or cloud folder, a USB stick, or a PDF they obtained in encrypted form and now want decrypted. In the last case the attacker holds an encrypted PDF they can’t read, tampers with its unencrypted title page, and sends it to someone who can, i.e., turning the victim into a decryption oracle.

The evil title page:

• contains an invisible PDF form,
• reads the contents of the decrypted pages into a PDF form element, on the event that decryption of the PDF body succeeds, and then
• submits the form via HTTP to evil.com

So when the victim recipient enters her password into the PDF reader to decrypt the document, the evil title page is exfiltrates the decrypted content to evil.com via a PDF form.

What went wrong? The core issue here was that the unencrypted contents of the document were not authenticated. That is, an attacker could modify the unencrypted pages of the document without detection.

One option is to use a MAC over all pages of the document. A downside of MACing the entire document is that, before rendering even a single page of the PDF, the reader would have to compute a MAC over the entire PDF. If the PDF consists of many thousands of pages, this could be expensive.

A better design uses two layers of integrity protection, at different granularities. The fine-grained layer authenticates each encrypted piece of the document on its own: rather than encrypting with a plain cipher, we encrypt each object with an authenticated-encryption scheme, so that every encrypted object carries its own MAC tag. A reader can then validate each object as it loads it, and can begin rendering without having read the rest of the file.

PDF applies encryption at the level of individual strings and streams rather than pages, so the natural unit here is the object, not the page. A single page is many objects, and an object may be shared across pages.

The fine-grained layer alone is not enough, though, and the reason is exactly the attack above. Per-object tags authenticate each object’s contents; they say nothing about which objects are present or how they refer to one another. An attacker who cannot forge a single tag can still delete an object, duplicate one, reorder the page tree, or—as in our attack—insert an entirely new unencrypted object. The evil title page requires no forgery at all, because nothing authenticates the fact that it was not there before.

So we add a coarse-grained layer: a single MAC over the structure of the whole document—which objects it contains and how they reference each other—under a key derived from the file encryption key. This is what detects the inserted title page. Note that the two layers are complementary rather than redundant. The structural MAC alone would force the reader back to hashing the entire file before rendering anything; the per-object tags alone would miss the attack entirely.

This is, in outline, the design that the PDF standard eventually adopted, but as optional extensions, so unauthenticated encryption remains legal and the attack is still prevented largely by patched readers that refuse to submit forms or open URLs from a document without the user’s consent.

11.2 Stream Encryption: Transport Layer Security (TLS)

The standard Internet data transfer protocols—TCP and UDP—provide no integrity or confidentiality whatsoever. To protect the data that we send over the network, we use the transport-layer security (TLS) protocol. The TLS protocol runs on top of TCP and aims to provide an encrypted “tunnel” between a client and server.

HTTPS is simply HTTP run over TLS.

While designing a stream-encryption protocol may seem straightforward at first glance, the task is much more subtle than it would seem. However, as is often the case in security, features and practical requirements make the situation much more complex.

Downgrade Attack

We now give one example of a downgrade attack—the sort of subtle security issue that can arise in protocol design.

The current version of TLS is TLS 1.3. An older version of the TLS standard, called SSLv3, is vulnerable to devastating attacks that allow an attacker to recover the plaintext traffic. However, for backwards compatibility, many TLS clients and servers still supported SSLv3 until quite recently.

When a TLS client connects to a TLS server, the two parties first exchange some messages to decide which version of the TLS protocol to use. In particular, the client will try to connect to the server using the latest version of TLS it supports. The server will respond back with either a confirmation (if the server supports the client’s proposed TLS version) or with garbage (if not). If garbage, the client will try to connect to the server with an older version of TLS.

An important point is that none of these negotiation messages are authenticated—the client and server cannot start using authentication (MACs, signatures, etc.) until they agree on which version of TLS to use! So, an active in-network attacker can simply replace all of the server responses in this protocol-negotiation phase with garbage until the client downgrades all the way to SSLv3. Once the client and server agree to use SSLv3, believing that this is their best available option, the attacker can then monitor and decrypt their traffic using known attacks on SSLv3.

The best defense against this attack is for both parties to disable support for SSLv3 completely.

TLS Structure

TLS consists of two main phases:

1. Handshake: In this phase, the client and server use a key-exchange protocol to agree on a shared key to use to encrypt their application-layer traffic. This step uses public-key cryptography, since the client and server initially have no shared secret.
2. Record protocol: This phase is where the actual application-layer communication happens. This phases uses the secret key that the client and server agreed upon in the handshake phase for authentication and encryption.

TLS Handshake Properties

In our definitions of public-key encryption, we had only two (relatively simple) properties: correctness and security. The TLS handshake, however, has a much more complicated set of goals:

• Correctness: Both parties agree on the same session key at the conclusion of the handshake.

• Security: adversary “learns nothing” about the secret key that the parties agree upon at the conclusion of the handshake.

• Peer authentication: At the end of the handshake, each party believes that they are talking to the other party.

• Downgrade protection: The parties agree on the same version of TLS that they would agree on absent an in-network attacker.

• Forward secrecy with respect to key compromise: if an attacker steals the secrets stored on the client or the server, the attacker cannot decrypt past traffic.

• Protection against key-compromise impersonation: If an attacker steals a client’s secret key, the attacker should not be able to impersonate other servers to the client.

• Protection of endpoint identities: The public keys of the two parties should never flow over the wire in the clear. For example, if a client is connecting to a website that uses a content-distribution network, such as Akamai or Cloudflare, an attacker should not be able to tell which website the client is connecting to—only that it is hosted on Akamai or Cloudflare.

To provide forward secrecy, modern cryptographic protocols use long-term secrets only for signing—not for encryption. These protocols use ephemeral (one-time-use) cryptographic keys for key exchange and encryption. Protocol participants delete these ephemeral keys on connection teardown and/or they rotate these keys often.

This way, if an attacker compromises a party’s secret key, the attacker can only sign messages on behalf of that party; the attacker cannot use the secret to decrypt past messages.

This is subtler than it looks, and we return to it below. Stealing the client’s key obviously lets the attacker impersonate the client to others; that much is unavoidable. The requirement runs the other way: it must not also let the attacker impersonate everyone else to the client.

Key-compromise impersonation. Suppose the two parties authenticated each other using static Diffie-Hellman keys: the client holds \(a\) with \(A = g^a\), the server holds \(b\) with \(B = g^b\), and the session key is \(k = H(g^{ab})\). Authentication here is implicit—only the two genuine parties can compute \(k\). Now suppose an attacker steals the client’s key \(a\). For any server, with public key \(B\), the attacker can compute \[ g^{ab} = B^{a}, \] using only the stolen key and the server’s public key. It can therefore impersonate every server on the Internet to this one client, without attacking any of those servers. The problem is that the shared secret is symmetric in the two long-term keys, so holding either one is as good as holding the other.

Signing fixes this. If the server authenticates by signing the handshake transcript, as in Section 11.2.4, an attacker holding \(a\) still cannot produce that signature: authentication becomes one-directional, and the server proves possession of something the client never had.

Note that this is the same design choice described above. Using long-term secrets only for signing, and ephemeral keys for key exchange, buys forward secrecy and key-compromise-impersonation resistance together—one decision, two properties.

TLS 1.2 permitted static Diffie-Hellman and static RSA, and practical attacks of this kind were demonstrated against real browsers (Hlauschek et al. 2015). TLS 1.3 removes those modes entirely: every handshake uses ephemeral key exchange and a signature over the transcript, so the property follows from the shape of the protocol rather than from configuring it correctly.

TLS Handshake

The TLS handshake is very carefully designed to achieve these properties. A grossly simplified version looks something like the following:

1. At the start of the handshake, the TLS client holds the public \(\pk_{\text{CA}}\) of a certificate authority. The TLS server (for example, MIT) holds its secret signing key \(\sk_\text{MIT}\) and a public-key certificate \(\text{cert}_{MIT}\) binding its public key \(\pk_\text{MIT}\) to its domain mit.edu.
2. Client Hello: The client sends the following values to the server:

   • a random nonce,

   • list of supported ciphersuites, and

   • an ephemeral Diffie-Hellman public key. (The client constructs this Diffie-Hellman public key using its preferred ciphersuite. If the server does not support the ciphersuite the client picked, the client will have to re-run this step using a different ciphersuite.)

3. Server Hello: The server sends several values to the client, choosing a ciphersuite to use and completing the Diffie-Hellman key exchange. That is, the server sends:
   • a random nonce,
   • a ciphersuite to use,
   • and an ephemeral Diffie-Hellman public key
4. Both partices compute a shared session key \(k\) using Diffie-Hellman key agreement on the ephemeral keys they exchanged.
5. Under encryption using the keys derived from the session key \(k\), the server sends the certificate for mit.edu as well as a signature over all messages sent so far, using its long-term secret key \(\sk_\text{MIT}\). The client then checks that:
   • the certificate has been signed by one of the client’s trusted CAs, and
   • the signature from the server matches their own record of the messages.
6. Finally, the client and server run the TLS Record Protocol to exchange encrypted and authenticated application data.

A ciphersuite contains all of the cryptographic parameters needed to perform key exchange, hashing, authentication, and encryption. For example, one possible ciphersuite for TLS 1.2 is ECDHE-RSA-AES256-GCM-SHA384. This indicates use of ephemeral elliptic-curve Diffie-Hellman key exchange (DHE) with RSA signatures (RSA), 256-bit AES encryption in GCM mode (AES256-GCM), and SHA2-384 as the hash function (SHA384).

This simplified toy version of the TLS handshake does not provide many of the features that the real TLS handshake provides. But it should give you a flavor of what the real handshake looks like.

Ephemeral keys, forward secrecy, and transcript signing

Three design choices do most of the work in the handshake above. We describe them at a high level here; the real protocol is considerably more involved.

Ephemeral keys. The Diffie-Hellman keys exchanged in steps 2 and 3 are ephemeral: each party runs the ordinary key-generation algorithm of the Diffie-Hellman key exchange lecture afresh for every connection, sampling \(x \getsr \{1, \dots, q\}\) and sending \(g^x\). These keys have nothing to do with the server’s long-term signing key \(\sk_\text{MIT}\). Note that the client must generate and send its share before it has heard anything from the server, so it guesses which group the server supports; if it guesses wrong, the server says so and the client generates a new keypair for the group the server names.

Both parties then compute the shared secret \(g^{xy}\), but neither uses it as a key directly. The secret is passed through a hash function—in TLS, a key-derivation function—to produce the encryption and authentication keys for the rest of the connection. The derivation also mixes in a hash of every handshake message sent so far, so the two parties end up with the same keys only if they saw the same conversation.

Forward secrecy. When the handshake completes, both parties delete their ephemeral secrets \(x\) and \(y\). This is what gives forward secrecy: an attacker who records the encrypted traffic today and compromises the server tomorrow finds nothing on the server that decrypts the recording, because the only secret that could have done so no longer exists. The long-term key \(\sk_\text{MIT}\) is of no help, since it was never used to encrypt anything.

Signing the transcript. How, then, does the client know it ran the key exchange with mit.edu and not with an in-network attacker? The ephemeral public keys are unauthenticated when they are sent, so the server authenticates them after the fact: in step 5 it signs a hash of the entire handshake transcript—every message exchanged so far, including both ephemeral public keys, the nonces, and the negotiated ciphersuite—under \(\sk_\text{MIT}\), and sends that signature together with its certificate.

Signing the whole transcript, rather than just the server’s own ephemeral key, is what makes the other handshake properties fall out. An attacker who tampered with any earlier message—say, to force a downgrade to an older version of TLS—changes the transcript, so the signature no longer verifies. And because the signature is itself sent encrypted under keys derived from \(g^{xy}\), the server’s certificate never travels in the clear, which is what protects the endpoint identities.

The client checks the certificate against its trusted certificate authorities and checks the signature against its own record of the messages. If both pass, the client knows that the party on the other end of this particular key exchange holds \(\sk_\text{MIT}\).

Observe that \(\sk_\text{MIT}\) is used only to sign, never to encrypt or to derive keys. This single choice is what buys both forward secrecy and resistance to key-compromise impersonation: an attacker who steals a long-term key can sign messages from that point on, but cannot read past traffic and cannot impersonate anyone else.

11.3 Properties that TLS does not provide

Authenticated End-of-File. TLS does not provide any end-of-file authentication, or “clean closure.” To explain what this means by example:

A popular tool to install the toolchain for the trendy systems programming language Rust is rustup. To use the tool and install the Rust toolchain, the recommanded method is to run the command “curl https://sh.rustup.rs | sh.” This downloads a shell script from the internet over HTTPS and immediately runs it using the shell sh.

Imagine that the contents of the downloaded script create a temporary directory, copy things into it, install some things, and finally delete the temporary directory with something like rm -r /tmp/install. An in-network attacker, who knows the structure of the rustup install script, could drop all of the packets in the stream immediately after the characters rm -r /. Eventually, the TLS connection will timeout, an shell will run the command rm -r /, deleting the user’s entire file system.

To protect against this, script writers try to design their scripts such that if the stream is cut off in the middle of a download, nothing happens. For example, the install script might consist of a single function definition that is called at the very end of the function.

Plaintext Length Obfuscation. As we have discussed, encryption reveals exactly the length of the plaintext. This becomes dangerous when attacker-controlled data and secret data are compressed together before being encrypted, since the compressed length then depends on the content of the secret. This is the CRIME attack (Rizzo and Duong 2012), which recovered TLS session cookies one character at a time.

The attacker makes the victim’s browser issue requests whose path it controls, so that the guess and the secret cookie land in the same compression window. A correct guess compresses to a back-reference and the ciphertext is one byte shorter; a wrong guess is not. Reading off the length turns a \(256^{16}\) search into a \(16 \times 256\) one. Note that the cipher is never broken—the entire leak is in the length, which is exactly what our CPA security definition declines to protect when it requires \(\abs{m_0} = \abs{m_1}\).

TLS compression was removed in response, but the follow-up attack BREACH (Gluck et al. 2013) applies the same idea to HTTP response compression, which cannot simply be turned off; defending against it requires application-level measures such as not reflecting user input into responses that contain secrets.

Gluck, Yoel, Neal Harris, and Angelo Prado. 2013. “BREACH: Reviving the CRIME Attack.” Black Hat USA.

Hlauschek, Clemens, Markus Gruber, Florian Fankhauser, and Christian Schanes. 2015. “Prying Open Pandora’s Box: KCI Attacks Against TLS.” USENIX Workshop on Offensive Technologies (WOOT).

Müller, Jens, Fabian Ising, Vladislav Mladenov, Christian Mainka, Sebastian Schinzel, and Jörg Schwenk. 2019. “Practical Decryption Exfiltration: Breaking Pdf Encryption.” ACM CCS.

Rizzo, Juliano, and Thai Duong. 2012. The CRIME Attack. Presented at the ekoparty Security Conference.