21. Advanced material: Signatures over data

21.1. Internals of inline signed messages

Inline signed messages are one of the forms of OpenPGP data signatures. An inline signed message joins the signed data and its corresponding data signature into a single OpenPGP message.

OpenPGP defines two variant forms of inline signed messages:

1. One-pass signed messages This is the commonly used format for inline signed messages. A signer can produce and a verifier can verify this format in one pass.

2. Prefixed signed messages This format predates[1] one-pass signed messages and is conceptually slightly simpler. However, it is now rarely used and can be considered a legacy format.

21.1.1. One-pass signed message

This is the commonly used format for inline signed messages.

21.1.1.1. Structure

A one-pass signed OpenPGP message consists of three segments:

1. One-pass signature packets: These one or more packets precede the signed data and enable signature computation (both creation and verification) in a single pass.

2. OpenPGP message: This contains the original payload data (e.g., the body of a message), which is signed without additional interpretation or conversion. Internally, a signed message consists of one or more OpenPGP packets. This payload is typically stored as either a Literal Data Packet, or a Compressed Data Packet.

3. Data signature packets: These contain the cryptographic signature corresponding to the signed data.

Fig. 34 The structure of a one-pass signed message.

Note

Despite its name, a one-pass signature packet is not a type of signature packet.

Instead, it’s a type of auxiliary packet that can be used in conjunction with signature packets, to enable efficient generation and checking of inline signed messages.

The structure of a one-pass signature packet closely mirrors an OpenPGP signature packet. However, it does not contain a cryptographic signature.

21.1.1.2. The function of the one-pass signature packet

The purpose of this packet is efficient handling of inline signed messages in stream processing mode. This is particularly important when the signed message is large and exceeds available memory in size.

Without this packet, the position of signature packets within an inline signed OpenPGP message constitutes a trade-off:

• The producer of a signed OpenPGP message wants to streamline the signature calculation process in such a way that allows to emit a copy of the signed data while calculating the cryptographic signature. On the signer’s side, the signature packet is therefore easy to store after the signed data.

• The verifier, on the other hand, needs some information from the signature packet to perform the signature verification process. In particular, the verifier needs to know which hash algorithm was used to calculate the signature, to perform the same hashing operation on the message data.

As a consequence, without a one-pass signature packet, either:

• The producer would need to process the input data twice:

  • once to calculate the cryptographic signature, and

  • a second time to emit the signed data (this format result is a Prefixed signed message), or

• The verifier would need to process the OpenPGP message twice:

  • once to read the signature packets at the end to determine the hash algorithm, and

  • a second time to process the body of the message, and calculate the hash verifying the signature.

The one-pass signature packet solves this issue by allowing both the creation and verification of a signed message in a single pass. The one-pass signature packet effectively contains an advance copy of the data in the signature packet, but without the cryptographic signature data.

The signer can easily emit the metadata in the one-pass signature packet before processing the full message. For the verifier, availability of this metadata at the start of the signed message enables processing of the message body.

Even in stream processing mode, signers can efficiently generate one-pass signed messages, and verifiers can efficiently check them.

21.1.1.3. Creation

To produce a one-pass inline signature, the signer decides on a hash algorithm and emits a one-pass signature packet into the destination OpenPGP message. This contains essential information such as the fingerprint of the signing key and the hash algorithm used for computing the signature’s hash digest. The signer then processes the entirety of the signed message, emitting it as a series of one or more packets into the message as well. Once the data is processed, the signer calculates a cryptographic signature using the calculated hash value. Lastly, the result is emitted as a data signature packet to the output message, and the whole packet sequence can be efficiently stored or transmitted.

21.1.1.4. Verification

For efficient verification, an application must understand how to handle the OpenPGP message prior to reading from it. This requirement is addressed by the one-pass signature packets located at the beginning of inline signed messages. This setup enables the verifier to process the data correctly and efficiently in a single pass.

Strictly speaking, knowing just the hash algorithm would be sufficient to begin the verification process. However, having efficient access to the signer’s fingerprint or key ID upfront allows OpenPGP software to fetch the signer’s certificate(s) before processing the entirety of the - potentially large - signed data. This may involve downloading the certificate from a keyserver. In case fetching the signer’s certificate(s) fails, or requires additional input from the user, it is better to signal the user about this before processing the data.

one-pass inline signed messages enable efficient verification in one pass, structured as follows:

1. Initiation with one-pass signature packets: These packets begin the verification process. They include the signer’s key ID/fingerprint, essential for identifying the appropriate public key for signature validation.

2. Processing the OpenPGP message: This step involves hashing its data, preparing it for signature verification.

3. Verifying signature packets: Located at the end of the message, these packets are checked against the previously calculated hash digest.

Important to note, the signer’s public key, critical for the final verification step, is not embedded in the message. Verifiers must acquire this key externally (e.g., from a key server) to authenticate the signature successfully.

21.1.1.5. Nesting of one-pass signatures

A one-pass signed message can actually contain multiple, nested, signatures.

Formally, this is the case because in the OpenPGP message grammar when an input OpenPGP message is one-pass signed, the resulting sequence of packets is in turn also considered an OpenPGP message.

Thus, this signed message can be one-pass signed yet again. This construction means that all signature packet pairs bracket the innermost message, and the outermost one-pass signature packet corresponds to the outermost signature packet.

21.1.1.5.1. Two semantics of nested signatures

There are two different use cases and semantics for nested one-pass signatures:

• Multiple signers issue independent cryptographic signatures that are stored in one shared (and thus space-efficient) inline signed message. In this case, each signer makes a cryptographic statement about just the signed message. The signatures are independent of each other.

• Alternatively, a signer can sign not just the input message, but also include previous signatures in their signature. In this case, the signer makes a cryptographic statement about the pre-existing signature(s) combined with the signed message. This means that the new signer attests the previous signature(s)[2].

21.1.1.5.2. How to pick one

When nesting one-pass signatures, the default expectation would be that each enclosing signature makes a statement about the complete message it contains, including any one-pass signatures within the inner message.

Issuers of signatures can choose the semantics of their signature, using the “nested” flag in the one-pass signature packet. The “nested” flag has a value of either 1 or 0.

Meaning of the “nested” flag:

• 0 means that the one-pass signature that this signature encloses is not signed/attested. The new signature doesn’t make a cryptographic statement about the directly enclosed signature. If the directly enclosed one-pass signature also has its “nested” flag set to 0, the enclosing signature also doesn’t include the subsequent inner signature in its hashing, and so on.

• 1 means that this one-pass signature makes a cryptographic statement about the full message that it encloses, including all enclosed signatures, if any.

A typical pattern of use is to set the “nested” flag to 1 on the innermost signature and to 0 on all enclosing signatures. With this pattern, all signatures are independent of each other. Each signature makes a statement about just the innermost message payload (which is stored in a literal data packet).

21.1.1.5.3. Examples

As a practical example, consider the following notation:

• LIT("Hello World") represents a literal data packet with the content Hello World.

• COMP(XYZ) represents a compressed data packet over some other packet XYZ.

• OPS₁ represents a one-pass signature packet with the nested flag set to 1. Analogous, OPS₀ has the nested flag set to 0.

• SIG represents a signature packet.

A normal, one-pass signed message looks like this: OPS₁ LIT("Hello World") SIG

Here, the signature is calculated over the payload Hello World. The signature doesn’t change if the signed message is instead stored as: OPS₁ COMP(LIT("Hello World")) SIG (also see Hashing the signed payload of an inline signature).

A message, where multiple independent one-pass signatures are calculated over the same payload looks the following: OPS₀ OPS₀ OPS₁ LIT("Hello World") SIG SIG SIG - all three signatures are calculated over the same payload Hello World.

By contrast, a message, where the signer attests an already signed message has the following format: OPS₁ OPS₁ LIT("Hello World") SIG SIG. While the inner signature is calculated over the usual payload Hello World, the outer signature is instead calculated over OPS₁ Hello World SIG.

21.1.2. Prefixed signed message

A prefixed signed message consists of signature packet(s) followed by the message. For the verifier, processing one-pass signed and prefixed signed messages are equally convenient. However, on the signer’s side, it takes more resources to generate a prefixed signed message.

This is a legacy format. Not all modern implementations support it. However, for example, GnuPG 2.4.x can validate messages with this signature format.

21.1.2.1. Structure

In this format, the signature packets are stored ahead of the message itself:

1. Data signature packets: These one or more packets contain the cryptographic signature corresponding to the original data.

2. OpenPGP message: This contains the original data (e.g., the body of a message), without additional interpretation or conversion.

Fig. 35 Structure of a prefixed signed message.

Compared to a one-pass signed message, there are no one-pass signature packets in this format, and the (otherwise equivalent) signature packet(s) are stored ahead of the signed data.

Note

Even when a prefixed signed message contains multiple signature packets, each signature packet contains an independent signature of just the message payload. Signatures do not include subsequent signatures in their hashes, every signature is only over the raw payload data of the message.

21.1.2.2. Format is inefficient for the signer

For verification, this format is equally convenient as the one-pass signed message form.

However, when a signer creates a prefixed signed message, the signed data must be processed twice:

• once reading it to calculate the cryptographic signature, and

• once more to store the data in the generated OpenPGP message, after the signature packet(s).

21.1.3. Hashing the signed payload of an inline signature

When inline signing a message, the hash for the signed content is calculated over just the raw payload contained in a literal data packet. No metadata of the literal data packet is included in the signed hash. Even if a compressed data packet wraps the literal data packet, the inline signature is still calculated over the uncompressed content of the literal data packet.

The calculation of inline data signatures is unusual in two regards:

• Most OpenPGP signature calculations include packet metadata, but for literal data packets, only the payload is hashed.

• Packets are usually hashed without transforming the packet content for hashing. Decompressing the content of a compressed data packet for hashing is an exception to this pattern.

However, this approach means that detached signatures and inline signatures are calculated on exactly the same data.

One format can be transformed into the other, after the fact, without requiring the private key material of the signer. A compression layer can be inserted or removed without disturbing the validity of an existing signature.

[1]

One-pass signing was first specified in RFC 2440. The format was not supported in PGP 2.6.x. For one discussion of the feature in the lead-up to the standardization of RFC 2440, see here.

[2]

It’s unclear to the authors of this text if any real-world use case for signatures that notarize inner signatures exists.