Question 2 of 7: Public-Key Communication, the Certificate Authority, and When to Prefer RSA over Diffie–Hellman

Question 2: Public-Key Communication, the Certificate Authority, and When to Prefer RSA over Diffie–Hellman (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Part a) — both parties already have a public/private key pair. With each side already holding a public/private key pair, they can communicate securely with public-key (asymmetric) cryptography directly: the sender encrypts a message with the recipient's public key, so only the holder of the matching private key can decrypt it, and/or signs it with their own private key so the recipient can verify the sender's identity and that the message was not altered. Because asymmetric operations are computationally expensive for bulk data, real systems use a hybrid scheme: the public keys authenticate each other and establish a fresh, random symmetric session key (e.g. via a signed key exchange), and all subsequent traffic is protected with a faster symmetric cipher (Question 1) — the approach used by TLS/HTTPS.

Part b) — the certificate authority. A certificate authority (CA) is a trusted third party that verifies an entity's identity and then issues a digital certificate binding that identity to its public key, signing the certificate with the CA's own private key. Anyone who trusts the CA and holds its public key can verify the certificate's signature and thereby trust that the enclosed public key really belongs to the named party, without ever having had prior contact with that party. This directly prevents a man-in-the-middle (impersonation) attack: without a CA, an attacker sitting between the two parties could substitute their own public key while claiming to be the other party, silently intercepting and re-encrypting all "secure" traffic; because the attacker cannot forge a CA signature over a certificate binding the victim's identity to the attacker's key, a client that checks the certificate against a trusted CA rejects the substituted key.

Part c) — when to prefer RSA over Diffie–Hellman. RSA (used as a public-key algorithm for key transport, not key agreement) is the better choice, rather than a DH key exchange, when: (1) the communication is non-interactive / store-and-forward — the recipient may not be online at the moment of sending (encrypted email/S-MIME, encrypting a file or a symmetric key for later retrieval) — because RSA lets the sender unilaterally encrypt a session key under the recipient's already-published public key in a single message, with no live two-way exchange needed, whereas basic DH requires both parties to be present to exchange values before a shared key exists; (2) forward secrecy is not required and simplicity matters more — RSA key transport reuses one long-term key pair and needs no per-session ephemeral-exponent generation or extra round trip, which is cheaper on constrained clients and simpler to implement/audit; (3) the deployment already has an RSA-based PKI (certificates, signing, and encryption all under the one algorithm and one certificate) and adding a second, separate DH capability is not justified; and (4) the protocol needs the same key pair to both encrypt and sign, which RSA supports natively (DH keys are only usable for key agreement, not signing, without pairing them with a separate signature scheme such as DSA/ECDSA).