TechKnowSurge
CompTIA Security+ 1.4 ISC2 CISSP 4.3 CompTIA SecurityX 2.3 Cisco CCST Cybersecurity 1.4 ISC2 CISSP 3.6 CompTIA Network+ 4.1
VideoSecurityFree

TLS Handshake Protocol

The TLS handshake protocol establishes the cryptographic parameters, key exchange, and authentication required before secure communication can begin. This content covers how TLS 1.2 and TLS 1.3 handle that process, with particular focus on the improvements TLS 1.3 brings in speed and earlier encryption.

Complete this video to capture a CTF flag worth 1 point.

About this video

The TLS handshake protocol is responsible for three critical functions before any application data is exchanged: negotiating cryptographic parameters, performing the key exchange, and authenticating the parties involved. It operates after a TCP connection has already been established, picking up at the point where layer 4 connectivity exists and building the secure session on top of it. Without the handshake, TLS record protocol could provide confidentiality and integrity for data in transit, but there would be no way to confirm the identity of the remote system or agree on the cipher suite and keys to use. TLS 1.2 completes the handshake through a sequential exchange — client hello, server hello, certificate, server key exchange, server hello done, client key exchange, change cipher spec, and finished messages on both sides — but encryption doesn't begin until the later stages, meaning a significant portion of the handshake is transmitted in the clear. TLS 1.3 addresses this by restructuring the handshake into three phases that reduce round trips and push encryption much earlier in the process. In the key exchange phase, Diffie-Hellman parameters are included directly in the client hello and server hello, allowing both sides to derive a shared key and begin encrypting immediately. For environments where devices have communicated before, a pre-shared key can be used instead, though Diffie-Hellman is standard for general web traffic. Once the initial key exchange is complete, the server parameters phase establishes the cipher suite, TLS version, and session keys that will protect all subsequent data — moving from the computationally intensive asymmetric exchange to faster symmetric encryption for the long-term session. The server selects parameters that fall within the capabilities the client advertised in its hello message. The final authentication phase uses PKI certificates to verify the server's identity: the server presents its certificate along with an intermediary certificate, which chains up to a trusted root certificate already installed on the client system. This chain of trust confirms that the server is who it claims to be, completing the verification that the TLS record protocol alone cannot provide. Client authentication is also supported in this phase, though it is optional depending on the deployment.

What you'll learn

What's covered

TLS Handshake Protocol

Aligned to

CompTIA Security+
1.4 Explain the importance of using appropriate cryptographic solutions.
ISC2 CISSP
4.3 Implement secure communication channels according to design.
3.6 Select and determine cryptographic solutions.
CompTIA SecurityX
2.3 Given a scenario, implement appropriate cryptographic protocols and algorithms.
Cisco CCST Cybersecurity
1.4 Explain encryption methods and applications.
CompTIA Network+
4.1 Explain the importance of basic network security concepts.

Key terms

Transport Layer Security
TLS
A cryptographic protocol that provides secure communication over a network, successor to SSL.
Key Exchange
A method used to securely share cryptographic keys between parties over an insecure channel.
Authentication
The process of verifying the identity of a user, device, or system.
Public Key Infrastructure
PKI
A framework of hardware, software, policies, and standards used to create, manage, and distribute digital certificates.
Digital Certificate
An electronic document that uses a digital signature to bind a public key with an identity.
Cipher Suite
A preconfigured set of algorithms specifying the symmetric cipher, key length, mode of operation, hashing algorithm, and key exchange method used to secure TLS communication.
Diffie-Hellman
A key exchange algorithm that allows two parties to independently generate a shared secret over an insecure channel using two private keys, two public keys, and a shared value, without ever transmitting the secret itself.

Topics

Tls Cryptography Key Exchange Tls Handshake Network Security Tls 1 3

Transcript

For TLS to work it needs to establish some parameters in communications, and it does that all with the TLS handshake protocol.

We're going to jump back and forth a little bit between TLS 1.2 and TLS 1.3. We're mainly going to be talking about TLS 1.3, because that's the most recent version, however TLS 1.2 gives us some perspective.

Why We Need the Handshake Protocol

When it comes to the TLS record protocol, we satisfy confidentiality and integrity. But how do we know we're connecting to the right device — that authenticity piece, or authentication piece — and how do we coordinate the cipher suite and the keys? The answer to that is through this handshake protocol. We need the TLS handshake protocol for that key exchange, for negotiating cryptographic parameters, and for the authentication piece.

The TLS 1.2 Handshake Process

Realize that we're going to pick things up after TCP has already made its connection. This client has already reached out to the server and made a TCP connection, and the three-way handshake has already happened from a TCP perspective. So we already have layer 4 connectivity and we have established a connection. Then that's when TLS steps into place.

TLS is going to start out with the client sending a client hello. The server will send a server hello back, a certificate, a server key exchange, and a server hello done. Then the client is going to send a client key exchange, a change cipher spec, and a finished. Then the server is going to send a change cipher spec back with a finished, and finally data can be sent back and forth.

The white lettering is where we have encryption happening, so we don't get to the encrypted data until much further along in this process, which can cause a few different problems that people see with some of the exchange that's happening. So we want to cut down on that.

The 1.2 Handshake in Wireshark

What I'm showing you here is a snippet from Wireshark. It's not digging into the actual packets, but it's the packets that are flowing back and forth. Here we see that TCP handshake right from the get-go — the three-way handshake that establishes a communication. Then that's when TLS version 1.2 comes into play. We see the client hello, the server says hello back, sends a certificate — that's for that RSA verification piece that happens there — there's a server key exchange and a server hello done. From there the client steps in and sends a client key exchange, a change cipher spec, and an encrypted handshake message. This is where we start the encryption, so the finished part of this is encrypted when it is sent. You can see how this lines up with everything we just talked about.

What TLS 1.3 Changes

As I already mentioned, we don't get to the encrypted part until further on in the message, and we also have a lot of communication that happens before we actually start communicating the data. TLS 1.3 sets out to have less communication back and forth and encrypt things faster.

So what do we see with TLS 1.3? First of all we see a client hello, we see the server says hello, we exchange the server parameters and authentication, we have the client sending an authentication, and then data is getting back and forth. So transferring the data is happening much sooner, plus we have encryption happening much sooner as well.

The handshake for TLS 1.3 can be thought of as three different phases. First of all we have the key exchange, then we get into the server parameters, and then the authentication piece.

  • The key exchange is that asymmetric cryptography. This is that initial encryption, sending encrypted data much sooner in the process.
  • The server parameters are going to set up that long-term encryption, the cipher suite that we're going to use to encrypt things — that symmetrical encryption that we're going to use long term.
  • Then we have that authentication piece. The server needs to prove that they are the site that they say they are, and there is an optional client verification as well.

Phase One: The Key Exchange

We want to get things encrypted right away, so the exchanging of keys needs to happen right away as well. We need to have it right here in the client hello and the server hello, and that's what we see here. Here's the first packet right here, which is the client hello, and here's the second packet, which also includes the server hello, and within here are the key exchanges. So this could be a pre-shared key or it could be one of Diffie-Hellman, and so it will exchange that and then start using those keys right away for the encryption, encrypting starting at that server parameters.

When it comes to TLS 1.3 there are a few different options for key exchange. A couple of them are a couple different types of Diffie-Hellman, and one of them is a pass key. The Diffie-Hellman is for servers that have never communicated before, versus a pass key would be set up in advance — you set up a pass key on each of the devices. When it comes to visiting websites, you're pretty much talking about the key share being Diffie-Hellman; that is, you're probably not going to have a pre-shared key set up with these different websites.

So let's take a look at what that looks like. This is the client hello packet, a Wireshark capture. We can see all the information that falls into this client hello packet, but most of the information is closed. We're not taking an in-depth look into anything here, we just want to take a look at the key share. This is the client doing the key share, the Diffie-Hellman key share with the server, and the server is going to reciprocate that. Then it uses that Diffie-Hellman key exchange to come up with a unique key that they're going to start encrypting data right away with.

Phase Two: Server Parameters

Using the Diffie-Hellman key exchange to have a shared key amongst the servers is not the ideal way of encrypting all your data, because it's too slow and time intensive. What we need to do is establish the next step, and so this server is going to determine what those parameters are for encrypting data from here on out. That's where we see the server parameters. It's going to come up with the TLS version that's going to be used, it's going to come up with the cipher suite that will be used, the keys that will be used, and possibly ask for client authentication if it wants to.

So it's going to come up with these server parameters and tell the client. But it does need to establish a TLS version and a cipher suite that's going to be acceptable to the client, so it needs to know what the client capabilities are. Within this client hello packet we also see some information about the client's capabilities — the TLS versions it can use and the supported cipher suites. If we take a look at the client hello packet again, we can see what the client capabilities are. We see the cipher suites that it's capable of handling, and we also see in this supported versions down here the supported versions of TLS. It doesn't show all of them here, but we can see right at the bottom here that it has different TLS versions that it will support.

Then the server will send back the server parameters, and we can see that in here. We see the server hello right here, then we have a change cipher suite, and then we see application data — and remember, the application data at that point in time is encrypted. We can see that it's encrypted right here. I would like to open this up and show you what's actually inside, but you can't see it on here because it's all encrypted.

Phase Three: Authentication

Then we have phase three, the authentication phase. In this phase we go through the authentication process, and the server will authenticate itself, and then there's this optional client authentication.

I can't really show you what TLS 1.3 looks like because it's all encrypted at this point in time — that communication is encrypted, so it just looks like encrypted data. So let's take a look at TLS version 1.2, which is not encrypted at this point in time in the communication. Here's the Wireshark. We can see the client hello and then the server hello, and then we get to the certificate, because really that's largely — at least if you're visiting a website — what you're authenticating: that server certificate, that PKI, the public key infrastructure.

Let's take a look at the certificate and what that looks like. We actually have two certificates here. Here's one certificate and here's the second certificate within this packet. I just chose a random site for this, that's governmentjobs.com. The first certificate is for the site itself, and then the second certificate is going to be the intermediary certificate. We can see the information for that right there.

What the End User Sees

What does this look like from an end-user perspective? They've visited the site, this packet has come over with these two certificates in it. If they were to click the link next to the URL and view these certificates, they would see the two certificates and all the information for the certificates. This is the intermediary certificate that we saw come over in that certificate packet, and this is the site certificate. I can click one of these and it would list out all of that information here in the fields section, and so I can view the different information that these certificates have inside of them. I wouldn't even need to go into Wireshark and take a look at it.

I also have this root certificate right here, which is installed on my machine. That root certificate would validate this intermediary certificate, which would then validate this site certificate. Now I have a certain verification that, as long as I trust this root certificate, and that root certificate trusts the intermediary, and the intermediary trusts this site, I have a certain sense that the site is legitimate.

What I've done is I've just proven the authenticity of the information coming to me from these servers, and that these servers are the servers that should be delivering that information. The handshake has also accomplished transferring that server information to establish the parameters for the cipher suites that we're going to use, which will then establish our confidentiality and integrity.

About TechKnowSurge

TechKnowSurge builds IT and cybersecurity professionals through hands-on, concept-first training built around real understanding — not memorization. Free interactive tools, structured programs, and 25+ years of real-world experience, all in one place.

Explore free tools and programs →