CPUs, Cores, and Hyper-Threading

We're running into some barriers when it comes to processing power. Processing power is measured in hertz — how many cycles we can do something in any given second — and we've reached kind of a peak on what we can do, so we're having a hard time pushing past that peak. What we do is we do things like cores and hyper-threading, and there are other ways that we start creating ways to get more processing power.

Where the speed limit comes from

As we've seen over time, the number of transistors we have within our chips has been drastically increasing. But if you actually look at the CPU processing speeds, which are measured in cycles per second, or hertz, what you can see is that we've kind of reached a plateau here. We see that we've started to exponentially grow, and then at some point in time we've just kind of remained the same. There are some really fast processors out there, but for the most part, generally what we use is stuck at a certain range that we've been stuck at for a while.

The biggest limitation to this is just the heat that it generates. As we increase the amount of processing power that a chip has and get it to go faster and faster, we have a problem, because it's a small amount of area that's being passed this electricity and doing a large amount of work, and we can't dissipate the heat fast enough. We've made increases with our speed, but we've increased the speed in areas where we've really brought the temperature way down. If we bring the temperature way down in the environment that the processor is processing in, then we can bring the speeds up, which is just not as feasible when you're talking about our machines sitting at home or in our businesses.

Hertz

This speed is measured in hertz. What is hertz? It's cycles per second. We measure how many cycles a chip can go through every second, and we typically measure it in something like gigahertz. Let's give an example of three gigahertz, because that would be a fairly fast machine nowadays. If this is operating at 3 gigahertz, then that means it's three billion cycles every second. There's some sort of clock that will time this chip, and every time there's that cycle it will do some sort of calculation. It might have to go through many different calculations to be able to have some sort of output, but it'll go through all these calculations before it gives that output.

Threads

There are some things that we've done to try to overcome the CPU speed limitations. For instance, in some of our machines we put multiple processing units, so we can process more stuff at any given time. Or we've implemented hyper-threading, or we've done multicore processing. Let's take a look at what those mean. But before we get into that, let's talk about what a thread is.

A thread is some data that needs to be sent through and processed. We have this line of code, or we have some data, or we have something that needs to happen that needs to get sent into the chip so that it can be processed. That would be a thread. Here's where that thread measures up with the hertz: as the thread goes in to be processed, it will take many cycles to do that processing, and how many cycles just depends on what that thread is trying to do, and then it will have some sort of output.

Getting more threads through

So how do we increase the speed if we're kind of stuck at the same speed, the same clock rate, on these CPUs? Number one is we could just increase the number of processors that we have. I can take this and I can duplicate it. I can have two of these on the same motherboard, or maybe four of them on the same motherboard, and so now I can process multiple threads at the same time.

Another thing I could do is something called hyper-threading technology, or HT. Hyper-threading technology can take in multiple threads at the same time into the chip and be able to process those separately and then have some sort of output to it. So usually it's like a couple of threads that can go into this processor and be processed and sent out. Hyper-threading was invented by Intel, and so only Intel can really use that name. This is also called simultaneous multithreading, or SMT, by any other vendor who's putting out the same technology.

Then in 2004 we came out with multicore processing. Multicore processing is when we take one of these chips and we add extra components, duplicates of these components, in here. What it looks like right here is that the core of this is duplicated, and now it'll accept two different threads. Notice that the outside is still the same here, and that's going to represent that there are still shared components among this. So this chip will have shared components and then it will duplicate some of the components, so that it can take in more than one thread at once.

A freeway analogy

How can we increase the processing power of a computer? I'm going to give a little analogy of a freeway, or a highway. Let's say the freeway is set to a speed limit of 70 mph — that would be 112 km per hour. That is the top speed, and that's the top speed that the chips can process data.

We have some options here. We can add more lanes: that's what the multicore CPU is, it's adding more lanes to the highway or freeway that we already have. What we also could do is build another freeway, so that is the multiple CPUs right there. The hyper-threading is more like letting people do the lane splitting and drive down the middle, or use the emergency lane. So this is using the emergency lane, or other lanes that don't quite really exist, but we're going to sneak them in somehow.

Single- versus multi-threaded applications

Just because we increase the amount of threads we can process at any given time doesn't mean that we actually are increasing the speed. It just depends on the scenario. Let's take a look at single- versus multi-threaded applications.

Let's say we have an application, and many applications are only single-threaded. What they do is, they need processing that needs to happen, so they submit the thread that needs to be processed, it goes into the chip and gets processed, and then the output comes out and gets sent back to the program. The program then figures out what's the next thing that needs to be processed, and the whole cycle starts over again. It's just a single thread that can be processed at any given time.

So for single-threaded applications, we're kind of stuck at the speed of whatever the processor can operate at. Even if it can accept multiple threads at any given time, the application might not be equipped to be able to handle that. You have to have specific multi-threaded applications in order to be able to handle that.

PCs versus servers

Let's just see how common these technologies are. Let's take a look at a PC versus a server — a PC being your desktop machine, your laptop, or maybe it's a Mac or whatever, and the server being something that's serving many different people.

What we have with just our desktop machine, generally speaking, is just one CPU running in there. We generally, on our personal devices, don't have more than one CPU that's running. The number of cores would be four to six, and hyper-threading is sometimes incorporated into it and sometimes it isn't — some chips have it, some chips don't.

With servers, generally speaking, we have two to four CPUs, but there can be many more CPUs on a server. The number of cores per CPU could be between 8 and 30, and it usually has some sort of hyper-threading technology turned on on it. So maybe with our desktop machine, with one CPU and six cores, maybe we could just do like six processes, calculate six things at any given time. Versus our server, we've got 4 times 30, so that's 120 that we have there, plus we do hyper-threading, so maybe that's over 200 different processes that can happen at any given time.

Why do servers need to be so much more? Number one is that generally with servers we have a lot more processes, a lot more programs, a lot more things that are running on a server than we do on a regular desktop. Also, a lot of our server applications can be multi-threaded software applications, so they're designed for multi-threading — not that a lot of games and a lot of things on desktops aren't nowadays, but it's more common for these server applications. And then it also has a lot of different users that have processes and things that need to be calculated at any given time. So the need to do multiple threads of calculation on a server is much greater.