# Dennard, Amdahl, and Moore: Identifying Limitations to Forecasting Laws

### Will Moore’s Law Hold Up?

As the hallmark law of technology forecasting (and often, the only case that people are familiar with) a debate rages around Moore’s Law and its validity–will it hold true? Will it fail? Will it plateau and then see breakthroughs? Fact of the matter is, that all of these are true statements…depending on what the exact metric you’re measuring is. In fact, the precise measured metric and how to choose one is going to be the focus of a later post, but for now I’d like to address what most people think of when they say Moore’s law, and what they expect, which is computers seeing drastic gains in raw speed performance from a processor level (disregarding improvements on the part of other parts of the system like the speed improvements from Solid State Drives).

If you go by that metric, Moore’s law has failed to keep up. There’s no two ways about it. I’m not saying the sky is falling, and I’m certainly not saying that this won’t change. All I’m saying is that, for now, the raw speed improvements in computers has failed to keep up. Why is that?

Well, there’s a corollary of Moore’s Law called ‘Dennard Scaling’. Simply put, Dennard Scaling states that as transistors get smaller their power density stays constant, or that total power per transistor decreases linearly. This means that if you cut the linear size of a chip by half in two dimensions, the power density will decrease by 1/4. If this wasn’t the case, 3 Moore’s law doubling cycles (ie an 8x improvement in number of transistors in a given area) would mean an 8x higher power density.

Dennard Scaling is what’s broken down. More details are explained here, but the gist of it is that the smaller the transistors get, the more static power loss there is. The more static power loss there is, the more the chip heats up, leading to even more static power loss, which is a self-reinforcing cycle called thermal runaway. Another problem occurs when the static power loss (which is a signal) is greater than the gate voltage, leading to errant activation of transistors, meaning faulty operation.

To avoid this, manufacturers began producing multicore chips (which you may have observed in the last few years). This is a valid approach, and also led to the push in parallelized code. However, while there are a number of architectural issues above my head here, there is one important fact about building multicore instead of single core system. What is it?

### The Problem

This is a general law called Amdahl’s Law. Amdahl’s Law states that the time T(n)  an algorithm takes to finish when being executed on n threads of execution with a fraction B of the algorithm that is strictly serial corresponds to:

$T(n)=T(1)(B+\frac{1}{n}(1-B))$

Amdahl’s law at 50%, 75%, 90%, and 95% parallelizable code. Source: http://en.wikipedia.org/wiki/Amdahl’s_law#mediaviewer/File:AmdahlsLaw.svg

As can be seen in the graph, even if your code is 95% parallelizable, as n approaches infinity (and infinite number of processors) you only get a 20x speedup…or just over 4 Moore’s Law cycles (8-10 years).

This article isn’t meant to try to convince you that these issues won’t be solved. In fact, for what it’s worth, I’m strongly of the opinion that they will be solved–new computing architectures and substrates mean that we will likely resume some form of rapid growth soon (this may be influenced by a degree of hope, but there are certainly enough alternatives being explored I find it somewhat likely). While it’s an interesting problem to look at, I think it’s a more useful example of how every technology forecasting law has associated theorems and roadblocks, and that finding these is important to a forecast.