New Technology

It may not seem like much, but to me the new class of memory devices - Solid State Drives - that are just entering the consumer market are very significant because they very much improve the efficiency of the home computer; and that means eventually, that new classes of applications may emerge.

In the modern processor, the fastest component is the CPU - the central processing unit that actually performs the computations. To get the instructions and data it needs to operate, the CPU relies on a hierarchy of memory systems. This is necessary because memory that is fast — near-CPU speed — is expensive.

The current hierarchy of memory subsystems is
L1 cache
L2 cache
L3 cache
RAM
Hard disk

Latency measures the time it takes between the CPU making a request for something from memory and starting to get the reply. Here is an excerpt from a table from Anandtech for Intel's latest processor, the Core i7 (Nehalem); I expect the numbers are typical for any processor running at a few gigahertz.

L1 latency - 4 CPU cycles
L2 latency - 11 CPU cycles
L3 latency - 39 CPU cycles
RAM latency - 107 CPU cycles
Hard disk latency - approx 10.5 million cycles.

( The average latency of a 7200 rpm hard disk is 4.2 milliseconds - the time for a half-rotation of the disk. A 2.5GHz processor has a cycle time of 0.4 nanoseconds. That is where the 10.5 million cycles comes from. There are faster disks and clever arrangements of disks one can make to reduce this somewhat, but the number remains of the order of millions of cycles. )

SSDs offer to fill the gaping hole that exists in the memory hierarchy. For instance, the just-released second generation of Intel main-stream (as opposed to enterprise class) SSDs, the Intel X25-M has a latency of 65 microseconds, or

SSD latency - approx 160,000 cycles - that is a hundred times faster than hard disk.

Apart from the speed of starting to fulfill the CPU request for data, we are interested also in getting a large throughput, because it would not do much for performance if the CPU had to wait for a long time for the request to complete. The Intel SSD has impressive performance numbers, but it appears that going through the standard processor-to-hard drive interface (SATA) robs a lot of performance potential. The Fusion ioextreme that uses the computer's PCI express bus instead, highlights that potential. See this review.

-----

In my opinion - if "intelligence" is in part speed of reaction, then the CPUs of today are greatly underperforming because of this memory performance wall they crash into. Even SSDs are a thousand-fold slower than RAM. Intelligence or its computer simulacrum is receiving input from the world, interpreting it on the basis of a model of the world, and taking actions accordingly. Any reasonably complete model of the world must be fairly large, and so to have affordable intelligent systems in the home, the processors of those systems cannot be spending most of their time twiddling their thumbs for bringing the representation of the model to the processor.

Our brains get over this problem with much slower components by being massively parallel. Human designers of computers have great trouble with even mild parallelism. Even to get computers to parallelize themselves on such a scale, we would (IMO) need fundamental breakthroughs in the state of the art.

-----

PS: to put the issue of latency into a human context - let us say humans effectively operate at the level of speech or writing at one cycle per second. Imagine two humans conversing. Then L1 cache is like a really slow conversation, taking 4 seconds to receive the response to a question. RAM latency is like waiting for two minutes. Hard disk latency is like waiting for a whole year!

PPS: In this analogy, the SSD latency is equivalent to roughly two days; a processor second is somewhere around 200-250 years.