 MEMORY > BRAIN VS THE COMPUTER

Memory management is an essential skill that every programmer develops over time. To develop
that skill you need to have a sense of how computer memory works. Surprisingly, memory is not
a simple thing. It is actually quite complex and only a few experts actually understand all of the
details. In this post, I make an analogy between computer and human memory to explain the
basics of both. This leads me to dive into the core of both computers and human minds as these
are deeply intermingled.

As stated by wikipedia, memory refers to the physical devices used to store programs
(sequences of instructions) or data. Both computers and humans have multiple forms of memory.
I don’t know whether human memory inspired the design of computers but the similarities are
striking as you are going to see.

 RAM or Random Access Memory and the Hippocampus

RAM or Random Access Memory

When someone refers to computer memory, it usually means RAM. RAM or Random Access
Memory was designed to provide a quick access to series of numbers. To limit access time,
RAM is organized around arrays of electronic components that allow very quick access to any
location in the memory. The main advantage of RAM is speed (typical RAM bandwidth today is
around 5-10 GBytes/s). The constraints on speed have historically limited the overall size of
available RAM on computers (today typical RAM memory is around 8 GBytes). The main
disadvantage of this form of memory is its volatility : you turn off your computer and it is gone.
 Where is the hippocampus in our brain?

Humans do have a form of RAM and it is called short-term memory. Interestingly, short-term
memory is usually associated with a very specific region of the brain called thehippocampus.
Short-term memory is the memory you will use to recall immediate actions (like the series of
words you are actually reading).

The hippocampus by Camillo Golgi

We don’t really understand how the hippocampus does it but I have always been puzzled by this
structure homogeneity. It is composed of billions of cells that are very densely packed, like on an
array as can be seen on the Golgi staining in the attached image. Reminds you something?

It is traditionally thought that the memory content stored in the hippocampus is transferred to
another brain area for long term storage. This process is supposedly happening when you are
turned off (your sleep). Again another similarity..
 The hard drive and the cortex.

A hard drive from Western digital

As for the hippocampus, RAM storage was too volatile so engineers came out with a
complimentary form of storage for long term memory in computers : the Hard Drive or HD.
Hard drive have low bandwidth (usually 100 MBytes/s) but can store much more (computer
usually comes with 500 GBytes of hard drive). This is where all of your data and programs is
located.

Human long-term memory is presumably located in the cortex, exactly why and how is a matter
of intense research nowadays. I am sure you have experienced that recollecting old memories
can take quite some time (sometimes minutes). This process is usually variable as it is when
reading from the hard drive. Hard drive can have very different read/write speed depending on
where the memory is stored.

Long term memory is supposedly transferred to the cerebral cortex via the hippocampus.

As for humans, when you put a computer to sleep, RAM memory is transferred to the hard drive
for long term storage. This process is reversed when you turn it on to make the best usage of the
fast RAM. I suppose the same thing is happening when we wake up in the morning (in my case,
this process is especially slow).

So far so good. Nothing too fancy here as the distinction between RAM and HD is usually
known.

There are still 3 forms of computer memory that we need to understand, so hold on.

 CPU Cache and neuronal network attractors

Transferring memory from the RAM to the CPU cache

Computers have something called CPU cache. CPU or Central Processing Unit is probably the
main thing a computer have that we don’t. This is the center of your system where all
calculations are being done. In essence a CPU is just a ton of operators packed in a very tight
space, they can combine bits of information in very specific ways. Thanks to decades of work,
CPU operates at very very high frequencies, in the GHz range. Most CPUs are now 64 bits
which means they can process 64 bits of information at each clock cycle. Because the frequency
is so high, it became essential to have a high bandwidth to provide information to the CPU. If
you check the numbers, it appears that the RAM is not fast enough to feed these monsters (the
order of magnitude is 24 GBytes/s). That’s what the CACHE was designed for. The total
memory content of the CACHE is very small as we really want this thing to be dead-fast (by
small I mean in the MByte range). The CACHE is a very little memory space packed at the close
proximity of your CPU to limit access times. There are multiple levels of cache (L1, L2 as
depicted on the associated image but there are more) to provide intermediate levels of storage to
make the CPU work smooth and organized.
Our brain is not clocked at such speeds. A typical neuron emits a new spike at about 10-40 Hz.
While it can go higher, we can say we have a 40 Hz clock running in our skull. This is why our
computer screens are only refreshed at 60 Hz. Why would you go faster, we don’t see it…

A neuronal network is a distributed cache memory

But as for more modern computer architecture, things are done in parallel in our little head. With
such a low clock speed, information is transferred very slowly from one side of the head to the
other so we can’t afford to have a single memory storage.

Indeed, Memory in the brain is distributed all other the place. I see the cortex as a gigantic
CACHE storage. Each bit of information that is processed wake up the local neuronal network,
reactivate its memory and compare this memory with the provided input. The beauty of neuronal
networks is that both the program (what we usually script in Matlab in M-files) and the
associated memory (the program parameters or your grandmother face) can be stored in the same
place.
 Protected memory and explicit memory
Human memory is further divided into two major classes
called explicit and implicit memory. Explicit memory is the memory you are consciously using,
like when you remember where you left your doors keys the day before, while implicit
memory is not necessarily conscious : you know how to walk but you don’t have to think to
walk. Still you had to learn and store it somehow. I have no idea how this separation is done at
the neuronal level but I do know that something very similar exists in computers and it’s
called memory protection. Memory protection is built-in in any modern OS. When you write a
program that needs some memory for its behavior, the OS allocates some space to it. It does so
with some protection mechanism to prevent the program from accessing any other memory. This
technique obviously prevent bugs and promote overall stability of your system. It does seem to
me that explicit and implicit memory fullfill the same purpose : stability of our brain “Operating
System” and protection of important memories.

 Memory swapping and writing

It sometimes happens that a program asks for more memory than what is available in the RAM.
When this happen, your operating system has no other solution than to use what’s available :
your hard drive. In that case, your OS will allocate some space on the hard drive for your
program to work. This is usually refers as “swapping”. Since hard drive are several order of
magnitude slower, this leads to great loss in performance. This is probably the number one
reason things can go slow within Matlab. Your program will not be able to tell if its data is stored
on the hard drive or in the RAM as this process is seemingly transparent. I personally believe
that if your program start to swap, it is because your program was not designed properly. It is
your job to be able to work within the memory limits of your computer.
Swapping does not exactly exists for human mind as our mind was extremely well designed to
work within its memory limits. However, in some ways we do swap all the time.

Indeed, what do you think you are doing when you take notes during a class?

The memory content of the class is too high for your brain to take it straight like this. You need
an intermediate data storage device to allow you to assimilate the new data set. So you swap the
data to a piece of paper using a pen. In some way, we human compensated our lack of a
swapping device via writing..