Marcos Guillen

Marcos Guillen:
Artificial Development, Inc., CCortex

Human Neural Network Simulation

at the Accelerating Change Conference 2003
Stanford University, September 12-14, 2003

Edited by Peter Y. Chou

Marcos Guillen, founder, president & CEO of Artificial Development, Inc. gave a powerpoint presentation on "Human Neural Network Simulation". Previously, Marcos was co-founder and CEO of Red Internauta, Spain's largest independent dial-up ISP (1999-2002). Marcos apologized to the packed audience in the Cypress Room that this is his first presentation in English, so he's a bit nervous. But he appeared confident once his talk began. 28 slides were shown followed by a Q&A session. Here are my notes augmented by Guillen's PowerPoint Presentation which may be found and downloaded from his web site,


CCortex: Building a 20-billion neuron
emulation of the human cortex.

Artificial Development, Inc.
800 West El Camino Real
Mountain View, CA 94040

(2) CCortex: Building a 20-billion neuron emulation of the human cortex.
1. What is CCortex?
2. Why a brain emulation?
3. Is it feasible?
4. How does it Works?
5. Applications

(3) What is CCortex?
A software emulation of the brain
• 20 billion neurons
• 20 trillion connections
• Powerful, fast, spiking neuron engine
• Distributed parallel architecture
• Tons of data
• Very large three-dimensional neural networks architecture

(4) Why an Emulation?
• Scientists can only partially describe how the brain works.
• A number of brain functions are not yet understand.
    Most mind theories are, so far, untested.
• A few groups have tried partial, two-dimensional
    emulations of small portions of the brain.
• Small neural networks are unable to test current mind theories.

(5) Why an Emulation?
• Current mind theories are based on three-dimensional
    interactions between very large neuron populations.
• Theories describing chaotic interactions between
    Competing, spiking neural networks are, so far, untested.
• Here comes the engineering approach: modelling the brain.

(6) Why an Emulation?
• CCortex use a Teraflops class supercomputer
    to model a complete version of the human cortex.
• CCortex will support three-dimensional modelling of
    the most complex, challenging mind theories.
• CCortex holds the promise of unlocking the solution
    to problems that has been eluding us so far.

(7) Brain Emulation Constraints
• Computer power.
• Lack of detail in our knowledge of neural
    structures and circuitry.
• Researchers have come forward with
    a number of memory algorithms, and...
• Consciousness theories have not
    been successfully tested so far.

(8) The CCortex Approach
• Minimize computer power needs, thanks
    to highly optimized neural algorithms.
• Maintain a live database of neural structures,
    incorporating past & new discoveries into the model.
• Collaborate with researches to test different
    memory and consciousness theories.

(9) Is a strong AI feasible now? The brain in numbers.
• Average human brain contains about 85 billion neurons.
• Of these, only 12 to 15 billion are telencephalic
    neurons. (Shariff 1953) [J. Comp. Neurol. 98:381-400]
• 70 billion are cerebellar granule cells. (Lange 1975)
• Fewer than 1 billion are brainstem and spinal neurons.
• Some neurons can fire at up to 200 times per second.
• We assume that most neurons fire 40-60 times per second.

(10) Is a strong AI feasible now? No cerebellum?
• CCortex will not implement a full version of the cerebellum.
• The cerebellum contains up to 70% of all brain neurons.
• Traditionally considered the lesser brain, it is just
    a motor brain to control fine movements.
• New studies indicate that it may be involved in a wider variety
    of tasks, but we know that removing the cerebellum from
    young individuals often causes few obvious behavioral
    difficulties, suggesting that the rest of the brain
    can learn to function without it. (Bower, Parsons 2003)

(11) Is a strong AI feasible now? The emulation in numbers
• Neurons: 20,000,0000,000
• Connections: 1,000
• Average updates per second: 50
• Maximum theoretical computer power needed
    to emulate the brain: 1,000 Teraflops (8-bit),
    or 250 Teraflops (32-bit)
• Roughly the power of 62,500 Intel Pentium 4 chips

(12) Is a strong AI feasible now? The emulation in numbers
• Actually, 250 Teraflops (32-bit) is not that much
    for such a conservative Maximum theoretical.
• The proponents* of such a conservative Maximum
    Theoretical do not intend those measurements as
    predictions of when the strong AI will be possible,
    but of when it would be inevitable.
• Next, we are going to compare it with a similarly
    conservative Pentium 4 emulation.

*Kurzweil, Moravec

(13) Is strong AI feasible now ? The emulation in Numbers
Pentium 4 emulation:
• Transistors: 100 Million
• Clock Speed: 4 Ghz
• Average updates per second: 50
• Maximum theoretical computer power needed to
    emulate a Pentium 4: 400,000 Teraflops (1-bit)
    12,500 Teraflops (32-bit)
• Roughly the power of 3 million Intel Pentium 4

(14) Is strong AI feasible now ? The emulation in Numbers
• It "may" take up to 62,500 Intel Pentium 4's
    to emulate the human cortex.
    — Suggested alternative: do not update the complete
        system 50 times per second, update only the part
        of the system that is changing.
• It "may" take up to 3 million Pentium 4's
    to emulate a single Pentium 4
    — Suggested alternative: buy an AMD processor!

(15) Why these conservative estimates are not realistic predictions
• While emulating the brain, we are performing
    very simple, 8-bit operations.
• Most structures in our brain are redundant,
    and can be greatly simplified.
• Chunks of post-synaptic weights data
    can be easily compressed.
• Simple algorithms allow us to operate
    over compressed data
• We don't need to calculate the state of every single neuron.
    Only neurons receiving outside input or presently connected
    to an active neuron needs to be updated.

(16) CCortex: How does it work?
• Massive computer power.
• 500 dual processor cluster nodes with 1.000 processors.
• Up to 4 Teraflops.
• Terabytes of RAM.

(17) CCortex: How does it work?
• Neuron database.
• Post-synaptic weight database.
• Circuitry database.
• Neuron-type database.

(18) CCortex: How does it work?
• Neuron database:
    — Each neuron has an entry on the database:
        • Neuron type
        • Neuron position
        • Neuron firing status
        • Number of connections

(19) CCortex: How does it work?
• Post-synaptic weight database:
    — Each pair of connected neurons
        has an 8-bit value entry
    — Values hold information about
        the post-synaptic weight, vector, and
        its positioning relative to the cell body

(20) CCortex: How does it work?
• Circuitry database:
    — It holds shortcuts and connections
        between different neural networks
    — Controls CCortex input-output and the way
        different brain structures communicate
        with each other

(21) CCortex: How does it work?
• Neuron-type database:
    Properties of different neuron types
    and populations. Some fields are:
        — Adaptation properties
        — Firing rate/shape
        — Number of connections
        — Positioning markers

(22) CCortex: How does it work?
Spiking Neuron Engine (SNE): The neuron as an algorithm.
• Only neurons receiving input or recently stimulated are updated.
• Once all the pertinent neurons has been updated, it starts again.
• Easy to update. Do we have new theories to test?
    Just re-configure the algorithm.
• Each cluster node is running several SNE, updating its own chunk
    of data, and communicating with each others (to maximize
    multithreading capabilities, dual processors, and memory reads).
• The cluster has thousands of SNE running in parallel at any given time.
    Some SNE oversees (and sometimes overrules) the others.

(23) CCortex: How does it work?
The 'universal neuron' algo uploads its properties from a database.
    • Interpolates action potential shape.
    • Calculates if the firing potential has been reached.
    • Fires accordingly, choosing the right spiking rate
        and action potential rate.
    • Updates neuron status data, and, if needed,
        post-synaptic weights data.

(24) CCortex: How does it work?
• Data compression
    — Most data is compressed.
    — Database index and cached data are stored in RAM.
• Low cost of mathematical operations
    Basic operations are relatively simple for a modern processor.
    — Some operations are performed over compressed data.
    — Some operations are not performed at all: it is more cost
    — efficient to look then up on a database or in cached data.

(25) Applications
• CCortex Cluster is a research version
    — CCortex Cluster installation is a "test bed," not the final product.
    — By modelling the cortex we aim at improving our basic
        understanding of the human intelligence, and to
        develop new, patentable algorithms to implement it.
• CCortex future versions: CCortex appliances and boards
    — The final CCortex product will more likely be implemented using
        appliance hardware, FPGA boards, or custom hardware.
    — After all, custom hardware has allowed 3D graphics chip
        manufacturers to develop 300 Gflops graphic processor
        retailing at $300. A custom chip may be CCortex future.

(26) Applications
Some basic lines of research we are pursuing with the CCortex Cluster are:
• Cracking the neural memory algo.
• Reproducing mammalian level decision capabilities.
• Reproducing human level cognitive capabilities.

(27) Applications
• Cracking the neural memory algorithm commercial applications:
    — New, radically improved Linguistic User Interfaces (LUI)
        for computers, cell phones and vehicles.
    — Quantum leap for data retrieving technologies, including
        relational databases and Internet search engines.
    — Pattern recognition technology orders of magnitude
        more powerful than anything available now.

(28) Applications
• Reproducing mammalian level decision-
    making capabilities has innumerable
    commercial applications. It would simply
    reshape the computer industry.

• Reproducing human level cognitive
    capabilities would have an unpredictable
    economic impact: Futurists like John Smart
    could loose their jobs!

Marcos Guillen's talk began at 11:45 am and ended at 12:10 pm



Q: (inaudible)
A: You wouldn't move the mouse.
    The mouse will move itself.

Q: Is yours an academic or commercial enterprise?
A: We're all computer guys— computer programmers.
    We don't have neurologists on board.
    There are tons of research out there.
    No one has put them together.

Q: How is your company funded?
A: I was successful in dot-com in Spain.
    and am funding myself right now.

Q: Brains can optimize their functions,
    so neurons shorten their pathways.
A: (answer inaudible or too quick to jot down)

Q: On the topic of mimicking mammalian brains,
    you need to consult neurologists.
    The brain has 1021 operations/sec,
    so your factors (20 x 109) are way off.
A: We're just starting. We're not there yet.

Q: (Ben Goertzel): How will your brain computer model linguistics?
A: That problem is way ahead of us. Right now, we're trying
    to get as close to the human brain as possible.

| Top of Page | ACC2003 Conference-2 | ACC2003 Conference-1 | Singularity Books | Singularity Resources |
| Brain Resources | Brain Books | Human Spirit Stories | Robo-Man Poem | Wisdom Stories | Psychology |
| Stanford Lectures | Art & Spirit | Books | Enlightenment | Sages Directory | Poetry | A-Z Portals | Home |

© Peter Y. Chou,
P.O. Box 390707, Mountain View, CA 94039
email: (9-14-2003)