Whobbes.com/blog

Creative Scientist

How to create a mind

Book written in 2012 by Ray Kurzweil famous futurist but also CS engineer. Its main subject is how we could create a machine that replicate our brain, and then make it better. By touching such an important topic - the final frontier - he also reflect on philosophy and ethics in a simple and direct way. The information is very accessible and also quite general but nice example from his career are mentioned. Overall, I found the book very inspiring and a must read.

92%


Summary

About

  • Neocortex aka the new brain

    • Make 80% of the volume of brain and his the latest to have been added to our body organs. Made of 10ˆ10 neurons with each 10ˆ3 or 4 connections.
    • Recent study show that it is made of very similar units made of a few hundreds neurons that are stacked at regular intervals in all three dimensions.
    • The author call them pattern recognizer and estimate that there are 100s of millions of them organized in layers of increased abstraction
    • Speed is slow - max 100Hz - but everything run in parallel so a detection from visual input takes about 50ms
    • Size and time: as we go in higher and higher layer the size of detectors augment and their processing speed decrease as time window augment too and enable to deal with bigger pattern to recognize.
    • Some regions are said specialized as their threshold and reaction to learning can vary i.e. hyper parameter tuning type of thing but still same structure. This comes from important plasticity of brain e.g. people with part of brain damaged can re-learn the missing skills.
  • Pattern Recognizers

    • Their functioning is relatively simple
    • Input are dendrites that connect to sensor organs or sub-layer of recognizer
    • The internal processing add inputs with each have weights and expected value ranges, e.g. length of sounds for spoken language, and decide if a global threshold is reached
    • If yes an output is fired on the axon to connected units and also a input is sent back towards input which In turn lower the threshold as a sort of expectation signal. This enable incomplete feature to still be detected. Those are excitatory signals which are responsible for expectation.
    • In the same way for a no threshold inhibitory signal can be sent to some features e.g. if mustache detected then probability of person being my wife is lower.
    • Each unit can be connected to 10 or 100 thousands other units
    • Redundancy is high to insure detection and avoid specialization of the system.
  • Memory

    • use the same brain part than learning and detection!
    • Memories are in fact stored like a linked list where one pattern start firing it literally replay the patterns matched in order hence why alphabet is remembered only in one direction.
    • Also some memories make word appears or have sound or images and some not depending on the abstraction of the pattern in the linked-list.
  • Old brain

    • Made of different parts and compete with neocortex for control of body in a fusional relationship as they have evolved one after the other so both system are now inter-woven.
    • A lot of output from old brain are emotional reactions
    • Processing time are faster hence the saying that reflex live in reptilian brain. That being said reflex learn can change so the neocortex can reprogram the old brain.
    • Size of neocortex is bigger than for any other animals and evolved bigger and bigger with time
    • General sensing is actually fairly low bitrate and a lot of pre-processing is done in organs and old brain. Also the neocortex is very good at giving illusion of data completeness to the point that we can experience things that do not exist but brain predicted could be there.
  • Creativity

    • Neocortex is nature best invention. Art is neocortex best invention.
    • Skills can be learned by everyone but not at same speed, see hyper parameters tuning idea.
    • To get new ideas i.e. be creative one must remove old ones. Author said that courage to go against orthodoxy is a great skill to develop.
    • Biology learns over thousands of years new patterns and skills, e.g. rats learning to go through maze get better after a lot of generations, but brain can do similar development in matter of hours!
  • Consciousness

    • Clear definition is not possible as of now
    • Law and moral systems are mostly built on avoid suffering of conscious beings, how conscious are different animals or even toddlers?
    • One theory is that everything is conscious on a gradient of intelligence based on complexity
    • Babies would not pass Turing test yet they are not machines
    • How to know what is conscious? We often resort to compare with ourselves and try to estimate if we can have an emotional response to entity to test. But for example giant squid can act “fear “even if very different body, is it conscious?
    • The rational to based on ourselves is because we don’t have better referential but what is a conscious being if not interested in human-like communication, behaviors and simply don’t care about Turing test?
    • We need to look deeper as basic considerations are quite limited in scope. What are we conscious of?
  • Free Will

    • Like consciousness something important for many humans who believes in it without being able to define it or tell for sure if it actually exists!
    • boils down to know if the world is deterministic
    • Cellular automate show that even deterministic world can be unpredictable - see Category IV of automates. The only way to know the future is run step-by-step, no formal way of estimating step x only n+1 based on n
    • If our world is like this then we can’t predict as the computer is part of the world
    • That being said unpredictable does not mean free will. the latter implies to be able to do choices that are not influenced wrote Schopenhauer. One can only will one thing at a time so for him the concept of will exist but not free will.
  • Quantum mechanics

    • Eastern philosophy starts with consciousness and Western physical world. This creates two views on quantum mechanics.
    • East: particles exist when observed
    • West: particles are in essence functions and measurement is the result of the particle field interacting with the measurement tool field
  • Simulating the brain

    • Given the highly redundant structure of brain if seems that it can be fairly easily modelled. As of today we can simulate parts of brain of rats at 11000 speed with good accuracy.
    • Issue is then what data to feed into the model? Even if perfect simulation exist it would take years before the simulation learn any useful skill
    • Advantage of being able to store what is learned and copy it easily e.g. Google train big networks of neurons and make the result available to the masses
    • Criticism that the brain is much more complex at a molecular level: true but does this mean we needed all this complexity to create the brain in the first place or that the brain adapted and learn with those constraints present, current experiments of molecular level simulation seem for example to indicate that electrical crosstalk between neurons is a byproduct and probably not required to make a brain.
    • Quantum mechanics Has been used to show why brain cant be simulated but weak argument as if it were the case we could build a quantum computer.
    • How can we ensure that data can be transmitted in the first place knowing that silicon have a non zero probability of error in bits transmission and that trillions transaction occurs every minute in the brain? Shannon invented Information theory which state that if probability of error is not exactly 50% one can transmit data with any accuracy through any channel albeit not at any speed as redundancy is used to ensure bit error rate.
    • Turing defined the simplest machine that can compute: program, CPU, memory. Interesting fact: he proved that there is a similar number of problems that can be defined that are solvable and insolvable which have unique solutions. So some problems are per definition not solvable even if solution is unique -> One can never understand math, just learn to cope with it.
  • Conclusion

    • Classic problem of definition that if one person is copied then the copy is another person. But if one was to replace each part of the body one by one by machine the result is entirely machine but also the same person.
    • The brain mostly reasons with linear phenomenon. Exponential growth is hard to predict intuitively.
    • Intelligence is the ability to solve problems with limited resources including time.
    • Discussion on the argument is often given that a computer with a rulebook for language who can perfectly use the language but does not “understand” it is not true intelligence. This is for me completely wrong as our brain does not “understand” language more that it can use it perfectly.
    • Fascination concept that the brain uses the same structures for processing, learning and storing information. It evens learn continuously as it is being used without a hard line between the learning and using of skills.