Imagine taking statistics on the connectivity of transistors in a Pentium chip and then trying to make your own chip based on those statistics. The Visual6502 team reverse-engineered one of the chips used in the early Atari video game system in this manner using connectivity statistics.
Some neuroscientists tried to use techniques from neuroscience to reverse engineer Atari chips. They couldn’t. They are able to find transistors which uniquely crash one of the games but not the others. Here is the abstract of their paper:
There is a popular belief in neuroscience that we are primarily data limited, that producing large, multimodal, and complex datasets will, enabled by data analysis algorithms, lead to fundamental insights into the way the brain processes information. Microprocessors are among those artificial information processing systems that are both complex and that we understand at all levels, from the overall logical flow, via logical gates, to the dynamics of transistors. Here we take a simulated classical microprocessor as a model organism, and use our ability to perform arbitrary experiments on it to see if popular data analysis methods from neuroscience can elucidate the way it processes information. We show that the approaches reveal interesting structure in the data but do not meaningfully describe the hierarchy of information processing in the processor. This suggests that current approaches in neuroscience may fall short of producing meaningful models of the brain.
Even unlimited data would not help, they say, with the tools and techniques we’re using. You can get as much “behavioral” and phenotypic data as you want from a 6502 chip, but it doesn’t help. Trying to learn what’s going on from the equivalent of brain lesions was, for example, not too informative.
Lesions studies allow us to study the causal effect of removing a part of the system. We thus chose a number of transistors and asked if they are necessary for each of the behaviors of the processor (figure 4. In other words, we asked if removed each transistor, if the processor would then still boot the game. Indeed, we found a subset of transistors that makes one of the behaviors (games) impossible. We might thus conclude they are uniquely responsible for the game – perhaps there is a Donkey Kong transistor or a Space Invaders transistor. Even if we can lesion each individual transistor, we do not get much closer to an understanding of how the processor really works.
This finding of course is grossly misleading. The transistors are not specific to any one behavior or game but rather implement simple functions, like full adders. The finding that some of them are important while others are not for a given game is only indirectly indicative of the transistor’s role and is unlikely to generalize to other games. Lazebnik  made similar observations about this approach in molecular biology, suggesting biologists would obtain a large number of identical radios and shoot them with metal particles at short range, attempting to identify which damaged components gave rise to which broken phenotype.
Ray Kurzweil, to pick one bold futurist, has us by 2019 with a much (much!) better understanding of the brain than we currently have. Things are really going to have to pick up, let me tell you.
According to Kurzweil, we are less than a decade away from nanobot-driven flying cars (whatever that means)! Makes me wonder if these futurists are aware of the state of nanotech. Five years ago we could make nanorods. Now we can make… hollow nanorods? Nanobots should be just around the corner, surely!
There are a few more problems with interpreting a wiring diagram of the brain (assuming one could be obtained).
- The wiring diagram is not static– we know that synapses are constantly being formed and destroyed
- Synapses can be excitatory or inhibitory. Electron micrographs of synapses can’t tell one from the other
- The neurotransmitters of greatest interest to thought, emotion and mood — norepinephrine, dopamine, serotonin — are released diffusely into the brain extracellular fluid, not locally at synapses.
For more on these points please see here.
Not only those issues but changes in synaptic strength, which would not be visible, are where much of the rubber meets the road.
Even with simple, 14-cell networks (see the Crustacean STG as one example), we still have difficulty in replicating their output, even knowing their wiring diagram, firing properties, ionic conductances, and effect of neuromodulators on many of the points in the diagram.