Ministry of defence in America:
Interactive Computation at BioInfoMicro Interface
Interactive Neuronal and Nanoelectronic/photonic circuits J. Anderson, B. Connors, J. Donoghue, B. Kimia, A. Nurmikko, J. Xu Brown University, Providence RI 02912
Goals:
Coupling of Brain to Micro/Optoelectronic Chips: Approved for Public Release, Distribution Unlimited: 01-S-1092 Computing/Information Processing Technology ca. 2030?
If "invent by increment" :Today’s digital microelectronic computer: Si-based, 0.18
mm) basic hardware architecture "locked-in" Moore’s law (ca. 2020)?: limits to microprocessor size/speed Molecular electronics (self assembly) device size (CMOS) interconnects packaging & dissipation conductors (DNA wire?) switches, active devices on molecular scale ? Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Photonic superchip? (Photonic Superchip: Ultrafast Signal Router/Processor• aim for ultrafast all-optical packet and binary switching• aim for wide wavelength range on chip-performance• aim for large arrays >1000x1000Future Computing Paradigms ?
Two examples:
•
coherent superposition (phase)entanglement (many spins) quantum parallellism
•
Spatially distributed (global) information processing internet and optical networks/communication
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface
"our paradigm": Biological processor Nanoelectronic/photonic processor Interaction and physical integration Neural network (e.g. brain) Nanoscience/nanotechnology:
as a massively parallel, spatially distributed analog computer large-scale non-silicon based interactive, monolithic transducer/receiver arrays
• 6 faculty, 2-3 senior staff, 10-12 Ph.D. students (joint advising)
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Develop massively parallel interfaces for (i) studying neuronal connectivity and spatial organization; (ii) mapping that information onto nanoelectronic/photonic chips for study of collective features to look for new computational functions Brown Bio/Info/Micro Interdisciplinary Team:
• Institute for Brain Science, Laboratory of Engineering and Man Made Systems, Center for Advanced Materials Research
• Central Research Facilities
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface
Technical Structure and Task Organization of Program: Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Target Milestones
Year 1:
·Demonstration of electrical recording from cortex slices by carbon nanotube electrodes ·Real time imaging of cortex slices by fluorescence optical probes based on compact semiconductor micro-optical light emitters/detectors ·Development of signal acquisition and information processing strategies for the manmade arrays
Year 2:
·Fabrication of carbon nanotube arrays and initial studies of synaptic correlations with these novel nanoprobes ·Fabrication of high density arrays of blue/green/UV LEDs and photodiodes and development of prototype high
spatiotemporal resolution imaging of cortical activity ·Studies of neuronal activity by the nanoelectronic/microphotonic arrays ·Development of information theories of spatially distributed computing
Year 3:
·Demonstration of carbon nanotube arrays as spatially distributed, interactive sensors/transducers of cerebral cortex; proof of concept experiments in spatially distributed computation. ·Demonstration of microphotonic arrays as spatially distributed, interactive sensors/transducers of cerebral cortex; proof of concept experiments in spatially distributed computation. ·Connecting the theory concepts of spatially distributed computing with initial input from experiments
Year 4:
·Integration of nanoelectronic and microphotonic sensor/transducer arrays ·Incorporation of results from theoretical models to guide the experimental system design
Year 5:
·Demonstration of a proof-of-concept computational device at the Bio/Info/Micro interface with integration of cerebral cortex and nanoelectronics/microphotonics
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Technical Example: Recording CortexRecording and Decoding of Neuronal Ensembles in Motor Cortex Surgical implantation using a pneumatic Si microelectrode, tip ~ ì m dia
impulse inserter
Chronic intracortical multi-electrode array(University of Utah; Bionics Technologies, Inc)
We have performed 18 implants in either primary motor cortex (MI) or supplementary motor cortex (SMA)
(Donoghue, Hatsopoulos et al) Simultaneously RecordedExtracellular Action Potentials(Mean/Standard Deviation)
Channel: 42Channel: 43Channel: 52Channel: 53Channel: 54Channel: 75Channel: 85Channel: 92Channel: 9390 mmV-1.5-0.50.5020406002040020400501000510020400102001020010203003060020406005010002040050100movement onsetfrequency (Hz)time (s)rostralcaudalmediallateralActivity profiles (peri-movementhistograms) from 13 Simultaneously recorded neurons Coupling of Brain to Micro/Optoelectronic Chips:Interactive Computation at BioInfoMicroBioInfoMicro Interface Interface(Donoghue, Hatsopoulos et al)
Decoding of Discrete and Continuous Movements of the Arm Movements to Visual Targets
Y position (cm)
reconstruction actual position
17 16 15 14 13 12 11 10
9 10111213141516
18
X position (cm)
• A chronic physical interface with neural populations is feasible. • Signals from randomly selected motor cortical neurons provide extensive information about discrete motor behaviors as well as continuous movement parameters (hand trajectory) • Decoded cortical signals can be used to drive physical devices (robot arms, computer cursors) in ~real time. Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicro Interface
microelectrode
Technical Example: Electrically coupled networks of neurons: Inhibition, synchronyInhibition, rhythms, and synchrony neocortex (linear array) 20 ì m RS RS LTS
thalamocortical slice fast spiking in the recording chamber interneuron (excitatory)
Information dynamically represented by correlated pattern of activity in small groups of neurons
The activated, synchronized LTS network generates stronginhibitory potentials in other neurons in the local circuit (top);inhibition is itself synchronized across the circuit (bottom).
+ACPD
c
LTS
a 0.6
0.0
20 2 mV FS -0.6 250 ms LTS vs FS
-400 -200 0 200 400
Time (ms)
d
e 0.8
RS R S 1 mV 0.6
0.0
-0.6 RS vs R S -400 -200 0 200 400 (2000)
Cross-correlation Cross-corr
elation Time (ms) Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface 0.80.60.40.20.0-0.2-0.4-0.62mV2s5002500-250-5000246Time(s)10:m300:m0246600:mTime(ms)Normalizedcross-correlation6024aRSRSCoupling of Brain to Micro/Optoelectronic Chips:Coupling Interactive Computation at BioInfoMicroBioInfoMicro Interface InterfaceSynchrony of rhythmic inhibition ranges over a wider spatial domain of cortex thandoes irregular inhibitory patternsCoupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicro Interface Technical Example: Nanoelectronic ProbesNanoelectronic Interactive Probes in andin Coupled Neural and Nano Systems
•
new generation of large area, high spatial resolution electrical (potential) probes
Neuronal Probing and Probe Design using Carbon Nanotubes Arrays
• In vivo recordings. • In vitro recordings. • High spatial resolution, broad area coverage. compatibility with Si electronics, high conductivity, strength, chemical inertness, and flexibility Explorations of Systems Couplings • Neuronal correlation / Passive recordings. • Active recordings via neuronal stimulation • Bio-neuronal and Nanoelectronic Intersystem Super-Coupling
(J. Xu et al)
Collective Behavior Nanoelectronic Nanoelectronic under the influence of imported bio neural connectivity Non Von Neumann Computing ? AlAuwires1-3um100 umSiO2CNTsTi (20A)/NiFe/100A/Au(150A)Ti (150A)/Au (2000A)SiO2Ti/NiFe/AucontactsCoupling of Brain to Micro/Optoelectronic Chips:Coupling Interactive Computation at BioInfoMicroBioInfoMicro Interface Interface(J. Xu et al)
Probe Design: Two Approaches
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface
System Design: The Probe at the Bio/Nanoelectro Interphase
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Technical Example: MicrophotonicMicrophotonic Arrays for Interactive CircuitryInteractive Imaging of Cortical Circuitry
Goal
interactions in context of parallel processing
blue/NUV compact semiconductorlight emitters (LEDs, diode lasers)
LED/laser and photodiode arrays:
• High spatial resolution (<10ì m)
• High speed (<<msec real time)
• Large area arrays (>mm2
establish a two-way "wireless" communication between neural networksand high speed, large scale optoelectronic probe/excite arrays (Nurmikko et al)transducer array imaging array • new (and unique) technology element: • study and exploit collective, long range )
• monolithic integration of ultracompact microphotonic transducer/receiver arrays
(b) Optical probing of neuronal circuits
• voltage sensitive dyes
• Ca2+
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Current Use of "Photonics" in Neurobiology
(a) Imaging of action potentials of single neurons
sensitive indicators
(c) Photostimulation of neuronal activity (e.g. photolysis of caged glutamate)
S. Antic et al J. Neurophys. 82, 1615 (1999): vertebrate neurons in brain slices
need of compact NUV/deep blue light sources
Our approach: compact, high intensity, programmable arrays of planar GaN-based NUV/blue/green semiconductor LEDs and lasers Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Blue/NUV Light Emitting Diodes: ultracompact sources for fluorescence imaging and photoexcitation of neurons • InGaN/GaN quantum well semi-conductor heterostructures • planar processing technology • wavelength range 370-520 nm • employed in time resolved spectroscopy (cultured cells) • output powers up to >10 mW (20 micron diameter) • current progress towards microcavity devices: RCLED and VCSELs for added spatial and temporal coherence • compatible with large array design and processing presently: 1024 element array Rudimentary Array of Blue LEDs (20 ì m dia; ë=460 nm)
Next Generation of Blue LED Arrays for Cortical Imaging
• 10 ì m individual device diameter, 50 ì m spacing, ~ 1.5 mm
2
total area
LED array layout
• 32x32 element individually matrix addressable array (1024 LEDs)
Electrical (contact) layout
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface
Anderson, Kimia
Technical Example: Computation with Massive Parallelism: The Nervous System and NanostructuresNanostructures
Present Status and Issue:
Objective: Modeling Neuronal Activity and Spatial Computation while developing needed theoretical support to the analysis and understanding of experimental input provided by the new sensor/transducer nanoarrays
rare for high level neural theory to interact with physical level experimental observationsdue to lack of detailed spatial and temporal recording over a large neuron ensembleWe have two types of modeling approaches to spatially extended computation: both get most of their power from the lateral propagation of information and formation of "interference patterns" when propagating information collides (a) "network of networks" (Anderson): an array of elementary units that are small nonlinear attractor neural networks and interact locally with neighbors (b) "shockwave" based spatial computational technique (Kimia) for object recognition as means of forming "symmetry based" representations Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface
Notes (general):
Traditional computers do not compute what we do: Different hardware leads to different engineering solutions.
• Computers are good at: Excel spreadsheets, bank balances, boring detail (trees)
• Human-like computers are good at: intuition, association, plausible inference (forests).
Nanostructures or any brain-like computational architecture will build "human like" computers.
One current approach to "cognitive computation" uses properties of dynamical systems with attractors as a way to do computation.
• Neurons (single units) are not the elementary unit of neural computation but groups of neurons are.
• Small attractor networks are the basic functional units
(single units are only of interest in as they give rise to the distributed activity patterns in the attractor networks)
• Attractor networks connect and states in one network "interference patterns" formed when patterns collide become key information processing elements. They are formed from combinations of lower level features
We have developed a computational paradigm for deriving and representing
for vision that relies on
"Shock wave" model: Technique
• Wave Propagation (Eikonal Equation) spatial relationships among edge
• Interference Patterns or Shock Waves elements
• Transformation of these patterns
• Detection of optimal paths of transformation
The need to organize "geometrically" related structure is not unique to vision, but also
applies to other domains such
as touch and sound, as well as
motor maps.
Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface Radar Signal Classifier Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation atInteractive at BioInfoMicroBioInfoMicro InterfaceInterface : Who is watching me? The "Network of Networks" ( (J. Anderson, and J. Sutton, Harvard) Coupling of Brain to Micro/Optoelectronic Chips: Interactive Computation at BioInfoMicroBioInfoMicro Interface • discretize underlying geometry • architecture found e.g. cortical colums (mammals propagate into neighbors. Wave-ParadigmWave-Propagation: A Computational Paradigm (B. Kimia) Interactive Computation at BioInfoMicro Interface The Neural Connection
•
The proposed framework requires intra-layer "horizontal" activity as well as inter-layer "vertical" activity.
•
Intra-layer wave propagation proposed here is intriguingly consistent with existing psychophysical evidence and neurophysiological recordings.
•
or parabolic PDE
Traditional Receptive Field (RF) models is equivalent to "convolution" Coupling of Brain to Micro/Optoelectronic Chips: models. Interactive Computation at BioInfoMicro Interface
Goals
•
Parallel implementations of wave propagation and corresponding experiments take into account neuronal circuit constraint
•
- evidence of interference pattern
- shock wave propagation
- role of scale
Provide a model to seek dynamic activity in large scale recording
•
- account for propagation velocity variations
- dependence on local context
- region based segmentation
Extend model to spatially variant Eikonal equation
•
Interactions between the "shockwave" and the "network of networks" models.
Coupling of Brain to Micro/Optoelectronic Chips:
:
(1) Aim at discovery and implementation of new computational paradigms acquired from interaction of a biological processor (brain) and man-made nanoscale device arrays, with emphasis on collective phenomena.
(2) Develop next generation of interactive, "smart" nanoprobe-based sensor array technology with ultra- high spatio/temporal resolution for a broad range of neuroscience imaging applications.
(3) Develop new theoretical approaches that bridge neuroscience and computer engineering, with emphasis on spatially distributed computing
