Award Abstract # 1734887
NCS-FO: A Computational Theory to Model the Neurobiological Basis of a Visuo-Cognitive Neuroprosthetic

NSF Org: SMA
SBE Office of Multidisciplinary Activities
Recipient: THE RESEARCH FOUNDATION FOR THE STATE UNIVERSITY OF NEW YORK
Initial Amendment Date: August 7, 2017
Latest Amendment Date: August 7, 2017
Award Number: 1734887
Award Instrument: Standard Grant
Program Manager: Dwight Kravitz
dkravitz@nsf.gov
 (703)292-4502
SMA
 SBE Office of Multidisciplinary Activities
SBE
 Directorate for Social, Behavioral and Economic Sciences
Start Date: August 1, 2017
End Date: July 31, 2022 (Estimated)
Total Intended Award Amount: $949,928.00
Total Awarded Amount to Date: $949,928.00
Funds Obligated to Date: FY 2017 = $949,928.00
History of Investigator:
  • Stephen Macknik (Principal Investigator)
    macknik@neuralcorrelate.com
  • Susana Martinez-Conde (Co-Principal Investigator)
Recipient Sponsored Research Office: SUNY Health Science Center at Brooklyn
450 CLARKSON AVE
BROOKLYN
NY  US  11203-2012
(718)270-2680
Sponsor Congressional District: 09
Primary Place of Performance: SUNY Health Science Center at Brooklyn
450 Clarkson Avenue
Brooklyn
NY  US  11203-2012
Primary Place of Performance
Congressional District:
09
Unique Entity Identifier (UEI): NJ14V2NZYM68
Parent UEI:
NSF Program(s): IntgStrat Undst Neurl&Cogn Sys
Primary Program Source: 01001718DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 8089, 8091, 8551
Program Element Code(s): 862400
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.075

ABSTRACT

Evoking high quality visual perception in a blind person, via direct microstimulation of the brain, poses great difficulties. One major obstacle has been that electrical stimulation of the brain typically affects neuronal populations that are mutually suppressive, which subverts proper neuronal signaling. The visual system has two antagonistic information channels that encode either the perception of lightness, in ON cells, or darkness, in OFF cells. Inappropriate coactivation of these two channels results in nullification of contrast, and deprived visual perception. It follows that high-quality prosthetic stimulation systems must avoid unwanted coactivation of mutually suppressive neurons, just as the natural visual system does. This is a challenge because the antagonistic neurons typically lie within microns of each other in the brain. The project aims at transformative advances in viral transfection and imaging methodology, computational theory, and cortical prosthetic neuroengineering design for the purpose of restoring vision by genetically modifying neurons in the brain and then stimulating them with light, a method called optogenetics. The expected results and methodology will form the scientific basis to build a breakthrough neuroprosthetic, with transformative potential to further brain research in sensory, motor, and cognitive parts of the cortex and to advance human medicine. To promote the development and availability of derived products to the public, the team will disseminate the discoveries to general audiences through public lectures and publications in popular science magazines. The investigators will supervise trainees from underrepresented groups, including postdoctoral fellows, graduate students, undergraduates, and high school students. The investigators are faculty mentors for The Children's Aid Society (CAS) Workforce Development Department Summer Youth Employment Program (SYEP), which provides summer research opportunities to disadvantaged and minority youth in NYC to inspire them to pursue STEM careers.

Recent research has shown that, for any given retinal position, the ON and OFF cell inputs to the brain's visual cortex are purely excitatory, concentrate in a specific layer, and are laid out in a pattern that can be targeted with light from outside the brain. First, the team will modify these neurons genetically, to turn them into a novel type of photoreceptor, embedded within the brain. The team will then target light stimulation to the identified ON and OFF cells, determining the precise balance of activation to either channel to generate high-quality prosthetic vision based on a video camera's signal. This technology can then be used to bypass the eye to stimulate the brain from the camera. The project aims to develop the computational model to drive an optogenetic brain stimulation system that will optimally activate neural responses in the primary visual cortex. By comparing the neuronal responses of sighted nonhuman primates viewing natural visual stimuli to prosthetic responses in the same neurons, the work will optimize stimulation patterns that evoke naturalistic visual perception. The balanced targeting of appropriate ON and OFF inputs at each position in visual space is expected to achieve maximal contrast perception at the highest attainable acuity, with full stereoscopic binocular vision. The team's computational model of spatiotemporal visual inputs into the cortex will also account for the effects of eye movements on early visual responses, a novel approach to visual prosthetics tested here for the first time.

This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 34)
Alexander, Robert and Mintz, Ronald and Custodio, Paul and Macknik, Stephen and Gindina, Sofya and Martinez-Conde, Susana "Gaze behavior during the averted detection of a simulated faint star" Journal of Vision , v.17 , 2017 https://doi.org/10.1167/17.10.1186 Citation Details
Alexander, Robert and Venkatakrishnan, Ashwin and Chanovas, Jordi and Ferguson, Sophie and Macknik, Stephen and Martinez-Conde, Susana "Why did Rubens add a parrot to Titians Fall of Man? An eye tracking investigation reveals attentional focus while viewing Italian Renaissance paintings" Journal of Vision , v.22 , 2022 https://doi.org/10.1167/jov.22.14.3162 Citation Details
Alexander, Robert and Waite, Stephen and Bruno, Michael A. and Krupinski, Elizabeth A. and Berlin, Leonard and Macknik, Stephen and Martinez-Conde, Susana "Mandating Limits on Workload, Duty, and Speed in Radiology" Radiology , v.304 , 2022 https://doi.org/10.1148/radiol.212631 Citation Details
Alexander, Robert G and Macknik, Stephen L and Martinez-Conde, Susana "Microsaccades in applied environments: Real-world applications of fixational eye movement measurements" Journal of Eye Movement Research , v.12 , 2020 https://doi.org/10.16910/jemr.12.6.15 Citation Details
Alexander, Robert G. and Macknik, Stephen L. and Martinez-Conde, Susana "Microsaccade Characteristics in Neurological and Ophthalmic Disease" Frontiers in Neurology , v.9 , 2018 https://doi.org/10.3389/fneur.2018.00144 Citation Details
Alexander, Robert G. and Macknik, Stephen L. and Martinez-Conde, Susana "What the Neuroscience and Psychology of Magic Reveal about Misinformation" Publications , v.10 , 2022 https://doi.org/10.3390/publications10040033 Citation Details
Alexander, Robert G. and Mintz, Ronald J. and Custodio, Paul J. and Macknik, Stephen L. and Vaziri, Alipasha and Venkatakrishnan, Ashwin and Gindina, Sofya and MartinezConde, Susana "Gaze mechanisms enabling the detection of faint stars in the night sky" European Journal of Neuroscience , v.54 , 2021 https://doi.org/10.1111/ejn.15335 Citation Details
Alexander, Robert G. and Venkatakrishnan, Ashwin and Chanovas, Jordi and Ferguson, Sophie and Macknik, Stephen L. and Martinez-Conde, Susana "Why did Rubens add a parrot to Titian's <i>The Fall of Man</i> ? A pictorial manipulation of joint attention" Journal of Vision , v.24 , 2024 https://doi.org/10.1167/jov.24.4.1 Citation Details
Alexander, Robert G. and Venkatakrishnan, Ashwin and Chanovas, Jordi and Macknik, Stephen L. and Martinez-Conde, Susana "Microsaccade dynamics mediate perceptual alternation in Monets Impression, Sunrise." Society for Neuroscience Conference 2018 , 2018 Citation Details
Alexander, Robert G. and Venkatakrishnan, Ashwin and Chanovas, Jordi and Macknik, Stephen L. and Martinez-Conde, Susana "Microsaccades mediate perceptual alternations in Monets Impression, sunrise" Scientific Reports , v.11 , 2021 https://doi.org/10.1038/s41598-021-82222-3 Citation Details
Alexander, Robert G. and Waite, Stephen and Macknik, Stephen L. and Martinez-Conde, Susana "What do radiologists look for? Advances and limitations of perceptual learning in radiologic search" Journal of Vision , v.20 , 2020 https://doi.org/10.1167/jov.20.10.17 Citation Details
(Showing: 1 - 10 of 34)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

We developed an optogenetic technology to stimulate connections within the visual pathways of the brain, in which we genetically turned brain cells within the visual system into photoreceptors, so that we could stimulate them with a video projector to produce artificial vision that did not require input from the retinas. The system included a novel brain activity measurement system using genetically encoded multicolor bioluminescence in V1 cells, suitable for use as a real-time control feedback mechanism to adjust brain stimulation using light for optimized power input. We have designed the initial nanophotonic hardware—based on our empirical findings and simulations—to combine light emission and detection in a novel planar photonic chip for implantation into the brain. Through experimentation and modeling of light scatter in the cortex (modeled as a scattering medium) we have achieved the proof of concept for these tools, and shown that their combined stimulation/detection usage is suitable for implementation as an implantable prosthetic device to restore vision in blind patients. The Optogenetic Brain System (OBServ) is an integrated nanophotonic implantable device that aims to restore foveal vision in the blind by mimicking naturalistic visual input patterns. By transducing brain neurons that normally receive input from the optic nerves--which is unavailable in certain forms of blindness--neurons with light-sensitive proteins, the system can stimulate the connections that enter the visual cortex without unwanted co-activation of inhibitory neurons, theoretically resulting in maximal contrast sensitivity. By targeting these specific localized brain inputs, naturalistic vision can theoretically be restored by providing a pattern of stimulation that mimics natural visual input. This project provided the proofs of concept suggesting that the project can in theory function. OBServ will track eye movements in the blind patients to account for oculomotor effects by adjusting the contemporaneous stimulation of the brain inputs that we artificially stimulate to mimic the effects eye movements. This prosthetic technology does break through the membrane covering the brain and holds the promise of restoring vision in the blind at the highest attainable acuity, with maximal contrast sensitivity while being the most non-invasive implant on the market.

 


Last Modified: 07/31/2023
Modified by: Stephen L Macknik

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page