30. Paulk AC, Kfir Y, Khanna A, Mustroph M, Trautmann EM, Soper DJ, Stavisky SD, Welkenhuysen M, Dutta B, Shenoy KV, Hochberg LR, Richardson M, Williams ZM, Cash SS. (2022) Large- scale neural recordings with single neuron resolution using Neuropixels probes in human cortex. Nature Neuroscience. 25:252-263.
- Main paper pdf url doi
- Supplemental Materials pdf_supp_mats
- Supplementary video 1 (Movie of raw and interpolated Neuropixels data) mov
- Data on Dryad
- Code on Github and Code on Zenodo
29. Deo DR, Rezaii PG, Hochberg LR, Okamura AM, Shenoy KV*, Henderson JM* (2021) Effects of peripheral haptic feedback on Intracortical brain-computer interface control and associated sensory responses in motor cortex. IEEE Transactions on Haptics. 4:762-775. pdf url
28. Simeral JD, Hosman T, Saab J, Flesher SN, Vilela M, Franco B, Kelemen J, Brandman DM, Ciancibello JG, Rezaii PG, Rosler DM, Shenoy KV**, Henderson JM**, Nurmikko AV, Hochberg LR (2021) Home use of a wireless intracortical brain-computer interface by individuals with tetraplegia. IEEE Transactions in Biomedical Engineering. 68:2313-2325. pdf url
27. Willett FR, Avansino DT, Hochberg LR, Henderson JM*, Shenoy KV* (2021) High-performance brain-to-text communication via imagined handwriting. Nature. 593:249-254.
- Publication materials
- Nature Cover Caption: "Character Building. Brain–computer interfaces (BCIs) have the potential to restore communication to people who have lost the ability to move or speak. To date, the focus has largely been on motor skills such as reaching or grasping. In this week’s issue, Francis Willett and his colleagues present the results from an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time. The researchers worked with a man who is paralysed from the neck down, asking him to try to write by imagining he was holding a pen on a piece of paper. The BCI used a neural network to translate the neural signals into letters, allowing the man to reach a writing speed of 90 characters per minute with an accuracy of 94.1%. The cover features aggregated images of the alphabet derived from the study participant’s neural activity as he thought about writing. Cover image: K. Krause / Nature adapted from F. R. Willett et al. Nature 593, 249–254 (2021)."
- A bundle with all paper materials and a few select news pieces: (1) pdf 24 MB version and (2) pdf 8 MB Adobe Acrobat reduced file size version
- Main paper pdf url Supplementary material pdf url Peer review file. pdf url
- Video 1: Copying sentences in real-time with the handwriting brain-computer interface. In this video, participant T5 copies sentences displayed on a computer monitor with the handwriting-brain computer interface. When the red square on the monitor turns green, this cues T5 to begin copying the sentence. url
- Video 2: Hand micromotion while using the handwriting brain-computer interface. Participant T5 is paralyzed from the neck down (C4 ASIA C spinal cord injury) and only generates small micromotions of the hand when attempting to handwrite. T5 retains no useful hand function. url
- Video 3: Freely answering questions in real-time with the handwriting brain-computer interface. In this video, participant T5 answers questions that appear on a computer monitor using the handwriting brain-computer interface. T5 was instructed to take as much time as he wanted to formulate an answer, and then to write it as quickly as possible. url
- Video 4: Side-by-side comparison between the handwriting brain-computer interface and the prior state of the art for intracortical brain-computer interfaces. In a prior study (Pandarinath et al., 2017) participant T5 achieved the highest typing speed ever reported with an intracortical brain-computer interface (39 correct characters per minute using a point-and-click typing system). Here, we show an example sentence typed by T5 using the point-and-click system (shown on the bottom) and the new handwriting brain-computer interface (shown on the top), which is more than twice as fast. url
- News & Views
- Shared resources
- Please see News Articles for several press articles, videos and podcasts
- Altmetric score (> 4,400) url
26. Rastogi A, Willett FR, Abreu J, Crowder DC, Murphy B, Memberg WD, Vargas-Irwin CE, Miller JP, Sweet J, Walter BL, Rezaii PG, Stavisky SD, Hochberg LR, Shenoy KV, Henderson JM, Kirsch RF, Ajiboye AB (2021) The neural representation of force across grasp types in motor cortex of humans with tetraplegia. eNeuro 10.1523/ENEURO.0231-20.2020. pdf url
25. Wilson GH*, Stavisky SD*, Willett FR, Avansino DT, Kelemen JN, Hochberg LR, Henderson JM**, Druckmann S,** Shenoy KV** (2020) Decoding spoken English phonemes from intracortical electrode arrays in dorsal precentral gyrus. Journal of Neural Engineering. 17:066007 pdf url
24. Even-Chen N*, Muratore DG*, Stavisky SD, Hochberg LR, Henderson JM, Murmann B**, Shenoy KV** (2020) Power-saving design opportunities for wireless intracortical brain-computer interfaces. Nature Biomedical Engineering. 4:984-996. pdf supp_mats url
- Editorial (2020) The painstaking pace of bioelectronic interfaces. Nature Biomedical Engineering. 4:933–934. pdf url
- News & Views: Slutzky MW (2020) Increasing power efficiency. Nature Biomedical Engineering. 4:937–938 pdf url
- Associated paper: Nason SR, Vaskov AK, Willsey MS, Welle EJ, An H, Vu PP, Bullard AJ, Nu CS, Kao JC, Shenoy KV, Jang T, Kim H-S, Blaauw D, Patil PG, Chestek CA (2020) A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces. Nature Biomedical Engineering. 4:973–983. pdf url
23. Stavisky SD, Willett FR, Avansino DT, Hochberg LR, Shenoy KV**, Henderson JM** (2020) Speech-related dorsal motor cortex activity does not interfere with iBCI cursor control. Journal of Neural Engineering. 17:016049 (13pp). pdf url
22. Willett FR*, Deo DR*, Avansino DT, Rezaii PG, Hochberg LR, Henderson JM**, Shenoy KV** (2020) Hand knob area of motor cortex in people with tetraplegia represents the whole body in a compositional way. Cell. 181:396–409. pdf supp_mats url
- Supplemental video 1. Real-time, discrete neural decoding of attempted movements from among 16 possible directional movements spanning the wrists and ankles (related to Fig. 6).
- Supplemental video 2. Real-time, discrete neural decoding of attempted movements from among 32 possible movements spanning the hands, arms, feet, and legs from both sides of the body (related to Fig. 6).
21. Rastogi A, Vargas-Irwin C, Willett F, Abreu J, Crowder DC, Murphy B, Memberg W, Miller J, Sweet J, Walter B, Cash S, Rezaii PG, Franco B, Saab J, Stavisky SD, Shenoy KV**, Henderson J**, Hochberg LR, Kirsch R, Ajiboye AB (2020) Neural representation of observed, imagined, and attempted grasping force in motor cortex of individuals with chronic tetraplegia. Scientific Reports. 10:1429. pdf url
20. Stavisky SD, Willett FR, Wilson GH, Murphy BA, Rezaii PG, Avansino D, Memberg WD, Miller JP, Kirsch RF, Hochberg LR, Ajiboye AB, Druckmann S, Shenoy KV**, Henderson JM** (2019) Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis. eLife. 8:e46015. pdf figure-supplements url
19. Willett FR, Young DR, Murphy BA, Memberg WD, Blabe CH, Pandarinath C, Stavisky SD, Rezaii PG, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral J, Jarosiewicz B, Hochberg LR, Kirsch RF, Ajiboye AB (2019) Principled BCI decoder design and parameter selection using a feedback control model. Scientific Reports. 9(1):8881. pdf supp_mats url
18. Milekovic T, Bacher D, Sarma A, Simeral J, Saab J, Pandarinath C, Yvert B, Sorice B, Blabe C, Oakley E, Tringale K, Eskandar E, Cash S, Shenoy KV, Henderson JM, Hochberg LR, Donoghue JP (2019) Volitional control of single-electrode high gamma local field potentials (LFPs) by people with paralysis. Journal of Neurophysiology. 121:1428-1450. pdf url
17. Young D, Willett F, Memberg W, Murphy B, Rezaii PG, Walter B, Sweet J, Miller J, Shenoy KV, Hochberg LR, Kirsch R, Ajiboye AB (2019) Closed-loop cortical control of virtual reach and posture using cartesian and joint velocity commands. Journal of Neural Engineering. 16:026011 (14pp). pdf url
16. Nuyujukian P*, Sanabria JA*, Saab J*, Pandarinath C, Jarosiewicz B, Blabe C, Franco B, Mernoff ST, Eskandar EN, Simeral JD, Hochberg LR**, Shenoy KV**, Henderson JM** (2018) Cortical control of a tablet computer by people with paralysis. PLoS One. 13:e0204566 pdf url
15. Stavisky SD, Kao JC, Nuyujukian P, Pandarinath C, Blabe C, Ryu SI, Hochberg LR, Henderson JM, Shenoy KV (2018) Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements. Scientific Reports.8:1635.7 pdf url
14. Pandarinath C, O'Shea DJ, Collins J, Jozefowicz R, Stavisky SD, Kao JC, Trautmann EM, Kaufman MT, Ryu SI, Hochberg LR, Henderson JM, Shenoy KV, Abbott LF, Sussillo D (2018) Inferring single-trial neural population dynamics using sequential auto-encoders. Nature Methods.15:805-815. pdf supp_mats url
- Batista AP, DiCarlo JJ (2018) Deep learning reaches the motor system. Nature Methods. News & Views. 15:772-773. pdf
- Seminar talk: Sussillo D (3/222018) LFADS seminar talk, Simons Institute for the Theory of Computing meeting, UC Berkeley. Video. url
- SuppVideo1 Generator initial states inferred by LFADS are organized with respect to kinematics of the upcoming reach. The video depicts the initial conditions vectors for each individual trial of the ‘Maze’ reaching task for monkey J, mapped onto a low-dimensional space (3D) via t-SNE (as in Fig. 2c). Each point represents the initial conditions vector for an individual trial (2,296 trials are shown). Colors denote the angle of the endpoint of the upcoming reach (colors shown in Fig. 2a), and marker types denote the curvature of the reach (circles, squares, and triangles for straight, counter-clockwise curved, and clockwise curved reaches, respectively). As shown, the initial conditions exhibit similarity for trials with similar kinematic trajectories (both for trials whose reach endpoints have similar angles and for trials with similar reach curvature). Since structure in the initial conditions implies structure at the level of the generator’s dynamics, this analysis implies that LFADS produces dynamic trajectories that show similarity based on the kinematics of the reach type for a given trial, despite LFADS not having any information about reaching conditions.
- SuppVideo2 LFADS reveals consistent rotational dynamics on individual trials. The video contains two sequential movies showing the trajectories in neural population state space during individual reach trials for monkey J (Fig. 3). The first movie illustrates the single-trial trajectories uncovered by smoothing the data with a Gaussian kernel. The second movie illustrates single-trial trajectories uncovered by LFADS. 2,296 trials are shown, representing the 108 conditions of the ‘Maze’ task.
- SuppVideo3 Multisession LFADS finds consistent representations for individual trials across sessions. The video contains six sequential movies showing the trajectories in state space during individual reach trials for monkey P (Fig. 4). The first video shows single-trial GPFA factor trajectories for all trials estimated for a single session. The second and third videos show single-trial LFADS factor trajectories estimated from all trials using a single-session model and from the stitched model, respectively. The fourth, fifth, and sixth repeat this sequence but show single-trial trajectories for 42/44 sessions (2 were omitted for ease of presentation). Colors represent eight reach directions. Multisession movies include approximately 14,500 trials, 38 separate electrode penetration sites and spanned 162 d from the first to the last session. Each trajectory begins at the go cue and proceeds for 510 ms into movement, which occurs at varying times due to reaction time variability. For GPFA and single-session. LFADS factors, the trajectories from individual sessions were concatenated and the projection yielding the CIS and first jPCA plane was estimated (Methods). This provided a set of common trajectories against which each individual session’s data were regressed. These regression coefficients provided projections of the individual sessions’ trajectories that were maximally similar to the common trajectories. In contrast, for stitched LFADS factors, we simply estimated the projection yielding the CIS and first jPCA plane from all of the sessions together, as the factors are already in shared space.
- GitHub LFADS run manager
- GitHub LFADS
13. Milekovic T, Sarma A, Bacher D, Simeral J, Saab J, Pandarinath C, Sorice B, Blabe C, Oakley E, Tringale K, Eskandar E, Cash S, Henderson JM, Shenoy KV, Donoghue JP, Hochberg LR (2018) Stable long-term BCI-enabled communication in ALS and locked-in syndrome using LFP signals. Journal of Neurophysiology. 120:343-360. pdf url
12. Willett FR, Murphy BA, Young D, Memberg WD, Blabe CH, Pandarinath C, Franco B, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral JD, Jarosiewicz B, Hochberg LR, Kirsch RF, Ajiboye AB (2018) A comparison of intention estimation methods for decoder calibration in intracortical brain-computer interfaces. IEEE Transactions in Biomedical Engineering. 65:2066-2078. pdf url
11. Even-Chen N, Stavisky SD, Pandarinath C, Nuyujukian P, Blabe CH, Hochberg LR, Henderson* JM, Shenoy* KV (2018) Feasibility of automatic error detect-and-undo system in human intracortical brain-computer interfaces. IEEE Transactions in Biomedical Engineering. 65:1771-1784. pdf url
10. Brandman D, Hosman T, Saab J, Burkhart M, Shanahan B, Ciancibello J, Sarma A, Milstein D, Vargas-Irwin C, Franco B, Kelemen J, Blabe C, Murphy B, Young D, Willett F, Pandarinath C, Stavisky S, Kirsch R, Walter B, Ajiboye A, Cash S, Eskandar E, Miller J, Sweet J, Shenoy KV, Henderson JM, Jarosiewicz B, Harrison M, Simeral J, Hochberg, LR (2018) Rapid calibration of an intracortical brain computer interface for people with tetraplegia. Journal of Neural Engineering. 15:026007. pdf url
9. Even-Chen N, Stavisky S, Kao J, Ryu SI, Shenoy KV (2017) Augmenting intracortical brain-machine interface with neurally driven error detectors. Journal of Neural Engineering. 14:066007 (16pp). pdf url
8. Pandarinath C*, Nuyujukian P*, Blabe CH, Sorice B, Saab J, Willett F, Hochberg LR, Shenoy KV**, Henderson JM** (2017) High performance communication by people with paralysis using an intracortical brain-computer interface. eLife. 6:e18554 pdf url
7. Willett FR, Murphy B, Memberg W, Blabe C, Pandarinath C, Walter B, Sweet J, Miller J, Henderson JM, Shenoy KV, Hochberg LR, Kirsch R, Ajiboye AB (2017) Signal-independent noise in intracortical brain-computer interfaces causes movement time properties inconsistent with Fitts' law. Journal of Neural Engineering. 14:026010. pdf
6. Willett F, Pandarinath C, Jarosiewicz B, Murphy B, Memberg W, Blabe C, Saab J, Walter B, Sweet J, Miller J, Henderson J, Shenoy KV, Simeral J, Hochberg LR, Kirsch R, Ajiboye AB. (2017) Feedback control policies employed by people using intracortical brain-computer interfaces. Journal of Neural Engineering. 14:016001 (16pp). pdf
5. Jarosiewicz B, Sarma AA, Bacher D, Masse NY, Simeral JD, Sorice B, Oakley EM, Blabe C, Pandarinath C, Gilja V, Cash SS, Eskandar E, Friehs G, Henderson JM, Shenoy KV, Donoghue JP, Hochberg LR (2015) Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Science Translational Medicine. 7:1-10. pdf supp_mats
4. Gilja V*, Pandarinath C*, Blabe CH, Nuyujukian P, Simeral JD, Sarma AA, Sorice BL, Perge JA, Jarosiewicz B, Hochberg LR, Shenoy KV**, Henderson JM** (2015) Clinical translation of a high performance neural prosthesis. Nature Medicine. 21:1142-1145. pdf url
- Composite video of representative Radial-8 and mFitts1 task trials from participant T6. Video1.mp4
- Composite video of representative Radial-8 and mFitts1 task trials from participant T7. Video2.mp4
- A free-pace free-choice typing task with the Dasher keyboard interface. Video3.mp4
3. Blabe C, Gilja V, Chestek CA, Shenoy KV, Anderson K, Henderson JM (2015) Assessment of brain-machine interfaces from the perspective of people with paralysis. Journal of Neural Engineering. 12:043002. pdf
2. Pandarinath C, Gilja V, Blabe CH, Nuyujukian P, Sarma AA, Sorice BL, Eskandar EN, Hochberg LR, Henderson JM*, Shenoy KV* (2015) Neural population dynamics in human motor cortex during movements in people with ALS. eLife. 4:e07436. pdf video1
1. Chestek CA, Gilja V, Blabe CH, Foster BL, Shenoy KV, Parvizi J, Henderson JM (2013) Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas. Journal of Neural Engineering. 10:02602 (11pp). pdf