US 20210138249A1

INI
( 19 ) United States
( 12 ) Patent Application Publication ( 10) Pub. No .: US 2021/0138249 A1
Howard (43 ) Pub. Date : May 13 , 2021
( 54 ) SYSTEM AND METHOD FOR NEURAL 2019 , provisional application No. 62 /896,563, filed
STIMULATION USING SPIKE FREQUENCY on Sep. 5 , 2019 , provisional application No. 62/896 ,
MODULATION 571 , filed on Sep. 5 , 2019 , provisional application No.
62 / 912,515 , filed on Oct. 8 , 2019 .
( 71 ) Applicant: Newton Howard , Providence, RI (US ) Publication Classification
( 72 ) Inventor: Newton Howard , Providence, RI (US ) (51 ) Int. Ci .
AWIN 1/36 (2006.01 )
( 21 ) Appl. No .: 16 /944,963 A61N 5/06 ( 2006.01 )
A61N 1/372 (2006.01 )
(22 ) Filed : Jul . 31 , 2020 G16H 20/30 (2006.01 )
( 52 ) U.S. CI .
Related U.S. Application Data CPC A61N 1/36139 (2013.01 ) ; A61N 5/0622
( 63 ) Continuation - in -part of application No. 15 / 988,315 , (2013.01 ) ; A61N 1/36067 ( 2013.01 ) ; G16H
filed on May 24 , 2018 , which is a continuation -in -part 20/30 (2018.01 ) ; A61N 1136071 ( 2013.01 ) ;
of application No. 15 /495,959 , filed on Apr. 24 , 2017 , A61N 1/37252 ( 2013.01 ) ; A61N 1/36082
Continuation - in -part of application No. 16 / 545,205 , ( 2013.01 )
filed on Aug. 20 , 2019 , Continuation -in -part of ap ( 57 ) ABSTRACT
plication No. 16/ 785,969 , filed on Feb. 10 , 2020 . Embodiments may comprise receiving electrical and optical
( 60 ) Provisional application No. 62 /326,007, filed on Apr. signals from electrophysiological neural signals of neural
22 , 2016 , provisional application No. 62 /353,343 , tissue from at least one read modality, wherein the electro
filed on Jun . 22 , 2016 , provisional application No. physiological neural signals are at least one of Spike fre
62/ 397,474 , filed on Sep. 21 , 2016 , provisional ap quency modulated or Spike frequency demodulated, encod
plication No. 62 / 511,532 , filed on May 26 , 2017 , ing the received electrical and optical signals using a
provisional application No. 62/ 534,671 , filed on Jul . Fundamental Code Unit, automatically generating at least
19 , 2017 , provisional application No. 62 / 560,750 , one machine learning model using the Fundamental Code
filed on Sep. 20 , 2017 , provisional application No. Unit encoded electrical and optical signals, generating at
62/ 658,764 , filed on Apr. 17 , 2018 , provisional ap least one optical or electrical signal to be transmitted to the
plication No. 62 / 665,611 , filed on May 2 , 2018 , brain tissue using the generated at least one machine learn
provisional application No. 62 / 719,849 , filed on Aug. ing model , wherein the generated signals are at least one of
20, 2018 , provisional application No. 62 /783,050, Spike frequency modulated or Spike frequency demodu
filed on Dec. 20 , 2018 , provisional application No. lated , and transmitting the generated at least one optical or
62/ 726,699 , filed on Sep. 4 , 2018 , provisional appli electrical signal to the neural tissue to provide electrophysi
cation No. 62/ 881,220 , filed on Jul. 31 , 2019 , provi ological stimulation of the neural tissue using at least one
sional application No. 62 / 883,983 , filed on Aug. 7 , write modality.

802
804
806
External Interrogator for wireless charging and
base - station for wireless communications

Electronics Platform for wireless charging

and processing wireless communications

KIWI implant containing CNT electrodes

800
Patent Application Publication May 13 , 2021 Sheet 1 of 159 US 2021/0138249 A1

LFTOANRGUOAGME C,OGNCIETPVESDOWNTOTHE ,MSOYLNEACPUTLIACR ANDATOMIC TINHERACTIONS BGRUAIDNE DAEVNLOPMDET FUNCTION

OFNDSTEUVCEHLOSPMWFUNLTRDNFIETUSQOVLGREICANS T,ASFAHOEIRNOMTSIDCTAGLHUFBNREDAMOSLTWNIRDHYKG AIFCBTNURPEOHNLICTGRED'ONSYT OFM-LAOEUDVLETLI

INEINXFCORHMANTIGOE SBIYOLTGECMASL
M
A
INA T H E I C A L Y
PTARHORCES RIC-ELNOTASELRDY ANDCANBEDESCRIBED MUNAIFOERM

)
Ah
-
sizh
NBA
yn
1
.
FIG

TSGACOTNREICAHDTS OFU"ENRKITOWRNYS~RAMONYCAJAL(1)A8N5T2I-A19G3O4 O Ze ag
CISAW"THEBORNSAI LIT DNG OFUANEXMPBLOER D ?

ZnEkstraß
Patent Application Publication May 13 , 2021 Sheet 2 of 159 US 2021/0138249 A1

)(LalngXoryIistOhm 200
AANLYSI Lxainoglu ye [Link] Nnaetulwryoasik

Fig
2
.

(umB)MoasnvietmdoSnriyNgt E
(
)
lf)i(remMsaognRiIcge ectronphalGgr y
L
c
p i
o
a nrg u
t
s e nigc
CDOLAECTIAON dvtVeaoricnaeol VALIDTON Fmuancgtieoalc
Patent Application Publication May 13 , 2021 Sheet 3 of 159 US 2021/0138249 A1

Advantges Broader Aplicaton BAectureacy LReoswuerc QLeuaircnkeg andTrain g Consumpti

3
.
Fig

SCAownatrexusl Aenatlimyesnt SAiwtaurenosl O/Qrucahelsitfreo IBLneatsrneidtg

M-mAnuoaldytasil
IMnfarstgeucmn
Brainos
Patent Application Publication May 13 , 2021 Sheet 4 of 159 US 2021/0138249 A1

Welns enhacmt Memory Atenio Langu Reasonig

fdMevoicreasl cognitve
High adptbilyNeuromphicarchiteus spiking NNs
.
esp
( to
)
dev lop learnig human machine interfacs
Health Neurolgica dsgneoatics t/reatment wDiagntoshic mheidstcoarly andbrain fleodbacpk dintaegrtaion &Stimulation treamnt presciton Combin g dsiouparcets Multimoda waniltyshi autom ed pieline rQunitcmke retain galdopwting nptoeatuireanlt respon

4
.
Fig
Transpoti tv,rehaifcliecs am n gedm nt logist c
Autonms Behaviorl anlysi Resource intent architeus -celnotruidc o p t i m z a o n d
I re n
i tv f e
y irn g pHe rif o g
m a n
h c e f
S c a
r l b
o i mt y e
to
m b d e

Security sVurivedlanoc indtihfreatns comunites Contex asemnt adetncdion situaon detcion oRnbapridng forfacial thelearnigenviromt in-msaeknisneg ambiguos situaons
a
t h r
n e a
dt d
ofe t c i o n t
dehprndaet D
o a
b njg e rco u
t s a
r e c n
o g i
dt o n n
a
Sept w
i o r
k e
s for
fit
p r o m i s n g

Stofwlarse ML
/
Al
for
IDEs
and
devlopmnt devlopmnt parmets Architeu selction
tAdovanlcesd Aoutmfaion netuwroalk Auto-tuning lNottoimited
-bydeeplearning
evolutin
methods
(,
.
.e )AGuotogMlLe fDynamic strouctres sgenratig porlutsiaoln
User Experinc Anthropm iciuntserfacresPersonalityemulation Artifcal emotinalintelgnceBhaviorl anlytics personalizt
Features
De p Sofelction awrcheitlures tosuited apernc pdtion
Brainos Why
i
an tel y s
i o
focus
D
&
R
on n
reasonig eoflment b)ehavior (asdriving
Patent Application Publication May 13 , 2021 Sheet 5 of 159 US 2021/0138249 A1

2
PEaxreclutor
MPlaner Scohedulr Selctor
O
Infrastuce DBModel

DBEvent Models FOustipount Problem Formalize

Fig
.
5
DBBRHerisatnorcsyh MToradienlsg
WKnowrledg Model Registry
DBPersona 1
/PM*rocdeslosr Orchestaor 3

Text2World
CDhantelas
Adaptors
Video Images
Patent Application Publication May 13 , 2021 Sheet 6 of 159 US 2021/0138249 A1

Recration Dmain RelationshpDomain

Family Hurian Welbing Enviromet Domain
Health Domain
Persona Domain Education
3
Mental Physical Spir tual
Brašnos
6
Fig
.
WUCelasnes ASenpatliymsecnht Analysis
Word
,
Tone
(
)

RSuicsdkal Asemnt
PaArlkzihnesomr Detcion Ccontrol Demntia /Detction Controi forpAorseiablse colabrtin
2

Diet,EWxericgihste WOetarhbelrs Sub-DMeodmiac ln D,Metrcairone yControl 3-EPahraosued

andTEMS devlopmnt CD,eotncrioln
P2-Mhigarsien
1-PainPhase Detcion
Weather ?
1

BD(Aieopmtairocens
,FGWairtimbcihnt
Patent Application Publication May 13 , 2021 Sheet 7 of 159 US 2021/0138249 A1

EPSP
1
-
MNRG -PAORSVILTBUMVIEN IBNATESRKUOTN
GABA CA1

N--EUROPSIN IPSP
PCYREAMLIDA

CA3
FIG
.
7
EPSP
1
-
MNRG -PAORSVILTBUMVIEN IBNATESRKUTON
GABA CA1

NEUROPSIN
/
+ PSP
CPYREAMLID
NEUROPSIN
CA3
Patent Application Publication May 13 , 2021 Sheet 8 of 159 US 2021/0138249 A1

EIwfacnxihteronlgeiradtsno
E
Pf
wl
c e
i
h a
rt
o f
lg
ne
r
i s
cm
cwf-sboimtaruonesclitreons cwpoamirunelstdiog 800
eCciKlomnItpNalriWoTdnIestg
8
.
Fig

804
802

806
Patent Application Publication May 13 , 2021 Sheet 9 of 159 US 2021/0138249 A1

TO CONT
.
FIG.9
910
906

ANDAPPKWCBOMICUNWTCAIHSE ORONTMARBTLPHEONT KIWIOFEBCCSI,EEG)(VXNTFEORANLMAS RATE,:[Link]

BCCSOFIKIWI)S(NPETFIUOKRINAMLSG O,BPXRYLOGESNDAOTUIYREN,RBPTEOMASTRDHUIEYNRG)&I(E,AMNXOTUVRNEDAMLINVOACLTE ,BISWFPIRDLOECTIOSWNAL NR1(EEG,[Link] [Link]
[Link] ? )
ETC
TEMP
PULSE
,
ECG
ETOBDTS5,[Link]
CASMRLTPNHOESUDABLT
[Link]/AHEBONLEDTESI [Link] [Link] ISAND/[Link]
C,TOTHELALPOTUODP ETC),I(SNPTEIRKVAELS TOCP,LHOUNDE TOKIWI(VIABCCS)
[Link]
[Link]
BC S CAONTIUDOS E
OF
C O
X N
T D IRNOALS
WTO/FIRELOSM TOBUKNCIWTSI
O(BCP)-SCOPTHEBIRVYELCSROTGVISECOAMRWL
MDIESBORLUANE I
PN D
OUW
C TE
IVR SUBTRAE OFPTIGCAULUPRONCEIS TNG RKEIAWDIS WARINTEDS ALLLAYERS
VI
-
I
FROM

9
.
FIG
SOFTGELL(HCNTSU)POBRLICGDAHNST
Al
mPFC NPA1RYEUGARMIODNASL

CSOWACTNEDFOPITBECRASL PINODUWCTEIRV VCSEL POWER

VE CMOS PROCES R FPGA BISECTED L1 L23 LO

(KW)[Link] ANTENA ELCTRODES LDEILNAEY CNTS

DELAY
LINE CMOS
/
CPU
CNTS
SISANOKPBHILEARWOTIED GP,IKENODUWRCATEIVRON WB-RINEADCUTHLIREVGSYD
, 902
900 VIEW PERSPECTIVE MBADO VIEW SIDE
Patent Application Publication May 13 , 2021 Sheet 10 of 159 US 2021/0138249 A1

9 CONT
.
FIG .

911 912 914

?
G
DEPMIND
CLOUD Go gle
CTOTKABSMOLNIRHTEPWCUEDOTISN ISPAL)(ORTONTHEENOHRAFBLCOYRANSMEILTSDINTHEPANDDSISCRTOLEORSAUIGPNDE CTO/FWILREOLUSMD -
~

&[Link]
SETCNSUORFELOW INTENT API

TO/FROMWIRELES /TBCCSTOPAHBOLNET
[Link] PSFCKOEHRIVAOWGBAELMID [Link]
[Link]-SHOAURLGDE [Link]
DEVICES [Link]
2.
E ACR B U D
S
PHONE SDK CLOUD
FROM FIG.9
Patent Application Publication May 13 , 2021 Sheet 11 of 159 US 2021/0138249 A1

CE ANTENA VCSEL
ELS1
?
EL
?
. 1

SCT PCROMESOR FIG

10
.
CSOWATNEDFOIPTBECRASL
VPERISECTWV Toot
t .

..*** 2

Core
LDEILNAEY SPOENITSORN TCROANDSPU ERT
YSA
.

C
YA 15
SIA
estateSL
22

*w923
Se

?
CMOS late 02
CSOWACTNEDFOPITBECRASL ANTENA
LDEILNAEY VCSEL
p1
*****
- *****
LINU

WN

VSIEDW
1002
1004 1008 1010
10 6
W 102
ar
"

TOPVIEW
Patent Application Publication May 13 , 2021 Sheet 12 of 159 US 2021/0138249 A1

SDTIOGRAEL CONTRLE MICROPHNE PULSE OXIMETRY

AKWCBOMICPNUWTDAIHSE ORSONTMARBTLPHEONT DIAP DSPGSPDKIONUWERTECIRE
AMOTPBIUECXNATHL SENSORS
6
(
ECG
) BCCSIKIWIOFEV,(EEG)XNITFEORANLMAS
KIWIIBCCSOF(NS)PEFTIUOKRINMASLG
[Link] MDIEGMTASL
BDO
LBLUUETOEHTO TH g FIG
11
.
EABRMLCTNUGRED
BC S GYROSCPEACELROMT AGNEOMTR MICMDIEGMTASL TRANSDUCER
906
Patent Application Publication May 13 , 2021 Sheet 13 of 159 US 2021/0138249 A1

3. ANALYTICS - LOCAL & REMOTE

SMARTPHONE CONNECTS BCCS AND KIWI TO THE CLOUD

LOCAL ANALYSIS IS PERFORMED ON THE PHONE (OR TABLET)

AND STORAGE AND DEEP PROCESSING IS IN THE CLOUD .

9124

PHONE
WIRELESS TO /FROM CLOUD

SECURE

INTENT
IN
TENSORFLOW
DEEPMIND
SDK API FCU
CLOUD
Google

CLOUD

FIG . 12
Patent Application Publication May 13 , 2021 Sheet 14 of 159 US 2021/0138249 A1

0
"
RE C TI F ER LOAD

RÈSRESONATNT
COISRWLUEIATROS
ooog VS

SOURCE
[Link] DPSFCKOEHRIVAOWCGBELMISD 13
.
FIG

[Link]
[Link]-SHOAURLGDER [Link]
[Link] [Link]

914
Patent Application Publication May 13 , 2021 Sheet 15 of 159 US 2021/0138249 A1

[Link] anensmoxcdtulrianmtelsg [Link]

hsoitismuchd,oinallLEDs•iauretcdpieuornts for
therwt,butuanehvidlaeunstgclehds
thanLEDsandaremorebm•Louadlskeyrs dwsinthep•Inihurtaelbcipstnley .andyforgisevenlLEDserolew onrw itmmmof1ls•Wipa0neick-wtdie3rats0lh [Link] A•fmayberdtoieqltmuioernvasled
.fevenptuortwhearl
S•ofhandcaoinrdezwcateors
14
.
Fig
LDisadvnetargs modulatr LDisaEdvnDtages

no-talkandcofistmthathreaonsrces [Link]
oftheinelf icoiwenscy itisforbeamd•Ifnotusingfibero,epltivecrsy sitetothetlhighieasydnaiergnshiecty
of0.1nmorbls•Nipeantrcwiredoatwlh forA85-90
%.range
withlathighsm•Canbedoiprutelcaetldy Verys•m,canandldbeoniomruagencltheltdy andfors•Sarenoptecaibf lacstioenlys
D•orinchieuistpglouinthaslg c•Bwihornaetviolnsegcitnhey [Link] stpioen [Link]'ons S•rsforedhxmtiqnrauezsglmiethdy
[Link] isrspeqciufrcietdy lowinher ntly
LAdavsnteagrs in
free
space
.
LAdvEantDges probe
.
Patent Application Publication May 13 , 2021 Sheet 16 of 159 US 2021/0138249 A1

suchas,[Link] r
m
d
-
M
.
t
,eaph
n
te r
a
i
d e
on
sr
l
t
dhaveto
Probe
•p(
a
be
seeeornsbinlt'eymLind,[Link].2012forexample)butneedstobe in
m
t
are
r
not
the
byaen
h u
mmcsnioaeltvuiayosbentld
f
eoa cv
it u
erd c
m
by
off
s
and
is
this
size
a
oun
rscp
iye
t c
dst [Link]
inatohdandp•Foiratfengitdcaluety [Link]•Tafohriescntfrsuel ionofftheitok,areyunthciotrbwoiuntm pns•Wbandhaego'iceryvtnodzges
c
brain
in
cells
Smith
(
2008
K.
,
.
)ha n el s
It'seosneifbilte50umfromthewrbelosttoteiactshiounes
could
bthat
CNT
of
p.
the
lot
a
.dms•Piuofapwirdceulryt

CDisaNdvnTtges cslientcianlg adres d :/?[Link]/198453 [Link] F

f
-
i
D l
m
s a
r
opd v e
t
nai g e s
15
.
Fig

,L.&-KBarea(andhighscoaunedrceftkaivecnty r-tonsi,tmeaopdhtgruincvasoincelg
BL.
;
2012
Y.
,
Hanein
Keren
-2014a
&areket Kotov
Ec•/[Link] rs•[Link] •CNTaren,tsbfialocerxumspoicagbtnlhealsowandnon-fhblaeriocvtdgrlainchegl
N.A.
,
;
2009
.
al
et
C.M.
SVoge
&
J.P.
tegeman byesfortheuseofCNTcanalmactlrodewrs •[Link]
Mienixkoantoi
P(
&
S.
.
)
2011
N.
, •[Link] g
•Ecxpornedlucrtinegs Ai-fFdlmvorpanteignts
CAdvNantTges 2011
.
)
Patent Application Publication May 13 , 2021 Sheet 17 of 159 US 2021/0138249 A1

WPIORELS SBREOICTDVOEMR
3
PCB
OF
)

3 } ? 3
RESONATCIRCUIT
$ 3 }
AF

}
TLEICHNOLAGYCWIHRAELGSRL1C423 X
2
mm

3 3 $ 3 3
RECHAGBL BATERYQ)(UALCOM
3 3 $ } 3

POWER REGULATORS

ICUNRTIELCPU1.0mm BLELUETO H(ACCEL6-AXISSENSORANDGYRO)80kbSRAM,SESOCQF3UL8A4RSkKHb

KIWI
PLAN
(
A
)
2
-
9.8
(
9.7
X
10
mm

DOIGUTAL FOPTIGCAL DOIPGT CAL

3 1 1 1 } I

OUTIINNOUT
DAC
SIGNAL
FIG
.
16
OPTICAL SEL CONDITIONING
LDEILNAEY
LINE
DELAY SEL OPTICAL
MULTIPLEXING ELECTRICAL
SEL OPTICALOE -MULTIPLEXING DE-MULTIPLEXING

OPTICAL FROMRICEUN SEL ELECTRICAL MULTIPLEXING

SPL SPLHOM
}

PREAMPUFICATION UNT

OPTICALSTIMULA ON
3

OFPITBIECR FSPLRITOEMRSONR
O-CSPWTACRONEDT SWCNT
COPOTAROTDEQDH O-CSPWTACRONEDT O-CSPWTACRONEDT
-
O,PTICAL
READING ELCTRIALSTMUAION ELCTRIALREADING EF-SLWICTBRNOEDTRS ESF-LIWCBTREONDTOS E-SFLWICTBRONEDTRS FSE-LIWCBTRENODTS
Patent Application Publication May 13 , 2021 Sheet 18 of 159 US 2021/0138249 A1

WPIOREWLSRBRS(EOICTDVOEMR
OF
PCB
)

3 3
RESONATCIRCUIT TEUCOHNCELOAGWCUIYRHRATELGSRLTC4123
3 1 3
2X2mm

1 3 } E
RECHAGBL BAT ERY)(QUALCOM
}

POWER REGULATORS
AVRCSAEYL
SIGNAL
KIWI
PLAN
(
)
2
-
B
mm
10
9.7
X
9.8
(
CONDITIONING
CNUTREIL CPU1.8mm BLELUETO THA6-AXISSECNSEOLRANDGYRO)80kbSRAM,SESOCQF3UL8A4RSkKHb
SEL ELECTRICALDE- MULTIPLEXING
17
.
FIG
LINE
DELAY
3 3

ON ?.? SIGNAL
CONDITIONING
FROMRICEUN SEL MULTIPLEXING UNIT
MOPDTULIACORS SEL
OPTICAL
MULTIPLEXING
SEL ELECTRICAL MULTIPLEXING
FOIPBTEIRCFSPLRITOEMRS
IND
PREAMPLIFICATION UNT
SPL SPL
SPL

OPTICALSTIMULA ON SWCNT-COATEDOPTRODESELCTRIAL STIMULA ON [Link] ELCTRIALREADING

O-CSPWTACRONEDT O-CSPWTACRONEDT SWONT
-COPTARTOEDE PHOT DI ES D)O(FETPIETBCIEORCSONRFROM F-SELIWCBTRENODRTS F-SELIWCBTRENODRTS F-SELIWCTBROENDRTS EF-SLIWCTBROENDRTS
Patent Application Publication May 13 , 2021 Sheet 19 of 159 US 2021/0138249 A1

1806 1804 1806 1804

1808 1804 1806 1808
18
.
Fig
GR G G
5
/
R
S $
/
R
S
VR
G
R
R.
Ref
1808 1806 1808
1804
1802 1806 1804 1806
Patent Application Publication May 13 , 2021 Sheet 20 of 159 US 2021/0138249 A1

VS *

FU $
B
8

:
3
RAS 83 Y
X

BSBS RAS
S 89
:
2
%

X 3

X
GRSASA RS RSR
$

RES 3
RX
9 1/8
HAS

19
Fig
.
K

GT
$
G
*

RS RY Re
5 G

OS 3 MASKIS SR9 NASR

R $
AXX

WS K
20
US A WS
9

33

23

.
5,5
%

3 Bisnis GORG ROMINS

28
*
WR SBS GA RS
SRS 6
&&&R
G36 89 78
Patent Application Publication May 13 , 2021 Sheet 21 of 159 US 2021/0138249 A1

20 2 20 2

urios
20
.
Fig
Patent Application Publication May 13 , 2021 Sheet 22 of 159 US 2021/0138249 A1

Dmiesoblruane Inpduoctwievr Substrae FopPtiGcUaluprnoceistng Soft

gel
l(
holds
CNTs
u)pbricgahnt
21
.
Fig
Patent Application Publication May 13 , 2021 Sheet 23 of 159 US 2021/0138249 A1

rKeIaWdIsawrnitdesalllayers
22
.
Fig
nprAleyuagmoidnasl
Patent Application Publication May 13 , 2021 Sheet 24 of 159 US 2021/0138249 A1

2304
Electrods Hermtic enlosur 2302

23
.
Fig CNTS
Patent Application Publication May 13 , 2021 Sheet 25 of 159 US 2021/0138249 A1

1, 2
24
.
Fig

444 Q -ALND {leuoijuan003 Q - AINO euopuanuo

Patent Application Publication May 13 , 2021 Sheet 26 of 159 US 2021/0138249 A1

(nm)Wavel ngth
5

25
.
Fig
Patent Application Publication May 13 , 2021 Sheet 27 of 159 US 2021/0138249 A1

Scale
bar
=
2
um
m 1

26
.
Fig
oa 00
Patent Application Publication May 13 , 2021 Sheet 28 of 159 US 2021/0138249 A1

2710
PROCESNGCIRUTY 270

2708 ??? 2707

2706 MUX
N

27
Fig
.

2704 PREAMLIF PREAMLIF 2704

2702
ELCTRODS SFWICBNETR 2702
ELCTRODS FSIWBCENRT
Patent Application Publication May 13 , 2021 Sheet 29 of 159 US 2021/0138249 A1

2814
PROCESIN CIRUTY
G 2816 280

2810 OPTICPALRPROOCESCES 2812

280 MU 8 X

28
.
Fig

2806 MODOPTICAL 2806 OPTICAL .

MOD

2804 SPLITE R
2804 SPLITE
2802 SWCNT COATED 2802 SWCNT COATED
Patent Application Publication May 13 , 2021 Sheet 30 of 159 US 2021/0138249 A1

2914
PROCESIN CIRUTY
G 290

2910 ADC
2912
2908 SCU
29
.
Fig
2906 PHOT DIODE
290 MU X
N

?
2902 SWCNT COATED 2902 SWCNT COATED
Patent Application Publication May 13 , 2021 Sheet 31 of 159 US 2021/0138249 A1

ta

t3

30
.
Fig
WPVCoaltyvecnfmir t2

t1
Time

to

Potential Cathodic
Patent Application Publication May 13 , 2021 Sheet 32 of 159 US 2021/0138249 A1

CONT
31
.
FIG
TO CONT
31
.
FIG
TO CONT
31
.
FIG
TO CONT
31
.
FIG
TO .
FIG
TO
CONT
31 CONT
31
.
FIG
TO CONT
31
.
FIG
TO CONT
31
.
FIG
TO

3114
N1ANALOGMUX N
N-1ANALOGMUX N1ANALOGMUX

3112

3106A
1

1
HIDH
2

NO
N REF

1
+
N +
N
22 2N
M

REF
3106B
3106M
1

- NQ
NO
1000 1001
|
N
SID

- 1000
2
N REF
31
.
FIG

3104
NEUROSENSE CONTROL SENSE
NIEURO
CONTROLLER / SELECTION ZONE
CONTROL NEUROSENSE CONTROL
3102
S1E,N0SE0 CHANELS
Patent Application Publication May 13 , 2021 Sheet 33 of 159 US 2021/0138249 A1

.
COMM
TO
PLATFORM 3118
2Mbps
PISO 3110
2N Kbps 2N Kbps 2N Kbps
3108A
10 X
COMPRESIN COMPRESIN
ENGINE 3108B 10 X ENGINE 3108M 10 X
COMPRESIN ENGINE FIG CONT.
31
.

20 N
200N10Kbps
BITS 10BITS
200NKbps 10
BITSKbps
Nx20Kbps Nx20Kbps
3116 ADC Nx
20Kbps ADC ADC
31
.
FIG
FROM 31
.
FIG
FROM FROM
31
.
FIG 31
.
FIG
FROM 31
.
FIG
FROM 31
.
FIG
FROM -
31
.
FIG
FROM 31
.
FIG
FROM
Patent Application Publication May 13 , 2021 Sheet 34 of 159 US 2021/0138249 A1

32
Fig
.

2
Patent Application Publication May 13 , 2021 Sheet 35 of 159 US 2021/0138249 A1

100
ms
)
When Wahulu
???
33
.
Fig tdniefwurontsPttosreoaipsmnidoeng
Athen
Me
huu
Muu
LUL
ANAK
A

nyl
60
Patent Application Publication May 13 , 2021 Sheet 36 of 159 US 2021/0138249 A1

mV 50

FR

34
.
Fig APNumber
.
4

.
10

Procabuty Firing

il-njection 1481 il
-njection Light
Patent Application Publication May 13 , 2021 Sheet 37 of 159 US 2021/0138249 A1

mV 10
1s
shall
w(nm)avel ngth
35
.
Fig
500
ChR2
0.5
)max to normalized(
spectrum excitation

?
Patent Application Publication May 13 , 2021 Sheet 38 of 159 US 2021/0138249 A1

V C h R 1
C-hrimson COhron s *ChR2 T*sChR
700
36
.
Fig W(nm)avel ngth
600500

0
400
1.0 0.5
charge
cumulative Normalized
Patent Application Publication May 13 , 2021 Sheet 39 of 159 US 2021/0138249 A1

3712 CSOWATNEDOPTRDE 3712 CSOWATNED OPTRDE 370

3710 MOD OPTICAL 3710 OPTICAL .

MOD

3708 DELAY LINE

37
.
Fig N

3706 DE MUX 3714

3704 OPTICAL PROCES

3702
PROCESNGCIRUTY
Patent Application Publication May 13 , 2021 Sheet 40 of 159 US 2021/0138249 A1

3818 CSOWATNED OPTRDE 3818 SCOWATNED OPTRDE 3818 SCOWATNED OPTRDE 3818 SCOWATNED OPTRDE
380
3816 OPT. DE MUX 3816 OPT. DE MUX

38
.
Fig
382 3814 OPTICAL MOD.
OPTICAL
3814 MOD

3812 DELAY LINE

3806 SCU N

3804 DAC 3810 LED 3810 LED

3820
3802
PROCESNGCIRUTY 3808 DE MUX
Patent Application Publication May 13 , 2021 Sheet 41 of 159 US 2021/0138249 A1

390
3910 CSOWATNED OPTRDE 3910 CSOWATNED OPTRDE
39
.
Fig 3908 DE MUX
3912
3906 SCU

3904 DAC

3902
PROCESNGCIRUTY
Patent Application Publication May 13 , 2021 Sheet 42 of 159 US 2021/0138249 A1

CFORM PLATFORM
40 2 SIGNAL CLK
SIPO
BITS
5 BITS
5 BITS fan agh
5

ii DAC DAC
ii DAC
40
.
FIG
D1+NAMULOXG D1-NAMULOXG N

N1DAMNAULOXG N

40 4B
40 4A 40 4M
1
+
N
+2N
2
IN
NEURO STIMUL S CONTROL NEURO STIMULS CONTROL NEURO STIMUL S
-

4008 CONTROL
CONTROLLER / SELECTION ZONE

4006
Patent Application Publication May 13 , 2021 Sheet 43 of 159 US 2021/0138249 A1

410
DIKMEPVLWIACNET
41 4 TEMP/PM
MODULES 41 6 /CSOTNFAITG MODULE
41
.
Fig
.CONFIG
COMANDS
N4E1URON8S 4104
NEUROALMDUATION MODULATIN COMANDS 4106 COLMPRESIODN COMANDS DS F E D B A C K 41 0 [Link] CONTR L 4
G A1T E
2 W 0
Y
PM/CRONDETSURLIOGCIRUTY READING COMANDS MCGOAMTUNEIWY ODULE
41 2

4102
NEUROAL ECORDING RECOD D A T A 4108 MODULE
Patent Application Publication May 13 , 2021 Sheet 44 of 159 US 2021/0138249 A1

42
.
Fig
rR(ne]);acodr_uingreboufner

for(Nn:){euronsr_to nead r);(ceompdings_budfaetr

t
)
time
-
rate
sampling
/
1
(
leep

rL<Nn;eurios_tnresad sIanmtpelgienrr;><BLecoyidngts_buferBfs;otalpesand while(!stopped){

; time
)
(
;

}
Patent Application Publication May 13 , 2021 Sheet 45 of 159 US 2021/0138249 A1

43
.
Fig
m(n,s]);toieudlatilrona_boutfner
){tos:nfor(Nteiumrloantse );tos(nuptdeaium_rtloaingse
L
S
<s
>t
b;i m u l a fs
o n C t
emiarn
d
f
s
d
; tui
lrm a
ot i on
CS{toimrulatcniodfi;nlteonasityist;munlaiot_ype <NL>n;eurois_tsroimtunlae while(!s){top ed =neuros_timlae
}

} }
Patent Application Publication May 13 , 2021 Sheet 46 of 159 US 2021/0138249 A1

4 10
output 40
4 09
System
4 08 4 12

44
.
Fig

4 06
CIontpruotl Measurd
4 04 Contrler
4 02 Er or
Target
Patent Application Publication May 13 , 2021 Sheet 47 of 159 US 2021/0138249 A1

45
.
Fig
cn
(a
;
) l
d
e=urto
a n
_ d u
ir a
t o n
rcd();nhaeultsanr_eodlsanc=(a);elxutr_osnexdat tosc,(athipmunlaetiyosn
t)
<
state
next
(
if
{hreshold
List<C>ctoread;han els ,c<CL;toshtiamnusleats
while
(
s)
!{
top ed
d);uration ????

}
}
Patent Application Publication May 13 , 2021 Sheet 48 of 159 US 2021/0138249 A1

46
.
Fig
readc(to);han els
cn();aesuxltrco_anxsltdea dtie)/(rovram_tpoiern+bd*KDiKIkeonpriutkevagrptiusaovtler);,otos(cathuipmtnlpaetuilyosn
i)*e†(tnraerigon_atirlme
snedtrauxotaern enron
=

)tis(leireamtieopn
0=er or_prior
i©ntegral dnaeutraon
stop ed
!
(
while
{
) perior
thin
}
Patent Application Publication May 13 , 2021 Sheet 49 of 159 US 2021/0138249 A1

4714 470
PSapciket Buf er 4718
4716
4720
4712
4708
Compresin Predictve RunLength Encoder
47
.
Fig 4710
4704
TSDepticktoer emplate Memory Adaptive Thresolding Corelatin
4702
4706
BDuaftear
Patent Application Publication May 13 , 2021 Sheet 50 of 159 US 2021/0138249 A1

. ...

48a
.
Fig euclidean
,
e
(
pow
*
k
/
1
spike
norm
)
neighbor
;
^
a
2

(
nfor
in
{eiagrhbeosrt
)
k
,
spike [si],nteprginahcktboesrn canewarxi(9);//lsnudostim_egrlnsyt
t,cpofkn17Fortheehoiageranbigostre asiontdap//Wnreoibnhatcdgleihnoty
[Link]//lsnouiartdgnemtlrasdy =acC(datap>,S);Map<sloiugnis_rtnakedroemsly par1/Citsc,[Link]);//r(dx_i-ahotnd_ipsonmieltmys
);(dpa-aFivrtlwes_orpdiatgnces ani1Cpabsnteolrciuwantgleosh
pbdat//Cievolshcwrunategs
;si),FSMap<(ntlpreoinacgkthieosn
indfor(s){atp_ipokinets vtdf17Reiamploruetans in
i
(
for
,
0
range
0.01
0.2
{
)

>dList<S;atp_ipokinets
}
Oc:lputsiemranlg
} cluster
Patent Application Publication May 13 , 2021 Sheet 51 of 159 US 2021/0138249 A1

48b
.
Fig T*spinc[x_il,(]);noleruisglthaebirosn
pow[si]/x_i,n(eterignahctbiohsrn t//Iftheninoficbadulratgmoesabmtesrt//sf,[Link]
(
nfor
in
x_i
{eiagrhebosrts
)
k
, cCn[x_i
)=];leuisgthebros tmthehf,c1/Ineihrapgoeisatumesr
ctothebihowm//Cletoaugsnotemysgr
:).=max(Ccbliugstesrt ;){>tcif(bhlirugeshteoslrtd=coptlimuas_terisng
cstph17rleauvsimoetudylr
random
(
if
<
)
p
{

.//criterion
}

whet r
} }

????
}
Patent Application Publication May 13 , 2021 Sheet 52 of 159 US 2021/0138249 A1

49
.
Fig

)(d-nfeuaroit_mplbsr.fundoet_gmach
rxvloftheacb#doauenilftuasiedtnsr )(dpbats_pfbniultdyer
t=(5*mdatab)/0heur.df6sih7oae4lnr5d
d(r)aet_bunfer.oads d[
(
if
-1
t{
?
)ahrae_bsuhfoledr
tc#halrehusaoteilndg
#nfioltierseng
}
Patent Application Publication May 13 , 2021 Sheet 53 of 159 US 2021/0138249 AI

10LQODI
x

50
.
Fig
T
bin
time
in
-
non
/
firing
333-64
23
3

394786532 YE

*
1.I6I,Z5T8DO7ER0L2TA6NORE6I ? ???
Patent Application Publication May 13 , 2021 Sheet 54 of 159 US 2021/0138249 A1

51a
.
Fig
wooffrommustbe1xn,an:siprunksemd_etoapitwmyes
ofthesidsn1xn,a:epruimrokapneys /32xnsizenofdatabaAeitrunmcoapndesy ().n=sipfktel_sauipmt.nsey8p
bcns(,ipet.u_krdonaecimfdys tsfgbCirpenmacoitkdes
):w,intnnienu_rsoinzes :thewwinsforueinacmozdepldwe
sp().ftilkpaemi_tekmsn
n1
ofsav#Casperiakc_ktmzieosr
f,
.
)
(
sp_l.saiuptoknetsn8

tnofoue:amtrbaoelnrs ()n=euro_idns.flaten
andnIeuDroSn
Angs
: R:eturns ***
spnik_etm.zs
in
ms
times swizne
Patent Application Publication May 13 , 2021 Sheet 55 of 159 US 2021/0138249 A1

it
::

d(b=n)np,upi.ta_seconyidgtupo3_sare2tdy
516
Fig
. size
,
)
nshape
win
(
=
_neurons(
csnpo.inlksiebt_.ampnatd
"
0,0
)
topad
0
(
,
nopa.rinaty8
=
tdtype
.
) cv=0)oanlstuaenst rpieansk_hesp_aiompzsaet
np
wts=
(
.
)
32
ints
nu
,
i).,(nscetmatpu._rsioc_nktmiaedrsx
([Link])anmat=spiskpe._tmibay.okpsteyl )togetbitexb(sm#pnauictlkoredi_3cpmnela2gsyt
tom#R"abitsforpuearlhtsipalrpciartexon 31
**
2
,
create
)
b#
4
1
(
itvector bite=d)[Link](bsnaipcotde3kad2e
)bifore(ncc#moxutpafeoirskdguneoardlgs
spike_mat
(
= C#nof32-bitialuctmelbgdaterdsn_ints=[Link][0]//32
#Padwithtozgetanofrowsuemrboesr
%32nnXeurons 32-topadXneurons bit32
2
=
n(
**
)
32
[Link]
v#Utocmserpastroesix ,

#dby32iv sable :ifXneurons

edbrnacitoudrtean
Patent Application Publication May 13 , 2021 Sheet 56 of 159 US 2021/0138249 A1

52
.
Fig
w(c)!=tonhercitd_uolgaetw&&++n<muy){axb_ecro ft_ailures
0
;

();cstoarm_caenptdisg
bcto=fg;oantloecswtlaedy (rw);econati_nterval s();tar_ecording ();star_povis n g
if(ctog){oantecwtaeyd
(pifdone){rovis oni goffniaumnlbrets
{
else
}
}
}

start
{
)
(
plan
Patent Application Publication May 13 , 2021 Sheet 57 of 159 US 2021/0138249 A1

53
.
Fig

cps();troavm_icaoenptdisg ();staroe_pcvkiseontrg (cmd);cperxovemiscauontideg

(){,cpstroamv_iscaoenptdisg Rn(
=
!
{
)get
cmd
while
eExtS_TcoAmRaTnd
cmdcp;=normuvilasnod
s(){tar_povis n g
init_AP
)
(
;
;starto
}

}
}
Patent Application Publication May 13 , 2021 Sheet 58 of 159 US 2021/0138249 A1

54
.
Fig

;
1

pcCronfeaigutognPmar =AGCDTERPWSAYD 2;SPYOSWTERM W=3;AN_CREDNTIALS WC=4;IHRAELGISNG RB=5;ELAODCIKNSG

M

{ComnfiegursatComgend C{omnfeigusratonPgem cfgtypeP1;ar mType

2;vbayltuesEP{anr muTympe

}
}

} }
Patent Application Publication May 13 , 2021 Sheet 59 of 159 US 2021/0138249 A1

55
.
Fig
4
;

SC{Smtoiesulatnigdoerebfynct_iel stimulaon_Type
13

cbytehan el
;
2
=
=3;ebncoyde_tcomeand EL1;CTRICAL O=2;PTICAL C=3;HEMICAL
SeT{tinymulpatmieon
}
}

play
}
Patent Application Publication May 13 , 2021 Sheet 60 of 159 US 2021/0138249 A1

56
.
Fig

initacbms//hpenhicdvwpaefdtinlxsg
1
;

:mcrbhaeypntdielsg =2;mpbaytlroeiasxd
{RmecorsdingaDtehbeadyr_tm ke
matrix }
Patent Application Publication May 13 , 2021 Sheet 61 of 159 US 2021/0138249 A1

57
.
Fig

;
1

=1;vSrtaelpuseValuste =2;VSOEFRTSWIAORNE AT=4;CILVAETSD

SsptaurstTaypme pb23ayltoeasd S{etanusTympe =1)LBAETVERLY T=3E;MPERATURE

{SmtaesuDagte {StmaeusValgue
}

} }
Patent Application Publication May 13 , 2021 Sheet 62 of 159 US 2021/0138249 A1

58
.
Fig
pMVP,t;#Forthec/b=raheidtarmycn_etlgrvdeyl

();gte_mpraeture>ti){-pif(tmv_pterature b();gaet_ryle_v lb;cp-reahtv_bneagrtly

s();enid_gthnroatle s();end_stop_signal
tif
(
{
)
t2
>
emperature
pr;tev_m autre lmfbdiaoasntdcehumarlgye s();end_thro le_signal
p<
(
if
){redicted_time
61 spif(time<62){redicted();end_stop_signal lp;baretv_yletry
while
true
(
){
}
} } }

}
Patent Application Publication May 13 , 2021 Sheet 63 of 159 US 2021/0138249 A1

59
.
Fig

&&++n<mu){axb_ecroft_ailures
()!=truecwonhecit_dlokiew
);r(egist_acloudmnex();tgris_acloudtpbisher
lksfor();itwae_ndriatg
s();tar_povis n g
intnoff=0;auimlbure s cb=f;onaoecltds_kliew r(w);econati_netrval
ckiwi
(
if
{on ected
)
pif(done){reviosioni g
{
else
}
}
}

start
)
(
{

}
Patent Application Publication May 13 , 2021 Sheet 64 of 159 US 2021/0138249 A1

60
.
Fig
sta();r_webpfovingcurat ed(c,so);wrlneau_pdrbtlicakeys (kn);roil_wascmdxutrefos_kiw);(nladtpubishre_fokw
){(csetlupo_ dcnuertilo k:c(f){oinwet_odaki res
(){star_povis n g );(cstlorued_cluord
init_AP
;
)
(

} phy
}
Patent Application Publication May 13 , 2021 Sheet 65 of 159 US 2021/0138249 A1

61a
Fig
.
stc(m);_inderf.agxuPStmylionCad icn()t;[Link] =sct();_midn.egraftcPeuySlosatd
tca();mdr.geKtIW_Akdirwes gt(cie);ano_krwseifcmukaltwodn =ckt();[Link] );(cigeotn_kwdrafgeut_ckiwon ();-ctmad.rgeKItW_Akdirwes
p(OK);ublish_result (OK);publish_result
);(cgeotn_kiwsdaurefiwc );(spubtliasht_uas
c:KcIW_CaONFGUsRATeMD =fg_interfac
o(n_comand scmd:witch
cmd
{
)
:CSKcTIaMULWsATDIeON
stm_inerfac B;reak B;reak CMD:SKcTIaATWsUIeS istnterface Break
Patent Application Publication May 13 , 2021 Sheet 66 of 159 US 2021/0138249 A1

61b
.
Fig d(c);omwpn.lagey_tolpayUrodl ();campd.gleytC_onfiuraPtylond
();ctmad.rgeKItW_Akdirwes s=otai(pn);[Link]
r){(cigoent_kwdrsafmguelt_ckiown ki;wen_tsurmflatncioe
if(c)o{netd_catrge_koiw
g(c);eotn_kiwdarget_fkicw p(OK);ublish_result (OK);publish_result ra;ise_ro
:CGcONAFaTIMEUsRWDAeOYN
KIWIOTACMD:case ionterface
else
}
{
Break; break; }

???
}
}
Patent Application Publication May 13 , 2021 Sheet 67 of 159 US 2021/0138249 A1

61c
.
Fig
{(cg)eot_nkiwfdurateiwc){(cifonetd_cartge_kiowkr;iwne_tcofugracnioe {(cg)eotn_kiwsdaurefiwcif(co){netd_catrg_ekoiwrki;we_staunrfce (cif){onetd_cartge_kiow
rki;netwru_foatcne
*

ra;ise_ro ra;ise_ro (ciog){net_drkafigetwc r a

,i se _ r o
{
else
} else
}
{ {
else
}
} } }

} } }
Patent Application Publication May 13 , 2021 Sheet 68 of 159 US 2021/0138249 A1

62
.
Fig
p);c(lompurde_sbfaetr(c=enluorbydtie_cyunfkvpeltor
,
;
)
cif()||ml<=oaud_xbfer.taimnf_rgule (!pifkc,){uoimbwlr_esahdr_aets(ea);dnc_rtyop send_vqulop
kit=n()w;_dmaets.iwamestp
);kp(bc-rieulwfoc_eudsart a

){(koni_w id_at
}

phy

{ }
}
Patent Application Publication May 13 , 2021 Sheet 69 of 159 US 2021/0138249 A1

630

INTERFAC COMANDS
6318 C/SOTNAFIUGS 632 /SCTOANFUISG

63
.
Fig 6302 TOIMPLANT DEVICE 6314
STIMULAO INTERFAC COMAND EXCUTOR STIMULANCOMANDS
6316 6320 6304 TOCLOUD

6306
INTERFAC COMPRESIN GATEWY PUBLISHER
[Link]
D
6308 A T A 6310 BU F E R D
6312A T A
Patent Application Publication May 13 , 2021 Sheet 70 of 159 US 2021/0138249 A1

FIG
.
64
6404
640
GATEWAY ICKNTEFWRAGICE ISKNTIERWFAMICE DERKWIDATAREC CEXOMUATNODR PDUBALISTHEAR DBLOCAL OSLINUX
1

SECURITY LAYER
6408 DATA
S2??ERVICE
SERVICES MUANSGEMRNTSERVICES
SBUSIERVICE ECORNEPFSIAGTUCHNEORS CLOUD
C-MONDFISG CSTIMULDASON DREALTIMAE LSEACUYREITY SCEORMVAINCDE
1 TUIO LBOX
TCP / IP

6406
EDXAPOTRASERVICE 1 Al&ML
SMODELS ANELYRTICVSEICES
DKEIVWCIE CAONFIPGURAITON SATIMPULATION CSDTHRAENTMIALG RNECUODNIALG MNOEDURLATNIO CLONTSRPEO CDOMAPRETSION AL
&
ML
{
L/CRUITSNOUSXM

6402
Patent Application Publication May 13 , 2021 Sheet 71 of 159 US 2021/0138249 A1

7
;

65
.
Fig 1
;

=5;kKIWwReco_rdingCtlromand
G
g at e w y Co n
_ fi ucr m d
w

Idpletnoicfnkiaeslr
b//
200
the
of
each
Ofor
(
.
KIWI
a
in
)

kKIWwConf_igurcatmd
KCkwst=IW2;oSmtiualntido KIk=3;WAwcti_va onCcmtad KCkw=6s;IoWtmaStunsd
KkI=4;WwOt_aComtand
=8;gGatwe_yotaCmand
cetiu//KdnlrIhiotqWusfreidI
=
2
;

cetiuG//dnlarhiotequsfwreiady
identi=1;gsaterwaiyn_igd
birlenopctka3_ie2d {GmatewsyAdarges
;
string
1
=
id
kiwi

{Cmoesmangdoe{neof Sangat
{KImWeAsdraegs
} phong
} }
Patent Application Publication May 13 , 2021 Sheet 72 of 159 US 2021/0138249 A1

2
;

66
.
Fig

dK1;eIsWtiAnadreos
{KImWCeonfsigurat mend >c,s<monftiguraoin_pmegtrs
}
Patent Application Publication May 13 , 2021 Sheet 73 of 159 US 2021/0138249 A1

67
.
Fig

Kd=1;IeWstAidnareson SsTt2;imyulapotn_iey
E=L1;CTRICAL
scsp32oft//OoimuhnslaibtedosnIti3;mnultaeiong_valrue
=
{KImWSetisulaonCgmed stof//Theimyulpatieon ES t
{ i m
n u l a o n
m T y p e C=3;HEMICAL
O2,PTICAL

} }
Patent Application Publication May 13 , 2021 Sheet 74 of 159 US 2021/0138249 A1

68
.
Fig

,
1

{KImWAectisvaonCmgedKIWstAinareo ab2;cotilveatn

}
Patent Application Publication May 13 , 2021 Sheet 75 of 159 US 2021/0138249 A1

69
.
Fig

;
1

os2;taUrpditneUgrl
K{ImWeOtsacomgndedKeIsWtiAnaro
}
Patent Application Publication May 13 , 2021 Sheet 76 of 159 US 2021/0138249 A1

70
.
Fig

dKIe1,WsStainuComd
{KIWmRecordsingCtalme rb2;oecloeradn

}
Patent Application Publication May 13 , 2021 Sheet 77 of 159 US 2021/0138249 A1

71
.
Fig

;
1

K{ImWSetasuComgdKIeWstAinaro
}
Patent Application Publication May 13 , 2021 Sheet 78 of 159 US 2021/0138249 A1

2
;

72
.
Fig

1;d=GeatswinyAadtireosn
{GatmewyconsfiguraCmed >c,s<monftiguraoin_pmegtrs
Patent Application Publication May 13 , 2021 Sheet 79 of 159 US 2021/0138249 A1

73
.
Fig

1
;

]U

d{GamtewsyoCamgnedGeastwinyAros
osta2;_urpditen_ugrl

}
Patent Application Publication May 13 , 2021 Sheet 80 of 159 US 2021/0138249 A1

74
.
Fig

;
5

4
;

K1;sIWoAudrecs Rre=3;a_ditngyTpe
{KmIWesDagte
Ttime=2;imestamp
vfraleopute1s6driopatngIteravlER{eandiugTmype E=1;L CTRICAL
O=2;PTICAL C=3;HEMICAL

}
}
Patent Application Publication May 13 , 2021 Sheet 81 of 159 US 2021/0138249 A1

750

SEleasrtcih
7502 7504 7506
k&
akafka
f80ka Diocesng
IQntueriacy
75
Fig
.

Ge way MAoIdeRls
ES

9 7508
0 Viosualzftion
Patent Application Publication May 13 , 2021 Sheet 82 of 159 US 2021/0138249 A1

76
.
Fig
KeIf=3;//rdWIanpo_teSWscifurIhecd Rofds2;//Treaoiuhdnrtg_Tceayps
PIpat:efor1;//antndtaienctgf_iehrd//Ifore=2;KpantWeiIcaf_gtnherd
idBIbforall100K1;//dnleItoecWifgkIesr

{RemalTisDtaSogurce {KmIWesSouargce ER{eandiugTmype

E=L1;CTRICAL C=3;HEMICAL
O=2;PTICAL

KIWI Zone } } }
Patent Application Publication May 13 , 2021 Sheet 83 of 159 US 2021/0138249 A1

77
.
Fig

=1;pPreocsing_Ttype
{PrmepocsinagBleks EPr{epnocusimgType
BF=1;AINLDPTAESR
S2;MO THING
; ICA
PCA
=
3 ;
=
4

} }
Patent Application Publication May 13 , 2021 Sheet 84 of 159 US 2021/0138249 A1

78
.
Fig
3
;

tousemwIfor=1;//dlnoh_emtdiecfglie_hrd
ofb//Litwlsthasuorechiasteirfoutnldsy
ofd//Lforwitcanbeaiphseilsactehsd b=4;Sraetinp_rloiactnegds
m;m=1Iln_toedgl_eirdSo2;d-tel_rsicnptgo daSrpietslcanbegsd
{MLmBelsocakgse {MmLesodaegl
} }
Patent Application Publication May 13 , 2021 Sheet 85 of 159 US 2021/0138249 A1

79a
.
Fig
Vv=2;isualztonOpuiBlck 31

1
;
KkI=3;WiwCom_candBlock
f=FileOutlp Beock
{OmutepsBlaocgkoes{neof
=1;fSilte_roicantgo =2;pSertmris inogs
F{mileOustpaBlgocek
V{VmisuealztonOpgBecTvkisuaylzpton_e EVTi{snuyaluizptmeon LINE
CHART
1
=
; BAR CHART
=
;
2
S=3;CAT ER_PLOT
BPLOT=4;RAIN
5
;

}
PNELUORTN
} } } }
Patent Application Publication May 13 , 2021 Sheet 86 of 159 US 2021/0138249 A1

1
,

796
.
Fig

forall100K1;//IbdlenIotciWekfg_IsirdSsToft2;//imyuhlaptioen_type Oneofthe32p=3;//sItoinmsutlaeiognb_vlaeru

k{KImWCeosandBlgcekKirIwWe_dDpstano {KImWeDstinagoe cstoimulands ESt(imnulaonmType E=1L;CTRICAL C=3H;EMICAL

O=2;PTICAL

} stimulaon } phy
}
Patent Application Publication May 13 , 2021 Sheet 87 of 159 US 2021/0138249 A1

80
.
Fig

bersdwpf//Iahetnoiatusfnrlde ofc//Lpetsd,woahnitdoscatueilonhedts dsrRof2;//Teoapuhitnragce_Typd

=1;pIraentienagt_eird c=2;Sroetnpdritanoegsd
read,mustmeet
{BamtcehDsSaougrce
o{neof

be
}

readings }
Patent Application Publication May 13 , 2021 Sheet 88 of 159 US 2021/0138249 A1

81
.
Fig

M2,//smodpelP_cariemftlcsr
[Link]//Pereacinstage
Mmo=1,del_Ttype =3tIrani_estgplri =3PGRAOUCSEIANS
{MmLTreaisngBlockes EM{odneluTympe
N=1,ETUWROALK

parmets
SVM
=
,
2

} }
}
Patent Application Publication May 13 , 2021 Sheet 89 of 159 US 2021/0138249 A1

82
.
Fig

bctinulL//=1,ahsnogtiuecaogskdme

2code=String
{CumsetoBlagcke P=
,YTHON
1
2MATLAB

}
ELanguame }
Patent Application Publication May 13 , 2021 Sheet 90 of 159 US 2021/0138249 A1

83
.
Fig

Trs-a2;inumgS aryBlock =2;oSumpatryimOpoanrsy

=1;fFileiOutplBeock =1;fFileiOutplBeock

{OmutepsBlaogckes {TmraiensgSu yBloeck

} }
Patent Application Publication May 13 , 2021 Sheet 91 of 159 US 2021/0138249 A1

840

8406

CMAOSDULIENG
8408 8402
13 SHARED
84
Fig
.
SCHEMA Doplør
8404

MODULE
Patent Application Publication May 13 , 2021 Sheet 92 of 159 US 2021/0138249 A1

8506
ODUATPUAT
8404
8504
85
.
Fig

TMODULE RANSFOMI
8502
IN

IDNAPTUA
Patent Application Publication May 13 , 2021 Sheet 93 of 159 US 2021/0138249 A1

**

8406 8602
W

86
.
Fig

MCAOSDULIENSG 8604
Patent Application Publication May 13 , 2021 Sheet 94 of 159 US 2021/0138249 A1

TLOUT
OUT
:
JOB
n AP I
IN NIL

8408 8402
LFP ERP
EEG PetCT
PI ELINE ??
GENRIC SCHEMA 87
.
FIG
MEG
fMRI MRI DOPLER
para OUT LOUI
OUT
JOB
a APP
INI INI IN
Patent Application Publication May 13 , 2021 Sheet 95 of 159 US 2021/0138249 A1

8 08 80
8 10

AugstuMHCPam's SAemravzicoens
8 12
8 06
ES

88
.
Fig FRAMEWOKS LMEACRHNIEG LDEARNIPG DATA CSTREAITNSOG TRAIN G M.
.
2

8 04

8 02 E(R,BdVCoaONctslEkoCiOnIgxN
PSriocgesnailg MSaAcsXuTk UnAcareilmystv PIromceasgine MVSAaocxhitnoers MUanxiufonldg Koglsati DFAierstcouiws
Patent Application Publication May 13 , 2021 Sheet 96 of 159 US 2021/0138249 A1

8908 rVaelsiudtse 8910 Smtooadvel disk

I
890

8906 MLcTlars if enr

89
.
Fig

8904 Ap ly preocsing
8902
Rpaetaiedntfdraotma datbse
Patent Application Publication May 13 , 2021 Sheet 97 of 159 US 2021/0138249 A1

90 0
UN HUB

90
.
Fig
90 8
VSCercitoesnwal

VDTopioewn
9018 9012
WIT
III
VSCercitoesnwal

90 2
L|| I T
90 4
III
90 6
Patent Application Publication May 13 , 2021 Sheet 98 of 159 US 2021/0138249 A1

910
VDTopioewn

91
.
Fig
CSercotsio

9106
m
VSCercitoesnwal Di2bonenupdomispusmt

9102 9104
Patent Application Publication May 13 , 2021 Sheet 99 of 159 US 2021/0138249 A1

} I }

GATEWYMODULE 1 1 1 1 1
recodma
| I 1
simulatoncd I 1 1 1 1 1 1

SendQatToGwy
COMPRESIN MODULE I 1 I 1 1 1 1 1 I 1 1 1 1 1 1

92
.
Fig
Ky10
CONTRL
compresDat floudStirsancm
LCOSERD MODULE
serRcodChanl ArocesDat stimulaoncd stimulaeNron 1
Csiomulatnd
Nsetuimrolsaite 1 1 1 1

KEURAL MODULATIN
readNuot Dneautra

KOFSARTEICLMNUWDIOG
1 1 1 1 1 { 1

NEURALRECO NG ? }

SEilgencatlrisc } } } } ?? } {

ESliegcntarilcs 1

ESliegcntrailcs
PATIEN 1 I 1 1 1 I