Europe PMC Funders Group

Author Manuscript
Sci Robot. Author manuscript; available in PMC 2021 November 29.
Published in final edited form as:
Sci Robot. ; 6(54): . doi:10.1126/scirobotics.abd7935.
Europe PMC Funders Author Manuscripts

Robotic hand augmentation drives changes in neural body

representation
Paulina Kieliba1, Danielle Clode1,2, Roni O Maimon-Mor1,3, Tamar R Makin1,4,*
1Institute
of Cognitive Neuroscience, University College London, 17 Queen Square, London
WC1N 3AZ, UK
2Dani Clode design, 40 Hillside Road, London SW2 3HW, UK
3WIN Centre, University of Oxford, Oxford OX3 9DU, UK
4Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK

Abstract
Humans have long been fascinated by the opportunities afforded through augmentation. This
vision depends not only on technological innovations, but also critically relies on our brain’s
ability to learn, adapt and interface with augmentation devices. Here, we investigated whether
successful motor augmentation with an extra robotic thumb can be achieved, and what are
its implications on the neural representation and function of the biological hand. Able-bodied
participants were trained to use the Third Thumb over 5 days, including both lab-based
and unstructured daily use. We challenged participants to complete normally bimanual tasks
using only the augmented hand and examined their ability to develop hand-robot interactions.
Europe PMC Funders Author Manuscripts

Participants were tested on a variety of behavioural and brain imaging tests, designed to
interrogate the augmented hand’s representation before and after the training. Training improved
Thumb motor control, dexterity and hand-robot coordination, even when cognitive load was
increased or when vision was occluded. It also resulted in increased sense of embodiment over
the Thumb. Consequently, augmentation impacted key aspects of hand representation and motor
control. Thumb usage weakened natural kinematic synergies of the biological hand. Furthermore,
brain decoding revealed mild collapse of the augmented hand’s motor representation following
training, even while the Thumb was not worn. Together, our findings demonstrate that motor
augmentation can be readily achieved, with potential for flexible use, reduced cognitive reliance
and increased sense of embodiment. Importantly though, augmentation may incur changes to
the biological hand representation. Such neurocognitive consequences are crucial for successful
implementation of future augmentation technologies.

*
Correspondence to: [Link]@[Link].
Authors contributions:
P.K., R.M and T.M. conceived and designed the study; D.C. designed the Third Thumb; P.K. and D.C. performed the experiments;
P.K. and R.M. analysed the data; P.K and T.M wrote the manuscript with input from all co-authors.
Competing interests:
Authors declare no competing interests.
 Kieliba et al. Page 2

Introduction
Motor augmentation is a growing field aimed at extending our physical abilities. Engineers
are currently developing extra robotic fingers and even entire arms created to augment our
bodies by expanding our natural motor repertoire (1–6). These augmentative devices aim
to change the way we interact with the environment, which entails changes to how we
Europe PMC Funders Author Manuscripts

move and operate our biological body. Yet, despite the rapid advancements in augmentative
technologies, little notice is given to the crucial question of how the human brain might
support them. Here we asked whether human brain could accommodate motor control of
an extra robotic finger, focusing on its impact on the neural representation of the biological
hand.

The hand has a well-established functional representation in the brain, with each of the
fingers represented relative to the others. This neural fingerprint of the hand develops
very early on (7, 8). It is highly consistent within (9) and across (10) participants and is
preserved even after severe loss of motor functions due to e.g. stroke (10), spinal cord injury
(11), disability (12) or even hand amputation (13–15). Similarly, recent studies on motor
learning in adults show that while premotor and parietal regions show reorganisation of hand
representation in the early stages (1st week) of intensive motor training, hand representation
in the primary motor cortex (M1) remains stable throughout training (16, 17). At the same
time, hand representation has been suggested to reflect daily hand use (10), with studies
showing that it may be altered under constrained circumstances. Most notably in musicians’
dystonia, a clinical condition involving increased finger enslavement following intensive
skill practice, the individualised representation of single fingers has been shown to collapse
(18, though see 19).

Here we trained able-bodied people to use an extra robotic thumb (the Third Thumb,
Europe PMC Funders Author Manuscripts

designed by Dani Clode (20), hereafter “Thumb”) over the course of 5 days, including
both lab-based and ‘in-the-wild’, unstructured daily use. The Thumb is a supernumerary
robotic finger, with two degrees of freedom, controlled by pressure exerted with the big
toes, designed to extend the natural repertoire of hand movements (Fig. 1A-B; Fig. S1).
We examined participants’ ability to develop motor skill and dexterity with the Thumb
under daily life settings, across key aspects of hand-robot interactions, such as collaboration,
shared supervision and individuation. During training, we also tracked (biological) finger
co-use and compared it with normal hand use. We tested for changes in motor control and
embodiment of the Thumb, as well as hand-Thumb coordination before and after training.
Augmented participants were compared to a control group that underwent a similar training
regime while wearing a static version of the Thumb. We also examined how neural hand and
body representation changed following training. We hypothesised that successful hand-robot
cooperation will promote changes to finger co-use, and thus modify both biological and
artificial body representation.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 3

Results
The Third Thumb
The Third Thumb is a 3D printed robotic thumb (20), originally designed as an
augmentative, general-use tool for able-bodied people (see Video S2). The Thumb is worn
over the ulnar side of the right palm, opposite to the user’s natural thumb (Fig. 1A-B).
Europe PMC Funders Author Manuscripts

It is actuated by two motors, allowing proportional control of two independent degrees

of freedom - flexion/extension and adduction/abduction. The motors are mounted on a
wrist strap (Fig. 1A-1) and powered by an external battery pack worn on the upper arm
(Fig. 1A-2). The movement of the Thumb is controlled with pressure sensors fixed to the
underside of the big toes of the user’s feet (Fig. S1). The pressure sensors are powered
by the external batteries secured around each ankle (Fig. 1A-3). A wireless communication
protocol is used to send the signal from the pressure sensors to the motors which actuate the
Thumb. Pressure exerted with the right toe pulls the Thumb across the hand (flexion), while
the pressure exerted with the left toe pulls the Thumb up towards the fingers (adduction).
The extent of Thumb movement is fully proportional to the pressure applied. As such,
the Thumb can be used when sitting or standing, but not while walking. The wireless
design allows users to operate the Thumb in an unstructured environment, providing us
with the unique opportunity to encourage participants to use the Thumb outside the lab and
unsupervised.

Daily training improves hand-Thumb coordination, even with reduced visual information
and increased cognitive load
We first characterised motor performance of the augmented hand throughout the 5
days of usage. Augmentation participants completed five daily in-lab training sessions
(1.58±0.22hr; mean±std) and were additionally encouraged to use the Thumb outside the
Europe PMC Funders Author Manuscripts

lab in unstructured environment (2.61±1.18hr; self-reported). The average use time, as

quantified by the automatic usage logs, was 2.95±0.84hr per day, out of which a total of
1.37±0.49hr involved active Thumb movement.

During daily training sessions, participants were presented with a variety of reaching,
grasping and in-hand manipulation tasks designed to introduce complex hand-robot
interactions and to be purposefully challenging to perform with only one hand (see Video
S1). In the collaboration tasks, participants had to use the extra Thumb together with another
finger to pick up multiple objects. In the shared supervision tasks, participants had to use
the extra Thumb to extend the natural grip of the hand and to free up the use of their
biological fingers. Finally, in the individuation task participants had to work on the fine
motor control of the Thumb, while having their hand fully occupied with a task-irrelevant
object. Augmentation participants showed significant improvement on all the training tasks
(main effect of time for all tasks: p<0.001, ηp 2>0.5 Fig. 1D).

Motor control was further assessed using a hand-Thumb coordination task, requiring
participants to oppose the Thumb to their biological fingertips. Even though controlling the
Thumb with the big toes may seem unusual, participants were able to successfully perform
the hand-Thumb coordination task even at baseline (Fig. 2B), though this performance

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 4

was significantly improved after training. Significant improvements were observed both
during daily training (F(4,76)=28.24, p<0.001, ηp 2=0.6 Fig. 2A-C), and when comparing
the performance pre- and post- the 5 days of training, using a sequential variation of
the same task (see Materials and Methods). Here, augmentation participants showed
significant improvements, not only with vision (t(19)=8.96, p<0.001, ηp 2=0.81), but also
when blindfolded (t(19)=7.40, p<0.001, ηp 2=0.74, Fig. 2E), indicating improved Thumb
Europe PMC Funders Author Manuscripts

proprioception.

As the improvements described above could be skewed due to task repetition, we also
tested a group of 11 control participants, who underwent similar pre- and post- tests and
training regime but wore a static version of the Thumb for the same duration of time -
4.11±1.06hr per day (t(29)=0.526, p=0.6, BF=0.39), out of which 2.93±1.34hr were outside
of the lab (t(29)=-0.697, p=0.49, BF=0.42 for wear-time group comparison). Similarly to
the augmentation group, the control group, who had to develop 5-fingered solutions to the
same problems, showed proportional improvements in nearly all training tasks (Fig. S2).
As control participants did not have to learn to control a new robotic device, their training
performance was significantly better, as compared to the augmentation group, with the
exception of the shared supervision tasks. This indicates that given specific task demands,
the extended motor ability provided by the Thumb can also increase participants’ functional
efficiency. Importantly, the control group was only allowed to use the Thumb during the
pre-post sequential hand-Thumb coordination test. Although control participants showed
significant pre-post improvements (with vision: t(9) = 3.74, p=0.005, without vision: t(9) =
2.35, p=0.043), those were significantly lower than the ones observed in the augmentation
group, both with vision (significant effect of group revealed by ANCOVA F(1,27)=22.86,
p<0.001, ηp 2=0.44) and blindfolded (F(1,27)=11.96, p=0.002, ηp 2=0.28).

A key component for successful augmentation is being able to multi-task, even when not
Europe PMC Funders Author Manuscripts

paying attention to controlling the device. Importantly, augmentation participants’ motor

performance with the Thumb was not impacted by increased cognitive load. This was
examined during the first and last days of training, using a dual-task (21, 22), requiring
participants to perform simple arithmetic operations while simultaneously using the Thumb
to complete a collaboration task (building a Jenga tower). We found no significant cognitive
load x session interaction (F(1,16)=0.003, p=0.959, BF=0.34) and no main effect of
cognitive load (F(1,16)=2.465, p=0.136, BF=0.32, Fig. 2D) on the motor performance.
At the same time, participants made a modest number of arithmetic mistakes (on average
15-19% of trials per participant), showing that the dual task indeed increased the cognitive
load demands.

Together, these results indicate that participants learned to operate the Thumb under a
variety of circumstances, extending beyond their specific training, and performed similarly
with and without the increased cognitive load. Yet, with the exception of the shared
supervision tasks, training performance of the augmentation group was reduced relative
to controls, highlighting that extensive, long-term practice is needed in order to functionally
benefit from motor augmentation.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 5

Training enhances subjective sense of Thumb embodiment

We also assessed the perceived (phenomenological) sense of embodiment of the Thumb
following the training period, relative to baseline. During pre- and post- testing sessions,
participants were asked to respond to statements relating to key embodiment features (23,
24). Augmentation participants reported a significant increase of embodiment in each of
the four categories (body ownership: t(13)=6.57, p<0.001, ηp 2=0.77; agency: t(13)=4.07,
Europe PMC Funders Author Manuscripts

p<0.001, ηp 2=0.56; body image: t(13)=5.215, p<0.001, ηp 2=0.68; somatosensation:

t(13)=6.032 p<0.001, ηp 2=0.74; Fig. 2F). Importantly, the increased embodiment we found
in the augmentation group significantly exceeded that reported in the control group (agency:
F(1,21)=10.013, p=0.009, ηp 2=0.285; body image: F(1,21)=11.16, p=0.012, ηp 2=0.26;
body ownership: F(1,21)=4.07, p=0.057, ηp 2=0.16). These results indicate that active usage
is critical for developing proprioception and embodiment of the robotic Thumb. For the
perceived somatosensation scores, the group comparison was nonsignificant (BF=0.48).

Next, we examined potential changes to body image (perceptions and attitudes concerning
one’s body representation (25)). Those were tested while participants were not wearing the
Thumb. We found no significant pre- to post- changes in tactile judgements (t(18)=0.164,
p=0.87, BF=0.24). Similarly, we did not observe any convincing evidence for visual
judgement changes (see Supplementary Materials), as our findings were not specific to
the augmented hand (main effect of time: F(1,16)=6.89, p=0.018; hand x session interaction
F(1,16)=0.019, p=0.89, BF=0.326).

Together, these findings indicate that while hand augmentation impacts the sense of
embodiment over the device, it does not necessarily influence one’s own implicit body
image.

Hand augmentation impacts motor control of the natural hand

Europe PMC Funders Author Manuscripts

Next, we investigated the impact of hand augmentation on motor control of the augmented
(right) hand. We first examined the complexity of the hand movements (i.e. kinematic
synergies) captured with a Cyberglove during the in-lab training. We found that in general
more principal components were needed in the augmentation group, compared to the control
group, to explain the 80% of the total variance of the hand movements (F(1,22)=5.52,
p=0.03, ηp 2=0.2, Fig. S4B). This difference was, however, strongly driven by the amount
of variance explained by the first principal component, corresponding to the coordinated
flexion of all fingers (Fig. S4A). Indeed, the variance explained by this inter-finger synergy
was significantly decreased in the augmentation group compared to controls (F(1,22)=6.27,
p=0.02, ηp 2=0.22, Fig. 2G), while no difference was found between the first and the
last days of training (F(1,22)=2.57, p=0.12, BF=0.62). Since the remaining principal
components represent more intricate finger movements, the decrease of variance explained
by the first kinematic synergy suggests more finger individuation in the augmentation group.

To uncover more detailed changes in biological finger coordination, we assessed the degree
of kinematic coupling between individual digit pairs. Here again, no differences in finger
coordination were found between the first and the last days of training (main effect of
time: F(1,23)=1.3, p=0.27, BF=0.17). For the augmentation group, this finding indicates that

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 6

the strategies implemented for incorporating the Thumb into the motor repertoire during
day 1 were generally preserved throughout training. This is likely a consequence of our
experimental design involving repeating the same set of tasks over multiple days. Consistent
with the PCA results, we found significant differences in finger coordination implemented
across groups (group x finger-pair interaction: F(9,414)=2.66, p=0.005), with an overall
decrease in inter-finger coupling in the augmentation group relative to controls (main effect
Europe PMC Funders Author Manuscripts

of group: F(1,23)=6.98, p=0.01, Fig. 2H-I). Together these results demonstrate small, but
robust changes to finger coordination, likely corresponding to more complex movement
patterns acquired by the augmentation group during Thumb use.

Changes in inter-finger motor control were further investigated through force enslavement
(involuntary force production by non-instructed fingers), measured pre- and post- Thumb
use. No significant group differences in force enslavement were found (F(1,27)=0.06,
p=0.81), with the results providing only anecdotal evidence for the increase of enslavement
caused by the biological thumb in the augmentation group, post- compared to pre- training
(F(1,17)=3.36, p=0.08, see Fig. S5). Given the ambiguous nature of these results, no clear
conclusions could be drawn.

Biological hand’s representation shrinks following Thumb use

Having observed altered finger coordination patterns in the augmentation group, as
compared to controls, we sought to understand whether Thumb usage can impact the
biological hand representation in the sensorimotor cortex. We used fMRI to compare neural
hand representation before and after Thumb use, using the non-augmented (left) hand as a
within-participants control condition. During the scans, participants were required to make
individuated finger movements with their biological fingers. Note that due to MRI safety
considerations, participants were not wearing the Thumb during the scans.
Europe PMC Funders Author Manuscripts

To investigate changes to the augmented hand’s representation, we estimated the

dissimilarity between multivariate activity patterns elicited by individual fingers’ movements
in the sensorimotor cortex, as measured using cross-validated Mahalanobis distance (26).
Small inter-finger distances indicate that the representation of the two fingers is more
similar/overlapping, while larger distances imply more individuated finger representation.
This experimental approach is the current gold standard in the field, and has been
extensively used to study plasticity and stability of hand representation (9, 10, 13, 14).

Augmentation participants showed significantly reduced inter-finger distances of the

augmented (right) hand’s representation in the sensorimotor cortex following Thumb use
(t(25.1)=2.3, p=0.03, Fig. 3). In other words, the biological fingers became less distinctive
from each other following training. This shrinkage effect was specific to the augmented
hand, as demonstrated by a significant hand x time interaction (F(1,722)=12.89, p<0.001,
Fig. 3C). These findings show that using the extra Thumb not only alters the motor control
of the biological hand, but also impacts how that hand is represented in the brain. Crucially,
this effect was observed while participants were not using or even wearing the Thumb. A
computational simulation, elaborated in Fig. 4, confirmed that the observed results could be
driven by both adaptive and maladaptive neural plasticity mechanisms.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 7

We confirmed that the shrinkage of the augmented hand representation was not associated
with net differences in overall activity levels in the hand areas (hand x time interaction:
F(1,19)=1.95, p=0.18, BF=0.36), and that it did not impact the typicality of the
representational structure (t(19)=1.12, p=0.277, BF=0.4); see Supplementary Materials).
No significant changes to the hand representation were observed in the control group, as
demonstrated both in a pre-post comparison of the right hand’s representational structure
Europe PMC Funders Author Manuscripts

(t(10.4)=-0.245, p=0.81, BF=0.32) and the hand x time interaction (F(1,342)=0.71, p=0.4,
BF=0.95, see Fig. S6). This further indicates that without Thumb use, the biological hand’s
representation is relatively stable, as previously demonstrated in motor training studies
(16, 17). Note however, that the 3-way interaction (hand x time x group) did not reach
significance (F(1,1064)=2.02, p=0.16), likely due to insufficient statistical power.

To further test whether the observed shrinkage of the neural hand representation may depend
on recent Thumb use, additional analysis was conducted using a partial dataset acquired
in a follow-up scan (7-10 days after the end of training) from a sub-group of available
participants (n=12). The original pre-post hand x time interaction remained significant even
with this smaller subset of people (F(1,417.99)=4.8, p=0.03), though the initial difference
between the pre- and post- representation of the right (augmented) hand was ambiguous
(t(13.7)=1.36, p=0.19, BF=0.58). We found moderate Bayesian evidence (t(13.6)=0.45,
p=0.66, BF=0.31) for a null difference between the distances measured during pre- and
follow-up sessions, suggesting that in the follow-up scan the reduction of the inter-finger
distances was at least partially diminished. Note, however, that no significant difference
between the augmented hand’s representation in post- and follow-up scans was observed
(t(12.9)=0.71, p=0.5, BF=0.37).

Hand-toes functional relationship in the primary sensorimotor cortex remains stable

Finally, we ran a series of analyses exploring the relationship between neural representations
Europe PMC Funders Author Manuscripts

of the hands and the toes – the body-part controlling the Thumb’s movements
representations in the brain. We first focused on primary sensorimotor cortex and found
no significant changes relating to Thumb training (Fig. S7). Specifically, we examined
toes-specific net activity within the augmented hand area (t(19)=0.47, p=0.64, BF=0.26),
dissimilarity between multivariate activity patterns elicited by hand and toes movements
(hand x time interaction: F(1,19)=1.46, p=0.24, BF=0.38), and functional coupling between
sensorimotor hand and feet areas (resting state functional connectivity (27), t(19)=1.375,
p=0.185, BF=0.52). These results suggest that while augmentation might promote plasticity
locally (i.e. between fingers), the neural representation of the body at large remains
unchanged in the primary sensorimotor cortex.

Lastly, we examined inter-body-part representations in the Supplementary Motor Area

(SMA), involved in motor learning and coordination (28). SMA contains a much cruder
body-map compared to the sensorimotor cortex (28), providing a better substrate for
exploring changes to inter-body-part relationships. Using cross-validated Mahalanobis
distances, we found a significant reduction in the hand-toes distance following training,
specific to the augmented (right) hand (t(19)=3.56, p=0.002, ηp 2=0.4) and resulting in
a significant hand x time interaction (F(1,19)=9.13, p=0.007, ηp 2=0.33, Fig. 4). We

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 8

repeated this analysis with an additional body-part unrelated to Thumb’s control (lips)
and again found a significant hand x time interaction (F(1,19)=9.1, p=0.007, ηp 2=0.35)
with no significant three-way (hand x time x body-part) interaction F(1,19)<0.001, p=0.99,
BF=0.31). In other words, the reduction in the inter-body-part distance was similar across
hand-toes and hand-lips. This finding suggests an overall decrease in selectivity that could
be attributed to increased tonic inhibition, as examined in our computational simulation,
Europe PMC Funders Author Manuscripts

simulated the sensorimotor cortex (see Fig. 4).

Discussion
Here, we provide a comprehensive demonstration of successful motor integration of a
robotic augmentation device (the Third Thumb) and explore how augmentation impacts
the user’s hand function and representation. After only 5 days of Thumb use, participants
showed significant improvements in augmented hand motor performance across multiple
tasks. In addition to individuated control of the extra Thumb, participants were able
to integrate Thumb motor control with the movements of their natural hand, requiring
collaboration, shared supervision and hand-robot coordination. Motor performance was
greatly improved even without visual feedback and remained stable under increased
cognitive load, though note that increasing cognitive load demands even further is likely
to eventually lead to increased interference with the motor performance (21). The ability
to successfully coordinate between the Thumb and the biological hand across diverse task
demands is crucial for successful adoption of augmentation devices. We further show that
hand augmentation resulted in increased explicit sense of embodiment over the Thumb
- a key goal for successful augmentation (29), while implicit body image was found to
be stable. By demonstrating successful adaptation to motor augmentation under diverse
settings, our findings extend earlier pioneering proof-of-concept accounts of successful
usage of extra robotic fingers (1, 4, 6, 30–32) or arms (3, 5) under restricted circumstances.
Europe PMC Funders Author Manuscripts

Importantly, successful adoption of augmentative technologies relies not only on the user’s
proficiency in operating the robotic device. A further challenge for augmentation is to
ensure that the device usage will not impact users’ ability to control their biological body,
especially while the augmentative device is not being used or even worn. Therefore, a
critical question for anyone interested in safe motor augmentation is whether it would incur
any changes to the user’s biological body representation. This concern is rooted in previous
research of brain plasticity, demonstrating that our motor experience shapes the structure
and function of the nervous system (10, 33). As such, since motor augmentation is designed
to change the way we interact with the environment, it is reasonable to predict that it will
reshape the neural basis of our biological body. Moreover, since we were not born with
the innate capacity to control additional robotic body parts, successful motor augmentation
likely requires extensive long-term practice, as highlighted in our training results. With that
in mind, our investigation was focused on changes incurred to the body representation while
the Thumb wasn’t being operated. This approach allows for our findings to be generalised to
other forms of robotic thumb control.

Traditionally, body representation in the sensorimotor cortex is considered to be highly

adaptive even in the adult brain (34, 35) however recent research contributes a new

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 9

perspective on its malleability (13, 14). Tools have been suggested to update the biological
body representation, for example by tool-body integration (36–38). Yet, tools are normally
used to replace the capacity of the hand, rather than to accompany it. Therefore, when
using a tool, one is not required to radically alter their hand function (for example,
the user will choose a grip for the tool’s handle that fits the natural synergies of the
fingers). As such, tool-use does not entail an updated representation of the hand itself.
Europe PMC Funders Author Manuscripts

Conversely, motor augmentation invites the user to reinvent the way they use their own body.
Consequently, here, motor integration of the Thumb altered the natural finger coordination
patterns (kinematic synergies) of the augmented hand, with the augmentation group showing
more complex movement patterns than the control group. This challenge is more closely
akin to the acquisition of a new and complex motor skill – e.g. learning to play the
piano. Recent research has demonstrated that long-term training leads to changes in finger
representations (39). Specifically, trained pianists (over the course of many years, starting
in childhood) demonstrate altered hand representation (lower inter-finger representational
distances) relative to novices. This evidence further emphasises the need to examine how
long-term motor augmentation can impact the biological hand representation.

Here, we used a variety of pre- to post- measures to study changes in body representation
when the Thumb was not being used, or even worn. While some aspects of body
representation (e.g., body image, large scale connectivity profile) were found to be stable,
semi-intensive Thumb usage (2.3–6.3 hours per day) resulted in mild, yet significant
changes to the hand representation. Specifically, we observed a shrinkage of the neural
hand representation in the sensorimotor cortex. This is likely a consequence of the motor
adaptations that the users made to best cooperate with the augmentation device (10, 40,
41), as further supported by the follow-up scan taken 7-10 days after Thumb usage had
ceased. As mentioned in the introduction, inter-finger representation is highly stable even
after intensive motor training, so long as this training doesn’t introduce changed inter-finger
Europe PMC Funders Author Manuscripts

coordination patterns (16). Conversely, shrinkage of inter-finger distances was recently

reported in a finger ‘syndactyly’ study, causing abrupt and profound change to inter­
finger coordination (9). The reduced inter-finger distance was associated with maladaptive
perceptual consequences, i.e., reduced perceptual acuity to discriminate between tactile
stimuli across the fingers. This is in stark contrast to a recent report of individuals who
were born with a 6th (fully operational) finger and could harness processes of developmental
plasticity to establish normal motor control across all six fingers (42).

As illustrated through the computational simulation shown in Fig. 4, our findings are
compatible with both adaptive and maladaptive plasticity mechanisms. As we originally
hypothesised, the shrinkage effect observed here could be disruptive, e.g. akin to other
studies reporting neural correlates of decreased motor control (18). It could also result
from homeostatic plasticity mechanisms, aimed at stabilising brain activity in presence of
abrupt input changes (43). This interpretation is supported by the reduced inter-body part
distances found in the SMA. Alternatively, the shrinkage effect could also be directed at
establishing optimal representation of the Thumb relative to the rest of the augmented hand,
and as such involved in developing motor control over a new body part. For instance, by
inducing new kinematic synergies, learning to use the extra Thumb may be pushing the
network outside of its existing manifold (44) to allow for formation of new neural activity

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 10

patterns. In other words, the observed neural changes could be reflective of a more compact,
but not necessarily less functional hand representation. At this early stage of research, it
is not yet clear whether these changes are adaptive, maladaptive or epiphenomenal. The
behavioural evidence, examining the impact of hand augmentation on finger enslavement,
was unfortunately too ambiguous to determine the answer to this key question (see Fig.
S5). Yet, regardless of the specific mechanism, our evidence nevertheless suggests that
Europe PMC Funders Author Manuscripts

motor augmentation might incur some changes to the augmented hand’s representation.
Considering that neuroimaging results have previously been shown as reliable biomarkers
for behavioural outcomes, relating to both motor control (e.g. in stroke recovery, (45)) and
pain (46), we believe it is crucial to consider whether the observed neural shifts in the
biological hand representation could be incurred safely (29).

To conclude, emerging technologies designed to assist, substitute and even augment our
motor abilities hold tremendous promise for transforming the lives of both disabled and
healthy communities. Hand augmentation could benefit diverse groups of people, from
factory workers to surgeons, allowing them to perform their labour more safely, without
having to coordinate their movements with assistants or external devices; from healthy
individuals to those with temporary or chronic hand impairment (1), looking to improve
decreased hand functionality. This vision depends not only on the exciting technological
innovations, it also critically relies on our brain’s ability to learn, adapt and interface with
these devices. Therefore, as technology becomes more integrated with the human body,
we see new challenges and opportunities emerging from neural and cognitive perspectives.
Critical questions arise as to how such human-machine integration can be best achieved,
given expected neurocognitive bottlenecks of brain plasticity. Here, we demonstrate that
successful integration of motor augmentation can be readily achieved, with potential for
flexible use, reduced cognitive reliance and increased sense of embodiment. Importantly
though, such successful human-robot integration may have direct consequences on key
Europe PMC Funders Author Manuscripts

aspects of body representation and motor control, be it adaptive or maladaptive, which need
to be understood and explored further before this technology can be widely implemented.

Materials and Methods

Participants
36 healthy volunteers (23 females, mean age = 23.1±3.89, all right handed – Edinburgh
handedness inventory (EHI) score = 77.44±23.98) were recruited from internet-based
advertisements and randomly assigned to either augmentation (n=24, 14 females, mean
age = 22.9±4.12, EHI score = 80.52±17.71) or control (n=12, 9 females, mean age =
23.5±3.55, EHI score = 71.83±32.77) group. All participants were right-handed, between the
ages of 18-35, did not have any known motor disorders and reported no counterindications
for magnetic resonance imaging (MRI). Professional musicians were excluded from the
study. Handedness was confirmed using the EHI. Ethical approval was granted by the UCL
Research Ethics Committee (REC: 12921/001). All participants gave their written informed
consent before participating in the study.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 11

Due to scheduling conflicts, 1 control participant and 3 augmentation participants dropped

out of the study. Additionally, due to technical problems during data collection, 1
augmentation participant was discarded from the study.

Experimental design
To assess the effects of hand augmentation on body representation, we implemented a
Europe PMC Funders Author Manuscripts

longitudinal experimental design (Fig. 1C), involving 8 experimental sessions conducted

across 7-9 days. All participants undertook (i) a 1-hour familiarisation session, introducing
the equipment and the behavioural tasks; (ii), a 4-hour baseline (pre-test) session consisting
of behavioural testing and an MRI scan; (iii) 5 2-hour training sessions conducted over the
5 subsequent days (1 session per day); (iv) a final 4-hour post-test session corresponding to
the baseline session. Additionally, 12 of the participants from the augmentation group also
undertook a secondary follow-up MRI session conducted 7-10 days after the end of training.
Since the acquisition of the follow-up dataset was decided after the study onset, based on the
preliminary results, we were unable to collect the data from all of the study participants. Due
to scheduling issues, 1 augmentation participant and 1 control participant completed only
4/5 training sessions.

All study participants were asked to wear an extra robotic thumb (the Third Thumb; Fig.
1A-B) on their right-hand throughout the day. Participants were instructed to wear the
Thumb during the in-lab training sessions and to continue wearing it outside of the lab for
at least 4 hours per day. The augmentation group had full motor control over the Thumb and
needed to actively use it to complete the training tasks. They were also encouraged to use
it as much as possible outside of the lab for a free-style environment exploration (‘in the
wild’). The control group wore a static (not-moving) version of the Thumb and completed
the training tasks without being able to control it. Due to initial equipment issues,, the first 2
control participants did not wear the Thumb during training.
Europe PMC Funders Author Manuscripts

Usage measures ‘in the wild’

To monitor Thumb usage outside the lab, self-reported wear time and Thumb usage
examples were collected daily from all wearers/users. Daily reports were averaged across
days and an independent samples t-test was used to test for differences in wear-time between
the augmentation and control group. In addition, both pressure sensors were equipped with a
SD-card data logger. While the Thumb was on, both sensors were logging the corresponding
motor’s position and the associated timestamp to the SD cards. If the participant turned
the Thumb off during the day, the recording was paused and resumed after the motor was
restarted. Those recordings were used to further quantify the number of hours participants
spent using the Thumb per day. Use time was defined as the time spent wearing the extra
Thumb with the motors of the Thumb switched on, while movement time was defined as the
time spent actively exerting pressure with the big toes while the Thumb was switched on.
Due to initial equipment issues, the sensors’ data from first 3 augmentation participants were
not recorded.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 12

Training protocol
During the training sessions, participants were asked to complete a set of reaching and
grasping tasks. These tasks were designed to encourage the use of the Third Thumb and
allow the participants to develop complex hand-robot interactions. The task execution was
restricted to the augmented (right) hand. The augmentation group was instructed to use
the extra Thumb to complete the training tasks. The control group, wearing the static
Europe PMC Funders Author Manuscripts

version of the Thumb was instructed to complete the training tasks using only their natural
fingers, i.e. without using the Thumb. Training tasks required participants to (i) use the
Thumb in collaboration with another finger to pick up multiple objects (collaboration, e.g.
building a Jenga tower); (ii) use the Thumb to extend the natural grip of the hand and
to free up the use of the biological fingers (shared supervision, e.g. stirring cups); (iii)
use the Thumb individually, while having their hand occupied with task-irrelevant objects
(individuation, e.g. stacking tapes) or to (iv) oppose the robotic Thumb to one of the natural
fingers (hand-Thumb coordination). For all of the tasks, participants were seated at a desk
facing the camera recording their hand movements. Each task was conducted for 10-15
minutes and repeated on 2-4 separate training days, with the exception of the hand robot
coordination task (see Supplementary Materials), which was performed during each of the
training sessions.

To quantify the improvement of the augmentation group on each of the training tasks, the
outcome measure of each task was averaged for each participant and each training session.
As different participants had slightly different training regimes, in terms of distribution of
tasks across the days, we sorted the average scores based on the order of task repetition (i.e.
1st, 2nd, 3rd time the task was repeated regardless of which days it was repeated on). These
data were then analysed using a repeated measures ANOVA in SPSS.

Numerical cognition
Europe PMC Funders Author Manuscripts

To assess the cognitive load related to Thumb use, a numerical cognition task was performed
twice, on the first and the last training session (21). The task was adapted from previous
studies, showing that numerical cognition impacts motor performance while controlling
a virtual prosthetic arm (22) or a brain-computer interface (51). Participants were asked
to perform a cooperation task - building a Jenga tower (see Supplementary Materials),
while simultaneously presented with a set of low and high pitch auditory tones played
from a laptop. The tones were presented every 1-6s in a randomised order, for a total
duration of 1 minute per block. Starting with a number 10, participants were instructed
to add 1 to the current number after hearing a high tone, and subtract 1 from the current
number after hearing a low tone. After each mathematical operation, participants were
instructed to verbally respond with the resulting number. In order to assure participants’
engagement with the dual task, participants were explicitly instructed to pay closer attention
to the arithmetic operations, treating the motor task (Jenga building) as a secondary task.
Participants performed 5 blocks of the numerical cognition task during each session.
Numerical cognition blocks were always preceded and followed by 5 blocks of normal
(baseline) building a Jenga tower task (5 baseline blocks, 5 numerical cognition blocks, 5
baseline blocks). Note that the first 3 participants did not complete the numerical cognition
task.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 13

For each participant, the average number of Jenga floors built was calculated from all the
numerical cognition blocks in which the correct mathematical operations were performed.
Trials in which a wrong number was given were discarded (on average 15-19% of trials were
discarded per participant). Note however, that similar results were obtained when including
the erroneous trials (see Fig. S3). To determine whether the extra cognitive load caused
by the numerical cognition task had any impact on participants’ motor performance when
Europe PMC Funders Author Manuscripts

using the extra Thumb, the average score from the numerical cognition task was compared
to the baseline score. This was done separately for the first and the last day of training. The
baseline score was calculated as the average number of Jenga floors built across the two
baseline blocks (10 trials) that proceeded and followed the numerical cognition task.

Tracking hand movements

To assess the changes in finger coordination across all training tasks, we tracked the
kinematics of the augmented (right) hand using flex sensors embedded in a dataglove
(CyberGlove, Virtual Technologies, Palo Alto, CA, USA). Note that finger kinematics have
been previously shown to reflect the brain organisation better than EMG-derived measures
(10). Here, the sensors of the dataglove were associated with 19 degrees of freedom and
measured the joint angles of the metacarpal-phalangeal (MCP), proximal interphalangeal
(PIP) and distal interphalangeal (DIP) joints of the four fingers, the carpometacarpal (CMC),
metacarpal-phalangeal (MCP) and interphalangeal joint (IP) of the biological thumb, the
three relative abduction angles between the four fingers and the abduction angle between the
biological thumb and the palm of the hand. Sensors were sampled continuously at 100 Hz
using Shadow Robot’s ([Link] CyberGlove interface for the Robot
Operating System (ROS, [Link]

Participants wore CyberGlove underneath the extra Thumb throughout all of the training
sessions. Kinematics associated with each of the training tasks performed during a given
Europe PMC Funders Author Manuscripts

session were recorded onto a separate file. CyberGlove was calibrated for each participant at
the beginning of each training session, using a min-max pose calibration procedure provided
with the ROS CyberGlove package (see Supplementary Materials). Due to initial equipment
issues, the first 4 augmentation participants did not wear the CyberGlove during training.

Hand kinematics analysis

We focused the hand kinematics analysis on the data recorded during the first and last
days of training. The joint angles were smoothed using a 3rd order Savitzky-Golay filter,
with a window length of 151 samples. Angular velocities were then calculated from the
first difference of the filtered joint angle data divided by the time step. Since most of the
finger movements employed during the training tasks were executed using the PIP joints,
to simplify the analysis (10), only data from these five joints were analysed, resulting in 5
timeseries signals per session per participant. Due to acquisition errors, for 2 augmentation
participants and 2 control participants, the data recorded during the first day of training
was unavailable. Therefore, for these participants, the data from the second day of training
was used instead. Similarly, when the data from the last day of training was unavailable
(3 augmentation participants, 2 control participants), the data from the penultimate day

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 14

of training was used instead. Due to calibration issues, the hand kinematics data of 1
augmentation and 2 control participants were discarded from subsequent analysis.

To quantify the complexity of the hand movements across both groups, we first conducted a
Principal Components Analysis (PCA) of the angular velocities of the PIP joints. For each
participant and session, the 5 angular velocities were z-normalised (52) and decomposed
Europe PMC Funders Author Manuscripts

into subject- and session-specific kinematic synergies using PCA. 5 principal components
(PCs) were computed for each participant and session. The extracted PCs were matched
across subjects using normalised scalar product (dot product) and ordered according to
the amount of variance explained by each component. Consistent with the literature (53,
54), we found that the first PC accounted for more than 40% of total variance and
reflected a coordinated movement of all the fingers (see Fig. S4 for all the PCs). To
quantify the dimensionality of the hand movements, for each participant and day, we
recorded the number of PCs needed to explain 80% of total variance (55). These were
then compared across groups in a repeated-measures ANOVA with time (day 1, day 5)
as a within-subjects factor and group (augmentation, control) as between-subject factor.
To quantify the contribution of the alldigit movements to the complexity of the hand
kinematics, we compared the amount of variance explained by the first PC across both
groups using the same repeated-measures ANOVA design.

Next, to interrogate more detailed changes to the finger cooperation pattern caused by
the hand augmentation, we looked at the degree of coupling between digit pairs, adapting
the methods used in (53). We used linear regression to fit the angular velocity data of a
given digit as a function of the angular velocity of each of the other digits individually.
This yielded a single determination coefficient (R2) for each digit pair, expressing the
proportion of total variance of each digit’s angular velocity that could be explained by a
linear reconstruction, based on its paired regression with each of the other digits. Qualitative
Europe PMC Funders Author Manuscripts

comparison between the results obtained from the control group (who did not use the Third
Thumb during the training) and outcomes of previous hand kinematics research conducted
during free movement (10, 53), confirmed that the conducted analysis resulted in a typical
finger synergy profile.

To assess the effect of Thumb use on the finger coordination profile across groups,
we performed a linear mixed model analysis (LMM) with fixed factors of time, group
(augmentation vs controls) and digit pair, a random effect of participant and a random
participant-specific slope of time. The LMM was evaluated in R (version 3.5.2) under
restricted maximum likelihood (REML) conditions with Satterthwaite adjustment for the
degrees of freedom.

Pre-post testing protocol

To assess the neural correlates of hand augmentation we used a set of pre- to post­
training comparison measures, consisting of both behavioural and neuroimaging tasks. To
characterise the emerging representation of the extra Thumb, we probed the proprioception
and motor control of the Thumb using a sequential variation of the hand-Thumb
coordination task. We also assessed perceived (explicit) embodiment of the Thumb using
questionnaires. To interrogate changes to the natural hand representation, we measured

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 15

biological finger co-dependencies using finger kinematics and force-enslavement (see

Supplementary Materials). Changes to body image were probed using tactile distance and
hand laterality judgement tasks (see Supplementary Materials). Finally, fMRI was used to
track the hand representation in the sensorimotor cortex of the brain and to interrogate
changes to the relationship between the hand and feet representations. With the exception of
the hand-Thumb coordination task, participants were not wearing the Thumb during testing.
Europe PMC Funders Author Manuscripts

Hand-Thumb coordination (sequential)

To probe changes to implicit motor control of the Thumb, a sequential variation of the
hand-Thumb coordination (finger to Thumb opposition) task has been used. In this task,
participants sequentially opposed the Thumb to the tip of each of the five fingers of their
augmented hand, starting with the little finger. Participants were instructed to repeat this
movement cycle as many times as possible within a 1-minute block, while maintaining
high accuracy. The task consisted of 5 blocks. To assess the proprioception of the Thumb,
participants were further asked to perform 5 blocks of the same task while blindfolded.
The experimenter recorded the number of successful hits per block. For each participant,
an average score (number of hits) was calculated separately for each session (pre, post)
and vision condition (with vision, blindfolded). Due to a data acquisition mistake, 1 control
participant was not included in the analysis.

Embodiment questionnaires
To assess changes in the embodiment of the Thumb, participants were asked to complete
an embodiment questionnaire before the first and again after the last training session.
The questionnaire was focused on the explicit (phenomenological) aspect of embodiment,
concerned with whether the extra Thumb feels like a part of one’s hand (56). Due to data
collection issues, 5 augmentation participants and 1 control participant only completed the
Europe PMC Funders Author Manuscripts

post-training embodiment questionnaire. Participants were asked to rate their agreement

with 12 statements (based on (24)) on a 7-point Likert-type scale ranging from -3
(strongly disagree) to +3 (strongly agree). Statements were clustered into 4 main categories,
probing different aspects of embodiment, namely: body ownership, agency, body image and
somatosensation.

For each participant, questionnaire scores were averaged within each embodiment category.
Note that in the body ownership category, the opposite (negative) value of the ‘foreign
body’ statement has been used while computing the average. 1 augmentation participant was
discarded from this analysis, as their averaged agency score was classified as a statistical
outlier (different from the mean score by more than 3 standard deviations).

Statistical analysis
All statistical analysis was performed using IBM SPSS Statistics for Macintosh (Version
24), R (for linear mixed models) and JASP (Version 0.11.1). Tests for normality were
carried out using a Shapiro-Wilk test. Training data that were not normally distributed
were log-transformed prior to further statistical analysis. With the exception of hand
kinematics and force enslavement datasets, that were analysed using linear mixed models
(LMM), all the within group comparisons were carried out using paired t-tests or repeated

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 16

measures ANOVAs (training tasks data). Between group comparisons were carried out using
ANCOVAs with group (augmentation, controls) as a fixed effect and the pre-score used
as a covariate (57). All non-significant results were further examined using corresponding
Bayesian tests under continuous prior distribution (Cauchy prior width r=0.707).

Scanning procedures
Europe PMC Funders Author Manuscripts

Both pre- and post- neuroimaging sessions were comprised of the following functional
scans: (i) a resting state scan (see Supplementary Materials), (ii) a motor localiser scan (see
Supplementary Materials) and (iii) four finger-mapping scans. Additionally, a structural scan
and field maps were obtained during each scanning session.

Finger-mapping scans
Participants were instructed to perform visually cued movements of individual digits
of either hand, bilateral toe movements and lips movements. The different movement
conditions, as well as rest periods were presented in 9s blocks. The individual digit
movements were performed in the form of button presses on MRI-compatible button-boxes
(4 buttons per box) secured on the participant’s thighs. The movements of either of the
(biological) thumbs were performed by tapping them against the wall of the button box.
Instructions were delivered via a visual display projected into the scanner bore. Ten vertical
bars, representing the fingers flashed individually in green at a frequency of 1Hz, instructing
movements of a specific digit at that rate. Toe and lips movements were cued by flashing the
words “Feet” or “Lips” at the same rate of 1Hz. Each condition was repeated 4 times within
each run in a semi-counterbalanced order. Participants performed 4 runs of this task. Due to
timing issues 3 augmentation participants and 1 control participants completed only 3 runs
of the finger-mapping task. Additionally, due to a data acquisition issue, the finger-mapping
data of 1 control participant was discarded.
Europe PMC Funders Author Manuscripts

MRI data acquisition

MRI images were acquired using a 3T Prisma MRI scanner (Siemens, Erlangen, Germany)
with a 32-channel head coil. Functional images were collected using a multiband T2*­
weighted pulse sequence with a between-slice acceleration factor of 4 and no in-slice
acceleration. This provided the opportunity to acquire data with high spatial (2mm isotropic)
and temporal (TR: 1450ms) resolution, covering the entire brain. The following acquisition
parameters were used: TE: 35ms; flip angle: 70°, 72 transversal slices. Field maps were
acquired for field unwarping. A T1-weighted sequence (MPRAGE) was used to acquire
an anatomical image (TR: 2530ms, TE: 3.34ms, flip angle: 7°, spatial resolution: 1mm
isotropic).

MRI analysis
MRI analysis was implemented using tools from FSL (58, 59) and Connectome Workbench
([Link]) software, in combination with Matlab scripts (version R2016a),
both developed in-house (including FSL-compatible RSA toolbox (60)) and as part of the
RSA Toolbox (26). Cortical surface reconstructions were produced using FreeSurfer ((61,
62), [Link]).

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 17

fMRI pre-processing
Functional data was first pre-pre-processed using FSL-FEAT (version 6.00). Pre-processing
included motion correction using MCFLIRT (63), brain extraction using BET (64), temporal
high pass filtering, with a cut off of 150s for the finger-mapping scans and 100s for
resting-state and motor localiser scans, and spatial smoothing using a Gaussian kernel with a
FWHM of 3mm for the finger-mapping and 5mm for resting-state and motor localiser scans.
Europe PMC Funders Author Manuscripts

To make sure that the scans from the two scanning sessions were well aligned, for each
participant we calculated a midspace between their pre- and post- scans, i.e. the average
space in which the images are minimally reoriented. Each scan was then aligned to this
pre-post midspace using FMRIB’s Linear Image Registration Tool (FLIRT, (63, 65)). See
Supplementary Materials for details.

Low-level task-based analysis

For task-based datasets, voxel-wise General Linear Model (GLM) analysis was carried out
using FEAT, to identify activity patterns related to the movement of each digit/body part.
The design was convolved with a double-gamma hemodynamic response function (HRF)
and its temporal derivative. The six motion parameters were included as regressors of no
interest. In case of large movement between volumes (>1 mm) additional regressors of no
interest were included in the GLM to account for each of these instances individually.

For the finger-mapping scans, 14 contrasts were set up: each digit versus rest, all left/right
hand digits against rest, feet against rest and lips against rest. The estimates from the four
finger mapping scans were then averaged voxel-wise using fixed effects model with a cluster
forming z-threshold of 3.1 and family-wise error corrected cluster significance threshold of
p<0.05, creating 14 main activity patterns for each session and participant.
Europe PMC Funders Author Manuscripts

For the motor localiser scans, 4 main contrasts were set up: right/left hand against lips,
right/left foot against lips. The activity patterns associated with those 4 contrasts were then
used to define functional regions of interest (functional ROIs, see Supplementary Materials).

Regions of Interest (ROIs) definition

Changes to representational structure of the hand were studied using anatomical ROIs, as
previously practiced in related studies (14, 66, 67). Structural T1 images, registered to
the structural midspace, were used to reconstruct the pial and white-grey matter surfaces
using Freesurfer. Surface co-registration across hemispheres and participants was done using
spherical alignment. Individual surfaces were nonlinearly fitted to a template surface, first
in terms of the sulcal depth map, and then in terms of the local curvature, resulting in
an overlap of the fundus of the central sulcus across participants (68). The anatomical
sensorimotor ROI, used for the multivariate analysis were defined on the group surface using
probabilistic cytotectonic maps aligned to the average surface (69). These ROI was then
projected into the individual brains via the reconstructed individual anatomical surfaces.
Since we were primarily interested in the motor representation of the hand, we have focused
our anatomical ROI on M1, selecting all surface nodes with the highest probability for BA4
spanning a 2cm strip medial/lateral to the anatomical hand knob (14, 70). However, we

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 18

note that, given the probabilistic nature of these masks, the dissociation between S1 and M1
is only an estimate, and thus our ROI should be treated as a sensorimotor one. SMA was
defined as all surface nodes along the medial wall with the highest probability for BA6 (16,
17).

For our univariate analyses (resting state connectivity, net activity analysis), we also defined
Europe PMC Funders Author Manuscripts

a separate set of functional ROIs based on the sensorimotor representations of the left/right
hand and feet of each participant. See Supplementary Materials for details.

Multivariate representational structure of the hand (RSA)

We used RSA (71) to assess the multivariate relationships between the activity patterns
generated across digits and sessions. RSA was also used to quantify dissimilarity between
multivariate activity patterns elicited by hand and toes movements (see Supplementary
Materials). The dissimilarity between activity patterns within the M1 anatomical hand ROI
was measured for each digit pair using the cross-validated squared Mahalanobis distance
(26). We calculated the distances using each possible pair of imaging runs within a
single scanning session (pre, post) and then averaged the resulting distances across run
pairs. Before estimating the dissimilarity for each pattern pair, the activity patterns were
pre-whitened using the residuals from the GLM. Due to the cross-validation procedure,
the expected value of the distance is zero (or below) if two patterns are not statistically
different from each other, and larger than zero if the two representational patterns are
different. The resulting 10 unique inter-digit representational distances were put together in a
representational dissimilarity matrix (RDM).

To assess the effect of 5-day Thumb usage on the overall representation structure
(dissimilarity), we performed a linear mixed model analysis (LMM) with fixed factors of
time, hand and digit pair, a random effect of participant and a random participant-specific
Europe PMC Funders Author Manuscripts

slope of time. The LMM was evaluated in R (version 3.5.2) under restricted maximum
likelihood (REML) conditions with Satterthwaite adjustment for the degrees of freedom.

As an aid to visualising the RDMs, we also used classical multidimensional scaling

(MDS). MDS projects the higher-dimensional RDM into a lower-dimensional space, while
preserving the inter-digit dissimilarity values as well as possible. MDS was performed on
data from individual participants and averaged after Procrustes alignment (without scaling)
to remove arbitrary rotation induced by MDS. Note that MDS is presented for intuitive
visualisation purposes only, and was not used for statistical analysis.

Supplementary Material
Refer to Web version on PubMed Central for supplementary material.

Acknowledgements
We thank Dominic Stirling, Samuel Cousins, Lydia Mardell, Maria Kromm and Mathew Kollamkulam for their
help with data collection; Ekaterina Tupitsyna for developing the script for the kinematic data analysis; James
Kilner for providing us access to the Cyberglove; Joern Diedrichsen for the custom-made force keyboards;
Howard Bowman for introducing us to information theory measures; Gionata Salvietti for invaluable technical
help during piloting; Silvestro Micera, Juan Alvaro Gallego and Daan Wesselink for helpful comments on the

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 19

manuscript; Hunter Schone for proof-reading the manuscript; Domenico Prattichizzo for inspiring discussions about
supernumerary fingers; and our participants for taking part in this study.

Funding
This work was supported by an ERC Starting Grant (715022 EmbodiedTech), awarded to TRM, who was further
funded by a Wellcome Trust Senior Research Fellowship (215575/Z/19/Z) and by Sir Halley Stewart Charitable
Trust (580).
Europe PMC Funders Author Manuscripts

Data and materials availability

The data that support the findings of this study will be available from the Open Science
Framework upon publication.

References
1. Salvietti G, Hussain I, Cioncoloni D, Taddei S, Rossi S, Prattichizzo D. Compensating Hand
Function in Chronic Stroke Patients Through the Robotic Sixth Finger. IEEE Trans Neural Syst
Rehabil Eng. 2017; 25 :142–150. [PubMed: 26890911]
2. Wu, FY; Asada, HH. ASME Dynamic Systems and Control Conference; San Antonio, TX. 2014.
3. Sasaki, T; Saraiji, MY; Fernando, CL; Minamizawa, K; Inami, M. paper presented at the ACM
SIGGRAPH 2017 Emerging Technologies; 2017.
4. Meraz NS, Sobajima M, Aoyama T, Hasegawa Y. Modification of body schema by use of extra
robotic thumb. Robomech J. 2018; 5
5. Penaloza CI, Nishio S. BMI control of a third arm for multitasking. Science Robotics. 2018; 3
6. Shafti A, Haar S, Zaldivar RM, Guilleminot P, Faisal AA. Learning to play the
piano with the Supernumerary Robotic 3&lt;sup&gt;rd&lt;/sup&gt; Thumb. bioRxiv. 2020
2020.2005.2021.108407
7. Arcaro MJ, Schade PF, Livingstone MS. Body map proto-organization in newborn macaques. Proc
Natl Acad Sci U S A. 2019; 116 :24861–24871. [PubMed: 31732670]
8. Dall’Orso S, Steinweg J, Allievi AG, Edwards AD, Burdet E, Arichi T. Somatotopic Mapping of the
Developing Sensorimotor Cortex in the Preterm Human Brain. Cereb Cortex. 2018; 28 :2507–2515.
Europe PMC Funders Author Manuscripts

[PubMed: 29901788]
9. Kolasinski J, Makin TR, Logan JP, Jbabdi S, Clare S, Stagg CJ, Johansen-Berg H. Perceptually
relevant remapping of human somatotopy in 24 hours. Elife. 2016; 5
10. Ejaz N, Hamada M, Diedrichsen J. Hand use predicts the structure of representations in
sensorimotor cortex. Nature Neuroscience. 2015; 18
11. Flesher SN, Collinger JL, Foldes ST, Weiss JM, Downey JE, Tyler-Kabara EC, Bensmaia SJ,
Schwartz AB, Boninger ML, Gaunt RA. Intracortical microstimulation of human somatosensory
cortex. Sci Transl Med. 2016; 8 361ra141
12. Mancini F, Wang AP, Schira MM, Isherwood ZJ, McAuley JH, Iannetti GD, Sereno MI, Moseley
GL, Rae CD. Fine-Grained Mapping of Cortical Somatotopies in Chronic Complex Regional Pain
Syndrome. J Neurosci. 2019; 39 :9185–9196. [PubMed: 31570533]
13. Kikkert S, Kolasinski J, Jbabdi S, Tracey I, Beckmann CF, Johansen-Berg H, Makin TR. Revealing
the neural fingerprints of a missing hand. Elife. 2016; 5
14. Wesselink DB, van den Heiligenberg FM, Ejaz N, Dempsey-Jones H, Cardinali L, Tarall-Jozwiak
A, Diedrichsen J, Makin TR. Obtaining and maintaining cortical hand representation as evidenced
from acquired and congenital handlessness. Elife. 2019; 8
15. Bruurmijn M, Pereboom IPL, Vansteensel MJ, Raemaekers MAH, Ramsey NF. Preservation of
hand movement representation in the sensorimotor areas of amputees. Brain. 2017; 140 :3166–
3178. [PubMed: 29088322]
16. Beukema P, Diedrichsen J, Verstynen TD. Binding During Sequence Learning Does Not Alter
Cortical Representations of Individual Actions. J Neurosci. 2019; 39 :6968–6977. [PubMed:
31296537]

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 20

17. Berlot E, Popp NJ, Diedrichsen J. A critical re-evaluation of fMRI signatures of motor sequence
learning. Elife. 2020; 9
18. Elbert T, Candia V, Altenmuller E, Rau H, Sterr A, Rockstroh B, Pantev C, Taub E. Alteration
of digital representations in somatosensory cortex in focal hand dystonia. Neuroreport. 1998; 9
:3571–3575. [PubMed: 9858362]
19. Ejaz, N; Sadnicka, A; Wiestler, T; Butler, K; Edwards, M; Diedrichsen, J. paper presented at the
Society for Neuroscience Annual Meeting; 2016.
Europe PMC Funders Author Manuscripts

20. Clode, D. The Third Thumb. 2018. [Link]

21. Huang HJ, Mercer VS. Dual-task methodology: applications in studies of cognitive and motor
performance in adults and children. Pediatr Phys Ther. 2001; 13 :133–140. [PubMed: 17053670]
22. Witteveen HJ, de Rond L, Rietman JS, Veltink PH. Hand-opening feedback for myoelectric
forearm prostheses: performance in virtual grasping tasks influenced by different levels of
distraction. J Rehabil Res Dev. 2012; 49 :1517–1526. [PubMed: 23516055]
23. Maimon-Mor RO, Obasi E, Lu J, Odeh N, Kirker S, MacSweeney M, Goldin-Meadow S, Makin
TR. Communicative hand gestures as an implicit measure of artificial limb embodiment and daily
usage. medRxiv. 2020 2020.2003.2011.20033928
24. Longo MR, Schuur F, Kammers MP, Tsakiris M, Haggard P. What is embodiment? A psychometric
approach. Cognition. 2008; 107 :978–998. [PubMed: 18262508]
25. de Vignemont F. Body schema and body image--pros and cons. Neuropsychologia. 2010; 48 :669–
680. [PubMed: 19786038]
26. Nili H, Wingfield C, Walther A, Su L, Marslen-Wilson W, Kriegeskorte N. A toolbox for
representational similarity analysis. PLoS Comput Biol. 2014; 10 e1003553 [PubMed: 24743308]
27. Kieliba P, Madugula S, Filippini N, Duff EP, Makin TR. Large-scale intrinsic connectivity is
consistent across varying task demands. PloS one. 2019; 14 e0213861 [PubMed: 30970031]
28. Ruan J, Bludau S, Palomero-Gallagher N, Caspers S, Mohlberg H, Eickhoff SB, Seitz RJ,
Amunts K. Cytoarchitecture, probability maps, and functions of the human supplementary and
pre-supplementary motor areas. Brain Struct Funct. 2018; 223 :4169–4186. [PubMed: 30187192]
29. Makin TR, de Vignemont F, Faisal AA. Neurocognitive barriers to the embodiment of technology.
Nature Biomedical Engineering. 2017; 1 0014
30. Wu, FY, Asada, HH. Robotics: Science and Systems X. Berkley, USA: 2014.
31. Cunningham, J; Hapsari, A; Guilleminot, P; Shafti, A; Faisal, AA. 2018 7th IEEE International
Europe PMC Funders Author Manuscripts

Conference on Biomedical Robotics and Biomechatronics (Biorob); 2018. 665–670.

32. Zhu Y, Ito T, Aoyama T, Hasegawa Y. Development of sense of self-location based on
somatosensory feedback from finger tips for extra robotic thumb control. ROBOMECH Journal.
2019; 6 :7.
33. Dempsey-Jones H, Wesselink DB, Friedman J, Makin TR. Organized Toe Maps in Extreme Foot
Users. Cell Reports. 2019; 28 :2748–2756. e2744 [PubMed: 31509738]
34. Merzenich MM, Kaas JH, Wall J, Nelson RJ, Sur M, Felleman D. Topographic reorganization
of somatosensory cortical areas 3b and 1 in adult monkeys following restricted deafferentation.
Neuroscience. 1983; 8 :33–55. [PubMed: 6835522]
35. Nudo RJ, Milliken GW, Jenkins WM, Merzenich MM. Use-dependent alterations of movement
representations in primary motor cortex of adult squirrel monkeys. J Neurosci. 1996; 16 :785–807.
[PubMed: 8551360]
36. Iriki A, Tanaka M, Iwamura Y. Coding of modified body schema during tool use by macaque
postcentral neurones. Neuroreport. 1996; 7 :2325–2330. [PubMed: 8951846]
37. Maravita A, Iriki A. Tools for the body (schema). Trends Cogn Sci. 2004; 8 :79–86. [PubMed:
15588812]
38. Miller LE, Montroni L, Koun E, Salemme R, Hayward V, Farne A. Sensing with tools extends
somatosensory processing beyond the body. Nature. 2018; 561 :239–242. [PubMed: 30209365]
39. Ogawa K, Mitsui K, Imai F, Nishida S. Long-term training-dependent representation of individual
finger movements in the primary motor cortex. Neuroimage. 2019; 202 116051 [PubMed:
31351164]

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 21

40. Graziano MS, Aflalo TN. Mapping behavioral repertoire onto the cortex. Neuron. 2007; 56 :239–
251. [PubMed: 17964243]
41. Dempsey-Jones H, Harrar V, Oliver J, Johansen-Berg H, Spence C, Makin TR. Transfer of
tactile perceptual learning to untrained neighboring fingers reflects natural use relationships. J
Neurophysiol. 2016; 115 :1088–1097. [PubMed: 26631145]
42. Mehring C, Akselrod M, Bashford L, Mace M, Choi H, Bluher M, Buschhoff AS, Pistohl
T, Salomon R, Cheah A, Blanke O, et al. Augmented manipulation ability in humans with
Europe PMC Funders Author Manuscripts

six-fingered hands. Nat Commun. 2019; 10 :2401. [PubMed: 31160580]

43. Muret D, Makin TR. The homeostatic homunculus: rethinking deprivation-triggered
reorganisation. Current Opinion in Neurobiology. 2020
44. Oby ER, Golub MD, Hennig JA, Degenhart AD, Tyler-Kabara EC, Yu BM, Chase SM, Batista AP.
New neural activity patterns emerge with long-term learning. Proc Natl Acad Sci U S A. 2019; 116
:15210–15215. [PubMed: 31182595]
45. Stinear CM. Prediction of motor recovery after stroke: advances in biomarkers. Lancet Neurol.
2017; 16 :826–836. [PubMed: 28920888]
46. Brodersen KH, Wiech K, Lomakina EI, Lin CS, Buhmann JM, Bingel U, Ploner M, Stephan
KE, Tracey I. Decoding the perception of pain from fMRI using multivariate pattern analysis.
Neuroimage. 2012; 63 :1162–1170. [PubMed: 22922369]
47. Timme NM, Lapish C. A Tutorial for Information Theory in Neuroscience. eneuro. 2018; 5
ENEURO.0052-0018.2018
48. Alonso R, Brocas I, Carrillo JD. Resource Allocation in the Brain. The Review of Economic
Studies. 2013; 81 :501–534.
49. Lee V, Maguire J. The impact of tonic GABAA receptor-mediated inhibition on neuronal
excitability varies across brain region and cell type. Frontiers in neural circuits. 2014; 8 :3.
[PubMed: 24550784]
50. Recanzone GH, Merzenich MM, Jenkins WM, Grajski KA, Dinse HR. Topographic reorganization
of the hand representation in cortical area 3b owl monkeys trained in a frequency-discrimination
task. J Neurophysiol. 1992; 67 :1031–1056. [PubMed: 1597696]
51. Guthrie, MD; Brane, LJ; Herrera, AJ; Boninger, ML; Collinger, JL. paper presented at the
American Association of Physical Medicine and Rehabilitation Annual Meeting; San Antonio,
TX. 2019.
52. Jarque-Bou NJ, Scano A, Atzori M, Muller H. Kinematic synergies of hand grasps: a
Europe PMC Funders Author Manuscripts

comprehensive study on a large publicly available dataset. J Neuroeng Rehabil. 2019; 16 :63.
[PubMed: 31138257]
53. Ingram JN, Kording KP, Howard IS, Wolpert DM. The statistics of natural hand movements. Exp
Brain Res. 2008; 188 :223–236. [PubMed: 18369608]
54. Santello M, Flanders M, Soechting JF. Postural Hand Synergies for Tool Use. The Journal of
Neuroscience. 1998; 18 :10105–10115. [PubMed: 9822764]
55. Lambert-Shirzad N, Van der Loos HF. On identifying kinematic and muscle synergies: a
comparison of matrix factorization methods using experimental data from the healthy population. J
Neurophysiol. 2017; 117 :290–302. [PubMed: 27852733]
56. de Vignemont F. Embodiment, ownership and disownership. Conscious Cogn. 2011; 20 :82–93.
[PubMed: 20943417]
57. Van Breukelen GJ. ANCOVA versus change from baseline: more power in randomized studies,
more bias in nonrandomized studies [corrected]. J Clin Epidemiol. 2006; 59 :920–925. [PubMed:
16895814]
58. Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM. Fsl. Neuroimage. 2012; 62
:782–790. [PubMed: 21979382]
59. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, Bannister
PR, De Luca M, Drobnjak I, Flitney DE, Niazy RK, et al. Advances in functional and structural
MR image analysis and implementation as FSL. Neuroimage. 2004; 23 Suppl 1 :S208–219.
[PubMed: 15501092]
60. Wesselink, D; Maimon-Mor, R. rsatoolbox, version 20fbe05. 2017. [Link]
rsatoolbox

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 22

61. Fischl B, Liu A, Dale AM. Automated manifold surgery: constructing geometrically accurate and
topologically correct models of the human cerebral cortex. IEEE Trans Med Imaging. 2001; 20
:70–80. [PubMed: 11293693]
62. Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis. I. Segmentation and surface
reconstruction. Neuroimage. 1999; 9 :179–194. [PubMed: 9931268]
63. Jenkinson M, Bannister P, Brady M, Smith S. Improved optimization for the robust and accurate
linear registration and motion correction of brain images. Neuroimage. 2002; 17 :825–841.
Europe PMC Funders Author Manuscripts

[PubMed: 12377157]
64. Smith SM. Fast robust automated brain extraction. Hum Brain Mapp. 2002; 17 :143–155.
[PubMed: 12391568]
65. Jenkinson M, Smith S. A global optimisation method for robust affine registration of brain images.
Med Image Anal. 2001; 5 :143–156. [PubMed: 11516708]
66. Arbuckle SA, Yokoi A, Pruszynski JA, Diedrichsen J. Stability of representational geometry across
a wide range of fMRI activity levels. Neuroimage. 2019; 186 :155–163. [PubMed: 30395930]
67. Yokoi A, Arbuckle SA, Diedrichsen J. The Role of Human Primary Motor Cortex in the
Production of Skilled Finger Sequences. The Journal of Neuroscience. 2018; 38 :1430–1442.
[PubMed: 29305534]
68. Fischl B, Rajendran N, Busa E, Augustinack J, Hinds O, Yeo BT, Mohlberg H, Amunts K, Zilles
K. Cortical folding patterns and predicting cytoarchitecture. Cereb Cortex. 2008; 18 :1973–1980.
[PubMed: 18079129]
69. Wiestler T, Diedrichsen J. Skill learning strengthens cortical representations of motor sequences.
Elife. 2013; 2 e00801 [PubMed: 23853714]
70. Yousry TA, Schmid UD, Alkadhi H, Schmidt D, Peraud A, Buettner A, Winkler P. Localization of
the motor hand area to a knob on the precentral gyrus. A new landmark. Brain. 1997; 120 (Pt 1)
:141–157. [PubMed: 9055804]
71. Kriegeskorte N, Mur M, Bandettini P. Representational similarity analysis - connecting the
branches of systems neuroscience. Front Syst Neurosci. 2008; 2 :4. [PubMed: 19104670]
72. Knight Fle C, Longo MR, Bremner AJ. Categorical perception of tactile distance. Cognition. 2014;
131 :254–262. [PubMed: 24561189]
73. Schutt HH, Harmeling S, Macke JH, Wichmann FA. Painfree and accurate Bayesian estimation
of psychometric functions for (potentially) overdispersed data. Vision Res. 2016; 122 :105–123.
[PubMed: 27013261]
Europe PMC Funders Author Manuscripts

74. Parsons LM. Imagined spatial transformations of one’s hands and feet. Cogn Psychol. 1987; 19
:178–241. [PubMed: 3581757]
75. Greve DN, Fischl B. Accurate and robust brain image alignment using boundary-based registration.
Neuroimage. 2009; 48 :63–72. [PubMed: 19573611]
76. Hahamy A, Macdonald SN, van den Heiligenberg F, Kieliba P, Emir U, Malach R, Johansen-Berg
H, Brugger P, Culham JC, Makin TR. Representation of Multiple Body Parts in the Missing-Hand
Territory of Congenital One-Handers. Curr Biol. 2017; 27 :1350–1355. [PubMed: 28434861]
77. Hahamy A, Makin TR. Remapping in Cerebral and Cerebellar Cortices Is Not Restricted by
Somatotopy. The Journal of Neuroscience. 2019; 39 :9328. [PubMed: 31611305]
78. van den Heiligenberg FMZ, Orlov T, Macdonald SN, Duff EP, Henderson Slater D, Beckmann CF,
Johansen-Berg H, Culham JC, Makin TR. Artificial limb representation in amputees. Brain. 2018;
141 :1422–1433. [PubMed: 29534154]
79. Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K. A new
SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data.
Neuroimage. 2005; 25 :1325–1335. [PubMed: 15850749]
80. Behzadi Y, Restom K, Liau J, Liu TT. A component based noise correction method (CompCor) for
BOLD and perfusion based fMRI. Neuroimage. 2007; 37 :90–101. [PubMed: 17560126]
81. Whitfield-Gabrieli S, Nieto-Castanon A. Conn: a functional connectivity toolbox for correlated and
anticorrelated brain networks. Brain Connect. 2012; 2 :125–141. [PubMed: 22642651]
82. Zhang Y, Brady M, Smith S. Segmentation of brain MR images through a hidden Markov random
field model and the expectation-maximization algorithm. IEEE Trans Med Imaging. 2001; 20
:45–57. [PubMed: 11293691]

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 23

Summary
Skilful hand augmentation can be readily learned but may impact the user’s body
representation and motor control.
Europe PMC Funders Author Manuscripts
Europe PMC Funders Author Manuscripts

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 24
Europe PMC Funders Author Manuscripts

Fig. 1. Experimental design.

(A-C) The Third Thumb is a 3D-printed robotic thumb. Mounted on the side of the palm
Europe PMC Funders Author Manuscripts

(1), the Thumb is actuated by two motors (fixed to a wrist band), allowing for independent
control over flexion/adduction. The Thumb is powered by (2) an external battery, strapped
around the arm and wirelessly controlled by (3) two force sensors fixed to the underside
of the participant’s big toes. (D) Experimental design for the augmentation group. (E)
Examples of the in-lab training tasks used for hand-Thumb collaboration, shared and Thumb
individuation. Augmentation participants showed significant performance improvements on
all of the tasks across training session. Asterisks denote significant effect of time at ***
p<0.001. See Fig S2. For statistical quantification of the improvements seen in the control
group.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 25
Europe PMC Funders Author Manuscripts
Europe PMC Funders Author Manuscripts

Fig. 2. Behavioural correlates of hand augmentation.

(A-C) Augmentation participants showed significant daily improvement on the hand-Thumb
coordination task. (D) Motor performance with the Thumb was not impacted by increased
cognitive load during the first and last training days. (E) Augmentation participants showed
greater improvement than controls on a hand-Thumb coordination task conducted before and
after the training period. Participants showed improved performance even while blindfolded,
indicating increased Thumb proprioception. (F) Self-reported Thumb embodiment increased
significantly in the augmentation group following Thumb training. (G) Hand kinematics
data collected during the training sessions. The first principal component (synchronised
movement across all five fingers) captured less variance in the augmentation group
compared to controls, indicating less synchronised movements. (H-I) The augmentation
group showed lower inter-finger coupling, relative to controls during Thumb use, indicating
change to the natural finger coordination. The bars depict group means, error bars represent
standard error of the mean. Individual dots correspond to individual subjects’ average inter­
finger (D1-D5) coordination scores as predicted by the linear mixed model (see Materials
and Methods). Asterisks denote significant effects at * p<0.05, ** p<0.01 and *** p<0.001.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 26
Europe PMC Funders Author Manuscripts
Europe PMC Funders Author Manuscripts

Fig. 3. Biological hand’s representation shrinks following hand augmentation.

(A) The sensorimotor hand area was defined anatomically, based on a primary motor cortex
segmentation. (B) Group mean dissimilarity matrix of the right (augmented) hand pre-
and post- training. Each cell shows the Mahalanobis (cross-validated) distance between
the representational pattern of two fingers. (C) The average inter-finger distances of the
right (augmented), but not the left (non-augmented) hand decreased significantly following
Thumb use. The bars depict group mean, error bars represent standard error of the mean.
Individual dots correspond to individual participants’ average distance as predicted by the
linear mixed model (see Materials and Methods). (D) Multidimensional scaling (MDS)
depiction of the left and right (augmented) hand representational structures. Ellipses indicate
between-participant standard errors. Darker colours represent the post scan, whereas lighter
colours represent the pre (baseline) scan. Red = D1, Yellow = D2, Green = D3, Blue = D4,
Purple = D5. (E) SMA ROI was defined anatomically, based on BA6 segmentation (F) The
distance between the hand and the feet, quantified in SMA, decreases significantly for the
right, but not the left hand. (G) MDS depiction of the inter-body-part distances in the SMA.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 27

Darker colours represent the post scan, whereas lighter colours represent the pre (baseline)
scan. Blue = Toes, Orange = Hand, Black = Lips. Asterisks denote significant effects at *
p<0.05, ** p<0.01 and *** p<0.001.
Europe PMC Funders Author Manuscripts
Europe PMC Funders Author Manuscripts

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 28
Europe PMC Funders Author Manuscripts

Fig. 4. Outcomes of the computational simulation.

The adoption of an extra robotic thumb by an adult with a stable hand representation
Europe PMC Funders Author Manuscripts

promotes a change to existing brain organisation. Here, we used computational simulations

to explore multiple potential plasticity mechanisms that could trigger the observed shrinkage
of the hand representation. First, based on information processing theories (47, 48),
the integration of a new additional finger into the hand’s motor control could impinge
on the existing representation of the biological fingers (new digit model). Second, the
change in finger coordination, observed during training, may also lead to abrupt changes
in excitability profiles that can trigger homeostatic plasticity mechanisms and promote
increased tonic inhibition (homeostatic inhibition model (49)). Thirdly, the change to finger
coordination may also result in increased finger individuation, leading to increased cortical
representation of individual fingers via Hebbian learning (cortical magnification model,
(50)). Simulating “neuronal activity” over a fixed-size ROI split into finger specific areas,
we found that each of these processes is conceptually capable of causing the observed
reduction in representational selectivity. (A) Mean dissimilarity matrices computed from
10000 simulations of each of the models. (B) Average distance (dissimilarity) is significantly
decreased, as compared to the canonical hand representation in each of the models. Solid
lines represent the mean of 10000 simulations, dashed lines denote the 1st and 3rd quartile of
the data. Asterisks denote significant effects at *** p<0.001.

Sci Robot. Author manuscript; available in PMC 2021 November 29.

 Kieliba et al. Page 29

Table 1
Embodiment questions divided into 4 separate embodiment categories
Europe PMC Funders Author Manuscripts

Body ownership

1 “It seems like the robotic finger belongs to me”

2 “It seems like the robotic finger is a part of my hand”
3 “It seems like the robotic finger is a part of my body”
4 “It seems like the robotic finger is a foreign body” (negative)
5 “It seems like the robotic finger is fused with my body”
6 “It seems like I have six fingers”

Agency

1 “It seems like I can move the robotic finger if I want”

2 “It seems like I am in control of the robotic finger”

Body Image

1 “It seems like I am looking directly at my own finger, rather than a prosthesis”
2 “It seems like the robotic finger begins to resemble my other fingers”

Somatosensation

1 “I can feel temperature in the robotic finger”

2 “I can feel the posture of the robotic finger”
Europe PMC Funders Author Manuscripts

Sci Robot. Author manuscript; available in PMC 2021 November 29.