Can motion capture improve task-based fMRI studies of motor function post-stroke? A systematic review

Abstract

Background

Variability in motor recovery after stroke represents a major challenge in its understanding and management. While functional MRI has been used to unravel interactions between stroke motor function and clinical outcome, fMRI alone cannot clarify any relation between brain activation and movement characteristics.

Objectives

We aimed to identify fMRI and kinematic coupling approaches and to evaluate their potential contribution to the understanding of motor function post-stroke.

Method

A systematic literature review was performed according to PRISMA guidelines on studies using fMRI and kinematics in post-stroke individuals. We assessed the internal, external, statistical, and technological validity of each study. Data extraction included study design and analysis procedures used to couple brain activity with movement characteristics.

Results

Of the 404 studies found, 23 were included in the final review. The overall study quality was moderate (0.6/1). Thirteen studies used kinematic information either parallel to the fMRI results, or as a real-time input to external devices, for instance to provide feedback to the patient. Ten studies performed a statistical analysis between movement and brain activity by either using kinematics as variables during group or individual level regression or correlation. This permitted establishing links between movement characteristics and brain activity, unraveling cortico-kinematic relationships. For instance, increased activity in the ipsilesional Premotor Cortex was related to less smooth movements, whereas trunk compensation was expressed by increased activity in the contralesional Primary Motor Cortex.

Conclusion

Our review suggests that the coupling of fMRI and kinematics may provide valuable insight into cortico-kinematic relationships. The optimization and standardization of both data measurement and treatment procedures may help the field to move forward and to fully use the potential of multimodal cortico-kinematic integration to unravel the complexity of post-stroke motor function and recovery.

Introduction

Stroke is the second leading cause of death and disability worldwide, affecting more than ten million people worldwide each year [1]. Among survivors, more than 60% show sequels like language, motor or cognitive disorders, which makes stroke a major public health problem [1]. For example, a large study found that 51% of stroke survivors were unable to walk independently just after their stroke. After rehabilitation this amount decreased to 18% [2]. In contrast, upper limb dexterity is less frequently recovered, with some dexterity retrieved in 38% of the cases and complete functional recovery in only 11.6% after six months of rehabilitation [3]. Indeed, through rehabilitation, recovery of motor function is esteemed to be driven by brain plasticity, or the capacity of the brain to adapt itself after a lesion [4]. Plasticity is expressed by a change in brain activity over time, which can be studied via neuroimaging technologies [5] like electroencephalography (EEG), magnetoencephalography (MEG), functional Magnetic Resonance Imaging (fMRI), functional Near Infrared Spectroscopy (fNIRS), or Positron Emission Tomography (PET). Amongst these, fMRI has become a corner-stone in acute-stroke imaging [6], as well as in post-stroke research, as a result of its high spatial precision [7], its whole brain covering [8] and its continuously improving temporal resolution [9]. It has revealed global brain activity patterns that correlate with motor function [10], thereby advancing our understanding of post-stroke motor control [11, 12].

Nevertheless, the large variability in the amount of motor recovery after stroke raises numerous questions, particularly regarding how to facilitate brain plasticity to optimize recovery for each individual patient [13]. While the evolution of motor task-related brain activity has been clearly linked to recovery outcome as measured by clinical scales [14], only few studies investigated brain activity in relation to the characteristics of the performed motor task itself. In a neuroimaging review on upper-limb recovery after stroke, Buma et al. [11] highlight the need to control for task-related confounding factors during fMRI, especially in relation to the quality of task performance. They suggest controlling the execution of motor tasks to improve the understanding of the association between brain activity patterns and post-stroke motor control. Indeed, without appropriate information on how the movement is performed within the fMRI, imaging data cannot distinguish whether the brain activity observed reflects adaptive or maladaptive plasticity. This distinction is crucial for differentiating true motor recovery from behavioral compensation [15–17]. It has therefore been recommended to combine task-related imaging with standardized analysis of the task performance. The most fine-grained manner to obtain such information is by means of a kinematic analysis, or, the study of motion [17]. Kinematic analysis permits the characterization of the motor task in time and space, using a motion capture device. There are numerous kinematic parameters that quantify movement execution, and have been shown informative of healthy motor control as well as post-stroke [18]. Kinematics are better able to discriminate between different levels of post-stroke motor impairment than the Fugl-Meyer Assessment (FMA) [19], which has been the gold-standard to assess post-stroke motor impairment in rehabilitation research [20]. Studies using functional neuroimaging with kinematics to unravel the cortico-kinematic relationship, may thus provide further information on brain activity and motor control (deficits) after stroke [21–25]. Being a relatively novel field, this systematic review aims at analyzing the different approaches currently used and their related findings to identify the potential value of a combined task-fMRI and kinematic approach to study motor function after stroke.

Materials and methods

The systematic review meets the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) requirements [26].

Inclusion criteria

Inclusion criteria were English written, full-text studies using motor task fMRI of the upper or lower extremity after stroke, with kinematic assessment of the motor task by motion capture, regardless the type of motion capture device. All published studies and preprints meeting the inclusion criteria until August 2022 were included. Reviews and conference abstracts were excluded.

Search strategy

The literature search was performed by two authors (ZB and LvD) and supported by a third author in case of discussion (ELB). The following search terms were added to Medline, Embase, Web of Science, and IEEE Xplore: ((fMRI) OR (functional magnetic resonance imaging) OR (functional neuroimaging)) AND (Stroke) AND ((motor control) OR (movement)) AND ((motion tracking) OR (motion capture) OR (kinematics) OR (movement smoothness) OR (motion analysis)). We did not use automatic tools to also include papers in which the kinematics coupled with fMRI approach appeared as a secondary objective.

Assessment of methodological quality of studies

Methodological quality was assessed with an adapted version of the clinical methodological rounds [27] following Buma et al. [11], who systematically reviewed serial imaging studies to identify trends in the association between brain activity and functional upper limb recovery after stroke. To comply with our objective of analyzing the value of a combined fMRI/kinematics approach, items evaluating the internal, statistical, and external validity were modified accordingly. A fourth scale was added to evaluate the technological validity of each study. The criteria of internal validity were broadened to include both lower and upper-limb studies, whether cross-sectional or longitudinal, and limited to imaging by means of fMRI. We also added a criterion to the statistical validity, covering the integration of kinematics in the fMRI statistical analysis. A comprehensive description of each item is provided below, along with a justification for its inclusion in the scoring process. At the end we propose a Methodological Quality Assessment Checklist that provides a short description of each item for practical use (Table 1).

Table 1 Methodological quality assessment checklist

Comprehensive overview of the items for the methodological quality assessment

• 1. Measurements of motor function (0–1 point): Measurement of motor function had to be assessed with validated clinical measures like the Fugl-Meyer-Assessment of the Upper Extremity (FMA-UE) [20], Box and Block Test (BBT) [28], Nine-hole Peg Test (NHPT) [29], Action Arm Reach Test (ARAT) [30] or Wolf Motor Function Test (WMFT) [31] for the upper-limb, or with the 50-feet walking test [32], 10-m walking test, 6-min walking test, Motricity Index of the Lower-Limb or the Fugl-Meyer Assessment of Lower Extremity test [32, 33] for the lower-limb.

• 2. Clear presentation of fMRI parameters (0–1 point): Positive if fMRI parameters are clearly described: pre- and post-processing procedures, statistical analysis including cluster size and location, software, and brain atlas used.

• 3. Description of additional medical or paramedical interventions (0–1 point): Positive if the study reports the verification of additional medical or paramedical interventions which might have an impact on fMRI results (e.g., treatment with botulinum toxin).

• 4. Mirror Movement assessment (0–1 point): Positive if the study controls for mirror movements of the contralateral limb, assessed with either EMG, kinematics, or visually during unilateral motor tasks. Mirror movements of the contralateral limb during paretic limb activity biases the corresponding activity patterns and should be taken into account in the analysis [34].

• 5. Motor task monitoring (0–1 point): Positive when movement pace and amplitude are either fixed or monitored, because they impact the intensity of the BOLD signal [35, 36].

• 6. Multiple comparisons correction (0–1 point): Positive if P-values for activated brain areas are corrected for multiple comparisons, for example, by applying a Bonferroni correction or a Family Wise Error (FWE) correction.

• 7. Validity of applied statistics within and between subjects (0–1 point): Positive if the applied statistics for within and between subject analyses are in accordance with the number of participants and the question addressed by the test.

• 8. Combined kinematics and fMRI analysis (0–1 point): Positive if kinematic and fMRI results are confronted using a statistical approach.

• 9. Specification of relevant patient characteristics (0–1 point): Positive if participants’ age, gender, the type of stroke, its location, number of strokes, the time since stroke, the severity of stroke, and the patient's cognitive status are specified.

• 10. MRI strength (0–1 point): Higher magnetic field strengths improve the measurement of the BOLD response [37, 38]. The MRI strength is considered positive if the magnetic field is superior or equal to 3 Tesla.

• 11. fMRI spatial resolution (0—0.33—0.67—1 point): The precision of fMRI results increases with increasing spatial resolution [39]. The spatial resolution is defined by the transverse resolution plane (x,y) in mm, and the slice thickness (z) in mm. The following gradation has been applied:

  • 0 point if both x and y > 4

  • 0.33 point if both x and y ∈ [3:4]

  • 0.67 point if both x and y ∈ [1:2] and z > 2

  • 1 point if both x and y ∈ [1:2] and z ≤ 2

• 12. fMRI temporal resolution (0–0.33–0.67–1 point): Higher temporal resolution reduces physiological noise and the under sampling effect, increasing the SNR efficiency [40] and providing a better sensitivity [41]. The following gradation has been applied:

  • 0 point if TR > 3 s

  • 0.33 point if TR ∈ [2:3] seconds

  • 0.67 point if TR ∈ [1:2[ seconds

  • 1 point if TR ∈ [0.5:1] second

• 13. Constraining character of the motion capture device (0.3–0.6–1 point): To evaluate and quantify kinematics, various devices exist with more or less impact on the ecological character of the movement, or, whether the movement can be performed as natural as possible. For example, haptic gloves alter the sensory feedback of a movement and thus the way the movement is controlled [42]. Such devices are considered highly constraining and little representative of ecological motion. Subsequently, studies using non wireless devices or devices that are strapped to the participant are considered constraining. In contrast, wireless small markers for optical motion tracking interfere only slightly with the natural/ecological movement and are considered slightly constraining.

  • 0 point if highly constraining (robot, haptic glove, goniometer)

  • 0.5 point if constraining (non-wireless, large markers, strapped devices)

  • 1 point if slightly constraining (wireless, small markers ≤ 15 mm diameter)

Data collection

The following additional information was extracted from each study:

• The type of the study, e.g., a longitudinal, a cross-sectional, or a pilot-study.

• The aim of the study, e.g., evaluation of a therapeutic approach, understanding brain function, or a technological proof of concept.

• The calculated kinematics, or the amount and nature of the kinematic parameters assessed.

• The recording conditions: e.g., the time of kinematic acquisition (during the fMRI acquisition or not).

• The type of statistical analysis that was performed between brain imaging and kinematics.

• The results of the brain imaging with kinematic analysis.

Results

Literature search

The search terms yielded 404 papers, including 207 from Medline, 56 from Embase, 141 in Web of Science and 2 in IEEE Xplore (details are listed in Appendix A). Sixty-six papers were retrieved based on title and abstract screening, dismissing papers that did not respect inclusion criteria (Fig. 1). The full in-depth evaluation of these papers led to the final inclusion of 23 studies [21–25, 43–60], excluding those that were about fMRI alone, kinematics alone or that did not involve people post-stroke. No additional papers were found by citation.

Fig. 1

Flow-chart of study selection

Methodological quality of studies

The methodological quality varied largely over studies ranging from 0.28 to 0.89. With a mean score of 0.63 the overall quality of the included studies was found to be moderate. Especially technological and statistical validity were weak, with respectively a mean quality of 0.48 and 0.57. Because of the limited sample-size and the heterogeneity of studies objectives, we did not set a quality-based exclusion threshold. An overview of the methodological quality assessment results can be found in Table 2.

Table 2 Methodological quality assessment scores

Internal validity

Nineteen studies measured upper and/or lower-limb motor function with clinically relevant and validated tests [22–25, 43, 45–55, 59, 60]. Considering the upper-limb, the ARAT was the most used measure with five occurrences. Four studies used the BBT, two studies used the WMFT, and only one study used the NHPT as a measure of initial upper-limb motricity. Among the seven lower-limb studies, the walking speed was used five times to assess motricity. In addition, Casellato et al. [22] used the Motricity Index for the Lower Limb, and Huiquiong used the ten-meter walk test. Other gait parameters were the stride length or symmetry ratio between the two legs during walking. Only four studies [24, 25, 44, 59] reported medical or drug conditions in patients that could have interfered with the functional MRI results. In all cases they controlled either for post-stroke spasticity treatment with botulinum toxin injection or for hypertension treatment. Eleven studies took into account potential mirror movements [22, 24, 25, 44–49, 55, 56], by using EMG [25], visual inspection [44, 45, 47, 49], or motion capture [22, 24, 46, 48, 55, 56]. The movement frequency was paced in nine studies with either an auditory or visual signal [21, 25, 44–46, 48–50, 53, 56, 58]. Four studies constrained the movement amplitude with an orthosis, a cast, or a brace [46, 49, 57, 59]. One study was paced and constrained in amplitude [49]. Meanwhile nine studies used a free movement with no pacing and without fixing the body member of interest [22–24, 43, 47, 51, 52, 54, 60], of which seven controlled the amplitude through a motion tracking device [22, 24, 54, 60] or by visual control [47, 51, 52]. Eighteen studies evaluated brain activity using a block design fMRI protocol, signified by alternating periods of continuous movement with periods of rest [21–23, 25, 43–49, 51–54, 56, 59, 60]. Only five studies used an event-related design, defined by the repeated execution of one distinct task at certain defined times [24, 50, 55, 57, 58].

Technological validity

For the kinematic assessment, five studies used optical motion capture and were classified as “slightly-constraining” [22, 23, 44, 47, 50]. Seven studies were classified as “constraining”, among which one used electromagnetic motion tracking system [25], five used ultrasonic [43, 51–53, 60], and two used accelerometers [49, 56]. Finally, nine studies were classified as “very constraining”, among which two used data-gloves [24, 58], two used a rehabilitation robot [55, 57], four used a data-goniometer [21, 45, 46, 48] and one used a custom-made leg-press recorder [54].

Data collection

In the following section, we describe the extracted additional information related to: the study design, the functional task, the kinematic parameters assessed, and the joint analysis performed between fMRI and kinematic data. An overview is provided in Table 3.

Table 3 Overview of all studies data collection

Study design

Among the twenty-three studies, eleven were longitudinal studies [22, 25, 44–46, 49, 50, 53, 55, 59, 60], mostly evaluating motor recovery in a pre/post rehabilitation design. Twelve studies were cross-sectional [21, 23, 24, 43, 47, 48, 51, 52, 54, 56–58]. And five studies were pilot, case or feasibility studies, which were either longitudinal or cross-sectional in character [22, 23, 45, 47, 59]. The main objective of fifteen studies was the analysis of brain activity patterns, while four studies focused on the evaluation of a rehabilitation program. In addition, there were two feasibility studies evaluating the integration of an MRI-compatible kinematic system, and one study was about the prediction of rehabilitation efficiency.

Motor task configuration

The majority of studies were interested in motor function of the upper-limb with a variety of functional tasks performed during fMRI, including finger flexion [24, 25, 46, 55, 56, 59], wrist flexion [21], elbow flexion [60], finger tapping [22, 43, 51, 52], hand tapping [43], finger opposition [47], and handgrip [44, 53]. Five studies included a reach to grasp task that was performed outside of the MRI [25, 44, 47, 51, 52]. The seven studies interested in the lower limb used either a pedaling [54] or ankle flexion task [22, 23, 45, 48–50].

Kinematic parameters

To get an overview of the type of kinematic parameters used, we regrouped all kinematic parameters that were used within the seven domains described by Schwarz et al. 2019 [18], notably ‘efficiency’, ‘speed’, ‘smoothness’, ‘temporal posture’, ‘planning’, ‘accuracy’, and ‘spatial posture’ (Fig. 2). More than fifty percent of the kinematics covered the efficiency and speed domain. The efficiency domain was mainly represented by kinematic parameters that described the execution time and the movement amplitude. The speed domain was represented by both movement velocity and frequency measures.

Fig. 2

Number of studies per kinematic parameter, regrouped within seven domains as defined by Schwarz et al. 2019 [18]

Brain and movement analysis

Thirteen studies recorded fMRI and motion capture simultaneously [21, 22, 24, 45, 46, 48, 50, 54–58, 60]. The other ten studies used a non-simultaneous tracking in which motion capture of a comparable task was performed outside of the MRI.

We identified four different ways to use motion capture in a fMRI context (Fig. 3). First, kinematics were used at the first (individual) level of the fMRI analysis either to optimize the fMRI contrast paradigm using kinematics to define the on/offset of action and rest blocks [22], and/or by integrating kinematic variables as regressors or individual covariates [21–23, 55]. Second, kinematics were integrated at the second level of analysis, as group covariates [21, 25, 43, 44, 52–54]. Third, the results between modalities were intellectually compared [45, 47–49, 56, 57, 59, 60]. Fourth, kinematics were used to guide related therapeutic interventions like transcranial magnetic stimulation [51], Functional Electrical Stimulation (FES) [50], or to provide the participant with visual feedback [24, 46, 58]. Note that among the 13 studies where kinematics and fMRI were recorded simultaneously, only 6 performed a statistical analysis between both techniques.

Fig. 3

Use of Kinematics in the fMRI analysis. Note that one study can score positive on multiple items. Blue references refer to studies registering kinematics outside of the fMRI, whereas red references refer to studies registering them during the fMRI

By integrating kinematics as a covariate at group level, Bani-Ahmed et al. demonstrated that the activity of the primary motor cortex (M1) during a hand-grip task varied with the amount of trunk displacement during a reaching task chronically post-stroke [44]. Buma et al. demonstrated the additional recruitment of secondary sensorimotor areas as a function of finger flexion/extension smoothness [25]. Ameli et al. found that baseline ipsilesional M1 activity correlated with the functional improvement in finger tapping frequency following repetitive TMS [43]. Nowak did not observe any correlation between rTMS-modified activity of the contralesional M1 during hand grip movements of the affected hand and its amount of functional improvement [53]. Finally, Promjunyakul et al. were unable to identify any relationship between the lower-limb pedaling rate and the amount of brain activity [54].

By integrating kinematics at the individual level Casellato et al. demonstrated a greater difference in brain activity levels between rest and movement periods in the fMRI block-design when using kinematics to identify when the participant was in motion and when at rest [22]. Brihmat et al. showed that the amount of cerebellar activity decreased when the amplitude time-course during passive hand motion was regressed with the BOLD signal [21]. Moreover, in a longitudinal study, Saleh et al. showed that two out of four participants increased the correlation strength between ipsilesional sensorimotor activity and the angular velocity of finger flexion over rehabilitation [55]. Finally, Del Din et al. demonstrated that their participant’s improvement in walking correlated with a greater and improved activation of the affected hemisphere, as indicated by a larger proportion of active voxels [23].

Discussion

In this review we searched for papers that combined functional brain imaging and kinematics to better understand motor function after stroke. For clarity, references concerning the reviewed papers are identified by a star (*) throughout the discussion. We were particularly interested in the information that could be gained by such a cortico-kinematic analysis. Twenty-three studies met our inclusion criteria. This limited number of studies highlights the novelty of combining kinematics and fMRI to assess motor function after stroke. This field of research also faces technological challenges inherent to integrating kinematic recording and analysis with fMRI, as illustrated by the four technological feasibility studies *[22, 23, 47, 59]. The overall methodological quality of the included studies was sufficient. However, some studies lacked statistical power and had poor internal validity, particularly in considering potential interference from adjuvant medical interventions and monitoring mirror movements. Technological validity was low (with a mean of 0.48/1), which could be in majority explained by the fMRI’s low level of spatial and temporal resolution. Still, this was dependent on the time of data collection. Later studies showed higher resolutions following the continuous development of fMRI sequences. Also, only few studies used non-constraining motion capture devices. Overall statistical validity was impacted by the fact that 14 studies did not statistically confront kinematics with fMRI. In the following we discuss how kinematics were used in combination with fMRI data and what information such analysis provided. We will start with how the different studies captured movement, followed by how kinematics and fMRI analysis were coupled. Finally, we address its current and future challenges.

How to capture a motor task?

3D motion tracking is the most versatile way to register and analyze human movement, regardless of the technology (ultrasonic, optic, electromagnetic) [61]. Kinematics extracted from 3D motion tracking systems outside of the MRI that were confronted with brain activity enabled a fine analysis of brain activity patterns in relation to, for instance, movement irregularity *[25] or compensatory movement intensity *[44]. However, the relation between movements performed outside of the fMRI in an upright position vs. movements performed within the fMRI in a horizontal position as well as constraint in space is not univocal. To gain true insight in the cortico-kinematic relationship, both should be registered simultaneously. 3D motion tracking during fMRI was documented in 13 of the 23 studies. Of all 3D motion capture methods, optical tracking has been recommended for its precise and reliable kinematic analysis after stroke [17]. Within the fMRI, this is not as straightforward. Required to be non-magnetic, camera costs are elevated, and camera angles are limited by the MRI itself as well as by the environment (control windows, clear access to the fMRI for personal, medical equipment…). Currently only one study used optical 3D motion tracking. This feasibility study, published in 2010, on upper and lower limb movements of one stroke patient *[22], concluded that the kinematic acquisitions were reliable and enriched fMRI image information, allowing an evaluation of the cortico-kinematic relationship. Still, one case study is not much, this underlines the infancy of the field, while its growth may be held back by the high costs of a compatible MRI optical motion tracking system and the need for advanced and reliable reconstruction methods. The latter includes marker-labeling and gap filling methods to overcome the tracking constraints of optical motion tracking within the MRI-bore, like potential skin movement artifacts [62] and data-loss when markers are out of sight. Still, the non-magnetic passive character of optical tracking markers, as well as their small size, their easy attachment, and their adaptability to physical constraints induced by the stroke (e.g., hand-spasticity) argue in favor of optical motion tracking inside the machine. And although data-loss is less of an issue with rehabilitation-robots or electro-goniometers, these less expensive tools capture the movement of the robot rather than the movement of the limb within the robot. Having frequently limited degrees of freedom, directional variations are potentially under-estimated. For instance, Shirinbayan et al. [63] evaluated brain correlates of speed using a custom-made MRI-compatible gyroscope. It captured speed efficiently and was less costly, but it did not capture speed in an ecological manner, and using a cast or a glove provides direct tactile sensory feedback, which may alter the way a movement is controlled and thus the corresponding brain activations [42]. Using such tools within the MRI is valuable when evaluating the effects of related robot training on brain activity patterns [64] but might be less-representative of the irregular and variable character of movement post-stroke [65].

In ten studies, participants performed an unconstrained movement in space and time, while the thirteen remaining studies used a predefined pace (auditory signal) or amplitude (straps or orthosis). Post-stroke, motor impairment varies strongly over patients. By using a paced rhythm, task-reproducibility in terms of number of repetitions is indisputably higher. However, the more severe the impairment after stroke, the slower patients move and the more irregular their performance becomes [19]. The further a movement is away from the preferred frequency, the higher the energetic costs to perform such a movement [36]. Based on the principles of optimal control to move with maximal efficiency at minimal costs [66], an unconstrained movement could thus be more adapted to compare different persons with stroke, with different levels of deficits *[60]. We observed that most studies either controlled a fixed movement or monitored the rhythm and amplitude of a “free" movement. When the latter is done with adequate motion tracking systems, it might be preferable because of its higher ecological value, being closer to real live movements with functional relevance [67].

Which brings us to the next point that the type of motion tracking tool impacts the kinematic parameter that can be assessed, and thus the research question that can be addressed. The eight studies using a rehabilitation robot, focused mainly on kinematics from the efficiency domain of the categories described by Schwarz et al. [18], with only two studies including also a kinematic parameter quantifying movement smoothness. In contrast, although the efficiency domain was equally well represented in 3D motion tracking systems studies, they additionally included variables from different relevant domains, including speed, but also smoothness, planning, accuracy, and posture related kinematics. This is an important advantage of 3D motion tracking as these variables contain valuable information on hemiparetic movement. For instance, movement smoothness is known to be inversely related to the capacity level after stroke [68], and posture related variables contain information about potential compensation strategies [69]. Interestingly, recent work by the group of Grefkes *[52] proposed using a “kinematic motor composite score”, based on the principal component explaining the maximal kinematic variance across tasks and participants. The interest of such a composite score is that it may reflect the overall motor performance.

Finally, in the context of simultaneous kinematic recording, lower-limb fMRI studies present an advantage over upper-limb ones. During brain fMRI, participants are placed deeply within the MRI-bore. The more we approach the center of the magnetic field, the more difficult it is to integrate a motion capture system. A system with active markers which emits a signal will be perturbed by the strength of the magnetic field, while passive markers which reflect an emitted light are difficult to see when far in the MRI-bore. As the feet often protrude outside of the MRI-bore, their motion tracking is easy in contrast to the upper-limb that rests within the MRI-bore. For upper-limb tasks, tracking of hand and fingers movement has been shown to be reliable. However, finger tapping performance for instance does not necessarily reflect motor impairments under real world conditions [70], in which object manipulation is an important upper-limb function. Moreover, before being able to manipulate an object, the object needs to be reached. Hence, some authors favor evaluating extension of the elbow, being a main building-block of reaching *[61]. Nevertheless, both tasks are equally important, but functionally different, with different levels of complexity and proprioceptive feedback. Moreover, fine finger manipulation might be more difficult to evaluate early post-stroke, as recovery often evolves from proximal to distal [4]. Also, patients with a severe deficit might be unable to perform such movements, requiring alternative tasks or fine-grained motion tracking/force production measures to evaluate the intention-to-move and the related cortical activity pattern.

Coupling fMRI and kinematics

Kinematic motion capture and fMRI research after stroke was combined in four ways: (1) kinematics were integrated at the first level of fMRI analysis either to optimize the fMRI contrast paradigm using kinematics to define the on/offset of action and rest blocks [22], and/or by integrating kinematic variables as regressors or individual covariates [21–23, 55], (2) kinematics were integrated at the second level of fMRI analysis as group covariates [21, 25, 43, 44, 52–54], (3) kinematics and fMRI data were analyzed separately, allowing intellectual comparison of results between modalities, and sometimes (4) they were used to guide related therapeutic interventions like transcranial magnetic stimulation [51], Functional Electrical Stimulation (FES) [50], or to provide the participant with visual feedback [24, 46, 58]. Independent of the integration mode, studies used the combination of kinematics and fMRI to draw inferences about the underlying motor control *[21–23, 25, 43, 44, 51, 53, 55].

Kinematic integration at the first level fMRI analysis: controlling variability.

All studies integrating kinematics at the individual level, highlight that it, (a) allows to control for differences in task execution within and between subjects *[58], and (b) improves activity pattern precision by using kinematics to define the on and offset of the movement block in a blocked fMRI design *[21, 22]. However, integrating kinematics at the individual level had a different impact on signal intensity over different studies. As a nuisance regressor, it came at the cost of a decreased signal in the work of Brihmat et al. *[21], whereas Casellato et al. *[22] found an increased and optimized activation map by adding kinematic regressors of an active finger tapping and/ ankle dorsal-plantar flexion task as an interest regressor, while Saleh et al. *[24] observed both effects as a function of the level of impairment. Two patients with little recovery on clinical scales showed a decrease in the extent and intensity of the activity in the ipsilesional sensorimotor cortex during hand flexion. Contrarily, two patients that showed more clinical improvement, demonstrated increased correlation strength between the bilateral motor cortex activity and the mean angular movement velocity. Thus, the impact of integrating kinematics at the individual level might be modulated by the impairment level of the patient or their capacity to perform large and fast movements. Accordingly, it has previously been described that finger movement with large amplitude elicits significant brain activity, whereas small amplitude movements do not [35]. Also, the intensity of the BOLD signal is modified by the movement frequency [71]. Nevertheless, other factors, like task characteristics, e.g., passive regular and paced movement *[21] versus active irregular and unconstrained movement *[22] may also play their role. Notably, during passively paced movements, participants always adhere strictly to the task-paradigm.

Kinematic integration at the second level fMRI analysis: understanding motor control

The most significant upper-limb results were found at a group level analysis, by adding kinematic parameters as covariate to the second level analysis of the BOLD-signal. This allowed the identification of regions that varied in activity intensity with the kinematic parameter. For example, Buma et al. *[25], showed that patients with lower levels of hand aperture smoothness during a reach-to-grasp task, recruited additional secondary sensorimotor areas during finger flexion/extension within the fMRI. This was interpreted as a signal of adaptive motor learning strategies to compensate for motor impairments. Interestingly, the jerk being a direct measure of movement quality correlated stronger with brain activation than clinical scales like the Fugl-Meyer Assessment or The Action Research Arm Test. Schaechter & Perdue (2008) *[56], demonstrated that activity in the ipsilateral cortical network was enhanced as a function of task difficulty in stroke patients with good motor recovery. Likewise, Bani-Ahmed et al. *[44] demonstrated the dynamic recruitment of the ipsilateral M1 to be associated with the expression of compensatory trunk use. Hence, ipsilateral M1 was identified as a potential biomarker signaling behavioral compensation. However, methodologically, the tasks within the fMRI (voluntary hand grip force) and outside of the fMRI (reach to grasp) were quite different in nature. Ipsilateral M1 activity has previously been related to the control of complex or difficult motor tasks [72]. This would suggest that lower grip force and increased trunk compensation during reaching are expressions of the same underlying problem: difficulty of motor control after stroke. However, the control of hand grip force alone may differ from the control of a reach-to-grasp movement as the latter is a more complex task that includes not only grip force but also correct muscle synergies and intersegment coordination [67, 73, 74]. Thus, although challenging because of the spatial constraints and the limited field of view within the fMRI, the evaluation of a more comparable task within the fMRI might have been preferable, to allow more valid associations between both measures.

A direct analysis of movement kinematics was performed by van Dokkum et al. (2018) *[60] who measured the kinematics during an elbow flexion/extension task of the less affected upper-limb within the fMRI after stroke. Changes in kinematics were intellectually confronted with changes in fMRI results, facilitating brain activity patterns interpretation. Unfortunately, no statistical inference was performed between both measures. Contrarily, Brihmat et al. (2020) *[21] did include the normalized amplitude of the passive wrist-extension as a regressor at the second level fMRI analysis. This allowed them to draw a direct link between the activation observed and the task-specific changes in the BOLD signal when modeling the group level effect. The latter revealed a correlation between the movement amplitude and primary sensorimotor cortex activity.

Change in kinematics and fMRI to quantify rehabilitation gains.

Dobkin and colleagues (2004) *[49] established that the ankle dorsiflexion paradigm was a valuable physiological assay to identify the optimal type, duration and intensity of rehabilitative gait training. In line, most subsequent lower-limb studies used the combination of kinematics and imaging to evaluate the effects of various rehabilitation strategies. That is, in chronological order, ankle flexion with visual feedback (ankle tracking) produced training effects in both ankle function (kinematics) and brain reorganization (fMRI) *[45]. Telerehabilitation with ankle tracking showed larger gains in walking capacity than repetitive ankle dorsiflexion movements at self-selected pace *[48]. Biofeedback rehabilitation of passive and active ankle dorsiflexion equally modified fMRI parameters and gait, whereby the amount of change in both parameters was strongly correlated *[23]. This led the authors to conclude that fMRI is able to capture phases of motor learning after electromyographic biofeedback training. Finally, in a longitudinal pilot study, Gandolla et al. (2021) *[50] used kinematics to identify responders to a FES-based therapy, after which between group fMRI modeling was performed to identify the underlying brain organization that may explain why some people do respond to the stimulation and others do not. In all studies, kinematics served to quantify whether a participant showed functional progress and/or recovery, without being taken into account in the fMRI analysis itself. The only study since Dobkin et al. (2004) *[49] that did not evaluate a rehabilitation technique, evaluated the feasibility of a continuous, multi-joint pedaling motion task, rather than isolated ankle dorsiflexion. The proposed custom-made fMRI compatible pedaling device could indeed be used with fMRI to examine brain activations after stroke *[54]. Kinematics, like step length, walking velocity and between legs variables, were used to explain the reduced brain activation volume during pedaling post stroke. The only kinematic parameter that approached significance was the amount of work performed by the paretic limb during pedaling. It may thus not be surprising that the group’s next studies did not explicitly focus on kinematics. In these subsequent studies, however, secondary results showed that the strength of local and global network connectivity during pedaling was not correlated with walking speed [75], nor with the corresponding clinical measures [76]. Interestingly, they also observed that the Fugl-Meyer Lower Limb score was unrelated to pedaling rate. This makes one wonder whether the right kinematic parameters and the right task were used to unravel the relation between brain network modifications and lower-limb motor function after stroke. Especially when taking into account that pedaling after stroke is characterized by a more variable velocity profile with impaired interlimb coordination and impaired relative limb phasing [77], all these variables were attenuated by the use of a pedaling device with pedal interdependence. After stroke, it was also found that the increase in BOLD signal in motor regions negatively correlates with the foot-tapping rate outside the scanner [78]. We conclude that, especially in lower-limb studies, there is still a need to identify and evaluate the relevant kinematics (i.e., representing the structure of movement), under both fMRI and real-world walking conditions.

Current challenges and perspectives

Motion capture has been successfully applied to both upper and lower-limb protocols, although most studies focused on upper-limb movements. Numerous research questions remain unanswered in both areas. A key challenge is identifying individual brain and behavioral characteristics to optimize rehabilitation strategies for stroke patients. Another important area for future research is determining the optimal balance between ecological validity and traceability of movements during fMRI. Motion capture was mainly used to provide behavioral information on the speed and extent of movements, either to understand, for example, velocity control, or to monitor differences in movement frequency between participants in order to understand how the movement itself was executed. Interestingly, multimodal analysis has received less attention. Multimodal data integration can be defined as a technique that aims to extract information that may not be accessible through a single source, in our case, measuring the movement performed during the fMRI exam. When multimodal integration was performed, only simple linear approaches were used, whereas nonlinear relationships between cortical dynamics and movement kinematics can be expected [79, 80].

Half of the studies that performed statistical analysis between kinematics and brain activity, did not assess kinematics during fMRI. The elephant in the room is whether movements performed outside of the MRI are comparable to those within, when lying down in a physically restraint environment. Is walking velocity functionally related to pedaling speed? How does grip force relate to a reach-to-grasp task requiring multi-joint coordination? The kinematic quantification of standardized movement assays, as recommended for kinematic upper-limb assessment by stroke rehabilitation experts [17], should be implemented for both upper and lower-limb movements. The subsequent identification of kinematic markers that allow to distinguish neurological recovery from behavioral compensation can then be confronted with their expression during standardized movements within the fMRI at various time-points over recovery. As the choice of the task is likely the cornerstone of success, it is primordial to identify the optimal compromise between ecologically valid movements and their traceability during fMRI.

Finally, an optimal coupling of fMRI with kinematics requires a short repetition time and an efficient MRI-compatible tracking device. In an ideal world the sample frequencies of both modalities should be comparable. Yet, although imaging sample frequency is improving through imaging techniques like multiband fMRI [81], current repetition times vary between one and three seconds. This is far from the optimally recommended sample frequency for motion capture, to know 60 Hz [17]. Although this threshold may be debatable, as the frequency components of body movement remain generally below 20 Hz [82], thus valuable information may equally be obtained with lower sampling frequencies. Correlating the evolution of time-series with different sampling frequencies requires the down-sampling of the kinematic time series which may induce information loss, especially in the case of repetitive motion. For example, when the timing of the imaging volume coincides systematically with the zero-velocity turning point of the rhythmical motion, a false representation of movement is created. Approaches known in human motor control studies like the Nyquist-Shannon sampling theorem might thereby provide alternative solutions limiting information loss [83]. Another related challenge that merits further exploration is the link between the fast fluctuations of movement time series versus the slow and stable BOLD response that is maintained over rhythmic motion in a blocked fMRI design.

Thus, even though the value of combining kinematics with brain imaging has been underlined repetitively, and while it seems worthwhile to improve our understanding of brain plasticity over recovery in a holistic manner with increasing reliability, certain recommendations seem in order. Notably:

• Performing direct motion capture during fMRI, using minimally restricted motion capture devices like optical motion tracking.

• When direct motion capture is not possible, task correspondence within and outside the MRI should be maximized.

• Analyzing both shaping and structural kinematics, covering all kinematics domains.

• Using high field MRI with the lowest repetition time possible.

• At the individual level fMRI analysis, kinematics should be more explored as a time-series.

• Exploring non-linear relationships between kinematics and brain activity patterns.

Of course, these recommendations are based on the English literature of fMRI motor function studies after stroke. We recognize that there is broad range of other techniques available to measure brain activity, such as Electroencephalography (EEG) [84], Magnetoencephalography (MEG) [85], Near Infrared Spectroscopy (NIRS) [86] or Positron Emission Tomography (PET) [11]. Yet, a recent review on motion capture and EEG came to comparable conclusions [87]. Also, these techniques have limited spatial resolution and limited access to subcortical structures, whose role during finger tapping was unveiled by the multimodal integration of kinematics and brain imaging [88].

Conclusion

The present review explored studies using the combination of kinematics and fMRI to evaluate post-stroke motor function and/or recovery. First, kinematics were used in various manners, to know: integrated at the first level of the fMRI analysis, integrated at the second level of the fMRI analysis, analyzed in parallel for intellectual confrontation of results, and analyzed in parallel to feed external devices. Interestingly, independent of the way kinematics were confronted with the brain activity patterns observed, studies used the joint analysis to improve the precision of the fMRI analysis, and to draw inferences on underlying motor control (deficits) after stroke.

When kinematics of the task performed in the fMRI were captured, on and off-sets of the movements block could be defined precisely, and when integrated as regressor of no-interest they allowed to control for within and between subject variability. Contrarily, when used as a regressor of interest, they were suggested to highlight the neural substrates of such variability, explaining potential deficit related activity patterns. Especially when followed longitudinally, the combination of both might provide further insights on the complex issue of adaptive versus maladaptive plasticity in the context of behavioral compensation and true recovery. However, caution is warranted. The actual state of the art was marked by variable and explorative methodological and statistical approaches, evolving fMRI acquisition parameters, and limited exploration of diverse kinematic variables. While the field continues to develop, the optimization and standardization of both fMRI and kinematic acquisition parameters, as well as coupling analysis, could enhance the overall quality of future studies. These advancements might enable to move the field forward and to fully leverage the potential of multimodal cortico-kinematic integration to unravel the complexity of post-stroke motor function and recovery.

Appendix A

Selection by search terms

Term of interest	Number of results	Motor engine	Date of search
fMRI	260,542	4 databases	17/08/2022
+ Stroke	17,700	4 databases	17/08/2022
+ movement	3766	4 databases	17/08/2022
+ motion tracking	199	4 databases	17/08/2022