On the role of visual feedback and physiotherapist-patient interaction in robot-assisted gait training: an eye-tracking and HD-EEG study

Abstract

Background

Treadmill based Robotic-Assisted Gait Training (t-RAGT) provides for automated locomotor training to help the patient achieve a physiological gait pattern, reducing the physical effort required by therapist. By introducing the robot as a third agent to the traditional one-to-one physiotherapist-patient (Pht-Pt) relationship, the therapist might not be fully aware of the patient’s motor performance. This gap has been bridged by the integration in rehabilitation robots of a visual FeedBack (FB) that informs about patient’s performance. Despite the recognized importance of FB in t-RAGT, the optimal role of the therapist in the complex patient-robot interaction is still unclear. This study aimed to describe whether the type of FB combined with different modalities of Pht’s interaction toward Pt would affect the patients’ visual attention and emotional engagement during t-RAGT.

Methods

Ten individuals with incomplete Spinal Cord Injury (C or D ASIA Impairment Scale level) were assessed using eye-tracking (ET) and high-density EEG during seven t-RAGT sessions with Lokomat where (i) three types of visual FB (chart, emoticon and game) and (ii) three levels of Pht-Pt interaction (low, medium and high) were randomly combined. ET metrics (fixations and saccades) were extracted for each of the three defined areas of interest (AoI) (monitor, Pht and surrounding) and compared among the different experimental conditions (FB, Pht-Pt interaction level). The EEG spectral activations in theta and alpha bands were reconstructed for each FB type applying Welch periodogram to data localised in the whole grey matter volume using sLORETA.

Results

We found an effect of FB type factor on all the ET metrics computed in the three AoIs while the factor Pht-Pt interaction level also combined with FB type showed an effect only on the ET metrics calculated in Pht and surrounding AoIs. Neural activation in brain regions crucial for social cognition resulted for high Pht-Pt interaction level, while activation of the insula was found during low interaction, independently on the FB used.

Conclusions

The type of FB and the way in which Pht supports the patients both have a strong impact on patients’ engagement and should be considered in the design of a t-RAGT-based rehabilitation session.

Background

Over the last decade, robot-based rehabilitation has been increasingly accepted in clinical applications as an adjunct conventional therapy, assisting the physiotherapist in increasing the intensity and repeatability of rehabilitation sessions [1,2,3,4]. Commercially available robotic-assisted gait training (RAGT) devices are generally divided into stationary systems and overground systems [5]. The latter are robotic devices that allow patients to practise walking on a hard surface, whereas the former take advantages from the body weight support (BWS) [1] and can be divided into treadmill-based gait trainers with exoskeleton (t-RAGT) and end-effector gait trainers. The t-RAGT is a device in which the movement of the leg is generated by the exoskeleton worn by the patient. The end-effector gait trainer is a device with two independently moving footplates to which the patient’s feet are attached [5]. The movement of the plates induces the stance and swing phases of the patient’s gait. Of these two devices the t-RAGT is one the most widely used in rehabilitation centres [6] and there is evidence of its effectiveness [7] when used in conjunction with the conventional gait rehabilitation for central nervous system conditions [2, 8, 9] such as the Spinal Cord Injury (SCI) [2, 10].

t-RAGT offers several advantages such as task-oriented movements, precisely controllable assistance, the ability to modulate the weight-bearing effect on the lower limbs and objective and quantifiable measures of patient’s performance based on kinematic, kinetic and spatio-temporal data [11, 12]. t-RAGT provides automated locomotor training using a BWS system and actuators located on limbs that provide adjustable Guidance Assistance (GA) to assist the patient’s lower limb joints in performing a physiological gait pattern while reducing the physical effort of the physiotherapist [13]. In addition, a reproducible, rhythmic and physiological limb movement provides the optimal afferent input which is necessary to stimulate the spinal neural circuits to activate the lower limb muscles that cannot be moved voluntarily [14].

The addition of a robot as a third party to the one-to-one Physiotherapist-Patient (Pht-Pt) relationship has modified the classic dynamics between the therapist and the patient, moving from a dyadic to a triadic interaction paradigm. In addition, the lack of direct physical contact during rehabilitation poses a challenge for the therapists in providing feedback to patients on their performance [15], potentially compromising their involvement during therapy. However, this gap has typically been bridged by providing both the patient and the physiotherapist with visual feedback (FB) during therapy, informing them of the characteristics of the patient's motor pattern. The therapist is then able to use the FB information to provide the patient with enriched instructions, suggestions and corrections to reinforce what the patient is already receiving through the FB itself. This approach contributes to improve the effectiveness of the patients’ motor training by promoting motor learning strategies, improving the human–robot interaction and increasing motivation and engagement [16, 17]. In fact, FB (e.g., visual, auditory, haptic) is widely recognised as a key component for robotic neurorehabilitation, due to its ability to enhance a plasticity-dependent mechanism that improves motor learning and performance, especially in the context of t-RAGT [18]. FB also promotes active patient’s participation, which is indeed crucial for successful rehabilitation outcomes and needs to be facilitated and tailored to patient-specific requirements [19, 20]. Although the importance of FB in t-RAGT is well recognised, it is still unclear what the effective role of the therapist should be in the complex patient-robot interaction. In fact, the patient can be left to decode the information contained in the FB without any verbal support from the therapist, or the therapist can provide continuous real-time encouragement for the correct interpretation of the FB through a positive social relationship characterised by mutual respect, therapist’s empathy and patient’s trust [21].

To date, patient’s experience and participation in the rehabilitation process has been assessed using questionnaires that examine the perceived quality of care and services received [19], or exploring aspects of motivation, agency and satisfaction [22,23,24]. Patients report that training with the visual FB increases their motivation [25] and they express a desire to incorporate the FB into their training programme as they perceive that it accurately reflects their activity [23,24,25]. To improve the understanding of the dynamics that characterise patients’ engagement in terms of visual attention and cognitive processing during t-RAGT we propose to introduce objective measurements alongside the subjective information derived from questionnaires. Indeed, objective measurements can provide quantifiable data that improve reproducibility and reduce the bias and subjectivity inherent in questionnaire-based assessment. These measurements could be derived from bio-signals such as Eye-Tracking (ET) and Electroencephalography (EEG). Indeed, ET has proven to be a valuable tool for studying human behaviour by measuring and monitoring eye movements, which determine the stimuli on which the gaze falls [26]. ET metrics, derived from fixational and saccadic eye movements, are used to define observer engagement and determine how visual attention is distributed across the visual field. This is based on the eye-mind hypothesis, according to which the fixation of the eyes corresponds to the engagement of the mind [27]. The EEG records the electrical activity generated by neurons from electrodes placed on the scalp [28]. Advanced analysis of EEG signals has contributed to a better understanding of various cognitive processes (e.g. visual perception, attention, emotion), providing real-time data on brain function with excellent temporal resolution. The neural mechanisms underlying empathic interaction and mentalising have been investigated using EEG [29,30,31,32,33], demonstrating the relevance of specific brain structures such as the anterior cingulate cortex (ACC), superior frontal regions, parietal and somatosensory regions, the temporal-parietal junction and the posterior cingulate. EEG has also been used to assess social behaviour during key interactions in clinical settings [34] such as eye-gaze [35], cooperative decision making [31], verbal and non-verbal communication [36].

In this context, we conducted a feasibility study to investigate and quantify, using multimodal EEG and ET recordings, how the combination of different types of visual FB and different levels of Pht-Pt interaction would affect the patients’ engagement during t-RAGT.

Specifically, we hypothesise that a combination of the type of visual FB and of the way in which the therapist supports the patient during a t-RAGT rehabilitation session:

• i. would influence the patient’s visual attention by differently directing the patient’s gaze towards the monitor where the FB is provided, the therapist silhouette, or other potential sources of distraction (environment);

• ii. would elicit cognitive processes involving brain areas associated with social interaction when the physiotherapist’s support is more relevant.

To this end, ten individuals with SCI performed seven t-RAGT sessions using the Lokomat Pro (Hocoma AG, Switzerland). During these sessions, two experimental conditions were implemented and randomly combined: (i) three types of visual FBs available on the Lokomat and (ii) three levels of Pht-Pt interaction. ET and EEG data were acquired simultaneously during the sessions and used to assess the patients’ attentional allocation and engagement during the rehabilitation intervention. By monitoring eye movements and brain activity, we were able to determine where the patient’s attention was focused, and which brain regions were active during the different experimental manipulations.

Methods

Participants

We included 10 individuals with SCI (6 females, aged 18–85 years) in the study. Their epidemiological and clinical features are reported in Table 1. The patients were recruited from the Spinal Cord Unit rehabilitation ward at Fondazione Santa Lucia (FSL) in Rome, according to the following inclusion criteria: sensory-motor incomplete SCI [classified according to the International Standards for Neurological Classification of Spinal Cord Injury [37] as ASIA Impairment Scale (AIS) level equal to C or D], lesion level D12 or higher, traumatic or non-traumatic SCI aetiology, subacute or chronic SCI, absence of severe cognitive impairment such as to interfere with the protocol assessed by Montreal Cognitive Assessment (MoCA, cut-off > 26) [38] and psychiatric assessed by Beck Depression Inventory (BDI-II, cut-off < 30, severe depression) [39] and State-Trait Anxiety Inventory (STAI, cut-off < 40) [40].

Table 1 Epidemiological data and neurological features of individuals with SCI enrolled in the study

The exclusion criteria were other concomitant neurological disorders, severe concomitant diseases (metabolic disorders, severe cardiac impairment), severe symptomatic orthostatic hypotension, gross dystonic/involuntary movements, high level of spasticity (Modified Ashworth Scale—MAS higher than 3) [41], pressure sore of stage 2 or higher, debilitating diseases that cause exercise intolerance, or severe reduction in the lower limb joints’ range of motion.

The RAGT rehabilitation sessions were carried out by 3 different trained physiotherapists with at least 6 years of experience in neurorehabilitation field (1 male, 2 females, age 32.66 ± 5.50 years).

The study was approved by the local ethics board at FSL, Rome, Italy (PROT.CE/PROG. 883) and all the participants signed an informed consent. The study falls into the category of feasibility study as defined in [42]. The main results of the study, in fact, have contributed to the definition of specific criteria (FB type and modality of Pht’s interaction toward Pt) to be adopted in a Randomized Controlled Trial that we are currently conducting with the aim to investigate the effect of therapeutic alliance between patient and physiotherapist on the outcome of a robotic rehabilitation intervention (ClinicalTrials.gov ID NCT06531304).

Experimental design

We acquired data from individuals with SCI during seven t-RAGT sessions (SS1, SS2, …, SS7) with the Lokomat Pro (Hocoma AG, Switzerland), each lasting 45 min and covering an average period of 2 weeks within the rehabilitative plan of the patient as determined by the medical staff. Training was provided by a dedicated physiotherapist as add-on to the conventional rehabilitation care.

The Lokomat consists of a bilateral exoskeleton, a treadmill and a dynamic BWS, which helps to reduce the gravitational load, ensure safety, and provide balance. The system assists hip and knee flexion/extension movements and the pelvis rotation, while spring-based foot lifters passively assist ankle dorsiflexion. Hip and knee movement is assisted by an impedance controller, whose reference trajectory is defined by the Pht together with the Gait Speed (GS). The level of assistance is indicated as Guidance Force (GF) and is increased, according to the patients’ needs, by selecting a higher impedance. The higher the GF, the less the patients contribute to the walking task. The GF can be tuned from 0% (indicating no robot contribution, at least theoretically) to 100% (corresponding to full robotic assistance).

At study entry all the patients performed a familiarisation session (SS0) with the Lokomat to allow the physiotherapist to adjust the device parameters (BWS, GS, GF and joint trajectories), which then remained unchanged during the following seven sessions to avoid any influence on patients’ performance during the rehabilitation. In detail, SS0 started with the physiotherapist tailoring the GF and BWS levels and the GS ad hoc for each patient. For the whole cohort of patients, the mean (sd) of the selected parameters was as follows: i) BWS: 60.1% (15.0%); GF: 89.70% (9.96%); iii) GS: 1.24 km/h (0.30 km/h).

A flow chart of the study is shown in Fig. 1.

Fig. 1

Flowchart summarizing the study design. Dotted arrows identified the different time-points along the rehabilitation where specific assessments were carried on. Clinical evaluation included all the clinical scales reported in “Patients’ clinical assessment and functional evaluation” section. The acronyms PHT—physiotherapist and PT—patient refers to the subject of the assessment

Two experimental conditions, FB type and Pht-Pt interaction level (see paragraphs below), were randomly manipulated across the seven t-RAGT sessions as follows. In the first six sessions (SS1–SS6), each FB was used for two consecutive sessions and remained the same throughout the entire session (SINGLE-FB). Each SINGLE-FB session was timed by the therapist’s behaviour according to the 3 different levels of interaction. The level of Pht-Pt interaction changed every 5 min by following the fixed high–medium–low sequence in the first SINGLE-FB session and by randomising it in the second SINGLE-FB session for each FB. Within the last t-RAGT session (SS7) all three types of FB were randomly administered (MULTI-FB) and the Pht-Pt interaction was modulated according to only two levels (high and low) for each FB. Specifically, the three types of FB were administered according to a block-design approach, with each block lasting 5 min. Each block was divided into 2.5 min of high Pht-Pt interaction and 2.5 min of low Pht-Pt interaction. The order of the blocks was randomised across patients. At the beginning of the MULTI-FB session, patients underwent a 5 min baseline block during which they walked with the Lokomat without any specific task assigned by the therapist and without any experimental conditioning.

FeedBack type

Three types of FB were selected from those available from the manufacturer [43]: chart, emoticon and game (Fig. 2a). Although the FB can be displayed in visually different forms, it is based on human–robot information assessed by means of force sensors integrated in the Lokomat’s hip/knee linear actuators, which measure the joint interaction torques, i.e. those required by the Lokomat to keep the patient on the predefined gait trajectory. The time-varying joint torques measured by the device are then properly weighted by joint-specific weighting functions and averaged, during each gait cycle, separately for the swing phase and stance phase to obtain 8 different FB values after each stride: right/left hip and knee during stance and swing phases. Such an adjustment was proposed to take into account the primary rehabilitation concept according to which the useful joint torques exerted by the patients should have the same sign as the joint angular velocity, i.e. they should point in the same direction of the desired movement, with the sole exception of the knee during stance. Positive FB values are associated with an active participation of the patient in the exercise, whereas negative FB values indicate a passive behaviour or even a resistance to the robot motion [25, 43, 44]. The chart FB shows trend graphs containing these 8 values, with swing and stance data displayed in two different colour-coded lines, separately for four sub-graphs representing left/right hip and knee joints data. When using the chart FB, each point represents the FB value of one stride. Patients are instructed to walk actively to produce positive FB values on the chart.

Fig. 2

a Types of FB (game, chart, emoticon) selected for the study. b Experimental setup employed for the MULTI-FB session. c Identification of the three AoIs in the patient’s field of view recorded dynamically through the ET device: monitor, physiotherapist (Pht) and surrounding

The FB values from each joint are averaged into a single value accounting for the overall patient’s performance and displayed by a smiley in the emoticon FB, which smiles broader for higher and frowns for lower values of the FB during the most recent stride. Indeed, the emoticon facial expression is updated at each gait cycle and depends on the comparison between the FB value and a threshold set during the calibration phase: if the FB value exceeds the threshold, the emoticon mouth points upwards, otherwise it points downwards. In this case, the patient is expected to achieve the goal of a full smile. Finally, the game FB provides a virtual environment where the patient’s movements are mapped onto an avatar: the left/right FB values measured by the robot are used to control the steering of the walking avatar. When using the game FB, the patients’ task is to control the left/right movement of their avatar within the virtual environment by following a walking path during which they can collect coins or keys (i.e., rewards) that allow the avatar to reach the next level of the landscape.

Pht-Pt interaction level

The Pht-Pt interaction was modulated according to three levels (high, medium and low). In the high interaction level the physiotherapist constantly provides verbal support to the patients, enriching the information provided by the FB when it is poorly informative, translating it into a specific request when it clearly indicates that the patients’ movement is incorrect, giving them specific tasks that involve a single joint in a particular gait phase, drawing their attention to the screen where the FB is provided and constantly motivating them to actively perform the rehabilitation exercise. In the medium interaction level, the physiotherapist assigned to the patients a specific task, e.g. focusing on a specific joint (right hip, left hip, right knee, or left knee) and a specific gait phase (stance or swing) using the visual FB and only intervened when patients requested assistance. During the low interaction, verbal communication between the physiotherapist and the patient was interrupted, and the patient was guided only by the visual FB.

ET and EEG data acquisition

During the SINGLE-FB sessions, only ET data were collected from patients by means of a wearable eye-tracker system (Tobii Pro Glasses 3, Tobii, Sweden), properly calibrated, with a sampling frequency of 50 Hz. In the MULTI-FB session, a multimodal simultaneous ET and EEG recording was performed. The EEG data were sampled at 250 Hz and recorded using a 61-channel active electrodes system (LiveAmp, Brain Products GmbH, Germany) positioned according to an extension of the 10–20 International System. The ground and the reference were placed on the right and left mastoid respectively. The eye-tracking system and the EEG amplifier were synchronised via a jack cable which allowed the transmission of a TTL signal automatically provided by the ET system as soon as the device started recording. This signal generated common event markers that appeared simultaneously in the data streams of both devices to allow synchronisation during off-line data processing. Figure 2b shows the experimental setup used for the study (MULTI-FB session).

ET and EEG data analysis

The ET data collected during both SINGLE-FB and MULTI-FB sessions were pre-processed according to the steps reported in “Eye-tracking data processing” section. ET metrics describing the patients’ visual attention allocation among different parts of the scene were computed during the entire rehabilitation session (for the entire list and related definitions see “Extraction of ET metrics in specific AoIs” section) and subjected to the statistical analysis reported in “Statistical analysis” section. ET metrics extraction and statistical analysis were conducted separately for SINGLE and MULTI-FB sessions. In addition, ET data recorded during MULTI-FB session were also used to segment the simultaneous EEG data according to the different experimental conditions (see “EEG data pre-processing”).

Eye-tracking data processing

ET data were processed by means of Tobii Pro lab (Tobii, Sweden), a software package that allows the visualization of the recorded scenes and the extraction of eye-gaze metrics in specifically selected Areas of Interest (AoIs) (i.e., specific stimuli or elements in the video scene) during selected time windows [45]. The main features quantifying the motor behaviour of the eyes are fixations and saccades. Fixations are periods with duration between 100 and 600 ms during which the eyes are focused on a specific area [45]. Saccades are fast and accurate ballistic eye movements used to reposition the fovea to a new location in the visual environment [26].

The first step in the analysis of ET data was the segmentation of the traces according to the conditions of interest. In particular, the SINGLE-FB sessions were segmented into temporal windows, each containing the three different levels of Pht-Pt interaction while the MULTI-FB session was segmented according to both the type of FB administered and the two levels of Pht-Pt interaction. Subsequently, we drew three AoIs, importing a snapshot of the patient’s visual field. The selected AoIs were: (1) the Lokomat monitor, where the FB is provided to both the physiotherapist and the patient (monitor), (2) the physiotherapist silhouette (Pht), (3) the remaining elements of the field of view other than monitor and Pht, which can be considered as a potential source of distraction for the patient (surrounding) (Fig. 2c). An automatic assisted mapping approach was applied to map the gaze data on the snapshot and to assign the gaze point to a specific AoI for each frame in the considered time window. The misclassified frames were manually corrected. All segmentations were performed by considering monitor as the first AoI where the patient’s gaze fell, to ensure uniform calculation of the ET metrics across all patients and experimental conditions.

Extraction of ET metrics in specific AoIs

Once labelled the ET data with a specific AoI for each frame of the recording, as described above, the following metrics were extracted [46, 47]:

• 1. total duration of fixations: defined as the sum of all the fixation durations within an AoI normalized by the total duration of all the fixations in the selected time interval.

• 2. number of fixations: defined as the number of fixations occurring in an AoI normalized by the total number of fixations occurred in the considered temporal window.

• 3. average duration of fixations: defined as the sum of all the fixation durations in one of the AoIs divided by the number of fixations within the same AoI.

• 4. number of saccades: defined as the number of saccades occurring in an AoI normalized by the total number of saccades occurred in the considered temporal window.

• 5. time to exit saccade: defined as the time instant at which the patient leaves each of the defined AoI for the first time.

• 6. time to first fixation: defined as the time instant in which the first fixation takes place in an AoI.

Time to exit saccade was calculated only for the monitor area as a measure of the time patients employed to have a distraction and move their eye-gaze out of the screen where the FB was displayed. Time to first fixation was instead calculated only for Pht and surrounding AoIs as a complementary information to the one extracted from the time to exit saccade.

All the metrics were computed for each AoI, each FB and Pht-Pt interaction level. Moreover, once verified that no statistical differences occurred for the ET metrics extracted from the two SINGLE-FB sessions conducted with the same FB, they were averaged and considered as dependent variables in the statistical analysis (see “Statistical analysis”).

EEG data pre-processing

The synchronization of EEG and ET data acquired during the MULTI-FB session was performed through the EYE-EEG Matlab toolbox [48] that requires at least two common event markers present in EEG and ET data. The ET data between the first and last shared markers were linearly interpolated to match the sampling frequency of the EEG. The remaining shared events were used to evaluate the quality of synchronization. Once synchronized with ET data, the EEG signals were pre-processed by using the software Vision Analyzer 2.0 (Brain Products GmbH, Germany). EEG signals were band-pass filtered [1–20] Hz to reduce the muscular artefacts due to patient’s movements on the Lokomat. Then, noisy channels were identified and excluded from further analysis. The Independent Component Analysis (ICA) was employed to remove eye-blinks artefacts and subsequently the EEG traces were segmented according to the three FBs and two Pht-Pt interaction levels by using temporal information provided by the ET. Every condition was furtherly segmented in epochs of 1 s duration and an automatic procedure based on a voltage threshold criterion (± 100 µV) was applied to highlight residual artefacts. Then, if we found less than three artefactual channels per trial, we interpolated them by means of a spherical interpolation including all the other channels. Otherwise, the trial was removed. Only artefact-free epochs were considered for the subsequent analysis. To uniform the number of epochs across the different experimental conditions, 65 epochs were randomly selected for each condition.

Spectral analysis in source domain

EEG data of each patient were re-referenced to the Common Average Reference and then localized in the whole grey matter volume by applying the source localization algorithm standardized LOw-REsolution brain electromagnetic Tomography (sLORETA) [49] implemented in sLoreta software [50, 51]. This approach allowed to estimate the current density distribution of cortical and subcortical brain sources from the scalp electric potentials with zero localization error. As a solution of the forward model reproducing the electromagnetic propagation from the active sources to the EEG sensors, we used a lead field matrix obtained from the application of the Boundary Element Method [52] to the MNI152 realistic head model [53]. We used a 3D lead field matrix modelling the propagation of 6239 active sources distributed in the whole grey matter at 5 mm spatial resolution towards the EEG sensors. The regularization parameter λ used for sLORETA solution was computed by means of a cross-validation approach [54]. The solution of the source localization problem for each patient and each experimental condition consisted of a waveform for each of the 6239 dipoles used to model the grey matter. Power spectral density (PSD) was computed for each source in the grey matter by means of the Welch periodogram method on non-overlapping data segments of 1 s each tapered with the Hann window. PSD was then averaged in two frequency bands: theta [3–7] Hz and alpha [8–13] Hz. Beta band was excluded due to the high number of muscular artifacts. The spectral analysis was performed for each patient and each experimental condition (three FBs and two Pht-Pt interaction levels).

Patients’ clinical assessment and functional evaluation

A clinical and functional evaluation was performed to monitor patients along the rehabilitation path and check for the absence of undesirable effects or adverse events. These evaluations were administered at recruitment and at the end of the 7 training sessions by expert physiotherapists not involved in the t-RAGT and included the following clinical scales: Spinal Cord Independence Measure III (SCIM-III) [55] for measuring the level of independence in performing activities of daily living; Lower Extremity Motor Score (LEMS) from the ISNCSCI [41, 56] for assessing the strength of the five key muscles of the lower extremities; MAS [41] for measuring increased muscle tone in flexion/extension in the hip, knee, and ankle joints; Numeric Rating Scale (NRS) [57] for measuring lower extremity pain; Walking Index for Spinal Cord Injury version II (WISCI II) [58] as a measure of walking function; 10 Meter Walk Test (10MWT) and 6 Minute Walk Test (6MWT) for assessment of walking ability over short and long distances, respectively [59].

Empathic and relational evaluation of Pht-Pt interaction

Therapists were screened for their empathy level (defined as a combination of the ability to understand and share other people’s emotions) by means of the Empathy Quotient (EQ-short version—cut-off > 30) questionnaire [60]. To further explore the interaction with the physiotherapists, all patients were administered with the Consultation And Relational Empathy (CARE) scale [61] at the end of 3 SINGLE-FB sessions (SS2, SS4 and SS6). This scale allows the measurement of the patient’s perception of the physiotherapist's interpersonal qualities and the effects that the three FBs have on the interaction.

Patients’ workload evaluation

To evaluate patients’ experience during t-RAGT sessions, the NASA-Task Load Index (NASA-TLX) [62] was administered at the end of session SS2, SS4 and SS6 with the aim to determine the perceived workload during robotic therapy with different FBs, taking into consideration specific workload components (mental demand, physical demand, temporal demand, effort, performance and frustration).

Statistical analysis

ET data

Evaluating the effects of the experimental manipulations on the ET metrics

A two-way repeated measures Multivariate Analysis of Variance (MANOVA) was performed (significance level α = 0.05) considering as independent within-factors the type of FB (FB—three levels: chart, emoticon, game) and the Pht-Pt interaction level (IL—three levels: high, medium, low) and as multivariate dependent variables each ET metric in the three AoIs. The analysis was conducted for the following ET metrics: total duration of fixations, number of fixations, average duration of fixations, number of saccades, time to first fixation. Instead, the ET metric time to exit saccades, being computed only on monitor AoI, was considered as a unique dependent variable of a repeated measures ANOVA considering FB type and Pht-Pt interaction level as within factors. Data distribution normality was previously assessed by means of Lilliefors test (alpha = 0.05).

EEG data

Reconstructing EEG-based spectral activations

PSD values obtained for each dipole were statistically compared (as dependent variables) across patients (N = 10) between the two different Pht-Pt interaction levels (high vs low) by means of a non-parametric statistical approach to obtain spectral activation maps. A permutation test (N = 5000 permutations) for a significance level of 0.05 was applied using the paired t-value as a measure of distance between the two distributions. A non-parametric approach is preferred for this class of data since it corrects for multiple testing and does not rely on Gaussianity assumption [63].

Clinical/functional assessment and empathic/relational evaluation

Descriptive statistics (computation of mean and standard deviation) has been implemented on data collected during the patients’ clinical/functional evaluation (scores obtained from clinical scales administered as in “Patients’ clinical assessment and functional evaluation” section) and the Pht-Pt empathic (EQ scores) and relational (CARE scores) assessment.

NASA-TLX data

The distributions of the NASA-TLX scores obtained for the six subscales of the questionnaire were checked for normality by means of Lilliefors test (alpha = 0.05). We performed a statistical test (one-way repeated measures ANOVA or Friedman test, depending on whether the normality test was verified or not) considering each NASA-TLX subscale score as the dependent variable and the type of FB (three levels: chart, emoticon, game) as within factor.

Tukey’s test was used for post-hoc comparisons. The software Statistica 8.0 (StatSoft Inc., USA) and Matlab custom scripts (Matlab 2021b, Mathworks Inc.) were employed for all the statistical tests.

Results

Evaluation of the effects of the experimental manipulations on ET metrics

As described in “Experimental design”, in the SINGLE -FB sessions (SS1-SS6) each FB was administered for two consecutive sessions and remained the same throughout the entire session, while the level of Pht-Pt interaction was manipulated according to three levels in each session (5 min each). In the multi-FB session (SS7), we set three FB types (5 min each) and two interaction levels per FB (2.5 min each). Hence, ET data recorded in SINGLE-FB sessions have a longer duration per condition than those recorded in the MULTI-FB and so a greater amount of data was available on which computing the ET metrics. Therefore, we chose to report on the main text the analysis of ET metrics related to SINGLE-FB sessions being more consistent and reliable and to report in the supplementary materials the ET metrics results assessed in the MULTI-FB session for which we performed the same analysis.

The results of the repeated measures MANOVA on the main factors FB and IL, for each ET metric in the three AoIs, are depicted in Table 2 (multivariate test) and Table 3 (univariate test). Regarding Table 2, it is worth noting that both the experimental factors (type of FB and Pht-Pt interaction level) have an effect on the distribution of patients’ gaze across the three AoIs. In particular, the FB factor has a significant effect on all the ET metrics combined for the three AoIs, whereas the IL factor, alone or in interaction with FB, only shows effects on total duration of fixations and number of fixations. Regarding the univariate analysis, FB factor has a significant effect on all the ET metrics in the three AoIs while, the IL and the interaction ILxFB have interestingly an impact on the ET metrics computed only in Pht and surrounding AoI.

Table 2 Results of the multivariate test obtained by computing a two-way repeated measures MANOVA (F-values) considering the ET metrics extracted in SINGLE-FB session in the three AoIs (monitor, Pht, surrounding) as multivariate dependent variables and the FeedBack (FB) and the Pht-Pt interaction level (IL) as within factors

Table 3 Results of the univariate test obtained by computing a two-way repeated measures MANOVA (F-values) considering the ET metrics extracted in SINGLE-FB session in the three AoIs (monitor, Pht, surrounding) as multivariate dependent variables and the FeedBack (FB) and the Pht-Pt interaction level (IL) as within factors

For a qualitative observation of how patients’ gaze allocation across the three AoIs is influenced by the experimental factors, the number of fixations within the three AoIs for the three FB types and the three interaction levels are depicted in Fig. 3 (for other metrics refer to Figures S1–S3 in supplementary materials). As for the game FB, patients predominantly were focused on the monitor AoI since the number of fixations is almost around 90%, independently on the level of Pht-Pt interaction.

Fig. 3

Pie charts reporting the distribution of the number of fixations (related to SINGLE-FB session) across the three AoIs for each FB type and Pht-Pt interaction level

Fixations count in Pht AoI is slightly higher (6%) in the case of high interaction with respect to the other two interaction levels (2–3%). The increase of fixations in Pht AoI is done at the expense of monitor AoI. A comparable number of fixations in the other AoI is found for the three interaction levels. Instead, for the chart and emoticon FBs, the number of fixations on the monitor AoI decreased compared to the game FB remaining the highest one, i.e. around 70% independently on the Pht-Pt interaction level. The remaining 30% of fixations is distributed between Pht and surrounding AoIs with a different ratio according to the level of Pht-Pt interaction. Moving from the high to the low level of Pht-Pt interaction, the number of fixations in Pht AoI reduced from 15 to 5% while those in the surrounding AoI increased from 16 to 26% for the chart FB and from 23 to 26% for the emoticon FB. Similar conclusions could be drawn from the observation of the same pie charts obtained for other eye-game metrics (see Figures S1–S3 in Supplementary Materials).

Figure 4 reports the results related to the within factor FB for the total duration of fixations and the number of fixations values, respectively in three different AoIs. Tukey’s post-hoc test reveals that both ET metrics referred to monitor AoI are higher when the game FB was presented to the patients with respect to the use of the other two FBs. On the contrary, the highest values for the two metrics extracted for Pht and surrounding AoIs are obtained when chart or emoticon FBs were used.

Fig. 4

Boxplots reporting results of MANOVA univariate test related to the FB within factor for the ET metrics total duration of fixations (a) and number of fixations (b) computed in the three AoIs and extracted from SINGLE-FB session. The symbol (*) indicates statistically significant differences between different levels of FB factor as revealed by Tukey’s post-hoc test (p < 0.05)

Figure 5 shows that during the administration of chart and emoticon FBs, patients left the screen AoI toward the other two areas earlier than the game FB. Complementary, the time spent before patients made their first fixation in an AoI different from the monitor, computed considering as reference the beginning of the rehabilitation session, is comparable among the three FBs for Pht AoI (FB factor not significant in this AoI) whereas is higher in game FB with respect to the other two FBs for the surrounding AoI.

Fig. 5

Boxplots reporting results of MANOVA univariate test related to the FB within factor for time to exit saccade metric in monitor AoI (a) and for the time to first fixation metric in Pht and surrounding AoIs (b), both extracted from SINGLE-FB session. The symbol (*) indicates statistically significant differences between different levels of FB factor as revealed by Tukey’s post-hoc test (p < 0.05)

Figure 6 reports the results of MANOVA univariate test related to the effect of interaction factor FBxIL on the total duration of fixations index computed in Pht (panel a) and surrounding (panel b) AoIs. It is worth noting that the total duration of fixations in both AoIs is almost zero when the patient was using game FB, independently on the level of interaction established with the therapist, and thus it is significantly lower with respect to the other two FBs, where the index values are almost comparable. In addition, in the case of both chart and emoticon FBs, the duration of fixations in the Pht AoI decreased according to the reduction of the level of interaction, being maximum in the case of the high interaction between therapist and patient. On the contrary, the same index computed on surrounding AoI was lower in the case of high Pht-Pt interaction with respect to the other two levels in the case chart or emoticon FBs.

Fig. 6

Boxplots reporting results of MANOVA univariate test related to the FBxIL interaction factor for the ET metrics total duration of fixations in Pht (a) and surrounding (b) AoIs. The symbol (*) indicates statistically significant differences between different levels of FB and IL factors as revealed by Tukey’s post-hoc test (p < 0.05)

Lastly, the results obtained applying the two-way repeated measures MANOVA to the ET metrics extracted from the MULTI-FB rehabilitation session (SS7) are almost comparable to those of the SINGLE-FB sessions (see Tables S1, S2 and Figures S4–S9 in Supplementary Materials).

Reconstruction of EEG-based spectral activations

The spectral activation maps depicted in Fig. 7 for theta and alpha bands are the result of the non-parametric statistical permutation test (α = 0.05) performed between high and low Pht-Pt interaction levels for each FB. Specifically, for the game FB, we found a synchronization in temporo-parietal junction (TPJ) over the two hemispheres for theta band and in the left hemisphere for alpha band and a desynchronization of the insula in alpha band when high versus low interaction conditions are compared. When chart FB was used, we found a high vs low synchronization in bilateral posterior superior temporal sulcus (pSTS) and right superior frontal gyrus (SFG) in theta band and in right orbitofrontal gyrus (OFG) in alpha band. In addition, negative activations in bilateral para-hippocampal cortex (PHG)/insula and right fusiform gyrus (FG) resulted in alpha band. As for the emoticon FB, we found a synchronization of precuneus, ACC and dorsolateral prefrontal cortex (DPFC) in theta band and left orbitofrontal gyrus, right fusiform gyrus and left PHC in alpha band. In addition, HIGH vs LOW desynchronization resulted for left TPJ in theta band and for right insula in alpha band.

Fig. 7

Group statistical spectral activation maps in theta and alpha bands obtained performing a non-parametric permutation test (α = 0.05), comparing the PSD obtained in high vs low interaction levels. Red voxels report significant synchronizations (t-values when high > low) while blue for desynchronizations (t-values when high < low). Images were generated using the Loreta Viewer utility, available at (http://www.uzh.ch/keyinst/loretaOldy.htm)

Clinical and functional evaluation

Clinical and functional evaluation data collected before SS1 and after SS7 are reported, averaged across patients, in Table 4. The scores show a slight trend of improvement for independence in activities of daily living and muscle strength, associated with a slight reduction in lower limb pain and spasticity. Speed performance over short and long distances tends to improve, with an increase in the average score on the WISCI scale. This result indicates that individuals with SCI achieve slightly higher speeds but with less need for aids or physical assistance. Despite the lack of statistically significant improvement, average changes in 10MWT, 6MWT and SCIM are above the smallest difference that people living with SCI perceive as beneficial (i.e. the minimal clinically important difference—MCID [64]).

Table 4 Results of the clinical and functional evaluation executed on individuals with SCI before and after the rehabilitation period (7 t-RAGT sessions)

Empathic and relational evaluation of Pht-Pt interaction

The EQ values for physiotherapists (42.70 ± 1.53) were similar for all three, indicating a good level of trait empathy. With reference to Pht-Pt interaction, the patients’ CARE scores are almost close to full-scale level (46.96 ± 0.99) suggesting the establishment of a good therapeutic relationship between patient and physiotherapist. In addition, such scores were not influenced by the FB type.

Workload assessment

The evaluation of perceived workload during the administration of the 3 FBs, via NASA-TLX, was rather homogeneous (chart: 57.43 ± 13.33; emoticon: 56.71 ± 11.76; game: 52.33 ± 11.50). However, game FB elicited a lower (F(2,18) = 4.58, p < 0.05) mental demand (chart: 65.00 ± 21.16; emoticon: 49.50 ± 12.54; game: 38.00 ± 26.25), and a lower (F(2,18) = 4.06, p < 0.05) physical demand (chart: 81.00 ± 13.45; emoticon: 75.00 ± 9.00; game: 66.00 ± 14.64,) with respect to the chart FBs. No differences between chart and emoticon FBs resulted in these two subscales. Moreover, the three FBs elicited a comparable moderate level of frustration (chart: 9.50 ± 5.35; emoticon: 12.00 ± 5.56; game: 12.00 ± 17.08) [65].

Discussion

In this study we proposed a multimodal assessment including psychological, EEG and ET data to investigate whether the combination of different types of visual FB and different levels of Pht-Pt interaction would impact on patient’s engagement during t-RAGT rehabilitation. We hypothesized, in fact, that the type of visual FB selected to inform patients on their performance together with the verbal contribution of the physiotherapist to support the patient might have an impact on patients’ perception of the therapy and thus influence their engagement and adherence to the rehabilitation program.

Although the purpose of the study was not to investigate the effects of patients’ engagement on clinical outcomes or functional performance, a battery of rating scales was administered to monitor patients and assess the possible occurrence of adverse events. On the basis of the scientific evidence related to Lokomat’s efficacy in patients with both subacute and chronic incomplete SCI (AIS C or D) we expected to describe benefits associated with the training in speed, walking distance, functional mobility and independence in ADLs, if the t-RAGT is generally administered with a frequency ranging from 2 to 5 sessions per week, for a minimum of 4 weeks [1, 4]. We only found a trend of improvement, not statistically significant, probably due to the small number of sessions carried out by the patients between pre and post clinical measurements. Although not statistically significant data, some of the improvements obtained after SS7 were above the minimum threshold for demonstrating a clinical benefit. In the framework of SCI, the MCID thresholds are available for SCIM, LEMS, 10MWT, and 6MWT [64, 66, 67]. Our results showed that the 10MWT, 6MWT and SCIM improvements were above MCID value, suggesting a clinical impact and a related perceived benefit of t-RAGT for gait speed and functional independence in individuals with incomplete SCI.

The physiotherapists included in this study demonstrated a high level of empathy, indicating effective interaction with others. Additionally, the evaluation of the Pht-Pt interaction was favourable, indicating that the physiotherapists’ interpersonal qualities were judged as satisfactory by the patients.

The overall perceived workload was comparable among the three investigated FBs, however patients reported a low mental and physical demand when game FB was provided. This is in line with previous works evaluating the effects of immersive virtual reality on patients’ state during t-RAGT that highlighted how gamification of a FB is associated with lower physical demand and higher performance whereas temporal demand remains unchanged [68]. The ET data showed that the monitor AoI monopolizes the visual attention of the patient (90% of the total session duration), independently on the FB used and on the level of Pht-Pt interaction (Fig. 3). This finding suggests a high patient’s adherence to the requested rehabilitation exercise [43] and are in line with previous studies evidencing that the combination of a motor exercise with real-time visual FB improves patients’ motor performances [69,70,71,72]. In addition, we found a significant effect of type of FB and level of Pht-Pt interaction, on almost all the ET metrics extracted (Table 2). Interestingly, FB modulated patients’ visual attention in all the areas of interest defined while the level of interaction between Pht and Pt combined with FB type showed an effect only on the ET parameters computed in Pht and surrounding AoIs (Table 3) [25].

As for FB type, independently on the level of Pht-Pt interaction, the total duration and number of fixations in monitor AoI were higher when game FB is used whereas the same indices calculated in Pht and surrounding AoIs showed an opposite trend, i.e. they were higher for chart and emoticon FBs with respect to game FB (Fig.4). The strong focus of patient’s visual attention on the monitor in the game FB and the higher tendency to get distracted with chart and emoticon FB were also confirmed by: (i) the time taken from the patient to leave the monitor AoI for the first time since the beginning of t-RAGT session (Fig.5a) and (ii) the time passed before patients moved their eye-gaze for the first time in Pht or surrounding AoIs from the monitor AoI (both significantly higher in game FB) (Fig.5b).

All the three selected FBs can be considered “extrinsic” since they take the performers’ attention away from their own movements and direct it towards the effects of these movements. The approach based on extrinsic FB, in contrast to that based on “internal” focus, has been shown to promote a more effective learning [73, 74]. However, the level of abstraction of the focus is different for the three FBs. In our interpretation, Game resulted as the most attractive FB for two interconnected reasons. In the Game FB the interaction information (weighted interaction torques) at the hip and knee joints of the robot are not simply shown to the patients, but they are used by the patients to control the avatar’s movement. Self-controlled practice, in fact, enhances learners’ motivation because it leads to a more active involvement in the learning process by encouraging them to take charge of their own apprenticeship process [75,76,77]. In addition, the presence of a reward is mainly expressed when game FB (represented by money and keys to be collected) is used compared to the other tested FBs. According to the theory of motivation, a reward is a trigger for the motivation of the participant (external motivation) and enhances participation in the exercise [68]. In a similar way, the emoticon FB should have increased patient’s participation by promoting a self-controlled exercise where the patient’s objective was to make the emoticon smile. However, in this case the patient’s visual attention is more often diverted to the surroundings. This could be due to an excessive abstraction of FB as the reward to the patients (emoticon smiling) is given based on a global performance (averaged among all joints). Such a synthetic information does not provide clear indications on how to adjust the motor action when the FB is neutral or negative (emoticon not smiling). The chart FB represents the lowest level of externalization of the attention focus of the patients since the FB variable is directly provided to the patients in the form of a chart, without any cover story or adaptation triggering extrinsic motivation. In addition, chart FB is the only one in which the patient also receives information about previous gait cycles for each joint, and this could be perceived as confusing since their actions are not directly transformed in a FB change on the monitor. In this case, patients had difficulty to keep their visual focus on the monitor and got distracted. In our opinion, the high number of information simultaneously provided to the patients was difficult to interpret.

In addition, we showed that the Pht-Pt interaction level has a different impact on the patient’s visual attention depending on the different FBs used. In the game FB, fixation metrics are almost close to 0% in Pht and surrounding AoIs, regardless of the level of interaction (patient’s visual attention is focused exclusively on the monitor, Fig. 6a and b). With chart and emoticon FBs we found a decreasing visual attention in Pht AoI along the decreasing level of interaction between Pht and Pt (Figs. 6). The reduced visual attention on Pht AoI in low interaction blocks is in favour of the surrounding AoIs (Fig. 6b). The importance of the Pht in directing patients’ attention toward the key aspects of the rehabilitation exercise has been already demonstrated in a previous study where the effect of therapist and visual FB were comparable in increasing patients’ motor output [25]

The fact that the patients were less focused on the FB when the verbal support of the Pht guiding the exercise was interrupted (low interaction) is in line with previous work highlighting the importance of a task-specific rehabilitation exercise focused on individual joints or individual gait phases, as opposed to a general, unfinished encouragement to train a specific body part, or support provided only by the FB through the screen without any intervention from the physiotherapist [25, 78, 79]. Our results, to be taken with caution due to the limited number of patients included, show that the effect of the Pht on patients’ attention is different depending on the FB used: more game-like FBs are less influenced by the presence of the physiotherapist while more synthetic or less intuitive FBs require a higher contribution from the therapist to guide patients during rehabilitation. In this context, we believe that FB and physiotherapist are not mutually exclusive, but they could jointly contribute to improve patients’ focus on the exercise.

Our EEG data showed that neural activation in brain regions crucial for social cognition is modulated by different levels of Pht-Pt interaction [80, 81]. During high interaction levels, we revealed increased activation in the superior frontal region, the parietal region, and the posterior cingulate, whereas during low interaction we reported insula activation, independently of the FB type. For the game FB specifically, activations were elicited in the TPJ bilaterally, during the high level of interaction, and in the insula, when the communication between physiotherapist and patient is interrupted, respectively. In literature, the TPJ, a key region of mentalizing [82], has been proposed to be involved in activities that are important for social interaction: focusing attention, interpreting socially relevant stimuli, and determining social intentions [83, 84]. Thus, the activation of TPJ during collaborative action reflects the degree of cooperation between partners, which is relevant for determining the partner’s intent. Insula, particularly the anterior part, has been implicated in empathic abilities during negative emotion, is involved in auditory and facial affect processing [85,86,87]. Regarding the chart FB, by comparing again high versus low interaction levels, we observed activation in the bilateral posterior superior temporal sulcus, a brain region involved in representing goal-directed action [88] and associated with perception of the gaze of a face, in particular when the observer’s expectation is violated [89]. Moreover, in this condition, our data also reported a desynchronization in bilateral parahippocampus/insula and fusiform gyrus, brain regions associated with facial emotional expressions and negative experience [90, 91]. Such information, combined with the high visual attention in Pht AoI demonstrated by the ET data in case of high level of interaction, allows to claim that, when chart FB is used, the patients need more indications and support from the physiotherapist, as proved by the activation of brain circuits involved during a social exchange. Similarly, the emoticon FB evidenced pronounced activation in different social brain areas when Pht and Pt strongly interacted while again insula was involved when their communication was interrupted.

To summarise, altogether ET and EEG results complementarily highlight that the therapist focused the patient’s attention on the monitor and on himself interacting with the patient, as evidenced by spectral activation in areas typically involved during social interaction. Interestingly, when the Pht-Pt contact was interrupted we reported insula activation, independent of the type of visual FB, that we interpreted as a negative emotional experience.

Besides the neuroscientific implications of our findings which should be confirmed in a larger population of patients, some useful considerations could be drawn on methodological aspects. It is worth of note that the ET metrics extracted during the single t-RAGT session showed a strong consistency and reliability along different rehabilitation sessions carried on 2 or 3 weeks apart. We demonstrated that the same conclusions about the modulation of patient’s visual attention by FB type and Pht-Pt interaction level could be drawn by looking at training sessions planned in two consecutive days (SINGLE-FB session) and were consistent with the corresponding experimental blocks in the MULTI-FB session. Such result is corroborated by a strong concordance between EEG results and ET metrics extracted in non-simultaneous sessions (SINGLE-FB and MULTI-FB sessions). Results about ET metrics reliability are in line with what has already been demonstrated in other experimental paradigms and in other pathologies [92,93,94,95].

Although we implemented a feasibility study in preparation of a larger randomised clinical trial assessing the link between patients’ clinical outcome and factors related to the rehabilitation session itself (type of feedback or level of interaction between patient and therapist), we are perfectly aware that the current study has a main limitation in the low patients’ sample size. The obtained results should be confirmed in a larger population and should assess the impact of the different type of visual FB on patients’ rehabilitation outcome.

In addition, a further limitation consists in the lack of measurements relative to the actual patient’s physical exertion action during t-RAGT in the different conditions. Indeed, physiotherapist requested the patients to actively contribute to the motion by exerting active movements at the Lokomat joints level. The adherence to this request, which was meant to match with the actual visual FB available on the Lokomat and that it has been demonstrated to be the ground of a favourable rehabilitation outcome [25, 96], was not assessed in this study and hence its relationship with the considered psychological, behavioural and neuroelectrical measures was not investigated. In fact, by means of ET and EEG we only quantified patients’ engagement as a compound of visual attention and cognitive processing completely relying on the hypothesis that a high visual attention was directly correlated with an active action performed by the patient [97,98,99] without eventually verifying the real adherence of the physical activity to the FBs and physiotherapist demands. Future works should address this aspect by including the measure of active/resistive action exerted by the patients during human–robot interaction and link it with the factors modulated in this study (FB and Pht-Pt interaction level) and ultimately with patient’s rehabilitation outcome in a larger randomized controlled design.

Conclusions

To the best of our knowledge this is the first study in which a multimodal approach combining psychological, EEG and ET measures has been employed to evaluate patient’s engagement during t-RAGT sessions with the Lokomat. Obtained results demonstrate that the type of FB provided to the patients combined with a specific level of interaction with the physiotherapist has an effect in focusing patients’ visual attention and increasing their adherence to the prescribed motor exercise. The compound results, if confirmed in a larger population of patients, prospectively provide hints for the definition of new rehabilitation protocols in t-RAGT, including information on the type of FB to be adopted and the way the physiotherapist should interact with the patient, with the final aim to promote a strong engagement in the rehabilitation exercise, which is established as promising for a positive outcome. Our findings, if confirmed in studies with higher sample size, contribute to emphasize the crucial role of the physiotherapist in the rehabilitation process, which is not secondary to the human–robot dyadic relationship, and to further promote a richer therapist-mediated human–robot interaction in RAGT.