A fuzzy-logic approach for longitudinal assessment of patients’ psychophysiological state: an application to upper-limb orthopedic robot-aided rehabilitation

Abstract

Understanding the psychophysiological state during robot-aided rehabilitation is crucial for assessing the patient experience during treatments. This paper introduces a psychophysiological estimation approach using a Fuzzy Logic inference model to assess patients’ perception of robots during upper-limb robot-aided rehabilitation sessions. The patients were asked to perform nine cycles of 3D point-to-point trajectories toward different targets at varying heights with the assistance of an anthropomorphic robotic arm (i.e. KUKA LWR 4+). Physiological parameters, including galvanic skin response, heart rate, and respiration rate, were monitored across ten out of forty daily sessions. This data enabled the construction of an inference model to estimate patients’ perception states. Results expressed in terms of correlation coefficients between the patient state and the increasing number of sessions. Correlation coefficients showed statistically significant strong associations: a state of heightened engagement (formerly described as “Excited”) had $\rho = -0.73$ (p-value=0.01), and a more calm and resting state (namely “Relaxed” state) had $\rho = 0.70$ (p-value=0.02) with the number of sessions completed. All patients had positive interaction with the robot, initially expressing curiosity and interest that gradually shifted to a more “Relaxed” state over time.

Introduction

Recent advancements in robotic technologies have revolutionized clinical therapy for individuals with musculoskeletal [1, 2] and neuromuscular [3, 4] disorders. These systems offer distinct advantages by (i) providing objective measurements of a patient motor performance [5], (ii) actively engaging patients through biofeedback mechanisms [6] and immersive virtual reality games [7, 8], and (iii) dynamically adjusting assistance levels based on the user overall condition [9].

Despite the significant advancements and benefits offered by robotic technologies in rehabilitation, there is still a notable gap in fully characterizing a patient state during therapy sessions. While many studies in robot-aided rehabilitation have primarily focused on assessing user performance through kinematic [10] and biomechanical perspectives [11], these metrics provide only a partial understanding of the experience. Neglecting the PsychoPhysiological (PP) aspects, such as emotional states, stress levels, and overall perception of the technology, can limit the understanding of how patients truly engage with and benefit from robotic rehabilitation [12]. By broadening the focus beyond kinematics and biomechanics to include the patient’s PP state, it is possible to gain a deeper insight into the users’ perception of trust in robotic systems [13, 14]. This insight is essential in designing user-centered technologies and the optimization of rehabilitation protocols to enhance patient experience and outcomes [15].

According to the Russell circumplex model [16], the PP state of a user can be represented by two key dimensions: Arousal and Valence. Arousal reflects the level of a person’s engagement or reaction to a stimulus, while Valence indicates the positive or negative quality of the user’s experience. These dimensions are commonly assessed using the Self-Assessment Manikin (SAM) questionnaire [17], a visual tool designed to capture Arousal and Valence perceptions. Administered immediately after a stimulus or task, the SAM questionnaire provides a quick and intuitive method to collect data on participants’ emotional responses. Collecting this information immediately following a robot-aided rehabilitation session allows for a quick and intuitive assessment of patients’ perceptions regarding the impact of the robot on their emotional state [18]. However, relying solely on the SAM questionnaire at the end of each robot-aided rehabilitation session presents challenges. Firstly, the SAM questionnaire provides a general overview of the participant’s perception of the entire session, lacking the granularity to capture the specific emotional states experienced throughout the session. Moreover, the responses obtained from the SAM questionnaire are entirely subjective, lacking an objective foundation or measurable parameters. This subjectivity poses challenges in accurately capturing and quantifying patients’ emotional states. As a result, a more continuous and objective method is needed to assess patients’ PP states during robot-aided rehabilitation sessions. Such a method would allow for real-time monitoring of Arousal and Valence levels, providing a comprehensive understanding of patients’ emotional experiences and their impact on therapy outcomes.

In the literature, some approaches have been proposed to estimate the patients’ PP state by leveraging measurable processes such as physiological parameters [19, 20]. These parameters, closely tied to the autonomic response of the nervous system, can be readily associated with the user’s PP state. During robot-aided rehabilitation, physiological responses have been monitored, revealing distinct patterns based on the user interaction with the rehabilitation robot or haptic interface [21]. Studies have shown that users exhibit varying physiological responses corresponding to different levels of physical and cognitive workloads [22]. This suggests that the user’s PP state can be inferred from their physiological reactions during the rehabilitation session, providing valuable insights into their emotional and cognitive engagement with the technology [23].

Fuzzy Logic stands as a prominent method for continuous estimation of users’ PP states in rehabilitative contexts and various other scenarios. The Two-step Fuzzy Logic approach has been effectively applied to derive the PP state of users engaged with planar rehabilitation robots, enabling adjustments to the difficulty levels of virtual reality games [24] or adaptation of robot stiffness [25]. In [26], a method was proposed to continuously quantify the emotional states of gamers interacting with entertainment technologies. Through Two-step Fuzzy Logic, the participants’ physiological responses were translated into Arousal and Valence, further categorized into emotional states such as fun, excitement, frustration, challenge, and boredom. This approach allowed the mapping of imprecise inputs into emotions, offering an objective means to monitor user experiences. Another instance of Fuzzy Logic application in retrieving users’ PP states is highlighted in [27]. In this study, participants underwent treadmill-based exoskeleton-assisted walking, with synchronous collection of physiological parameters. The Two-step Fuzzy Logic method was utilized to estimate four PP indicators: attention, stress, energy expenditure, and fatigue.

Based on the existing literature, Fuzzy Logic has proven effective in translating physiological data into meaningful indicators of the user’s emotional and cognitive state, offering insights for enhancing rehabilitation strategies. However, existing studies mainly focus on estimating the PP state in a single session for user experiences. This reveals a gap in the literature regarding the application of these methods across multiple robot-aided rehabilitation sessions to track the evolution of PP states over time. Addressing this gap is crucial for longitudinal studies, enabling the assessment of how patient-robot interaction evolves. Currently, limited research in this area hinders understanding the long-term impact of robot-aided rehabilitation on patients’ PP responses.

Therefore, the objective of this study is to develop a psychophysiological state estimation approach and apply it during multiple robot-aided rehabilitation sessions to assess how the perception of the patient-robot interaction evolves over consecutive treatments. By analyzing the evolution of PP states, this research seeks to provide valuable insights into the long-term impact and effectiveness of robot-aided rehabilitation on patients’ psychophysiological responses. Furthermore, having a methodology for estimating the PP status of patients during the rehabilitation protocol allows the robotic treatment to be adapted to the needs of the specific patient.

The rest of the paper is structured as follows: Section 2 and Sect. 3 present the proposed psychophysiological estimation method and its experimental evaluation, respectively. In particular, the PP estimation approach, the experimental setup, and the protocol of the robot-aided rehabilitation session are presented. Section 4 shows and discusses the obtained experimental results. Lastly, Sect. 5 summarizes the principal outcomes of the research and draws the future work.

Psychophysiological estimation

The block scheme of the PP state estimation approach applied in this work is highlighted in Fig. 1. The patient PP state estimation approach takes as input the physiological parameters of the patients interacting with the rehabilitation robot. Specifically, the cardiorespiratory activity and the Galvanic Skin Response (GSR) are monitored during the entire rehabilitation session. The acquired parameters are then normalized to enhance the responses with respect to the subject’s resting condition, referred to in the following as the baseline, and given as input to a Fuzzy Logic model. This Fuzzy Logic model serves as an inference system implemented to map physiological responses onto the defined PP states by exploiting Russel’s circumplex model. It organizes affective states along two dimensions, Valence (pleasant-unpleasant) and Arousal (high-low), and provides a framework for understanding and categorizing the patient’s emotional and affective states during the rehabilitation process. By utilizing this model within the Fuzzy Logic system, it is possible to obtain a deep understanding of the complex patient state.

Fig. 1

Block scheme of the psychophysiological state estimation approach applied during upper-limb robot-aided rehabilitation

Physiological monitoring system

In the process of estimating the PP state of patients, it is paramount to gather physiological data during their interaction with the rehabilitation robot. This involves monitoring and quantifying both GSR, which reflects cognitive activity, and cardiorespiratory activity, providing insights into the physical aspect of the condition of the patient.

GSR is one of the most valuable physiological parameters for estimating a user’s Arousal level [28] and cognitive load [29]. This is due to the fact that the skin’s electrical properties undergo changes whenever an individual is stimulated by visual, acoustic, haptic, or physical stimuli. Starting from the raw GSR signal, it is possible to compute its tonic level called Skin Conductance Level (SCL). Specifically, the SCL can be retrieved by applying a 4th order Butterworth low-pass filter with a cutoff frequency of 0.1 Hz to the raw GSR. SCL represents the baseline GSR level in the absence of specific external stimuli and/or user activity. Furthermore, the rapidly changing phasic GSR component, also called Skin Conductance Response (SCR), can be computed from the raw GSR to detect instantaneous spiking activity, highly correlated with the administration of specific stimuli and useful to design reactive robot behaviors [30]. However, since the primary objective of this work is to quantify the overall psychophysiological state of the patients during the overall rehabilitation session, the methodology adopted in this paper follows the guidelines provided by [31] in which only the tonic component was employed.

On the other hand, HR and RR serve as fundamental indicators of the physical exertion or workload experienced by an individual [32]. When a person engages in physical activity, the body’s demand for oxygen and energy increases. Consequently, there is a natural tendency for both HR and RR values to rise, reflecting the body’s physiological response to this heightened demand. These changes in HR and RR are integral components of the body’s adaptive mechanism to meet the increased metabolic needs associated with physical exertion. Moreover, these physiological parameters are not solely influenced by physical exertion; they also respond to various stimuli of different natures [33]. Studies have shown that HR, measured in beats per minute [bpm], tends to decelerate following the administration of certain stimuli [34]. Conversely, an increased HR or significant variability compared to the baseline can indicate an excited or emotionally aroused condition. Similarly, changes in the RR, expressed in breaths per minute [bpm], can also be influenced by different stimuli [35]. While modifications in RR may occur at a slower pace than those observed in other physiological signals, it is important to recognize that both HR and RR can respond to a variety of stimuli, including emotional, sensory, or cognitive stimuli. These changes in HR and RR reflect the dynamic interplay between the body’s physiological responses to external stimuli and internal states.

Given the high intra- and inter-subject variability of physiological signals as a result of age, gender, time of day, and other factors, all the collected data required a normalization procedure with respect to a baseline value acquired from the volunteer blindfolded and acoustically isolated [36]. Given the physiological parameters vector defined as $X = \left[ SCL, HR, RR \right]$, the normalization procedure removes the baseline physiological condition of the patient as

$$\begin{aligned} X_r(t) = \frac{X(t)-X_{B}}{X_{B}} \end{aligned}$$

(1)

where t is the time stamp, X(t) is the physiological vector sampled at the t-th time instant, and $X_{B}$ is the mean physiological parameter vector computed during the resting baseline phase.

Psychophysiological estimation model

The physiological responses $X_r$, computed during the rehabilitation session, serve as input to a Fuzzy Logic estimation model [37]. A Fuzzy Logic inference model is a system that utilizes fuzzy logic to handle imprecise or uncertain information. Unlike traditional logic that deals with binary true or false values, fuzzy logic allows for degrees of truth, enabling a more flexible and nuanced approach to decision-making.

In this scenario, a Fuzzy Logic inference model is particularly useful for several reasons. Firstly, it allows for the integration of various physiological parameters, which often exhibit complex and overlapping patterns in real-world data. Fuzzy logic excels at handling such complexities by defining Membership Functions (MFs) that capture the fuzzy boundaries between different states of these parameters. Secondly, Fuzzy Logic models are adept at interpreting ambiguous or vague inputs, which is often the case with human physiological responses. By defining linguistic variables such as “LOW”, “MID”, and “HIGH” for each parameter, the model can effectively categorize and analyze the patient’s PP state. Moreover, the Fuzzy Logic inference model offers interpretability, as its conditional “IF-THEN” rules are grounded on human-understandable linguistic terms rather than purely mathematical formulations.

Three MFs are generated using data collected from all enrolled participants for each input signal of $X_r$. Gaussian membership functions were chosen for this application because they provide a smooth, continuous representation of physiological data and can handle uncertainty in boundaries more effectively. Although other types of MFs could have been considered (e.g., triangular or trapezoidal), Gaussian functions were selected due to their flexibility in capturing gradual transitions between different levels of physiological activation. This approach aligns well with the real-world variability present in human physiological responses. Even though the data is not assumed to follow a perfect normal distribution, Gaussian MFs offer a robust method for capturing the central tendency (mean) and variability (standard deviation) in the dataset, ensuring that overlapping patterns between levels are smoothly represented.

Following the methodology presented in [27], the physiological responses were divided into three levels, represented by the linguistic variables “LOW”, “MID”, and “HIGH”, following a structured process. First, the collected physiological responses are sorted in ascending order to arrange the data by gradually increasing activation levels. This sorted dataset is then divided into three equal parts, ensuring that each part represents one of the three levels of physiological activation. This method ensures that the division into “LOW”, “MID”, and “HIGH” is directly based on the distribution of the collected data, making the categorization proportional to the actual physiological responses observed in the participants. Given the three sets, the center of each Gaussian is the mean value of the distribution of collected physiological responses. Additionally, the standard deviation of these Gaussian functions is set equal to that of the data, ensuring an accurate representation of the variability within the dataset.

The outputs of the Fuzzy Logic estimation model serve to represent the PP state of the patient, expressed in terms of Arousal and Valence. These PP state values are constrained to a range from 0 to 1, providing a normalized scale for interpretation. In the output layer, five equally spaced Gaussian MFs are constructed, enabling the model to capture and characterize different levels of the patient’s PP state with greater detail. These five levels are linguistically represented by the activation levels “LOW”, “MID-LOW”, “MEDIUM”, “MID-HIGH”, and “HIGH”.

Once the MFs are defined, a set of fuzzy rules can be established and implemented to estimate Arousal and Valence. Following the literature, a total of 17 rules were carefully crafted to derive estimations of Arousal and Valence based on the physiological response vector $X_r$ [22, 24, 29]. Lastly, the fuzzy operators were defined in the MATLAB Fuzzy Logic Toolbox, and used to analyze the collected data (MATLAB R2020b). The Mamdani method was selected for its intuitive “IF-THEN” rules and ability to generate flexible, interpretable fuzzy outputs, which are ideal for handling complex physiological data. Unlike Sugeno, which provides precise outputs, Mamdani better captures general trends and variability in the patient’s psychophysiological state, making it more suitable for this application. The Boolean AND and OR operators were replaced with min() and max(), respectively, as they offer a smooth transition between overlapping membership values, which is more appropriate for physiological signals that naturally involve gradual transitions rather than strict binary distinctions. The implication and aggregation methods are min() and max() respectively. The implemented defuzzification process is the area centroid or center of gravity. The resulting Fuzzy Logic estimation module is schematically reported in Fig. 2. In particular, the input layer, made of the three MFs of the $X_r$, the Estimation Module itself implementing the conditional rules, and the two outputs are reported in Fig. 2 from the left side to the right, respectively.

Fig. 2

Fuzzy Logic inference system designed for the patients’ PP state estimation

The outputs of the Fuzzy Logic inference system, namely the Arousal and Valence values, serve as key indicators for interpreting the Psychophysiological (PP) state of the participants. In this study, four distinct PP states corresponding to each quadrant of the Valence-Arousal plane were defined as follows:

• High Valence AND High Arousal: This condition signifies a high level of participant engagement and positive affect. In this PP state, termed as “Excited”, patients exhibited curiosity and enjoyment while interacting with the robot. They were actively involved and found the experience stimulating and rewarding.

• High Valence AND Low Arousal: Lower Arousal values within the context of high Valence indicate a state of calmness and positive emotion. This relaxed and positive condition was labeled as “Relaxed”. Patients in this state were at ease, experiencing a pleasant and tranquil interaction with the robot.

• Low Valence AND Low Arousal: Participants falling into this quadrant experienced a state of “Bored”, characterized by low arousal and negative valence. This negative calm condition reflects a lack of interest or engagement, where patients may have felt disinterested or uninvolved during the interaction.

• Low Valence AND High Arousal: When a patient experiences high arousal along with low valence, it indicates a state of agitation and negative emotion. This PP state is termed as “Stressed”, where patients are agitated by strong negative emotions, possibly feeling overwhelmed or anxious during the interaction.

By categorizing the PP states in this manner, it is possible to quantify how participants responded emotionally and physiologically to the rehabilitation robot use.

As a synthetic indicator, the percentage of time that a patient spent in a specific PP state ($TP_{state}$) during each robot-aided rehabilitation session was computed. This calculation describes how much of the session time was occupied by each emotional and physiological state. The equation used to compute this percentage of time spent in a PP state ($TP_{state}$) is

$$\begin{aligned} TP_{state} = \frac{T_{state}}{T_{tot}} \cdot 100 \end{aligned}$$

(2)

where $T_{state}$ is the total duration of time spent in a certain PP state among {“Excited”, “Relaxed”, “Bored”, and “Stressed”}, and $T_{tot}$ is the total duration of the rehabilitation session.

Experimental validation

Experimental setup

The adopted robot-aided orthopedic rehabilitation platform is composed of: i) a 7-degree-of-freedom anthropomorphic robotic arm (i.e. the KUKA Light Weight Robot 4+); ii) a custom 3D printed end-effector, to support the patient wrist; iii) a purposely developed virtual reality environment developed using Unity and projected onto a 2D monitor to show to the patient the task to be performed and iv) a physiological monitoring system. Fig. 3 shows a participant wearing the physiological monitoring system while interacting with the robot.

Fig. 3

Experimental setup used to test the proposed psychophysiological estimation approach

The robot is controlled through Robot Operating System (ROS) Kinetic middleware on Ubuntu 16.04 LTS using a tunable interaction control described by the equation:

$$\begin{aligned} \tau _c = {\textbf{B}}({\textbf{q}}) \cdot \textbf{J}_{\textbf{A}}^{\dagger }({\textbf{q}}) [{\bar{\textbf{K}}}_{\textbf{r}} {\tilde{\textbf{x}}}_{\textbf{r}} + {\bar{\textbf{K}}}_{\textbf{t}}{\tilde{\textbf{x}}}_{\textbf{t}}] + {\mathbf{n}(\mathbf{q},{\dot{\mathbf{q}}})}) \end{aligned}$$

(3)

Here, ${\tau _c}$ denotes the torque applied to the robot joints. The terms $\mathbf {B(q)}$ and ${\mathbf{n}(\mathbf{q},{\dot{\mathbf{q}}})}$ account for inertial properties and dynamic effects including Coriolis and gravitational forces. $\textbf{J}_{\textbf{A}}^{\dagger }\mathbf {(q)}$ is the pseudo-inverse matrix of the analytical Jacobian. Additionally, ${\tilde{\textbf{x}}}_{\textbf{r}}$ and ${\tilde{\textbf{x}}}_{\textbf{t}}$ indicate the radial and tangential pose errors concerning the desired pose $\mathbf {x_d(t)}$.

The interaction controller offers two supportive components:

• ${\bar{\textbf{K}}_{\textbf{r}}} {\tilde{\textbf{x}}}_{\textbf{r}}$ maintains the robot end-effector close to the intended path.

• ${\bar{\textbf{K}}_{\textbf{t}}} {\tilde{\textbf{x}}}_{\textbf{t}}$ moves patients tangentially to progress along the trajectory within a predetermined timeframe.

Expanding upon this concept, ${\bar{\textbf{K}}}\textbf{r} = \mathbf {A^T K_r A}$ and ${\bar{\textbf{K}}\textbf{t}} = \mathbf {A^T K_t A}$, where ${\textbf{A}}$ represents the adjoint matrix. This matrix serves as the transformation from the pose error expressed in the base frame $[{\textbf{X}}_{B}, {\textbf{Y}}_{B}, {\textbf{Z}}_{B}]$ to the moving frame $[{\textbf{X}}_T, {\textbf{Y}}_T, {\textbf{Z}}_T]$, whose ${\textbf{X}}_T$ axis is tangentially aligned to the planned path. The gain matrices are defined as $\mathbf {K_r}=diag(0, k, k, k_{\phi }, k_{\phi }, k_{\phi })$ and $\mathbf {K_t}=diag(k_t, 0, 0, 0, 0, 0)$, adjusting the system’s stiffness radially and along the task direction, respectively. Specifically, $k_t=0$ N/m when the patient can move independently toward the target, and $k_t=k$ when the patient requires assistance. The influence of the controller gains on tracking errors was tested for two different values of $k=\{300, 1000\}$ N/m, in a previous study [25]. The mean tracking errors resulted to be $7.8\pm 3.9 \cdot 10^{-3}$ m to $17.7\pm 8.3 \cdot 10^{-3}$ m for $k = 1000$ N/m and $k = 300$ N/m, respectively. In this experiment, the robot motion planner generates three-dimensional point-to-point movement to be executed in a maximum of 7.5 seconds with the k and $k_{\phi }$ control gains set at 1000 N/m and 800 Nm/rad, respectively, because all the patients enrolled in this study were in the early stages of their rehabilitation treatment. At this stage, higher stiffness was prioritized to ensure maximum assistive intervention from the robot and guarantee controlled movements.

Safety measures were a key consideration during the design of the experimental setup. The control law implemented in Eq. (3) derives from an impedance control and ensures safe physical interaction with the patient, managing the interaction force for smooth and controlled movements. Additionally, software safety limits were set to stop the therapy if force or velocity thresholds were exceeded. More in detail, whenever the robot is asked to provide a force higher than 50 N or the end-effector is manipulated at a speed higher than 0.2 m/s, the software interrupts the session. A force limit of 50 N is typically considered safe for human-robot interaction, as it is below the threshold that could cause harm to soft tissues [38]. Similarly, a velocity limit of 0.2 m/s ensures controlled and smooth movements, reducing the risk of sudden, high-speed actions that might overwhelm or shock the patient [39]. An emergency stop button, operated by a technician present during all sessions, was also included to immediately halt the robot if necessary. These measures ensured that any accidental interactions were handled safely, with the robot automatically stopping when required.

The physiological monitoring system measures the GSR, the heart activity, and the respiration of the patients. The GSR is measured by using two electrodes of the Shimmer 3 GSR+ Unit placed on the index and middle fingers of the non-dominant hand. Such a difference of potential, collected with a sampling rate of 52.1 Hz, allows for retrieving information about the user’s electrodermal activity. Both the electrical and respiratory activities of the enrolled participants are monitored by using the BioHarness 3.0 chest belt, developed by Zephyr^TM Technology. Such a wearable device fuses capacitive and stretch sensors: the former assesses the electrical activity of the heart, and the latter measures the deformations of the rib cage induced by respiration. To better measure the electrical changes due to the heartbeats, the BioHarness sensor is worn against the skin, at the height of the sternum. The physiological data are acquired synchronously under Yet Another Robot Platform (YARP) [40] at 40 Hz.

Experimental protocol

Eight orthopedic patients (mean age $63.3\pm 11.8$) who suffered from musculoskeletal disorders were enrolled in this longitudinal study. All participants underwent a surgical procedure, whose characteristics are shown in Table 1.

Table 1 Demographic characteristics of the patients enrolled in this study

In particular, four patients underwent rotator cuff suture, while one patient had an open fracture of the humerus treated surgically. Following the surgical intervention, all patients adhered to a standard post-operative period of upper-limb immobilization. After this immobilization phase, patients began mobilization under the supervision of physical therapists using conventional rehabilitation methods. Only once patients achieved 90 degrees of active shoulder elevation without assistance were they eligible to be enrolled in the robotic rehabilitation study.

At this point, inclusion criteria were applied, ensuring that participants had no prior experience with robotic systems to avoid any bias in their psychophysiological responses. Additionally, all participants were required to have normal or corrected-to-normal vision to reduce the risk of discomfort or disorientation from the use of virtual reality. Once enrolled, the participants proceeded to the robot-assisted rehabilitation protocol.

Before starting the tests, participants were thoroughly informed about the experimental procedures, including the tasks they were required to perform and the purpose of the physiological monitoring. Detailed instructions were given to ensure that they fully understood the steps involved. The rehabilitation protocol included twenty daily robot-aided rehabilitation sessions spread over one month. Among these sessions, ten were specifically designated for the patients to be equipped with a physiological monitoring system to assess their PP state. Fig. 4 reports the diagram of the protocol used in these experiments.

Fig. 4

Diagram of the trial stages carried out in this study

At the beginning of each experimental session, a five-minute resting baseline was recorded to establish the physiological rest condition of each participant. During this baseline period, patients were instructed to sit comfortably, with their eyes blindfolded and in an acoustically isolated environment, to facilitate the attainment of a restful state [36]. The mean values of the physiological parameters collected during this baseline (denoted as $X_{RB}$) were used in Eq. (1) to compute the physiological responses $X_r$. Following the Resting Baseline recording, the participant progressed to the robot-aided rehabilitation session itself.

Throughout all twenty sessions, patients were instructed to perform nine cycles of nine three-dimensional reaching movements with their post-surgical upper limb toward targets in the Cartesian space and return to a resting position after each movement. During the first session of each enrolled participant, the robot was placed in gravity compensation mode ($k=0$ N/m), allowing the patients to explore their reachable workspace. In this phase, the target points were selected based on the positions the patient could comfortably reach at different heights without experiencing pain. The nine target positions were distributed across three angular directions and three different heights. Additionally, the resting position, where the patient’s shoulder was at approximately $0^\circ$ flexion and the elbow at $90^\circ$, was also recorded. Patients then followed a trajectory displayed on a screen with the aid of the robot. The robot control actions, as presented in Eq. (3), were presented to the patients before starting the interaction session. This structured activity was designed to engage the upper limb joints and promote the recovery of the range of motion. Fig. 5A shows an example of the three-dimensional reaching of the target and the subsequent return to rest position. Furthermore, an insight into the virtual reality displayed to the patients is provided in Fig. 5B. The position of the robot’s end-effector is mapped to the virtual wrist of the user’s avatar, thereby reflecting their movements in real-time. The target trajectory is highlighted in pink for the user to follow during the task. Lastly, Fig. 5C displays a graphical representation of the nine target points inside the virtual reality environment.

Fig. 5

A Example of a three-dimensional reaching and returning to the rest position execution. B Insight on the virtual reality game. C Graphical representation of the nine target points inside the virtual reality game

It is important to note that the duration of the robotic rehabilitation sessions was not fixed for all participants. The session lengths varied because the control strategy allowed patients to perform the movements at their own natural pace. This approach ensured that participants could follow the trajectory according to their physical condition and recovery progress. The robot only intervened when a reaching or return movement exceeded a predefined time limit, assisting as necessary to help the patient complete the task. Adopting Eq. (2) as a key performance indicator enables the management of varying session durations. This is achieved through the normalization of the time spent in each state with respect to the session duration $T_{tot}$.

The study was conducted under Ethical Committee approval (Ethical Approval N. 03/19 PAR ComEt CBM) and following the Declaration of Helsinki. All patients were adequately informed about the purpose of the study and gave their written informed consent.

Statistical analysis

A linear correlation analysis was conducted on the collected data to investigate potential changes in the $TP_{state}$ across the rehabilitation sessions. The linear Pearson correlation analysis was selected as the most appropriate method for measuring the strength and direction of linear relationships between two continuous variables. The objective of this study was to assess whether a linear trend existed between the percentage of time spent in each PP state and the number of rehabilitation sessions. Given the nature of the data, which involves session progression over time and proportional time spent in various PP states, Pearson’s correlation is an appropriate method for identifying potential linear associations. A test vector ranging from 1 to 10, representing the session number, was defined. For each PP state percentage, a linear Pearson Correlation Coefficient ($\rho$) and its associated p-value were computed. The significance level was set at 0.05.

The derived correlations were classified, according to [41], as:

• Very weak: $|\rho | \le 0.19$

• Weak: $0.20 \le |\rho | \le 0.39$

• Moderate: $0.40\le |\rho | \le 0.59$

• Strong: $0.60\le |\rho | \le 0.79$

• Very strong: $0.80\le |\rho | \le 1.0$

A correlation was considered statistically significant if the p-value was $\le 0.05$. This analysis aimed to identify any noticeable trends or patterns in the time spent in different PP states throughout the rehabilitation sessions, providing insight into the patients’ psychophysiological responses throughout the rehabilitation program.

Results and discussions

All participants successfully completed the robot-aided rehabilitation sessions without reporting any significant discomfort, indicating that the tasks were well-tolerated and within their physical capabilities.

Fig. 6 reports the raw data extracted from the robot during the first three cycles of a representative participant’s nine three-dimensional reaching movements. This participant was selected because his responses aligned with the overall trends observed in the study, providing a clear and representative example of the robot trajectories and physiological changes recorded during the rehabilitation sessions. More in detail, the desired and current positions, namely pd and p, along with the forces exchanged at the robot end-effector are reported in the left column of Fig. 6. The joint trajectories of the robot and a three-dimensional representation of the assigned reaching and returning to rest position tasks are presented in the right column.

Fig. 6

Raw data collected from the robot during one experimental session. The left column reports the desired pd and current p positions along x, y, and z and the forces exchanged at the end-effector, i.e. $F_x$, $F_x$, and $F_z$. The right column displays the trajectories in the joint space and a three-dimensional representation of the assigned reaching and returning to rest position tasks

Moreover, Fig. 7 reports a comprehensive overview of the physiological responses during the first 5 minutes of that robot-aided rehabilitation session. More in detail, the top panel displays the desired and actual trajectories of the robot along the z-axis, just to provide a reference with respect to the motion of the robot. The second panel shows the GSR in [$\mu S$] over time. As the robot starts delivering the therapy session, i.e. starts moving, there is a notable increase in the GSR levels. This rise suggests an elevated physiological response, possibly indicating increased engagement or arousal. The third panel illustrates the HR in [bpm] over time. Despite fluctuations, no distinct pattern of significant increase is observed, which could indicate substantial physical effort or stress. The fourth panel depicts the RR in breaths per minute [bpm] over time. Here, a noticeable decrease in RR is observed. This decline in RR may signify the onset of fatigue, possibly due to prolonged physical exertion during the rehabilitation session [42]. Lastly, the estimated $PP_{state}$ in terms of Arousal and Valence is reported. Here, it is worth observing that the proposed estimation approach is capable of computing per each time instant the PP condition of the user.

Fig. 7

Raw physiological signals collected during a robot-aided rehabilitation session along with the $PP_{state}$ estimated in terms of Arousal and Valence

The bar plot in Fig. 8 displays the $TP_{state}$ calculated for the eight participants across the ten robot-aided rehabilitation sessions. Each bar represents the mean value of $TP_{state}$ for the respective PP state, with error bars indicating the standard deviation. This plot provides a visual representation of how the time spent in each PP state varies across the rehabilitation sessions allowing for an assessment of trends or consistency in the distribution of time spent in different PP states throughout the sessions.

Fig. 8

Percentage of time spent in a specific PP state estimated during the ten monitored rehabilitation sessions. The coloured bars and the error bars represent the mean values and the standard deviation computed for the eight enrolled patients

Firstly, it is notable that the valence of the participants’ experience, as represented by the “Excited” and “Relaxed” states, has consistently been positive. This suggests that, overall, the participants felt positively engaged and relaxed during the rehabilitation sessions. This positive valence is an encouraging outcome in terms of patients’ acceptance of and satisfaction with the treatment. Additionally, there is no discernible temporal trend observed in the $TP_{state}$ of the PP states associated with negative valence, specifically “Stressed” and “Bored”. Throughout the rehabilitation treatment, instances of these negative emotions were infrequent and irregular, with their occurrences being minimal. This indicates that the participants rarely experienced extended periods of negative emotional states during the sessions.

The statistical analysis of the correlation between the $TP_{state}$ and the increasing number of sessions provides insights into the relationship between the patients’ states and their repeated interaction with the rehabilitation robot. Table 2 presents the correlation coefficients $\rho$ and the corresponding p-values resulting from the linear Pearson Correlation analysis. Statistically significant results (p-value$<0.05$) are reported in bold.

Table 2 Correlation coefficients between PP states and time

The correlation analysis revealed a strong negative correlation ($\rho = -0.73$) between the time spent in the “Excited” state and the number of completed rehabilitation sessions. This relationship was statistically significant with a p-value of 0.01. These results suggest that as patients progressed through more rehabilitation sessions, they tended to spend less time in the “Excited” state. This could indicate a potential decrease in the level of excitement or arousal experienced by patients as they became more familiar with the rehabilitation process or as their overall condition improved.

The “Relaxed” state demonstrated a notable positive correlation ($\rho = 0.70$) with the number of sessions completed, yielding a significant p-value of 0.02. This finding suggests a trend where, as the number of rehabilitation sessions increased, patients were more likely to spend increased amounts of time in the “Relaxed” state. This implies that the rehabilitation program may have had a calming and beneficial effect on the patients, leading to an increased sense of relaxation throughout the sessions.

The correlation analysis revealed interesting findings regarding the “Bored” and “Stressed” states. The “Bored” state showed a very weak positive correlation ($\rho = 0.13$) with the number of sessions completed, with a non-significant p-value of 0.71. This implies that there was no substantial relationship between the time spent in the “Bored” state and the number of completed sessions. Similarly, the “Stressed” state also demonstrated a very weak positive correlation ($\rho = 0.09$) with the number of sessions, with a non-significant p-value of 0.80. This suggests that the time spent in the “Stressed” state did not significantly vary as the number of sessions increased. These results indicate that neither the “Bored” nor the “Stressed” state showed a meaningful association with the progression of rehabilitation sessions.

In summary, the correlation analysis revealed distinctive patterns. Patients tended to spend less time in the “Excited” state as they completed more sessions, suggesting a potential decrease in arousal levels with increased familiarity or improved condition. Conversely, the “Relaxed” state showed a positive correlation with session completion, indicating a beneficial calming effect of the program.

These results suggest that the proposed estimation approach captures the modifications of the patient’s emotional states during the robot-aided rehabilitation sessions. This capability not only enables the quantification of the quality of human-robot interaction but also facilitates the assessment of the psychophysiological impact on individuals throughout the rehabilitation process. The estimation of patient perception regarding the use of the robot emerges as a valuable tool for initially evaluating how a specific technology is perceived as well as for studying the phenomena of adaptation to its use over time.

The correlation analysis presented in this study highlights the evolution of patients’ PP states throughout rehabilitation sessions, offering insights into their adaptation to both the technology and the rehabilitation process. In the initial stages of interacting with the technology, patients may display higher levels of excitement when encountering something new and potentially engaging. However, as indicated by the correlation analysis, this initial excitement tends to diminish in subsequent sessions, suggesting an adaptation or growing familiarity with the technology.

This adaptation could stem from various factors, such as an increased confidence in using the technology, a deeper understanding of its benefits, or simply becoming accustomed to the routine of the rehabilitation sessions. Similarly, the positive correlation observed between the “Relaxed” state and session completion suggests that patients may develop a sense of comfort and ease with both the technology and the rehabilitation environment over time.

This adaptation to the technology may indicate an enhanced understanding of how to interact with it effectively, resulting in a more relaxed and positive experience during sessions. In essence, the estimation of patient perception not only provides a snapshot of their emotional states during rehabilitation but also offers a dynamic view of how these states evolve and adapt over time. This knowledge is essential for designing user-centered technologies and optimizing rehabilitation programs to enhance the overall experience and outcomes for patients.

Although this study provides insights into psychophysiological responses during robotic upper limb rehabilitation, some limitations have to be acknowledged. The small sample size and non-normally distributed characteristics such as age, gender, and affected limb may limit the generalisability of the results and introduce potential sampling bias. A larger, more diverse sample is needed for future studies to improve robustness. In addition, this study focused exclusively on upper-limb robot-aided rehabilitation scenarios, and the obtained results may not be directly applicable or scalable to other types of rehabilitation robots. However, the proposed approach can indeed be applied to different systems. This study does not claim to provide generalizable results, but rather to provide evidence that certain adaptation phenomena may exist in patient-robot interaction. Furthermore, this method provides a way to quantify these phenomena, demonstrating its potential applicability in different rehabilitation contexts.

Conclusions

This paper introduces a psychophysiological estimation approach based on a Fuzzy Logic inference model to evaluate patients’ perception of robots during robot-aided rehabilitation sessions. Physiological parameters, including galvanic skin response, heart rate, and respiration rate, were monitored across ten out of forty robot-aided rehabilitation sessions. This data was used to construct an inference model capable of estimating the patients’ PP state. The percentage of time spent in a particular PP state was then calculated to succinctly represent the patient’s status within a single robot-aided rehabilitation session.

The results indicate that all patients had positive experiences during their interaction with the robot. Initially, as they began their rehabilitation treatment, there was a sense of curiosity and interest in robot-aided rehabilitation. However, as they spent more time with the robot, the initial excitement gradually decreased ($\rho =-0.73$ over time), giving way to a more “Relaxed” state ($\rho =0.70$ over time). This transition from excitement to relaxation suggests a positive adaptation to the robot and the rehabilitation process. Therefore, it is necessary to design rehabilitation treatments that evolve over time, ensuring continuous patient participation and engagement. The psychophysiological estimation approach presented in this study effectively captured all the changes in patient perception throughout the rehabilitation protocol. This model demonstrated its capability to track the evolving emotional states of the patients, from their initial curiosity and excitement to a more relaxed and comfortable state as they engaged with the robot over time.

Future developments will focus on enriching the parameters considered in the monitoring and estimation module to provide a more detailed description of the user state. For example, the phasic component of the GSR can be added to provide relevant information about the cognitive workload induced in users. In addition, future work will focus on applying this methodology to different robotic platforms and control strategies to assess their impact on patients over time. This will allow quantification of how different technologies and interaction methods affect the psychophysiological state of patients throughout the rehabilitation process. The development of technologies that are increasingly person-centered and tailored to the needs of the individual patient, also in terms of the perception of the technology, could indeed contribute to improving the effectiveness of the rehabilitation treatment itself.

Appendix A Fuzzy rules

This appendix includes the fuzzy rules implemented in the method described in Section 2.2 to transform the physiological measurements $X_r$ into the patient PP state.

• 1. If (SCL is “LOW”) then (Arousal is “LOW”)

• 2. If (SCL is “MID”) then (Arousal is “MID”)

• 3. If (SCL is “HIGH”) then (Arousal is “HIGH”)

• 4. If (HR is “HIGH”) then (Arousal is “HIGH”)

• 5. If (HR is “LOW”) then (Arousal is “LOW”)

• 6. If (SCL is “HIGH”) and (HR is “LOW”) then (Arousal is “MID-HIGH”)

• 7. If (SCL is “MID”) and (HR is “MID”) then (Arousal is “MID”)

• 8. If (RR is “HIGH”) then (Arousal is “MID-HIGH”)

• 9. If (RR is “LOW”) then (Arousal is “MID-LOW”)

• 10. If (SCL is “HIGH”) and (RR is “LOW”) then (Arousal is “MID-HIGH”)

• 11. If (SCL is “LOW”) and (RR is “HIGH”) then (Arousal is “MID-HIGH”)

• 12. If (SCL is “HIGH”) and (HR is “HIGH”) then (Valence is “LOW”)

• 13. If (SCL is “LOW”) and (HR is “HIGH”) then (Valence is “HIGH”)

• 14. If (SCL is “MID”) and (HR is “HIGH”) then (Valence is “MID-HIGH”)

• 15. If (SCL is “MID”) and (HR is “LOW”) then (Valence is “MID-LOW”)

• 16. If (SCL is “HIGH”) and (HR is “LOW”) then (Valence is “LOW”)

• 17. If (SCL is “LOW”) and (HR is “LOW”) then (Arousal is “LOW”) (Valence is “LOW”)