Skip to main content
Download PDF
Download ePub

Home-based guidance training system with interactive visual feedback using kinect on stroke survivors with moderate to severe motor impairment

Journal of NeuroEngineering and Rehabilitation volume 21, Article number: 189 (2024) Cite this article

Abstract

The home-based training approach benefits stroke survivors by providing them with an increased amount of training time and greater feasibility in terms of their training schedule, particularly for those with severe motor impairment. Computer-guided training systems provide visual feedback with correct movement patterns during home-based training. This study aimed to investigate the improvement in motor performance among stroke survivors with moderate to severe motor impairment after 800 min of training using a home-based guidance training system with interactive visual feedback. Twelve patients with moderate to severe stroke underwent home-based training, totaling 800 min (20–40 min per session, with a frequency of 3 sessions per week). The home-based guidance training system uses Kinect to reconstruct the 3D human body skeletal model and provides real-time motor feedback during training. The training exercises consisted of six core exercises and eleven optional exercises, including joint exercises, balance control, and coordination. Pre-training and post-training assessments were conducted using the Fugl-Meyer Assessment-Upper Limb (FMA-UE), Fugl-Meyer Assessment-Lower Limb (FMA-LE), Functional Ambulation Categories (FAC), Berg Balance Scale (BBS), Barthel Index (BI), Modified Ashworth Scale (MAS), as well as kinematic data of joint angles and center of mass (COM). The results indicated that motor training led to the attainment of the upper limit of functional range of motion (FROM) in hip abduction, shoulder flexion, and shoulder abduction. However, there was no improvement in the active range of motion (AROM) in the upper extremity (U/E) and lower extremity (L/E) joints, reaching the level of the older healthy population. Significant improvements were observed in both left/right and superior/inferior displacements, as well as body sway in the mediolateral axis of the COM, after 800 min of training. In conclusion, the home-based guidance system using Kinect aids in improving joint kinematics performance at the level of FROM and balance control, accompanied by increased mediolateral body sway of the COM for stroke survivors with moderate to severe stroke. Additionally, spasticity was reduced in both the upper and lower extremities after 800 min of home-based training.

Introduction

Existing home-based training systems use games as the primary trend to enhance the level of motivation [1,2,3,4,5]. In addition, the training efficiency, in terms of repetitions, may be lower when utilizing the home-based training system without feedback [6]. If the fun environment is enriched and the correct movement is guided by visual feedback during training exercises, this approach would be a feasible training approach. Cloud computing technology has advanced recently, cloud-based networks can easily connect the training from home to the center, and therapists can adjust the training protocol remotely and access the training data. Two markerless systems for motion tracking 3D depth sensor technique and RGB camera system. 3D depth-sensing cameras, employing technologies such as stereo vision, time of flight, or structured light, are now capable of identifying 3D body segments. Notable examples include Kinect [7,8,9,10,11], ZED [12, 13], Intel RealSense [14]. Another type of system using markerless’s AI-driven motion capture technology with RGB cameras (e.g., Theia3D) [15] to construct the 3D skeletal models for tracking joint movements and balance control. In this work, we raise the following questions: Is it possible to have an interactive guidance system for home-based users during training, and what is the effectiveness of motor recovery after 800 min of home-based training using computer-guided visual feedback?

The home-based training approach [16,17,18] provides training for chronic stroke and benefits stroke survivors with an increased amount of training time and more feasibility in the training schedule, especially in patients with severe levels. The depth sensor-based training systems (i.e., UINCARE Home + (UINCARE Corp., South Korea) [19, 20], MindMotion® GO (Switzerland) [21], EvolvRehab (Spain) [22], LongGood TeleRehabilitation System (Taiwan) [23]) provide the visual feedback with specific symbols or targets to guide the user in achieving the tasks. However, these training approaches display virtual objects in a video game or demonstration videos without providing feedback on the user’s real-time motions. In this study, we developed a home-based guidance training system for stroke survivors, enabling to provide the RGB-depth sensor to capture 25 artificial anatomical landmarks to reconstruct the body skeleton and real-time visual feedback on users' body segment movements and joint angles during training.

The purpose of this study was to investigate the improvement in motor performance for stroke survivors with moderate to severe motor impairment after 800 min of training using the home-based guidance training system with interactive visual feedback. With the technology of depth sensors, home-based training with computer-guided motion guidance in real-time may help users improve motor performance with minimal therapist assistance, which can greatly enhance the flexibility to facilitate their training time schedule.

Materials and methods

Subjects

Twelve patients with chronic stroke (age: 63.8 ± 12.8 years [range from 41 to 83]; poststroke duration: 41.8 ± 33.4 months [range from 12 to 131]; training duration: 5.82 ± 2.92 weeks [range from 2.14 to 12.86]; affected side: 5 left/7 right; type: 8 ischemia/4 hemorrhage) were recruited from The Hong Kong Stroke Association. All subjects were matched to the included criteria of (1) being diagnosed with ischemic brain injury or intracerebral hemorrhage shown by magnetic resonance imaging or computed tomography after stroke onset, and (2) at least in the moderate motor impairment in upper-limb and lower-limb (FMA≦79) [24]. In addition, all subjects provided informed consent, which was approved by the Joint Chinese University of Hong Kong—New Territories East Cluster Clinical Research Ethics Committee before the experiment. The participants were excluded if they could not follow the instructions or independently conduct the rehabilitation training using the home-based guidance system.

Home-based guidance system using kinect

A total of 25 artificial anatomical landmarks of the 3D skeletal model of the full human body, including body segments of the head, trunk, arm, forearm, hand, thigh, shank, and foot, were reconstructed using the body tracking software development kit (SDK) technology of Kinect for Windows SDK 2.0. This reconstruction was performed using a Microsoft Kinect V2 RGB-depth camera (sampling rate: 30 Hz, resolution: 512 pixels × 424 pixels, optimal distance of measurement: 0.5–4.5 m). The home-based guidance system was developed in C# programming with Microsoft Visual Studio (Visual Studio 2019, ©Microsoft Corporation, USA.). For each patient, the training exercise set consists of core exercises as well as personalized optional exercises. The core exercises of physical and occupational therapy included five exercises on shoulder flexion (i.e., arm elevation on the sagittal plane), shoulder abduction (i.e., arm elevation on the frontal plane), hip flexion (i.e., thigh elevation on the sagittal plane), hip abduction (i.e., thigh elevation on the frontal plane) and knee flexion (i.e., shank elevation on the sagittal plane), and one balance control exercise of lateral weight shift. The therapist would select optional exercises for personalized training based on patient’s motor function recovery. The optional exercises included eleven exercises, such as drawing a circle, sit-to-stand, trunk rotation, etc. (details are shown in Table 1). This home-based system using a Kinect camera provided real-time motor feedback by guiding interested joints or the center of mass (COM) to a specifically designed target during the training exercises (Fig. 1a and b). The training exercises were designed on the 2D plane, but using the 3D coordinates reconstructed with the 3D human body skeletal model. Therefore, human body movement tracking would not be limited to joint motion parallel to the camera, and the correct movement patterns, assisted by motor guidance, could be provided. After finishing motor training, the personalized kinematic performance, animation, and score were provided in the daily report, and the total training time and average training time in the latest week were provided in the weekly report. All the reports were provided to both stroke survivors and the therapist. The therapist could update and manage the personalized optional exercises based on the weekly report once patients completed every three training sessions. The animation provides a video replay of the 3D human skeletal model at a frame rate of 30 fps during training exercises. Furthermore, it supports mouse control for 3D views, enabling zoom-in, zoom-out, and rotation.

Table 1 The training exercise set consists of core and personalized optional exercises, and the therapists updated the personalized optional exercises based on the weekly report
Full size table
Fig. 1
figure 1

Illustration of real-time motor feedback by guiding interested joint/center of mass (white circle) to a specifically designed target (yellow circle) with corresponding kinematic performance in the training exercises of drawing a circle (Fig. 1a) and lunge (Fig. 1b) between pre- and post-training assessments using the home-based guidance system

Full size image

Experimental procedure

Before motor training, the computer-guided visual feedback home-based system was installed on a mini-PC (OptiPlex 7080 Micro, Dell Technologies) at the patient’s home. The Kinect V2 camera was placed in front of the user at 1 m when using the home-based guidance system. In the beginning, experienced trainers helped patients become familiar with the procedures of the training exercises of the home-based system by following the demonstration video.

The amount of time spent [25, 26] influences the effects of addressing the impaired motor performance of a limited range of motion (ROM) and balance control in stroke patients. A minimum therapy input of 16 h (960 min) within the first 6 months was recommended to facilitate improvement in stroke survivors' activities of daily living (ADL) through augmented exercise [27], and four other studies demonstrated enhanced motor performance in upper limb range of motion (ROM) [28, 29], lower limb joints [30], and balance [31] through the use of home-based training systems. The duration of these interventions ranged from 600 to 720 min. Furthermore, both 600 min (10 sessions, 60 min each) and 1200 min (20 sessions, 60 min each) of training significantly improved balance recovery and resulted in faster muscle response after stroke training [32]. Hence, all patients underwent training sessions with a total of 800 min (3 sessions/week) based on the above literature to evaluate motor performance after training using our home-based guidance training system, and the duration of each training session ranges from 20 to 40 min which was determined by therapists.

Clinical assessment for pre-training and post-training

All participants were arranged to test the clinical assessments for pre-training and post-training. The pre-training assessment was conducted before the first training within 2 weeks, and the post-training assessment was conducted within 1 week after finishing the last training session. In this study, the Fugl-Meyer Assessment (FMA), including Upper Limb (FMA-UE), Lower Limb (FMA-LE) and Total (FMA-Total), Functional Ambulation Categories (FAC), Berg Balance Scale (BBS), Barthel Index (BI) and Modified Ashworth Scale (MAS), was used to assess motor performance, motor skills, balance control, degree of independence and abnormal muscle tone in patients with poststroke hemiplegia. Therapists conducted and instructed all clinical assessments at the participant’s home.

Quantified evaluation of kinematic performance

Besides clinical tests, pre- and post-training training assessments were also evaluated for kinematic performance. The tested motions for the affected side included the joint motion activities of (A) hip abduction, (B) hip flexion, (C) knee flexion, (D) arm elevation on the sagittal plane, (E) arm elevation on the frontal plane, and balance control of (F) lateral weight shifting from the neutral position as much as possible. The patient was asked to execute all the tested motions in a standing position. The pre- and post-training assessments were also arranged to evaluate the motor performance in joint motion and balance control activities. The calculated variables, including active range of motion (AROM), displacement of the COM, fitted ellipse area (EA) with COM, and axis length of the ellipse, are described in detail as follows.

Kinematics evaluation in joint motion

The AROM of joint rotation angles was used to evaluate the joint kinematics in the joint motion activities between pre-training and post-training assessments. The joint rotation angles were obtained from the 3D human-body skeletal model reconstructed via the official Microsoft SDK Beta2 version (Microsoft® Visual Studio, Microsoft Corporation, USA.) [7]. In this study, the local coordinate system of the 3D skeletal model was determined with the positive x-axis directed to right, the positive y-axis directed superiorly, and the positive z-axis referred to posteriorly, described based on the spatial relation between the pose of the human body and the Kinect camera. The sagittal knee angle was calculated using in-plane angles between two adjacent vectors representing thigh/shank. The rotation angles in shoulder and hip joints were computed by using the Euler angle in z-x-y sequence, corresponding with flexion/extension, adduction/abduction, and internal/external, of the upper arm relative to the trunk and thigh relative to the pelvis, respectively. Compared to Vicon data, the accuracy of the mean value of root-mean-square error (RMSE) in joint angle measurement ranged from 3 to 17 degrees using the Kinect v2 camera [7].

Kinematics evaluation in balance control using COM

The body COM was calculated by weighting the 3D pose of 25 joints in the human-body skeletal model with Dempster’s coefficients, which provided an accurate body COM with a mean RMSE of 4.38 mm and mean Pearson correlation coefficient (CORR) of 0.94 comparable to that of the 3D infrared motion capture system [8]. This study used COM movements and human-body sway of the COM to quantify the balance control performance. The corresponding variables included right/left (R/L), superior/inferior (S/I), and posterior/anterior (P/A) displacements of the COM, major and minor axis lengths of the fitted ellipse with the COM, and fitted ellipse area (EA) with the COM. In addition, the fitted ellipse in the anatomical plane contains the COM trajectories with 95% confidence [33, 34].

Statistical analysis

At first, Shapiro–Wilk test was used to test the normality in the baseline of FMA clinical assessments. Paired t-tests were used to test the differences in outcomes of FMA-UE, FMA-LE, FMA-Total, FAC, BBS, BI, and calculated variables of AROM, L/R, S/I and P/A displacements of COM, EA, and axis length of ellipse between pre- and post-training assessments. The nonparametric Wilcoxon signed-rank test was used to examine the differences in MAS between pre- and post-training assessments. The score of MAS 1+ ranges from MAS 0 to MAS 1, and MAS 1+ was substituted by a scale of 1.5 [35]. All comparisons were performed using SPSS 20.0 (SPSS, IBM, Armonk, New York, USA.). The significance level was set at α = 0.05. In addition, the power of clinical assessments for sample sizes of twelve patients between pre-training and post-training was evaluated by post hoc power analyses using GPOWER.

Results

The p-values of normality tests in baseline motor performance of FMA-UE, FMA-LE, and FMA-Total were 0.513, 0.096, and 0.386, respectively. These results indicated that the data set of 12 stroke survivors showed no significant differences from the normal distribution. The post-training results, after 800 min of training at home, showed improvements in upper limb functions (FMA-U/E, AROM of shoulder joint), lower limb functions (FMA-L/E, AROM of hip and knee joints), as well as balance control performance (COM). In the clinical assessments, the mean (SD) of outcomes in FMA-U/E, FMA-L/E, FMA-Total, FAC, BBS and BI between pre- and post-assessment were 31.00 (12.77) and 43.58 (15.36), 18.08 (5.68) and 24.08 (5.6), 49.08 (17.2) and 67.67 (19.52), 4.92 (0.51) and 5.5 (0.8), 43.25 (10.56) and 46.0 (7.3), and 15.83 (2.95) and 17.17 (2.98), respectively (Table 2). The outcomes of clinical assessments showed significantly increased scores in post-assessment compared with pre-assessment, except for BBS (Table 2). Moreover, the effect size and power results in clinical assessments between pre-training and post-training are shown in Table 1. Most results showed a large effect size in the sample sizes of twelve with α levels of 0.05. Compared with the pre-assessment and post-assessment using the MAS, the results demonstrated significant decreases in spasticity scores for the upper extremities, lower extremities, and the total score (Table 3).

Table 2 Means (standard deviations) of the outcomes for patients with poststroke hemiplegia between pre- and post-training assessments after 800 min training in Fugl-Meyer Assessment of Upper Limb (FMA-UE), Fugl-Meyer Assessment of Lower Limb (FMA-LE), Fugl-Meyer Assessment in total (FMA-Total), Functional Ambulation Categories (FAC), Berg Balance Scale (BBS) and Barthel Index (BI)
Full size table
Table 3 The outcomes of the Modified Ashworth Scale (MAS) between pre- and post-training assessments, including flexor and extensor muscle tone for hip, knee, ankle, elbow, and wrist joints
Full size table

For the kinematic performance in joint motion, the mean (SD) AROM of the rotation angle in hip abduction, hip flexion, knee flexion, shoulder flexion and shoulder abduction between pre- and post-training assessments were 17.79 (6.72°) and 28.63 (6.9°), 37.83 (17.09°) and 64.23 (10.33°), 51.21 (28.2°) and 74.47 (31.14°), 103.68 (24.99°) and 133.98 (19.03°), 108.36 (21.65°) and 125.25 (18.58°), respectively (Fig. 2). The AROM of rotation angles for all joint motion exercises were significantly greater in post-training than in pre-training (Fig. 2). In addition, AROM in hip abduction, shoulder flexion and shoulder abduction after motor training achieved the upper limit of functional ROM (FROM), while no AROM in U/E and L/E joints achieved the level of the older healthy population (Fig. 2).

Fig. 2
figure 2

Mean and standard deviations between pre- (blue bar) and post-training assessments (red bar) of rotation angle in active range of motion (AROM) for hip abduction, hip flexion, knee flexion, shoulder flexion and shoulder abduction. The black dashed line indicated the AROM in the population of older healthy individuals [40, 41], and the green line showed the upper limit functional ROM for the performance of all activities of daily living (ADLs) [42,43,44,45,46,47,48,49,50,51,52,53,54,55]. The asterisk indicated the significant difference with α = 0.05

Full size image

For the kinematic performance in balance control, the mean (SD) of R/L, S/I and P/A displacement of COM in lateral weight shifting between pre- and post-training assessments were 107.18 (54.0 mm) and 166.47 (59.84 mm), 17.16 (8.77 mm) and 25.71 (11.01 mm), 27.2 (11.32 mm) and 29.25 (8.09 mm), respectively (Fig. 3a). Both R/L and S/I displacements of COM were significantly greater in post-assessment than in pre-assessment, while no significant difference was found in the P/A component (Fig. 3a). The corresponding parameters of the major and minor axis lengths of the fitted ellipse with COM in the R/L and S/I components and fitted EA were 189.31 (122.59 mm) and 283.25 (94.06 mm), 21.62 (14.07 mm) and 28.72 (15.71 mm), and 3937.09 (4149.07 mm2) and 6405.84 (3662.84 mm2), respectively (Fig. 3b). The major axis length was significantly increased in post-training assessment compared with the pre-training assessment, while no significant differences were found in the EA or minor axis length (Fig. 3b). A typical subject showed the major (pre: 250.96 mm, post: 292.00 mm), minor axis length (pre: 23.56 mm, post: 27.56 mm) and EA (pre: 4643.52 mm2, post: 6320.74 mm2) between pre- and post-assessment during 1 motion cycle of balance control activity (Fig. 3c). In addition, the mean (SD) of the inclination angle (intersection angle between the major axis and the horizontal axis) of the fitted ellipse between pre- and post-assessment were 0.32 (5.48°) and 3.35 (4.57°), and no significant difference was found in this comparison.

Fig. 3
figure 3

Mean and standard deviations between pre- (blue bar) and post-training assessments (red bar) of right/left (R/L), superior/inferior (S/I) and posterior/anterior (P/A) displacement of center of mass (COM) (Fig. 3a), and parameters of axis length and ellipse area (EA) obtained by fitting the trajectories of COM in R/L and S/I components with 95% confidence (Fig. 3b) during balance control task of lateral weight shifting. The asterisk indicated the significant difference with α = 0.05. In addition, an illustration of body sway with ellipses based on COM trajectories in R/L and S/I components for evaluating balance control performance. A typical subject showed the parameters of major (pre: 250.96 mm, post: 292.00 mm), minor axis length (pre: 23.56 mm, post: 27.56 mm) and EA based on COM with 95% confidence (pre: 4643.52 mm2, post: 6320.74 mm2) between pre- (blue) and post-training assessments (red) during 1 motion cycle of lateral weight shifting (Fig. 3c)

Full size image

Discussion

Compared to pre-training, significantly increased AROM in shoulder, hip and knee joints were shown in post-training assessment after training using the home-based guidance system. In this study, the improvement in mean AROM between pre- and post-training for the shoulder joint exercises of arm elevation on the sagittal plane (i.e., shoulder flexion) and arm elevation on the frontal plane (i.e., shoulder abduction) were 30.3° and 16.89°. The corresponding AROM values in the L/E joint exercises of hip abduction, hip flexion, and knee flexion were 20.59°, 26.4°, and 23.26°, respectively (Fig. 2). For U/E kinematic performance, previous studies found a median AROM improvement of 20° in the shoulder joint for both shoulder abduction and shoulder flexion after receiving intervention with physical therapy (PT) and VR game training using a home-based training system [56]. These patients were arranged to receive the combination intervention of 20 sessions of PT and 20 sessions of Kinect-based VR game training, and the training frequency for PT and Kinect-based VR game training was 1 h a day with a duration of 4 weeks and 1200 min in total [56]. Another study demonstrated mean changes of 5.4° and 17.2° in AROM for shoulder abduction and shoulder flexion, respectively, after receiving 10 training sessions with 45 min/day duration and 450 min in total [57]. These studies utilized video game-based interventions, such as the KineLabs software platform [56] or exergames [57], as motor training protocols in home-based systems. For L/E kinematic performance, a previous study found that the improvement in median AROM for hip abduction, hip flexion, and knee flexion was 5°, 5° and 10°, respectively, after receiving game-based therapy with 10 sessions of training with 27–37 min/day duration (270–370 min in total) [58]. Compared to game-based therapy, it appeared that our therapist-prescribed guidance system for home-based training demonstrated the potential advantages of motor improvement in L/E and U/E joints. Moreover, the results of MAS indicated that spasticity showed a reduction in all subjects with one or more spastic muscles after training (Table 3).

These results are likely due to several factors related to the design of the therapist-prescribed guidance system for home-based training. First, the amount of training spent, including minutes provided, frequency and duration, are the vital factors influencing the effect on motor improvement for stroke survivors using the home-based guidance system [27]. This study utilized the amount of training spent of 800 min with 20–40 min/session and frequency of 3 sessions/week within 3.25 months. This choice was based on previous studies on stroke training, specifically the augmented-based systematic review [27] as well as research on virtual reality training using home-based systems [28,29,30,31,32]. Also, more intensive motor rehabilitation was recommended for stroke survivors to show better motor recovery within the subacute phase [59] (3–11 weeks after stroke) [60, 61]. Second, the interactive visual feedback with kinematics data (i.e., joint center or center of mass) in real-time facilitates patients to perform exercise training correctly with improved motor performance with the assistance of motor guidance [62,63,64]. The type of training exercise differs between home-based therapy with therapist-prescribed exercises and game-based therapy. In clinical settings, patients trust the advice of professional therapists and follow the training protocols provided by therapists. In our home-based guidance system, we incorporate seventeen therapist-prescribed training exercises, including joint exercises, balance control, and coordination. Nevertheless, game-based therapy with an enriched and fun environment could also improve balance, motor performance and level of motivation in patients with stroke [65,66,67].

The analysis of COM movement has been commonly considered an essential variable to reveal the balance control mechanism underlying body stability during dynamic tasks in patients with motor dysfunction [68, 69], mobility recovery after stroke [70], people with deficits in movement [71] and the risk of falls in the elderly [72]. The human-body sway based on COM movements was also used to evaluate the balance control using Kinect as the clinical assessment [8, 73]. In addition, the fitted ellipse was used to describe the body sway based on COM trajectories in this study to qualify the displacement along the anatomical plane. The current results showed that significantly increased values were found in R/L and S/I displacements of COM as well as major axis length of the fitted ellipse with COM in R/L and S/I components during balance control activity of lateral weight shifting after motor training using the home-based guidance rehabilitation system. In this study, the improvements in mean displacements in R/L and S/I of COM between pre- and post-training assessments were 59.29 mm and 8.55 mm, respectively, and the corresponding parameter of major axis length was 93.94 mm (Fig. 3a and b). These findings indicated that therapist-prescribed guidance system for home-based training helped patients with stroke improve their balance control ability with increased COM displacements in the mediolateral and superoinferior directions and increased mediolateral body sway of the COM after motor training.

Compared to MDC (FMA-LE: 3.57 [37])/MCID (FMA-UE: 4.25 [36], FMA-LE: 6 [38]), the mean differences in FMA-UE, FMA-LE between pre-training and post-training were 12.58 and 6, which showing the greater or equal values. Moreover, mean differences in hip and knee AROM on the sagittal plane were 26.4° and 23.26°, showing greater improvement than minimal clinically important differences (MCID) of 5.81° [74] and 8.48° [75], respectively. By comparisons of the previous studies in the upper limit of the U/E and L/E rotation angles for the performance of ADL [42,43,44,45,46,47,48,49,50,51,52,53,54,55], the current results showed that patients with stroke achieved the upper limit of FROM level in hip abduction, shoulder flexion (i.e., arm elevation on the sagittal plane) and shoulder abduction (i.e., arm elevation on the frontal plane) after 800 min motor training by using the home-based guidance rehabilitation system with Kinect (Fig. 2). Nonetheless, the potential improvement in U/E and L/E joint kinematics between home-based training and the reference from the population in older healthy individuals [40, 41] still existed. Based on the baseline FMA-Total scores in pre-training assessment (Table 2), the stroke level of all the patients in this study was defined as ranging from moderate to severe (FMA-Total scores: 0–79) [24]. With the implicit outcomes of clinical assessments and kinematics evaluation, the home-based guidance system was proven to positively impact the improvement of motor performance in patients with moderate to severe stroke.

The current study aimed to investigate the improvement of kinematic performance for patients with stroke after motor training using the home-based training system with interactive motion guidance. The results showed that a home-based guidance rehabilitation system helps patients with moderate to severe stroke improve kinematic performance, reaching the level of MDC/MCID. Additionally, it improved FROM performance and increased body sway of the COM after training. These results may be further confirmed in the population of mild stroke patients after motor training using the home-based guidance rehabilitation system. The number of participants is relatively small. Further studies could recruit more subjects and be confirmed by randomized controlled trials (RCT) to provide more convincing clinical results. Table 2 showed that most patients showed mild improvement and clinically significant improvement in clinical assessments. More importantly, significant improvements were observed on the upper limb Fugl-Meyer Assessment (FMA) scores, with 91.7% showing clinically significant improvement and 83.4% showing clinically significant improvement on the FMA-Total. In addition, all patients exhibited an improvement (change > 0) in the active range of motion (AROM) of U/E and L/E joints. BBS did not exhibit significant differences in statistical analysis, with 75.1% showing improvement and 16.6% showing mild degeneration. Only 2 patients showed a level of mild degeneration in BBS, and less than 35% of patients exhibited no improvement in FAC, BBS and BI. It would be interesting to conduct further studies with a larger sample size, which could potentially enhance the training efficiency by incorporating personalized training protocols. In this study, home-based guidance training positively affected motor improvement after 800 min of training. To further investigate the effects of different training patterns, including home-based training, rehabilitation center-based training, and a hybrid combination of home-based and rehabilitation center-based training, developing stroke rehabilitation programs that incorporate guidance training systems at home and telemedicine would be beneficial.

Conclusions

The guidance system for home-based users using Kinect provided interactive visual feedback and significantly improved clinical data on joint range of motion and balance control performance. The system had embedded six training exercises, and therapists could personalize the training program for each individual for 2 months of training with 800 min. The results showed that this Kinect-based home-based system helps patients with moderate to severe stroke improve joint kinematic performance at the level of FROM and balance control with increased mediolateral body sway of the COM. In addition, spasticity was reduced in both the upper and lower extremities after 800 min of home-based training.

Availability of data and materials

All data generated or analysed during this study are included in this published article [and its supplementary materials]. Information on this clinical trial can be found at https://clinicaltrials.gov/study/NCT04638218.

References

  1. Kamnardsiri T, et al. An interactive physical-cognitive game-based training system using Kinect for older adults: development and usability study. JMIR Serious Games. 2021;9(4):e27848.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Zhang M, et al. Recent developments in game-based virtual reality educational laboratories using the microsoft kinect. Int J Emerg Technol Learn (iJET). 2018;13(1):138–59.

    Article  Google Scholar 

  3. Ballas A et al. Game-based system for learning labanotation using Microsoft Kinect. In: 2017 IEEE 6th global conference on consumer electronics (GCCE). 2017. IEEE.

  4. Bonnechère B, et al. The use of commercial video games in rehabilitation: a systematic review. Int J Rehabil Res. 2016;39(4):277–90.

    Article  PubMed  Google Scholar 

  5. Levin MF, Weiss PL, Keshner EA. Emergence of virtual reality as a tool for upper limb rehabilitation: incorporation of motor control and motor learning principles. Phys Ther. 2015;95(3):415–25.

    Article  PubMed  Google Scholar 

  6. Cameirão MS, et al. Coaching or gaming? Implications of strategy choice for home based stroke rehabilitation. J Neuroeng Rehabil. 2016;13:1–15.

    Article  Google Scholar 

  7. Yeung LF, et al. Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2. Gait Posture. 2021;87:19–26.

    Article  PubMed  Google Scholar 

  8. Yeung L, et al. Evaluation of the Microsoft Kinect as a clinical assessment tool of body sway. Gait Posture. 2014;40(4):532–8.

    Article  CAS  PubMed  Google Scholar 

  9. Fern'ndez-Baena A, Susin A, Lligadas X. Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments. In: 2012 fourth international conference on intelligent networking and collaborative systems. 2012. IEEE.

  10. Clark RA, et al. Validity of the microsoft kinect for assessment of postural control. Gait Posture. 2012;36(3):372–7.

    Article  PubMed  Google Scholar 

  11. Bonnechere B, et al. Validity and reliability of the Kinect within functional assessment activities: comparison with standard stereophotogrammetry. Gait Posture. 2014;39(1):593–8.

    Article  CAS  PubMed  Google Scholar 

  12. Sade S, et al. Development of an elliptical perturbation system that provides unexpected perturbations during elliptical walking (the EPES system). J Neuroeng Rehabil. 2023;20(1):125.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Nucita A, et al. Telerehabilitation with computer vision-assisted markerless measures: a pilot study with rett syndrome patients. Electronics. 2023;12(2):435.

    Article  Google Scholar 

  14. Siena FL, et al. Utilising the intel realsense camera for measuring health outcomes in clinical research. J Med Syst. 2018;42:1–10.

    Article  Google Scholar 

  15. Wren TA, Isakov P, Rethlefsen SA. Comparison of kinematics between theia markerless and conventional marker-based gait analysis in clinical patients. Gait Posture. 2023;104:9–14.

    Article  PubMed  Google Scholar 

  16. Triandafilou KM, et al. Development of a 3D, networked multi-user virtual reality environment for home therapy after stroke. J Neuroeng Rehabil. 2018;15:1–13.

    Article  Google Scholar 

  17. Wittmann F, et al. Self-directed arm therapy at home after stroke with a sensor-based virtual reality training system. J Neuroeng Rehabil. 2016;13:1–10.

    Article  Google Scholar 

  18. Bai J, Song A. Development of a novel home based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients. IEEE Access. 2019;7:9667–77.

    Article  Google Scholar 

  19. Lee JI, et al. Effects of the home-based exercise program with an augmented reality system on balance in patients with stroke: a randomized controlled trial. Disabil Rehabil. 2023;45(10):1705–12.

    Article  PubMed  Google Scholar 

  20. Shim GY, et al. A randomized controlled trial of postoperative rehabilitation using digital healthcare system after rotator cuff repair. NPJ Dig Med. 2023;6(1):95.

    Article  Google Scholar 

  21. Wiskerke E, et al. Determining the optimal virtual reality exergame approach for balance therapy in persons with neurological disorders using a rasch analysis: longitudinal observational study. JMIR Serious Games. 2022;10(1):e30366.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Ellis F, et al. Consideration-of-concept of EvolvRehab-Body for upper limb virtual rehabilitation at home for people late after stroke. Physiotherapy. 2022;116:97–107.

    Article  CAS  PubMed  Google Scholar 

  23. Liu X-T, et al. Feasibility of a kinect-based system in assessing physical function of the elderly for home-based care. BMC Geriatr. 2023;23(1):495.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Duncan PW, et al. Similar motor recovery of upper and lower extremities after stroke. Stroke. 1994;25(6):1181–8.

    Article  CAS  PubMed  Google Scholar 

  25. Clark B et al. The effect of time spent in rehabilitation on activity limitation and impairment after stroke. Cochrane Database Syst Rev. 2021(10).

  26. Cooke EV, et al. The effects of increased dose of exercise-based therapies to enhance motor recovery after stroke: a systematic review and meta-analysis. BMC Med. 2010;8:1–13.

    Article  Google Scholar 

  27. Kwakkel G, et al. Effects of augmented exercise therapy time after stroke: a meta-analysis. Stroke. 2004;35(11):2529–39.

    Article  PubMed  Google Scholar 

  28. Kwon J-S, et al. Effects of virtual reality on upper extremity function and activities of daily living performance in acute stroke: a double-blind randomized clinical trial. NeuroRehabilitation. 2012;31(4):379–85.

    Article  PubMed  Google Scholar 

  29. In TS, et al. Virtual reality reflection therapy improves motor recovery and motor function in the upper extremities of people with chronic stroke. J Phys Ther Sci. 2012;24(4):339–43.

    Article  Google Scholar 

  30. Mirelman A, et al. Effects of virtual reality training on gait biomechanics of individuals post-stroke. Gait Posture. 2010;31(4):433–7.

    Article  PubMed  Google Scholar 

  31. Cikajlo I, et al. Telerehabilitation using virtual reality task can improve balance in patients with stroke. Disabil Rehabil. 2012;34(1):13–8.

    Article  PubMed  Google Scholar 

  32. Junata M, et al. Kinect-based rapid movement training to improve balance recovery for stroke fall prevention: a randomized controlled trial. J Neuroeng Rehabil. 2021;18:1–12.

    Article  Google Scholar 

  33. Oliveira L, Simpson D, Nadal J. Calculation of area of stabilometric signals using principal component analysis. Physiol Meas. 1996;17(4):305.

    Article  CAS  PubMed  Google Scholar 

  34. Schubert P, Kirchner M. Ellipse area calculations and their applicability in posturography. Gait Posture. 2014;39(1):518–22.

    Article  PubMed  Google Scholar 

  35. Pandyan AD, et al. A review of the properties and limitations of the Ashworth and modified Ashworth Scales as measures of spasticity. Clin Rehabil. 1999;13(5):373–83.

    Article  CAS  PubMed  Google Scholar 

  36. Page SJ, Fulk GD, Boyne P. Clinically important differences for the upper-extremity Fugl-Meyer Scale in people with minimal to moderate impairment due to chronic stroke. Phys Ther. 2012;92(6):791–8.

    Article  PubMed  Google Scholar 

  37. Hiengkaew V, Jitaree K, Chaiyawat P. Minimal detectable changes of the Berg Balance Scale, Fugl-Meyer Assessment Scale, Timed “Up & Go” Test, gait speeds, and 2-minute walk test in individuals with chronic stroke with different degrees of ankle plantarflexor tone. Arch Phys Med Rehabil. 2012;93(7):1201–8.

    Article  PubMed  Google Scholar 

  38. Pandian S, Arya KN, Kumar D. Minimal clinically important difference of the lower-extremity fugl–meyer assessment in chronic-stroke. Top Stroke Rehabil. 2016;23(4):233–9.

    Article  PubMed  Google Scholar 

  39. Park C-S. The test-retest reliability and minimal detectable change of the short-form Barthel Index (5 items) and its associations with chronic stroke-specific impairments. J Phys Ther Sci. 2018;30(6):835–9.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Roach KE, Miles TP. Normal hip and knee active range of motion: the relationship to age. Phys Ther. 1991;71(9):656–65.

    Article  CAS  PubMed  Google Scholar 

  41. McIntosh L, McKenna K, Gustafsson L. Active and passive shoulder range of motion in healthy older people. Br J Occup Ther. 2003;66(7):318–24.

    Article  Google Scholar 

  42. Johnston RC, Smidt GL. 23 Hip motion measurements for selected activities of daily living. Clin Orthop Relat Res®. 1970;72:205–15.

    CAS  PubMed  Google Scholar 

  43. Korp K, Richard R, Hawkins D. Refining the idiom “functional range of motion” related to burn recovery. J Burn Care Res. 2015;36(3):e136–45.

    Article  PubMed  Google Scholar 

  44. Aizawa J, et al. Three-dimensional motion of the upper extremity joints during various activities of daily living. J Biomech. 2010;43(15):2915–22.

    Article  PubMed  Google Scholar 

  45. Artilheiro MC, et al. Three-dimensional analysis of performance of an upper limb functional task among adults with dyskinetic cerebral palsy. Gait Posture. 2014;39(3):875–81.

    Article  PubMed  Google Scholar 

  46. Carey SL, et al. Compensatory movements of transradial prosthesis users during common tasks. Clin Biomech. 2008;23(9):1128–35.

    Article  Google Scholar 

  47. Klotz MC, et al. Motion analysis of the upper extremity in children with unilateral cerebral palsy—an assessment of six daily tasks. Res Dev Disabil. 2014;35(11):2950–7.

    Article  PubMed  Google Scholar 

  48. Lee JH, et al. A comparison of young and old using three-dimensional motion analyses of gait, sit-to-stand and upper extremity performance. Aging Clin Exp Res. 2007;19:451–6.

    Article  PubMed  Google Scholar 

  49. Masjedi M, Lovell C, Johnson GR. Comparison of range of motion and function of subjects with reverse anatomy Bayley-Walker shoulder replacement with those of normal subjects. Hum Mov Sci. 2011;30(6):1062–71.

    Article  PubMed  Google Scholar 

  50. Namdari S, et al. Defining functional shoulder range of motion for activities of daily living. J Shoulder Elbow Surg. 2012;21(9):1177–83.

    Article  PubMed  Google Scholar 

  51. Raiss P, et al. Range of motion of shoulder and elbow in activities of daily life in 3D motion analysis. Zeitschrift fur Orthopadie und Unfallchirurgie. 2007;145(4):493–8.

    Article  CAS  PubMed  Google Scholar 

  52. Kasten P, et al. Three-dimensional motion analysis of compensatory movements in patients with radioulnar synostosis performing activities of daily living. J Orthop Sci. 2009;14:307–12.

    Article  PubMed  Google Scholar 

  53. Raiss P, et al. 3D motion analysis of the upper extremities and its application in shoulder arthroplasty. Obere Extremität. 2010;5:27–36.

    Google Scholar 

  54. Sheikhzadeh A, et al. Three-dimensional motion of the scapula and shoulder during activities of daily living. J Shoulder Elbow Surg. 2008;17(6):936–42.

    Article  PubMed  Google Scholar 

  55. Palmieri TL, et al. Alterations in functional movement after axillary burn scar contracture: a motion analysis study. J Burn Care Rehabil. 2003;24(2):104–8.

    Article  PubMed  Google Scholar 

  56. Aşkın A, et al. Effects of Kinect-based virtual reality game training on upper extremity motor recovery in chronic stroke. Somatosens Mot Res. 2018;35(1):25–32.

    Article  PubMed  Google Scholar 

  57. Adomavičienė A, et al. Influence of new technologies on post-stroke rehabilitation: a comparison of armeo spring to the kinect system. Medicina. 2019;55(4):98.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Miclaus RS, et al. Lower extremity rehabilitation in patients with post-stroke sequelae through virtual reality associated with mirror therapy. Int J Environ Res Public Health. 2021;18(5):2654.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Dromerick AW, et al. Critical period after stroke study (CPASS): a phase II clinical trial testing an optimal time for motor recovery after stroke in humans. Proc Natl Acad Sci. 2021;118(39):e2026676118.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Cramer SC. Repairing the human brain after stroke: I. Mechanisms of spontaneous recovery. Ann Neurol. 2008;63(3):272–87.

    Article  PubMed  Google Scholar 

  61. Rehme AK, et al. Activation likelihood estimation meta-analysis of motor-related neural activity after stroke. Neuroimage. 2012;59(3):2771–82.

    Article  PubMed  Google Scholar 

  62. Lee MH et al. Towards personalized interaction and corrective feedback of a socially assistive robot for post-stroke rehabilitation therapy. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 2020. IEEE.

  63. Zendehbad SA et al. Presenting a new muscle synergy analysis based mechanism to design a trackable visual biofeedback signal: applicable to arm movement recovery after ischemic stroke. In: IEEE Access, 2023.

  64. Weiss PL, et al. Video capture virtual reality as a flexible and effective rehabilitation tool. J Neuroeng Rehabil. 2004;1:1–12.

    Article  Google Scholar 

  65. Johar MN, Nordin NAM, Aziz AFA. The effect of game-based in comparison to conventional circuit exercise on functions, motivation level, self-efficacy and quality of life among stroke survivors. Medicine. 2022;101(2):e28580.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Cano-Mañas MJ, et al. Effects of video-game based therapy on balance, postural control, functionality, and quality of life of patients with subacute stroke: a randomized controlled trial. J Healthc Eng. 2020;2020:5480315.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Wang L, et al. Game-based virtual reality system for upper limb rehabilitation after stroke in a clinical environment: systematic review and meta-analysis. Games Health J. 2022;11(5):277–97.

    Article  PubMed  Google Scholar 

  68. Verrel J, Lövdén M, Lindenberger U. Motor-equivalent covariation stabilizes step parameters and center of mass position during treadmill walking. Exp Brain Res. 2010;207:13–26.

    Article  PubMed  Google Scholar 

  69. Black DP, et al. Uncontrolled manifold analysis of segmental angle variability during walking: preadolescents with and without Down syndrome. Exp Brain Res. 2007;183:511–21.

    Article  PubMed  Google Scholar 

  70. Kao P-C, et al. Dynamic instability during post-stroke hemiparetic walking. Gait Posture. 2014;40(3):457–63.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Grabiner MD, et al. Trunk kinematics and fall risk of older adults: translating biomechanical results to the clinic. J Electromyogr Kinesiol. 2008;18(2):197–204.

    Article  PubMed  Google Scholar 

  72. Lee H-J, Chou L-S. Detection of gait instability using the center of mass and center of pressure inclination angles. Arch Phys Med Rehabil. 2006;87(4):569–75.

    Article  PubMed  Google Scholar 

  73. Mazumder O et al. Posturography stability score generation for stroke patient using kinect: fuzzy based approach. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2017. IEEE.

  74. Guzik A, et al. Establishing the minimal clinically important differences for sagittal hip range of motion in chronic stroke patients. Front Neurol. 2021;12:700190.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Guzik A, et al. Estimating minimal clinically important differences for knee range of motion after stroke. J Clin Med. 2020;9(10):3305.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors are grateful for the support from participants, therapists and administrative staff in the Christian Family Service Centre, The Hong Kong Society for the Aged, Haven of Hope Christian Service, and The Spastics Association of Hong Kong. Thank you for all the research assistants and student helpers of CUHK that make this project possible.

Funding

This study was supported by the Social Innovation and Entrepreneurship Development Fund (Project Code: KPF22GWP34).

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong, SAR, China

    Hsuan-Yu Lu, Xiaoyi Wang, Chengpeng Hu, Cathy Choi-Yin Lau & Raymond Kai-Yu Tong

Authors
  1. Hsuan-Yu Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaoyi Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chengpeng Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Cathy Choi-Yin Lau
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Raymond Kai-Yu Tong
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Conceptualization: H.-Y .L. and K.-Y. T. and C.-Y. L.; Methodology: H.-Y. L. and K.-Y. T. and C.-Y. L.; Data curation: H.-Y. L. and X.-Y. W. and C.-P. H. and C.-Y. L.; Formal analysis and investigation: H.-Y. L. and X.-Y. W. and C.-P. H.; Writing—original draft preparation: H.-Y. L. and K.-Y. T.; Writing—review and editing: H.-Y. L. and K.-Y. T.; Funding acquisition: K.-Y. T.; Resources: K.-Y. T.; Supervision: K.-Y. T. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Raymond Kai-Yu Tong.

Ethics declarations

Ethics approval and consent to participate

This study was approved by The Joint Chinese University of Hong Kong-New Territories East Cluster Clinical Research Ethics Committee (CREC Ref. No.:2019.377-T and 2020.538-T). This trial was designed following the principle of the Declaration of Helsinki. All recruited subjects read and signed consent form before participation.

Consent for publication

The person in Fig. 1a and b has consented to the publication of the photograph.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, HY., Wang, X., Hu, C. et al. Home-based guidance training system with interactive visual feedback using kinect on stroke survivors with moderate to severe motor impairment. J NeuroEngineering Rehabil 21, 189 (2024). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12984-024-01479-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12984-024-01479-7

Keywords

Journal of NeuroEngineering and Rehabilitation

ISSN: 1743-0003

Contact us