Human factors considerations of Interaction between wearers and intelligent lower-limb prostheses: a prospective discussion

Abstract

Compared to traditional lower-limb prostheses (LLPs), intelligent LLPs are more versatile devices with emerging technologies, such as microcontrollers and user-controlled interfaces (UCIs). As emerging technologies allow a higher level of automation and more involvement from wearers in the LLP setting adjustments, the previous framework established to study human factors elements that affect wearer-LLP interaction may not be sufficient to understand the new elements (e.g., transparency) and dynamics in this interaction. In addition, the increased complexity of interaction amplifies the limitations of the traditional evaluation approaches of wearer-LLP interaction. Therefore, to ensure wearer acceptance and adoption, from a human factors perspective, we propose a new framework to introduce elements and usability requirements for the wearer-LLP interaction. This paper organizes human factors elements that appear with the development of intelligent LLP technologies into three aspects: wearer, device, and task by using a classic model of the human-machine systems. By adopting Nielsen’s five usability requirements, we introduce learnability, efficiency, memorability, use error, and satisfaction into the evaluation of wearer-LLP interaction. We identify two types of wearer-LLP interaction. The first type, direct interaction, occurs when the wearer continuously interacts with the intelligent LLP (primarily when the LLP is in action); the second type, indirect interaction, occurs when the wearer initiates communication with the LLP usually through a UCI to address the current or foreseeable challenges. For each type of interaction, we highlight new elements, such as device transparency and prior knowledge of the wearer with the UCI. In addition, we redefine the usability goals of two types of wearer-LLP interaction with Nelson’s five usability requirements and review methods to evaluate the interaction. Researchers and designers for intelligent LLPs should consider the new device elements that may additionally influence wearers’ acceptance and the need to interpret findings within the constraints of the specific wearer and task characteristics. The proposed framework can also be used to organize literature and identify gaps for future directions. By adopting the holistic usability requirements, findings across empirical studies can be more comparable. At the end of this paper, we discuss research trends and future directions in the human factors design of intelligent LLPs.

Background

Amputation can occur due to injuries but more often due to complications of vascular disease such as diabetes mellitus [1]. In the United States, approximately 1.6 million people were living with a limb loss in 2005, and the number is expected to double by 2050 to 3.6 million [2]. Among the amputee population, lower-limb amputation is the most common type, accounting for 65% of all amputations.

The loss of body segments greatly influences amputees’ physical and psychological health. Successfully employing lower-limb prostheses (LLPs) has a positive impact on the overall quality of life by regaining independence and a sense of self-efficacy [3]. Despite the potential benefits, not every amputee uses an LLP. It has been found that the adoption rate of LLP ranges from 49 to 95% [4,5,6] and users of LLPs who are younger age and have a distal amputation tend to utilize the device for more hours each day [7]. This is perhaps not surprising given the traditional LLPs usually are simple body-driven mechanical devices. Wearers can only rely on the physical interface between the residual limb and prosthetic socket to regulate the LLP’s behavior with their body movements, as well as to gain limited information about the LLP and how it interacts with the environment. When more segments of the lower limb are lost, it becomes more challenging for amputees to walk with their prosthetic legs, as indicated in walking speed and metabolic cost [8] and they are more likely to abandon the device [9]. Indeed, in a recent systematic review of therapeutic benefits of LLPs [10], it was found that devices with more advanced technology have the potential to provide more benefits. In particular, quasi-passive and active prostheses were much better than passive devices in enhancing amputees’ quality of life.

Intelligent LLPs refer to all computer-controlled lower limb prostheses equipped with advanced control systems and algorithms designed to minimize gait limitations [11] on various prosthetic devices, including quasi-passive devices [12] and active devices [11]. Intelligent LLPs continuously monitor their own status using embedded sensors and are able to make decisions based on predefined rules without direct instructions from wearers [13]. For example, a traditional ankle prosthesis’ stance dorsiflexion angle is optimized for level ground walking [14]. This setup forces wearers to adopt additional compensation efforts when they walk on a slope. An intelligent ankle prosthesis can be programmed to change its locomotion mode from level ground to ramp up based on measured foot orientation [14]. This mode change permits additional ankle dorsiflexion in the stance phase and makes it easier to walk on the ramp. The decision of mode change is made by the LLP directly based on its continuous monitoring of foot orientation and can be triggered by putting the foot on a slope without additional commands. As a result, the intelligent ankle prostheses allow wearers to walk on both level ground and slope naturally. Additionally, some technologies in intelligent LLPs offer channels for wearers to provide feedback and directly adjust their control rules [15]. These channels enable wearer inputs and additional interaction with the LLP. Alternatively, if wearers are unsure how to adjust the control parameters, they can use human-in-the-loop optimization [16]. In this process, wearers walk continuously for a period while the computer searches for the most effective control parameters by analyzing the wearer’s responses to large number of parameter combinations.

Two types of wearer-LLP interaction

The developed technologies generally affect the following two types of wearer-LLP interaction: (1) direct wearer-LLP interaction and (2) indirect wearer-LLP interaction through a user-controlled interface (UCI). Direct wearer-LLP interaction is a continuous process, which involves real-time information exchange between a wearer and an LLP when the LLP is in action. The ultimate goal of the direct wearer-LLP interaction is that wearers will eventually become intuitively involved in the interaction during locomotion. Efferent and afferent neural-machine interfaces are examples of technologies designed to enable and enhance direct interaction. The efferent neural interface recognizes user movement intent by decoding neuromuscular signals recorded from the residual limb, so the behavior of LLPs has adjusted accordingly (e.g., [17, 18]). The afferent neural interface restores somatosensory, which is lost due to amputation, on the residual limb with artificial electrical stimulations [19]. This afferent feedback is designed to inform wearers of the LLP status and its interactions with the external environment. Other forms of sensory substitution (e.g., via mechanical vibration, audio, or visual cues) have also been used to restore somatosensation [18].

Indirect wearer-LLP interaction through a UCI is initiated by the wearer when the LLP is not in action, with the wearer interpreting information from the LLP and delivering commands deliberately through the interface (see Fig. 1). Beyond the traditional ways of changing LLP settings through a third-party prosthetist, the field is striving to give wearers flexibility in customizing the LLP control through a UCI (e.g., [15, 20, 21]). The ultimate goal of the indirect interaction is to achieve optimal and preferred LLP control by the wearer adjusting the LLP settings on their own. Some UCIs have been developed and applied in both experimental devices (e.g., [20, 21]) and commercial devices (e.g., OttoBock). For commercial devices, the UCI allows the wearer to activate and change different modes to support various activities, as well as to adjust control parameters of the swing phase [15]. In experimental devices, the UCI aims to give wearers more options in adjusting LLP settings, including adjustment to the radial positions of the socket during ambulation[21] and defining the desired prosthesis knee impedance control [20].

Fig. 1

Wearer-LLP interaction is generally divided into two types: (1) direct wearer-LLP interaction, and (2) the wearer’s indirect interaction with the LLP through a UCI

A need for a new framework of wearer-LLP interaction

An effective application and successful adoption of these technologies cannot be achieved without taking human factors into account during the design process. As defined by the Human Factors and Ergonomics Society, “Human factors is concerned with the application of what we know about people, their abilities, characteristics, and limitations to the design of equipment they use, environments in which they function, and jobs they perform” [22]. Human factors is a discipline that studies the interrelations among three aspects: human, system or device, and the work environment, with the goals to improve safety, comfort, and productivity while reducing human errors.

Recently, some studies have started to take human factors considerations into the design process of LLPs. For example, Beckerle et al.[23] developed a framework that prioritizes specific requirements in the human and device aspects of the powered LLP design, including satisfaction, the feeling of security, body-schema integration, support, socket, mobility, and outer appearance. Based on survey data, Fanciullacci et al.[24] suggested that an ideal LLP should prioritize design requirements such as reliability, comfort, weight, stability, adaptability to different walking speeds, and functionality related to the wearers’ lifestyle. In both studies, the reported requirements are determined in the context of existing LLPs and prioritized by wearers or prosthetists based on their personal experience. Although practical, this method has difficulty identifying all critical requirements to emerging technologies, as both wearers and prosthetists generally lack experience in these emerging technologies. In this case, expertise developed in the other domains can provide guidance when intelligent systems are considered. Intelligent LLPs can make sophisticated decisions such as adapting prosthesis control to user intent and environment [25]. This aligns closely with other systems that leverages high levels of automation or artificial intelligence, such as autonomous vehicles or robots. For example, transparency informs users about the status and planned actions of the device has shown to facilitate appropriate trust and improve task performance because users understand system capability and can anticipate the behavior of the system [26,27,28]. For the same reason, we can speculate that providing transparency that informs wearers of the planned action will help to avoid disturbance and dangerous consequences when a decision from the LLP leads to actions that are not aligned with wearers’ expectations.

New elements also bring novel dynamics to the wearer-LLP interaction, which amplifies the limitations of the traditional ways of evaluating the interaction to ensure wearer acceptance and adherence. According to Nielsen’s Model of Attributes of System Acceptability [29], system acceptance is affected by usability and utility, as both are important aspects of usefulness. Usefulness is the perceived quality of a system that users’ goals can be attained through using the system. Utility focuses on whether the functions of the system can work properly to support user needs, and usability focuses on whether the functions are pleasant and easy to use. In the context of LLPs, utility is how well an LLP is supporting a wearer to achieve the goals of improved mobility, independence, balance, and stability (see functional user needs in [30]). Usability, on the other hand, is more related to factors influencing wearer satisfaction and the system’s ease of use. This includes considerations such as how quickly can a wearer learns to use an LLP and how much attention is required for body movements when walking with the LLP. The development of LLPs has devoted heavily to ensuring the functionality (i.e., utility) [31], while there is little effort on how easy and pleasant wearers can use the functions (i.e., usability). Even among the scarce efforts, the results are often inconsistent with their objective performance and, therefore, cannot provide accurate information for the design process (e.g., [32,33,34]). For example, in a study comparing a powered and an unpowered LLP, participants preferred the powered LLP because they felt it helped them walk longer and faster; however, the self-reported results could not be confirmed with the objective measures as the majority of them did not increase their daily step count and speed [33]. Therefore, the wearers’ subjective preference does not necessarily reflect the overall usability of the LLP. Although the limitation of using solely self-reported questionnaires in usability evaluation may be less salient in the non-intelligent LLP, which is designed to achieve basic mobility functions, lack of holistic evaluation on usability may largely compromise wearers’ acceptance of the intelligent LLP that has versatile functions [35].

To enable systematic examination of human factors issues in wearers’ interaction with intelligent LLPs, the current paper proposes a framework (1) to introduce new human factors elements that emerge with the development of intelligent LLPs, and (2) to adopt Nielsen’s five usability requirements [29] to evaluate the wearer-LLP interaction. The paper is organized into three parts. The first part summarizes relevant elements in the wearer, device, and task aspects that affect the two types of interaction (i.e., direct wearer-LLP interaction and indirect interaction through a UCI). Although an LLP supports various kinds of activities, the direct wearer-LLP interaction in this paper primarily focuses on the interaction in locomotion. Also, in indirect interaction, when we use the word “interface”, we refer to the UCI on a separate device used to adjust the LLP settings but not the physical socket interface that has been researched extensively [36]. In the second part, we redefine the usability requirements for wearer acceptance by adopting Nielsen’s five usability requirements for two types of wearer-LLP interaction. In addition, for each requirement, we systematically organize existing evaluation methods and methods from other domains that can be used to inform the usability requirement for each type of interaction. The last part introduces future directions with discussion of some ongoing research in understanding wearers’ preferences in indirect interaction and standardizing assessments in direct interaction.

A framework of two interaction types

In this framework, the goal is to achieve system acceptance and trust by improving the usefulness of the system. Acceptance is associated with successful introduction of and intention to use a technology [29]. Trust can be defined as the attitude that a system will help users achieve their goals in situations of uncertainty and vulnerability [37]. Both acceptance and trust are strongly related to users’ actual usage of the system. Usefulness refers to whether users’ goals can be attained through using the system, which has been proposed to be a key contributing factor to system acceptance in models such as the Technology Acceptance Model [38] and Nielsen’s Model of Attributes of System Acceptability [29]. Nielsen’s model further breaks usefulness down into two equally important components: utility and usability. Usability consists of learnability, efficiency, memorability, use error, and satisfaction [29]. Meeting these requirements means we are making both types of interactions (i.e., direct interaction in locomotion and indirect interaction through UCIs) easier to learn and remember with a lower likelihood to make mistakes as well as making LLPs more efficient and pleasant to use. As a result, the wearer will be more likely to have a positive attitude towards LLPs, which will help to ensure acceptance and trust.

To meet the five usability requirements, we need to consider human factors elements during the design process [29]. According to the classic model of the human-machine system [32], the elements influencing human-machine interaction come from three aspects: human, machine, and the task environment. Adopting this model, in wearer-LLP interaction, we organize the relevant elements into three key aspects: wearer, device, and task (see Fig. 2, left panel).

Wearer elements are the human characteristics that may influence the interaction quality, such as cognitive level and prior experience with LLP. Although these elements in the wearer aspect cannot be altered easily, they suggest individual differences and determine whether a design’s applicability is broad. Therefore, designers are recommended to consider the wearer elements with inclusive designs, and researchers should assess and report these characteristics as the context of the reported findings for appropriate interpretation. Device elements are the characteristics related to the usability of the device, which can be altered by interface design. In direct interaction, the device we focus on is the LLP; in indirect interaction, we focus on UCIs. Task elements are the contexts of under what situation and how the device is used which depend on the wearers’ goal. Interface design for an intelligent LLP to achieve high usability needs to be tested under as many contexts as possible under which the wearer will use this LLP to ensure adoption and long-term adherence.

In this prospective discussion, we only highlight the new elements that emerge as the new technologies in intelligent LLPs develop. These new elements include wearer elements (e.g., cognitive function, prior experience), device elements (e.g., transparency of an intelligent system), and task elements (e.g., walking environment). Other elements, such as the weight of an LLP which has been traditionally explored in prior research, are not considered in this discussion although they remain important for LLPs. Although we introduce individual elements respectively in each interaction type, the effects of these elements are not isolated from each other. These three aspects interact to affect how wearers feel and perform when using intelligent LLPs. It means that the results of one change in the device element may be specific to its wearer characteristics and task condition. Therefore, a good design should try to consider and report all three aspects of elements.

Fig. 2

Proposed human factors framework for two types of wearer-LLP interaction. The framework illustrates the new elements affecting the interaction and the requirements for system acceptance in intelligent LLPs. The left panel is adapted from the model of the human-machine system [32] and the right panel is adapted from Nielsen’s Model of Attributes of System Acceptability [29]. Elements affecting utility can also be organized into the wearer, device, and task aspects. Given that usability is less understood than utility, we focus on highlighting the elements influencing usability in this prospective discussion

Elements to be considered for direct interaction in locomotion

To achieve ultimate acceptance of emerging technologies developed to enhance direct interaction during locomotion, we need to consider elements that become more crucial to the direct wearer-LLP interaction as the automation level increases in the intelligent LLP. In this section, we review elements that may contribute to these outcomes from three aspects: wearer, device, and task (see Fig. 3, left panel). Some elements emerged with the development of the intelligent LLP technologies, such as system transparency of the device and prior knowledge of the wearer; elements such as executive function have been examined before, but it is more important now with more information to learn and process at the same time; task elements have been studied extensively with traditional LLPs, but with the new dynamics in the interaction, the previous results may not be generalizable and the influence of these elements may need to be re-examined.

Fig. 3

Examples of elements affecting direct wearer-LLP interaction in locomotion (left) and indirect interaction through a UCI (right)

Wearer elements

Example wearer elements include wearers’ executive functioning and prior knowledge. Executive function is one domain of cognitive functioning that supports goal-directed behavior and control of attention, which affects the cognitive resources available to manage task demands. Executive function plays a crucial role in determining how safely the wearer could operate an LLP, the wearer’s mobility performance, as well as rehabilitation outcomes after amputation (for a review, [39]). For example, O’Neil and Evans[40] examined post-amputation rehabilitation before limb fitting and followed up at six months, and they found that executive function negatively affected the hours of LLP use and mobility outcomes. Similarly, Hunter et al.[41] found that lower executive function scores at discharge from rehabilitation programs were associated with lower gains in gait velocity and functional mobility. Executive function declines with age or can be impaired as a result of chronic conditions. Considering that about 75% of lower extremity amputations happen in patients older than 65 years of age [42], possible age-related cognitive decline could place a high demand on those wearers.

Another wearer element is prior experience. Prior experience shapes the wearer’s understanding and expectations towards a new technology which would influence the acceptance of the technology [43, 44]. For example, if a wearer has an unpleasant experience with a new intelligent LLP, such as falling, they would be less likely to trust and fully engage with the technology for a long time. In addition, as traditional non-intelligent LLPs are still often used as the initial prosthetic legs for new amputees, many wearers could be exposed to emerging technologies with their knowledge and expectation of LLPs formed through prior experience with non-intelligent devices. A relevant question is whether the prior experience with non-intelligent devices would positively or negatively transfer to their understanding and expectations of the intelligent LLPs. Understanding how wearers’ prior experience affects trust and expectations can guide the design of the intelligent device and improve its usage and safety, such as how trust could be restored after an unpleasant experience and knowing how a wearer’s trust evolves with increasing experience with a device. Similar questions have been asked in the development of other intelligent systems [45,46,47,48] and LLPs will be no exception in the need to address these.

Device elements

Transparency refers to the idea that the wearers should be informed of the current status and planned actions of the LLP [49]. In a traditional LLP, it is difficult for a wearer to detect and predict the motion of the LLP because the wearer lacks accurate and reliable somatosensation such as joint angles or joint angle velocity from the residual limb to describe the status of the LLP. In addition, wearers are unable to acquire real-time information about the interaction of the LLP with the environment, such as contact points and amplitude of ground contact force, to handle unexpected disturbances [50]. The development of new technologies, such as the afferent neural-machine interfaces, can provide information to promote system transparency to keep wearers updated on LLP status and interactions with the external environment [18].

In addition to restore sensory information to inform wearers about the current status, another aspect of transparency is to inform wearers about the incoming actions. Newly developed technologies allow LLPs to automatically switch control based on recognized locomotion modes (e.g., [17, 18]) while wearers are often unaware of what the LLP is planning to do. If the locomotion mode changes unexpectedly, or the locomotion mode recognition has faults and mismatches with the wearer’s intention [51, 52], it could disturb the wearer’s task performance, negatively affecting trust and acceptance towards technologies. In this case, providing transparency to support the wearer’s understanding and prediction about the LLP’s upcoming motion can be beneficial.

Task elements

The task elements can affect walking by imposing high task demands. One source of the demand is related to the environment. For LLP wearers, walking on different surfaces, such as uneven terrains, stairs, and slopes, or walking in a dynamic environment is considerably more challenging than walking on the level ground [53]. Not only does the needs for biomechanical support vary for different terrain conditions [54], but from a cognitive perspective, amputees wearing LLPs are less stable and more likely to fall when crossing or avoiding an unexpected obstacle than non-disabled individuals without LLPs [55]. Therefore, it is reasonable to speculate that the demand for the walking task is highly dependent on the environment in which the task is carried out. Considering the effects of different terrains and dynamics of the environment on cognitive and physical demands would benefit the development of emerging LLP technologies.

Furthermore, in everyday activities, wearers often carry out concurrent tasks such as looking for a coffee shop while walking. This additional task can heighten task demand and impair walking performance in amputees [56]. In the example of navigating while walking, people need to perform two concurrent tasks; one is walking that requires the integration of the basic visual and sensory feedback and control of physical movements, and the other is navigation, which also involves significant mental efforts. This process is common in our everyday life, such as crossing roads while watching for traffic, as well as walking while talking or texting on the phone. Compared to non-disabled individuals, amputees wearing LLPs experience a more adverse impact on gait performance from performing a concurrent task when walking [57]. Therefore, another element to consider when developing effective emerging technologies is the concurrent tasks in daily walking. By reducing the cognitive demands of operating LLPs, wearers can have additional cognitive resources to perform concurrent tasks while walking.

Elements to be considered for indirect interaction to define device control

With newly developed technologies used to adjust device control (e.g., UCI), the acceptance may require the technology to provide the desired adjustments without inherent difficulty to use and learn over time for people without prior knowledge. A wearer would most likely be performing adjustments to the settings of a device in a stable environment such as at home, thus its performance is less susceptible to the surroundings. Therefore, task elements are not considered for this interaction. Nevertheless, the wearer and device elements remain important for the interaction between the wearer and the LLP.

Wearer elements

One wearer element that affects the interface interaction is the domain-specific knowledge in tuning. Experts have much more elaborated declarative domain-specific knowledge than novices [58]. As a result, experts are more likely to perceive holistic meaningful patterns from an interface, whereas novices may focus the processing on individual pieces of information from the same interface [59]. An expert such as a prosthetist may understand the relationship between the control setting and the dynamic properties of the intelligent LLP, while a novice wearer may have to figure out this relationship through trial and error. Therefore, the design of an interface for novices needs to consider the minimal knowledge that a novice has and support exploration and recovery from errors. For example, some terminologies that are commonly used by clinicians can be translated into laymen’s terms with effective visualizations.

Another user element that affects the interaction is the wearers’ physical and cognitive limitations. Physical and cognitive abilities differ among individuals, and the way people interact with products is influenced by factors such as memory capacity, cognitive fatigue, and attentional ability. Wearers with any visual, hearing or cognitive impairments also need a display that is accessible to ensure they can use the interface properly, such as allowing customization for font, color, and text size [60]. Additional sensory channels such as audition and touch could also be utilized to compensate for deficits in visual communication for individuals with visual impairments. Considering the potential special needs of the different wearers while designing the interface would improve the accessibility of LLPs.

Personal experience is another user element. Users who have experience in wearing LLPs likely understand better the setting and LLP functions and may have calibrated their expectation of the technology. For these reasons, it can take fewer steps for them to achieve a satisfactory configuration. Experience with similar technology could also affect how a wearer expects the interface to work and compare the interface to their previous experience. For example, if wearers had prior experience with tuning another intelligent LLP and the interface was similar (e.g., an earlier generation of the same product), they may benefit from generalizing prior knowledge to the use of the current intelligent LLP. However, if another assistive medical device works differently (e.g., a hearing aid), wearers would be less likely to apply that knowledge to the tuning of LLPs. It is important to note that wearers may also compare the self-tuning process to the traditional tuning procedure performed by a prosthetist. If the self-tuning interface is complicated and the procedure is too time-consuming to reach a satisfactory setting, a wearer may not choose to use the tuning interface and instead opts for the traditional tuning methods.

Device elements

The first device element is the visual design. Having a cluttered interface makes it more difficult to interact with the device. Visual clutter refers to a disorganized visualization that often causes a high cognitive load and affects the efficiency of searching for key information [61]. Visual clutter negatively affects the efficiency of using the interface. To reduce visual clutter, strategies such as replacing texts with icons, removing unnecessary elements, and highlighting the most important information should be considered. In addition, an interface for prosthetists is likely designed for larger display, such as one on a desktop or laptop computer. A large display gives ample space to display various information and controls. In contrast, an interface for user-guided auto-tuning by an LLP wearer will likely be on a mobile device to allow tuning at any given moment. Such an interface would need to be largely simplified considering a small display size.

The second device element is how many and what control parameters are given to the wearers. While prosthetists with domain expertise can be given many parameters to improve the functionality and comfort of the LLP, considering the lack of expertise in adjusting such a device by a wearer, simplified tuning settings may be considered. The tuning interface could allow LLP wearers to adjust a few of the most essential control parameters, and the remaining settings are only accessible by prosthetists.

Requirements and evaluations of the wearer-LLP interaction

Each of the aforementioned factors has an impact on wearer-LLP interaction. Adopting Nielsen’s [29] five usability requirements, we propose using the following to evaluate wearer-LLP interaction to achieve system acceptance: learnability, efficiency, memorability, use error, and satisfaction. Efficiency is reflected by the task demand or the amount of effort a wearer exerts to perform a task such as walking with the device after the initial learning period. Use error relates to the wearer’s incorrect action or lack of action for an intended purpose (e.g., choosing an incorrect parameter setting). Learnability reveals how easily the first-time wearers bring off the tasks with the design. Memorability refers to the ability to recall relevant knowledge after not using the device’s UCI for a long time. Satisfaction reveals the level of fulfillment and pleasantness of using the device. In the following section, we apply these requirements with specific measures to evaluate the wearer-LLP interaction during locomotion and when using the UCIs (see Table 1).

Table 1 A summary of the evaluation components of wearer-LLP interaction

Evaluation of direct wearer-LLP interaction in locomotion

Efficiency Efficiency is defined as the demands on the wearer to perform a task. Because natural locomotion is automatic and poses minimal mental demand, an efficient LLP should enable a wearer to achieve stability and mobility without much effort. Efficiency can be evaluated by measuring both the performance in locomotion and the mental effort of a wearer. To assess the performance in locomotion, metrics such as metabolic cost, walking speed, and gait mechanics have been widely used [78]. There are also many human factors methods to evaluate mental effort.

One measure of mental effort is cognitive workload, which can be quantified using self-report questionnaires, physiological measures, and performance measures using single-task or dual-task paradigms. One commonly used self-report questionnaire is the NASA Task Load Index (NASA-TLX) [62]. NASA-TLX yields an overall workload score by rating six subscales, including mental demand, physical demand, temporal demand, overall performance, effort, and frustration. NASA-TLX evaluates different sources of workload, especially on the perceived level of workload [79]. Physiological measures provide a more objective and real-time assessment of workload. Some methods that have already been used to understand the workload associated with the use of LLPs are electroencephalogram (EEG; e.g., [63]) and functional near-infrared spectroscopy (fNIRS; e.g., [64]), both measuring brain activities. Other physiological measures, such as pupillometry from eye tracking systems (e.g., [66]) and skin conductance response (e.g., [67]), have also been used to measure workload of wearers using prosthetic devices. While these methods could provide valuable real-time estimates of workload, according to a recent review on physiological measurements of mental workload [80], many face challenges in studying the use of LLPs. Measurements in brain activities are generally expensive and requires complex signal processing. Pupillometric measures can be easily distorted by light changes and physical movement and skin conductance measures are highly sensitive to physical activities, making them more vulnerable to inaccuracy when measuring mental workload while a wearer walks with an LLP. For performance measures, a widely used method in wearer-LLP interaction studies is the dual-task paradigm. This paradigm requires participants to perform a primary task along with a cognitively demanding secondary task, such as a mathematical subtraction test (e.g., counting down by 7 [34]). By comparing the walking performance with and without a secondary task, or with a secondary task of varying demand, the dual-task costs on the primary or secondary tasks can indicate the cognitive workload that a wearer experiences. As each of the self-reported, physiological, and dual-task methods has its unique advantages as well as limitations, some research directly compared the effectiveness of those methods. For example, Shaw et al. [81] used EEG and dual-task paradigm to evaluate the cognitive workload in people with unilateral transtibial amputation and those with transfemoral amputations. In the study, while comparing walking and sitting with cognitive tasks under varied demands, neither amputee group showed a difference in EEG theta synchrony response with increased cognitive and physical demand. The dual-task paradigm was more sensitive in reflecting workload, where participants with transfemoral amputation had a worse performance on the cognitive task during walking than sitting, while participants with unilateral transtibial showed no difference between the walking and sitting conditions.

One concern of an LLP posing a high cognitive workload is that a wearer’s walking safety can be compromised because potential hazards in the environment, such as ground obstacles and other moving objects, are less likely to be noticed. Visuospatial attention guides a person’s observation of the environment and detection of hazards across the visual field. Using an LLP may negatively affect a wearer’s visuospatial attention because walking becomes unnatural and difficult. In some cases, such a negative impact is so significant that a wearer needs to use visual information to continuously guide gait movements, preventing the wearer from observing other areas of the visual environment. A well-designed prosthetic device should not pose a high cognitive workload so that a wearer would remain observant of the environment when using the device. To quantify the effect of walking with an LLP on attention, the Attentional Visual Field (AVF) [65, 82] task can be used to depict visuospatial attention across an extended visual field.

Use Error Use error describes a wearer’s inappropriate action or lack of action that prevents the intended purpose to be achieved. In intelligent LLP, there are occasional but inevitable system errors caused by faults in sensors or control commands, which could jeopardize a wearer’s walking safety (e.g., [51, 52]). Therefore, it is important to understand wearers’ reactions to different types of system errors to minimize potential use errors [47, 48, 79]. On the other hand, use errors can also happen even when the system behaves properly. There are three types of use errors: slips, lapses, and mistakes [83]. Slips are incorrect actions due to attentional failure, and lapses are memory failures in which someone forgets to take an action. Mistakes are planning failures as a result of an inadequate understanding of the device or poor judgment of the situation. Among these, mistakes are likely the major cause of use errors when using new technology in intelligent LLPs. For example, when a wearer is unfamiliar with how an intelligent LLP would behave under different circumstances, such as failing to trigger correct locomotion tasks and adopting unnecessary compensation strategies. In this case, providing instruction and training helps to solve the problem. Evaluating the use errors, such as calculating error rates and criticality of the errors, as well as designing to prevent errors are important to ensure safety when using an LLP. Specifically, the error rate can be calculated by recording the occurrence of errors in actions based on self-reports or observations by an experimenter, and the criticality of an error can be gauged based on how impactful the consequence is [68]. Furthermore, qualitative methods, such as interviews, can provide information on the cause of use errors [71].

Learnability Learnability indicates how fast the first-time wearers can successfully use the devices. A device with high learnability means it is easy to learn to use and can promote wearers’ adoption and long-term adherence to the device. Indeed, previous research has raised the issue of amputees not being able to adapt to the use of LLPs as an important reason leading to abandonment of the devices [84]. Although limited research has focused on the learnability of LLPs, it is worthwhile to investigate improvements in walking efficiency and gait performance over time. One previous study applied learnability metrics to compare two types of control in an upper limb prosthesis that calculated the learning percentage using the average time on completing the task across trials [85]. The same methodology can be utilized to evaluate LLPs. Learnability can be calculated by learning percentage using the performance on completing the task over time.

Satisfaction Satisfaction is the subjective feeling of whether an LLP has fulfilled wearers’ needs. One of the most common methods of assessing satisfaction is using a questionnaire, yet the operationalizations differ among existing questionnaires. For example, two major questionnaires that assess satisfaction-related issues in the LLP are the Trinity Amputation and Prosthesis Experience Scales (TAPES)[73] and the Prosthesis Evaluation Questionnaire (PEQ) [74]. TAPES contains questions on eight different attributes of satisfaction, including color, shape, appearance, weight, usefulness, reliability, fit, and comfort, and one additional question on overall satisfaction with a prosthesis. PEQ assesses specific attributes that contribute to satisfaction, including appearance, frustration, sound, utility, and residual limb health, as well as general feelings of satisfaction, satisfaction in walking over the past four weeks, and happiness with the current prosthesis.

As highlighted in a recent systematic review of existing questionnaires [86], a wearer’s overall satisfaction with an LLP is a complex construct that is influenced by many factors and could change over time as the wearer uses the device in more settings and for longer periods [30]. This calls for the consideration of additional factors to those currently captured in the existing questionnaires. For example, prosthesis embodiment represents the cognitive integration of the prosthesis and is associated with wearer satisfaction [87]. This integration of an external device into one’s own body is a marker of wearers’ full acceptance and effective use of the device. To quantify this important aspect, a Prosthesis Embodiment Scale for Lower Limb Amputees (PEmbS-LLA) [75] was developed.

When a wearer is not fully satisfied with a device, research methods such as the think-aloud protocol could provide diagnostics of reasons related to dissatisfaction [76, 88]. Using think-aloud, a wearer could verbalize their thoughts and feelings while or after walking with an LLP. For example, when comparing two types of devices, wearers can provide immediate feedback about their physical perception and subjective feelings. By analyzing the think-aloud data, it is possible to uncover what aspects of the LLP the wearers are not yet satisfied with and why they are unsatisfied, especially those that the researchers are not yet aware of.

Evaluation of indirect interaction through UCIs

Similar to the evaluation of wearer-LLP interaction, Nielsen’s five usability requirements [29] can be applied to evaluate the UCI for LLPs, through the use of various performance metrics and subjective reports.

Efficiency An efficient UCI should allow the wearers to reach their goals quickly and with little effort. There are two indicators to quantify efficiency. The first one is task completion time [68]. For example, this indicator can be used in a scenario where a wearer is asked to tune the knee profile with a specifically pre-defined goal, and the researchers can observe the duration it takes the wearer to correctly tune the profile. The task completion time can be used to compare different tuning interface designs and can also indicate if a user becomes more efficient in using the interface given accumulated experience. In addition to task completion time, efficiency can also be measured by the number of steps or iterations taken by a user to complete tuning.

With a more complex interface that has multiple pages to navigate while tuning, another applicable indicator of efficiency is lostness [69]. Lostness is calculated based on the number of different pages visited when performing the task, the total number of pages (including the revisited pages), and the minimum number of pages that a task is required to visit. A perfect lostness score would be zero. To reduce lostness, one could consider decreasing the total number of pages, keeping the interface organized, and grouping controls for the same goal within one or a small number of pages. Although a control interface of LLPs may not involve an excess number of pages, considering the amount of support and guidance information that may be added for all the interface controls on a portable device such as a smartphone, it is important to minimize the number of pages and keep all pages well organized and indexed.

Use Error Use errors can occur when a wearer has inappropriate actions with a UCI. For example, a user, due to lack of knowledge, may improperly tune a parameter to an extent that makes the device less safe to use. There are both qualitative and quantitative methods to identify errors when using the control interface (for a review, see [89]). The qualitative methods identify the nature of an error by analyzing the step-by-step tasks that a user needs to perform for a specific goal. For example, when an LLP wearer is feeling too much push on the residual limb from the device and would like to adjust it to a better fit, the adjusting procedure can be simply divided into three steps: identify the cause of the push, select the relevant parameter, adjust the parameter to a proper level. The potential errors could occur at each step and further analysis can reveal the reason for the errors. For example, when a wearer does not have sufficient knowledge of which parameter is related to the uncomfortable feeling, the wearer may adjust a wrong parameter causing a use error to occur. This error-identification process can be conducted based on the expert judgment of the researcher. Once an error is identified, its probability of occurrence and criticality can be estimated with expert judgments (e.g., [70, 72])

In general, the tuning interface should guard wearers against errors, especially critical ones, and if any errors occur, the interface should help them recover from errors. According to mistake proofing in Six Sigma [90], human errors may be reduced through three steps: mistake prevention, mistake detection, and fail-safe. Mistake prevention could be through design that forces wearers not to make mistakes, such as eliminating settings that could lead to reduced safety. For example, the designer can only allow a wearer to adjust the setting within a specific range. Mistake detection is helping wearers to realize that a mistake has been made. One way is to provide feedback through warning messages when wearers make mistakes. For example, when wearers set an invalid parameter, the interface should provide an error message. Lastly, the interface should have a function to recover the negative consequence of the error even if a mistake occurs (i.e., fail-safe). For instance, including a shortcut key on the interface to quickly reset the parameters to a safe value.

Learnability A UCI with high learnability should be easy to use for all levels of users. The UCI can be complicated for wearers who lack domain-specific knowledge of tuning, and therefore, the interface should guide to help wearers. To promote learning, a clear tutorial on the interface and guided tuning processes would be beneficial. The interface should also allow a wearer to revisit the tutorial and access guidance information (e.g., an explanation of a specific parameter or mode) when needed. Learnability can be assessed by performance over time [68]. Specifically, as wearers perform a set of tuning tasks on the interface over several trials, the time completing the task and the errors they make can be recorded for each trial. An interface with high learnability should have decreased task completion time and errors over trials.

Memorability The memorability of a UCI is the ability of a wearer to remember the interface after some time without using it. An LLP may not need to be adjusted very often, therefore, when the wearers have not used the interface for a while, it is important that they can re-establish their knowledge quickly and easily. A design that generates a strong mental model tends to have good memorability (e.g., [91]). For example, presenting a slider bar with meaningful numbers for users to adjust the control point would be helpful to recover knowledge quickly. Just like the slider bar commonly used in other interfaces, 0 can be the default setting, -1 means a decrease by one degree from the current position, and + 1 means an increase by one degree from the current position. Memorability could be measured by assessing the knowledge of the control interface when a wearer would need to use the interface again after some time since the last tuning experience [68].

Satisfaction Satisfaction indicates how fulfilling and enjoyable it is to use the UCI. Similar to the measure of wearer-LLP interaction, the satisfaction of the tuning interface can be measured with self-report questionnaires and qualitative methods. One common questionnaire that measures satisfaction in an interface is the System Usability Scale (SUS) [77]. SUS is a usability questionnaire that consists of 10 items. By adapting SUS to measure wearer satisfaction with a tuning interface, an example item is “I found the system unnecessarily complex”. Participants rate each item on a 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). A SUS score will be derived from the weighted scores of the questions and a higher score indicates better usability of the system. Qualitative methods, such as interviews and think-aloud protocol [92, 93], can be used in the development stage of the interface to understand wearer needs and preferences of the design. During a think-aloud study, participants can simultaneously report their reasons for an action and thoughts towards the interface design while using the interface. Through think-aloud protocols, we can understand the attitude toward the design and identify issues with the interface so that we can take the necessary steps to improve the design.

Future research directions

Some research on direct wearer-LLP interaction has attempted to assess both biomechanical and cognitive performance when evaluating locomotion experience (e.g., [63, 64, 81]). However, testing methods lack standardization, as highlighted in a recent literature review [56]. For example, the dual-task paradigm is commonly used to assess cognitive workload, but instruction for prioritization is often unclear. When the instruction is not standardized, some studies may prioritize the locomotion task while other studies prioritize the secondary cognitive task, making it difficult to compare results across studies. This consideration emphasizes the important of clear reporting of methodological details such as instructions and justification of why particular prioritization instructions were given.

In addition, we also need to consider demands on various cognitive aspects during locomotion, as certain cognitive functions are more critical to locomotion tasks. For example, how wearers allocate visuospatial attention when using LLPs is crucial to safety as wearers need to be able to detect and avoid obstacles and hazardous objects while walking. However, this topic has been under investigated likely due to the lack of a validated and viable measure of visuospatial attention while walking. To fill this gap, Yuan et al.[82] developed the Standing and Walking Visual Attention Field (SWAVF) task based on the well-established Attentional Visual Field (AVF) task that has been carried out on a computer in a sitting position (e.g., [65]). The SWAVF task focuses on assessing a person’s allocation of attention in the lower visual periphery during walking to inform safety [94]. During the task, participants fixate at a cross straight-ahead and use peripheral vision to locate the position of a target that briefly appears in conjunction with distractors. Current data supports the validity and reliability of the SWAVF task [82]. Future research could use the SWAVF task to quantify the attentional demand of specific prosthetic designs (e.g., higher demand leads to poorer visuospatial attention), especially as walking scenarios expand to different terrains and with more road objects such as other pedestrians.

In terms of indirect interactions through UCIs, a number of recent publications indicate a growing interest in developing user-guided tuning processes and interfaces (e.g., [20, 95]). However, most efforts focus on developing the algorithm and determining what can be tuned, with little research on how to design a wearer-friendly tuning interface. To achieve the usability of the LLP tuning interface, it is important to consider both device and wearer elements. As an ongoing effort of a joint research team of human elements psychologists and biomedical engineers, our study [92, 93] is using the think-aloud protocol to explore the preferences and tuning strategies during user-guided prosthesis tuning. In this study, non-disabled and amputee participants performed multiple iterations of self-guided tuning followed by walking with the tuned prosthetic leg. Their thoughts were elicited with the think-aloud protocol. Thematic analysis was conducted on the qualitative verbal data to identify participants’ preference, tuning strategy, and factors that contribute to the feeling of a natural fit. This research identifies initial considerations in the design of the tuning interface and tuning procedure. It also shows the promise of allowing prostheses to be user-adjustable based on environmental and situational context. On the other hand, this first step calls for further investigations on user elements such as how to use data from the intact leg or past LLP devices to predict user preference for the new prosthetic leg [92, 93].

In addition, what accompanies the autonomy brought by user-guided tuning is the concern of safety risk. It is essential to investigate the extent to which users should have control to meet both preference and safety needs so that the researchers could know the options and ranges that should be restricted. Several usability requirements can be used to quantify the safety risks, such as errors (e.g., setting a parameter to a value that could lead to a fall would be an error) and efficiency (e.g., high lostness when navigating the tuning interface or procedure suggests low efficiency). After the risks are identified, we need to know what methods (e.g., providing error warnings, supporting the recovery from errors, or providing training guides) are effective in guarding against use errors and inefficiency. Future design can benefit from research investigating various user and device elements using the same usability requirements mentioned earlier. For example, studies could examine the effectiveness of providing training guides to reduce errors. The goal is to identify interface design or training guides that particularly benefit users of low domain-specific knowledge (e.g., amputees as LLP wearers rather than prosthetists). This would enable evidence-based design decisions on allowing the option to provide training guides to reduce the error rate and safety concern, especially for people with low domain-specific knowledge.

Furthermore, recent technological advancements enable human-in-the-loop optimization in LLPs, allowing the devices to continuously learn and adjust high-level parameters based on real-time data from the wearer [16]. Human-in-the-loop optimization raises a lot of interesting questions. When these intelligent LLPs adapt to the wearer, the wearer is also learning to use the device, creating a dynamic interaction loop where both the machine and the human are constantly evolving in response to each other. It is reasonable to wonder what parameters should be optimized in these controls – what exactly should the machine learn? This question is closely related to system transparency, as highlighted in our framework. It is essential for wearers to understand the goals of the LLP such as minimizing metabolic cost or maximizing stability as suggested in [16], and how the machine’s behavior aligns with these goals, as this understanding can significantly influence wearers’ preferences and interaction with the device. Additionally, what characteristics of the wearer – such as age, physical and cognitive abilities, and prior experience – might impact their ability to learn and adapt to the LLP? These are critical research questions that warrant further investigation.

As the advanced LLPs become more intelligent and can dynamically accommodate wearer needs, the design considerations will be even more complex as research questions related to trust in automation emerge. For example, to improve a wearer’s trust in the devices, it is important to know how to inform the wearer of the intention and function of the algorithm to improve automation transparency. To understand how the type of methods used to communicate affect trust, further investigation can be conducted following our proposed evaluation methods. Using the same evaluation metrics across multiple studies would allow effective cross-study comparisons and meta-analyses.

Conclusion

To ensure wearers’ acceptance of intelligent LLPs, researchers and designers need to take into account the potential impact of new device elements and consider the specific wearer and task characteristics when interpreting the findings. The proposed framework can guide the organization of literature, interpretation of findings within the constraints of specific study setups, comparing findings across empirical studies, and identifying gaps in the literature and future directions. Further research is needed to investigate the specific wearer, device, and task characteristics and the potential interplay among these elements using comparable evaluation metrics to improve the wearer-LLP interaction.