A review of concurrent sonified biofeedback in balance and gait training

Abstract

Background

Sonified biofeedback is a subtype of auditory biofeedback that conveys biological data through specific non-verbal sounds. It can be designed to provide augmented biomechanical feedback in near-real-time when provided as “concurrent” biofeedback. As a practice that developed spanning across engineering and the arts, sonified biofeedback can extend beyond simple tones and beeps, towards more fully incorporating music in movement training. Sonified biofeedback may leverage the motivational aspects of music in movement training, the neuroplasticity benefits demonstrated from participation in music-based interventions, and neurological auditory-motor coupling, all while providing task-relevant cues to facilitate motor (re)learning. Sonified biofeedback may also provide similar benefits as rhythmic cueing (e.g., rhythmic auditory stimulation), or added benefits because sonified biofeedback does not impose a strict isochronous rhythm when it follows rhythms that are driven by outputs of the motor control system. In this review paper, the unique opportunity presented by concurrent sonified biofeedback as a movement training tool for balance and gait is introduced and discussed.

Results and discussion

This review paper brings together prior research from clinical, engineering, and artistic design sources using sonified biofeedback in balance and gait training across diverse end-users to highlight trends, reveal gaps in knowledge, and provide perspective for future work in the area. The goal was to review progress and critically assess research using sonified biofeedback during movement training for postural control or gait. 49 papers were selected based on their experimental investigation and statistical analyses of the effects of using sonified biofeedback to assist in movement training for feet-in-place balance tasks (20 papers) or gait tasks such as walking and running (29 papers). The sound design choices, experimental design features, and movement training results are summarized and reviewed. All but two studies reported at least one statistically significant positive effect of training with sonified biofeedback in biomechanical, clinical, or psychosocial measures. Conversely, only seven studies shared any negative effect on one biomechanical, clinical, or psychosocial measure (with five of these studies also reporting at least one other positive effect). After describing these encouraging findings, this review closes by sharing perspectives about future directions for designing and using sonified biofeedback in balance and gait training, and opportunities for more cohesive growth in this practice. One such suggestion is to pursue sonified designs and experimental designs that can translate to the neurorehabilitation field. This includes strategically selecting control groups and evaluation tasks to understand if improvements from training with one task transfer to additional relevant movement tasks. Additionally, it is important that future publications share details about the design processes and sound designs so researchers can more readily learn from one another.

Conclusions

Overall, this review shares the positive impact of using sonified biofeedback in balance and gait training. This review highlights the evidence of existing successes and potential for even more impactful future positive effects from using sonified biofeedback to help diverse populations with a spectrum of balance and mobility challenges and goals.

Background

Sonification is the practice of displaying data with non-verbal sound. More specifically, sonification seeks “to translate relationships in data or information into sound(s) that exploit the auditory perceptual abilities of human beings such that the data relationships are comprehensible.” [1] Sonification has been used to represent all types of scales of data, from microscopic COVID-19 molecules [2] to massive astrological datasets [3, 4]. In this review paper, we focus on the unique opportunity presented by sonification of biological signals, or “sonified biofeedback”, as a supportive technology for balance and gait training pertinent to the neurorehabilitation field.

Sonified biofeedback is a subtype of auditory biofeedback that conveys biological data through non-verbal sounds. For example, in balance training, concurrent sonified biofeedback can be used to convey the changing postural alignment of the body in near real-time, as measured and estimated by a wearable sensor system. The signals form the wearable sensor would be pre-processed to extract meaningful data features mapped to specific sounds. For instance, the desired state of improved posture as measured by a decreased postural inclination angle (the data feature) can be conveyed in near real-time through the increased loudness of a pleasant computer-generated musical instrument sound emitted from smartphone speakers. While there have been prior studies using sonified biofeedback technology for rehabilitation, it is not yet established as a common clinical practice. Therefore, by reviewing relevant prior experiments, this paper supports further exploration of this technology towards wider adoption of sonified biofeedback in clinical practices to train balance and gait.

Balance and gait training approaches seek to reduce the risk of injury or improve task performance for those with a range of movement abilities spanning from those with motor impairments to athletes. One major clinical priority is to reduce the risk of injurious falls [5]. To decrease fall risk, clinical approaches typically include strength, flexibility, and task-specific movement training progressions from simple to more complex movements. For instance, a rehabilitation protocol for those with mild cognitive impairment and balance issues may include stationary balance tasks (standing with two feet together, progressing to “tandem” stance with one foot directly in front of the other, etc.), and more dynamic gait and transitional maneuvers (e.g., walking around obstacles, etc.) [6]. Another priority for athletic training includes avoiding musculoskeletal injury during running (for instance, [7]). Reducing the risk for falls, improving functional gait for daily mobility, and running-specific movement training approaches can likely benefit from sonified biofeedback. Thus, this review paper seeks to answer the overall question- To date, how effective has using sonified biofeedback been in balance and gait training across diverse end-users?

The first examples of sonification were documented in the beginning of the 1900s, before the term “sonification” was popularized. The Geiger counter, first described by Rutherford and Geiger in 1908 [8], conveys radiation levels through audible “click” sounds. A second example is the “Optophone” presented by d’Albe in 1914 [9] that assisted those with vision impairment and blindness in reading text by conveying visual contrasts in printed text by varying the timing and tones of sounds. The practice of sonification and auditory display grew from there, though, not necessarily in a centralized manner. In 1992, the first International Conference on Auditory Display (ICAD) was organized [10]. Shortly after, the United States National Science Foundation requested a report about sonification, prepared by Kramer et al. in 1999 [11]. This report introduced and proposed sonification as a valid area of research after contextualizing: “by nature, the field of sonification is interdisciplinary, integrating concepts from human perception, acoustics, design, the arts, and engineering” [11]. However, even this report noted that the absence of a “Journal of Sonification” or research funding opportunities specific to sonification will dampen unified growth of sonification research [11]. The foresight provided in that report is relevant today, as there is still a need for review articles that bring together prior work fragmented across disciplinary siloes. Also prompted by reflecting on sonification’s historical roots, a notable feature of the Geiger counter is that subsequent research found that using audio-only data display (as in sonification) improved the ability to search for radiation levels when compared to using visual-only or audiovisual displays [12]. This finding helps segway towards introducing the distinct benefits of using sonification in biofeedback design as compared to other “augmented biofeedback” modalities.

Augmented biofeedback in movement training provides additional sensory cues to the user to facilitate motor skill acquisition towards motor learning, described as relatively permanent changes in motor behavior. Sensory cues embedded in human movement, or “intrinsic cues”, provide critical information to facilitate motor control and motor learning [13]. For example, those who have intact sensory systems can visually perceive their orientation relative to the environment during walking while feeling their ground reaction forces underfoot and hearing their footsteps, among other sensory cues that may or may not raise to the level of conscious perception. Sensory cues allow for feedback and feedforward (i.e., predictive) motor control behaviors that can reduce errors between the desired movement goal and the body’s current state or predicted future state. Augmented biofeedback that is provided by technology typically uses biological sensors and computing technology to detect and convey additional cues, depending on the user’s needs and the movement task. These biofeedback cues can emphasize sensory cues already perceived by intrinsic sensory signals. Alternatively, biofeedback can provide cues about features of movement that are not typically perceived, or those that are not perceived with enough accuracy or enough lead-time to benefit movement training. These biofeedback cues can use visual, audio, or haptic modalities in isolation or in combination as “multimodal” (e.g., audiovisual). In the case of sonified biofeedback, distinct neurological, psychosocial, and perceptual factors support its unique strengths in biofeedback, particularly applicable during balance and gait tasks [14,15,16].

Sonified biofeedback in balance and gait training

Sonified biofeedback may have distinct benefits as a modality of biofeedback. If it is designed with musical sounds, it can leverage the motivational aspects of music in movement training [17], and support the neuroplasticity benefits demonstrated from participation in music-based interventions [18,19,20,21]. Sonified biofeedback may also leverage auditory-motor coupling that minimizes processing time-delays and facilitates rhythmic entrainment [22, 23], while providing task-relevant biofeedback cues to facilitate motor learning through multisensory integration [24, 25]. Additionally, many traditional forms of music (singing, playing an instrument, etc.) are outputs of motor behaviors and many motor behaviors produce sounds [24, 26], which may allow sonified biofeedback to tap into intrinsic movement-sound linkages [24, 27,28,29,30]. As such, sonified biofeedback can overlap and integrate with intrinsic feedback, such as proprioception, which can support motor learning [24]. The following section reviews advantages and disadvantages of sonified biofeedback in movement training, with an emphasis on how it pertains to balance and gait tasks.

When sonified biofeedback provides an engaging and enjoyable music interaction experience, it can assist in motivating achieving the quantity of movements practice required for motor learning. Music therapy is an established practice across medical domains, which is growing as more evidence accumulates to support music’s therapeutic effects [27, 28, 31]. In physical rehabilitation, music is often used to motivate or help patients keep to specific rhythms during movement practice. For example, the MusicGlove successfully motivated persons recovering from Stroke to complete their home exercise program repetitions of opposing thumb exercises because it leveraged music and gamification. In the MusicGlove, each opposing thumb exercise was prompted with a visual cue to produce the movement in synchrony with the rhythmic elements of the background music (a game design like Guitar Hero^™) [17]. In sonified biofeedback, an enjoyable musical interaction may be achieved by using musical sounds, and when helpful, layering multiple musical sounds for a more engaging musical experience [14, 15]. For example, gait training can be facilitated with background music playback controlled by maintaining a specific gait speed, with each step taken triggering the playback of a drum beat through sonified biofeedback. If such a layered musical design was used in sonified biofeedback, care must be taken to ensure that it aligns with the movement training goals and the user’s capabilities (i.e., attending to multiple sounds concurrently can have adverse effects if cognitive resources are overwhelmed) [32]. For instance, a layered and complex musical biofeedback system would not likely be helpful if the user needs to carefully attend to and extract meaning from all concurrent auditory signals. Generally, as the number of stimuli to respond to increases, decision or reaction time increases logarithmically [33].

Auditory-motor coupling refers to the direct neurological connections between the auditory and motor systems [23, 34, 35] that tend to minimize processing delays, as compared to visual biofeedback [36,37,38]. The minimized processing delays between sound and motor systems supports the use of sonified biofeedback to convey complex and rapidly changing data. The innate connections between movement and music (e.g., movement driving musical expression, movement in presence of music, etc.) present an opportunity for sonified biofeedback to uniquely assist motor learning. However, these connections are poorly understood for realistic whole-body movements due to methodological limitations, presenting a rich area for future research as neural measurement systems improve. Additionally, certain sound and musical cues may align with metaphorical or actual spatiotemporal movement behaviors with which users have prior experience. Therefore, sound design choices can tap into these already-known relationships between movement and sounds [14]. In this way, processing delays to hear and extract meaning from sonified biofeedback cues may be minimized.

Specific to balance and gait training tasks in clinical or athletic settings, the auditory system is a relatively underutilized sensory modality [15], presenting a unique opportunity for sonified biofeedback. For instance, when walking around obstacles, there is need to visually assess the surroundings such that visual displays of biofeedback would be distracting. Additionally, during cyclic motor behaviors like walking, as the body oscillates step to step, sonified biofeedback may tap into the benefits of rhythmic cueing in movement training, as in Rhythmic Auditory Stimulation (RAS) [39,40,41,42]. Sonified biofeedback may provide [23,24,25,26] benefits beyond rhythmic cueing because sonified biofeedback does not impose a strict evenly-spaced rhythm [43]. Instead, sonified biofeedback can follow rhythms that are internally sourced, which has been suggested to be even more helpful than externally sourced rhythms imposed when following the beat of a musical song [44], or the beat provided by RAS.

When sensory signals from multiple modalities are congruent and/or overlapping, motor learning can be facilitated through different means. For one, if sonified biofeedback emphasizes and builds improved perception of intrinsic feedback by overlapping signals that exist without the presence of biofeedback, the guidance effect (when feedback is removed, performance declines) can be minimized [14, 15]. In other words, in sonified biofeedback, there is an opportunity to augment task-intrinsic cues (cues that are already available without feedback) rather than sonify “creative new parameters” to mitigate the guidance effect [14]. Additionally, congruency between multiple streams of sensory signals can improve perceptual abilities that are helpful in motor learning. For example, Schmitz et al. [25] found that congruent perception of pre-recorded body movements and sonified cues (congruent audiovisual signals) improved perceptual judgements of small changes in movement velocities as compared to incongruent audio-visual cues or visual-only cues. Congruent audiovisual stimuli engaged multisensory integration in neural areas important to the action observation system, thus improving the perceptual analysis of movement. Thus, sonification may have significant benefits to perceptual-based task learning [25] which is an important component of motor learning with biofeedback.

As in other biofeedback modalities, there are factors that may limit the efficacy of sonified biofeedback. To begin, sonified biofeedback is not as accessible to those with hearing impairments and those with amusia (“tone deaf”), who may not perceive features of the sonified biofeedback. Additionally, as in all biofeedback designs, if the sonified biofeedback design overwhelms cognitive resources [15] or if it is not easily perceived or understood by its user, neural processing delays or an inability to use the biofeedback cues would mitigate its efficacy. Specific to sonified biofeedback, if the goal is to leverage therapeutic strengths of music biofeedback, musical designs could rely too heavily on music perceptual abilities, sociocultural music contexts, or become increasingly complex, which could increase neural processing delays. Similarly, all biofeedback modalities need to evaluate and minimize “latency” which in the case of sonified biofeedback, is the time delay between the biological signal measurement and the sound emitted. If the biofeedback system latency is not properly evaluated and minimized, it halts the benefits of congruent multisensory signals, and may confuse the user if the signal received is not relevant at the time it is received, perceived, and understood.

Additionally, if sonified biofeedback uses musical designs, there is a risk of prompting psychosocial responses that are adverse in motor learning (e.g., triggering autobiographical memories that distract the user). Relatedly, there is mixed evidence about the ability for background music to assist [45] or distract [32] users from learning tasks (though, in biofeedback, the musical sounds are often task-relevant vs. background music). Further, using sonified biofeedback in environments that provide cues about potential danger through sounds (e.g., automotive traffic sounds) would not be safe. Finally, as in other biofeedback modalities, there is an issue of dependence on the biofeedback (e.g., the “guidance effect”), so designs should aim to minimize the chances of the movement improvements relying on the presence of the biofeedback (further discussed in Sect. "Perspectives for future opportunities for sonified biofeedback").

Overall, biofeedback needs to be accurate, timely, and trustworthy to effectively inform movement skill acquisition that transfers to sustained motor learning. Design considerations to achieve these goals are provided in Sect. "Introduction to sound dimensions commonly used in sonified biofeedback" and further discussed in Sect. "Perspectives for future opportunities for sonified biofeedback". As a practice that is not yet common in movement training, there are many open questions about if sonified biofeedback can provide benefits to overcome these limitations (some of which are general for all modalities of biofeedback).

Purpose of this review

The purpose of this literature review was to evaluate the prior use of concurrent sonified biofeedback as a rehabilitation technology to improve balance and gait training. This review paper brings together prior research from clinical, engineering, and artistic design sources using sonified biofeedback in balance and gait training to highlight trends, reveal gaps in knowledge, and provide perspective for future work in the area. To contextualize the effects of these prior experiments using sonified biofeedback in balance and gait training, we start by providing information about the sonified biofeedback designs and then discuss the associated experimental designs and findings.

Providing a backdrop and motivation for our review, there are prior articles about sonified biofeedback design [14, 46,47,48,49,50,51], a review about sonified biofeedback in physical therapy [52], reviews about rhythmic and music interventions [37, 40,41,42, 53,54,55,56], generic rehabilitation reviews that mention audio or sonified biofeedback [30, 57,58,59,60,61,62], and methodology papers that discuss sonified biofeedback [63, 64]. These prior papers provide individual pieces of relevant background information and suggestions for future sonified biofeedback research. However, there are no reviews specific to the designs of sonified biofeedback and the effects of using sonified biofeedback on balance and gait training spanning across end-user populations. Relatedly, the clinical usability and efficacy of sonified biofeedback in improving balance and gait is currently unknown. Therefore, our purpose is to review the effects of using varied sonified biofeedback designs to train balance and gait movements with diverse end-user populations.

Introduction to sonified biofeedback design approaches

Sonified biofeedback design theory follows strong evidence about how music can motivate, how sound perceptions lend themselves to intuitive metaphors in the linkages between movement and sounds, and that sonified biofeedback can tap into neural benefits of auditory-motor coupling [14, 15]. Sect. "Introduction to sonified biofeedback design approaches" seeks to demystify the basic approaches to design sonified biofeedback.

The sound design process includes carefully selecting the audible sound as well as how that sound is “mapped” to convey the biological data feature of interest. Following design guides and theoretical frameworks, it is best practice to customize the sonified biofeedback design to the end-user(s) and the nature of the biological data [1]. This user-centered design process maximizes the potential usability and alignment when key user groups are part of the design process. These end-users may include athletes, patients, coaches, clinicians, and caretakers. Designing specifically for their capabilities and preferences will improve the usability and translation to practice. In the Perspectives section (Sect. "Perspectives for future opportunities for sonified biofeedback"), we point readers towards additional helpful design guides and frameworks.

In sonified biofeedback, biological data are measured, pre-processed, and displayed through specified design parameters. A relatable example of a concurrent sonified biofeedback design is a heart rate monitor that makes an audible “beep” sound upon measurement of each heartbeat. In this example system, the heart rate measurement system detects the biological signal, pre-processes the data signal (e.g., filters raw data, compares the measured value to a threshold value to determine if a heartbeat occurred), and finally, uses its sound design settings specified in a “parameter mapping” design to synthesize an audible beep sound. Figure 1 provides a conceptual diagram of a concurrent sonified biofeedback system to illustrate how biological data are measured, pre-processed, and displayed in real-time to the end-user(s) as synthesized audible sound based upon design settings that link biological data to synthesized sound(s).

Fig. 1

Diagram to describe an example concurrent sonified biofeedback system loop. For ease of understanding, we explain three major steps in sequence, but note, this is a continuous loop of interaction between the human user and the designed system, so the starting point is arbitrary. First, biological sensors are used to measure the user’s actions in “Biological Data Acquisition”. Second, the biological data is pre-processed to extract the desired biological data feature(s) in “Biological Data Pre-processing”. Third, in the sound synthesis process, the desired biological data feature(s) are conveyed through sonified biofeedback through three sub-steps: a. the sound synthesis system receives the desired data features, b. using a pre-selected mapping design, the data feature(s) are mapped to desired sound dimension(s) (each with associated perception and signal qualities), and finally, c. sound synthesis parameters are controlled to achieve the desired sound dimensions. For example, a desired sound dimension could be the loudness of an audible sound and the sound synthesis parameter is the volume level controlled by an equalizer. The sonified biofeedback information is looped back to the user to influence the user’s next actions. The user’s next actions are measured by the biological measurement system, which was the first step in this sequence

This sonified biofeedback loop enables participant(s) to hear the biological data feature(s) of interest and thus, become aware or more aware of aspects of their movements that were not previously as evident. From this increased awareness, one may better understand the control of their movement, and hopefully, use this information to modify subsequent movements as part of their motor learning progression. To best support motor learning, designing sonified biofeedback requires priorities to achieve design benchmarks related to technological capabilities (e.g., low latency, accuracy, etc.), perceptual abilities (e.g., the ability to distinguish features of the sound biofeedback), and user attitude measures (e.g., understandability, neutral to positive user “attitudes towards using” [65], etc.) [1].

In sonified biofeedback, parameter mapping can be used to convey meaningful biological data features with perceptible changes in sound driven by sound synthesis parameters (e.g., volume levels, etc.) [1]. Revisiting the heartrate monitor example, the monitor makes an audible “beep” upon measurement to represent each heartbeat. In this case, the beep sound was mapped to convey each heartbeat as a meaningful biological data feature. Parameter mapping in sonified biofeedback can allow display of single or multiple features of biological data concurrently or in isolation through one or many auditory signals or sound synthesis parameters. The number of biological data features mapped to the number of sound synthesis parameters are categorized as either “one-to-one”, “one-to-many”, “many-to-one”, or “many-to-many”[1, 66]. In the case of the heart rate monitor, one heartbeat detected as a data feature is mapped to one beep sound (one-to-one design) with sound synthesis parameters that control the sound dimensions of pitch, loudness, and duration of this beep.

Introduction to sound dimensions commonly used in sonified biofeedback

Here, we introduce eight “sound dimensions” [1] that have perceptual descriptors, signal attributes, and potential use cases in sonified biofeedback for balance and gait training. These eight sound dimensions are most pertinent to the reviewed literature.

• 1. Pitch:

  • a. Perception: Many sound sources, including the human voice, plucked and bowed strings, woodwind, and brass instruments—to mention a few—generate periodic variations of air pressure. Sound travels in longitudinal acoustic waves across media (e.g., air, water, etc.). When the frequency of the sound wave is within the range of human hearing (approximately 20 Hz to 20 kHz), we perceive these sounds as having a pitch (or “tone”). Pitch is the perception of frequency, which can be defined as the sensation by which sounds can be organized on a musical scale from “low” to “high”. We typically perceive the “fundamental frequency” of a sound, which is the lowest frequency embedded in the sound as the loudest, so the fundamental frequency is perceived as the pitch of the sound [1, 67, 68].

  • b. Signal attribute(s): Pitch is a function of the fundamental frequency of the sound wave. For example, a pure tone of a sinusoidal wave with a frequency of 500 Hz has a higher pitch than one with a frequency of 50 Hz. In non-pure tones, a mixture of vibrations generates waves with differing frequencies that superimpose to generate the total acoustic wave. The lowest frequency is the fundamental frequency and additional frequencies are known as “overtones”. As the fundamental frequency increases exponentially, the perceived pitch increases linearly [1, 67, 68].

  • c. Potential uses in sonified biofeedback: Pitch may be helpful for displaying increases or decreases in sonified data features due to the familiarity in visualizing pitch as “low” to “high”[1, 14]. However, high pitches can elicit biochemical and arousal changes in individuals that are not conducive to motor learning, and certain populations may have a narrower range of perceived pitches (e.g., due to aging). In an example design, as the data feature increases, the sound synthesis parameter of the fundamental frequency can increase with a linear mapping function. In this design, as the data feature increases, the goal is for the perceived pitch to increase.

• 2. Loudness (or Volume).

  • a. Perception: Loudness is the perception of the energy, or the amplitude, of a sound. As the amplitude in a sound increases exponentially, we perceive the loudness increasing linearly [1, 67, 68].

  • b. Signal attribute(s): Amplitude or energy of the acoustic sound wave. In sound synthesis, the term “level” is used as a control feature for loudness. Volume is usually measured in decibels, which is often abbreviated as “dB” or “dB-SPL” [1, 67, 68].

  • c. Potential uses in sonified biofeedback: Loudness can be used to auditorily display the value of any biological data feature. However, while many people can accurately distinguish gradations of pitch, people are less accurate in distinguishing gradations of loudness [1]. This suggests that loudness may be used to convey biomechanical quantities in broad bands, such as low, medium, and high ranges.

• 3. Timbre

  • a. Perception: Timbre can be thought of as the “fingerprint” of a sound, or the “sound” of the sound. The definition of timbre is often described in the negative: timbre is the quality of a sound that allows a listener to distinguish between two sounds which have the same pitch and loudness, but do not sound the same. For example, if a particular pitch were played on a piano, and the same pitch were then played on a xylophone, the difference in timbre is what allows a listener to distinguish between these two musical sounds. Note, it is difficult to quantify the perception of timbre [1, 67, 68].

  • b. Signal Attribute(s): The timbre of a particular sound is often a complex function of several factors, including prominently the spectrum (the frequencies that exist within the sound, and their energy levels), how the spectrum changes over time, and the loudness envelope (how the loudness changes over the duration of the sound) [1, 67, 68].

  • c. Potential uses in sonified biofeedback: Sonified biofeedback can use different timbres to convey or distinguish different categories of data because timbre enables distinguishing between sounds. For example, if designs want to differentiate signals between right and left heel strikes while walking, one electronic instrument sound object (e.g., piano) can be synthesized upon right foot heel-strikes and another (e.g., xylophone) can be synthesized upon left foot heel-strikes, while maintaining similar pitch and loudness perceptions.

• 4. Brightness

  • a. Perception: Brightness can be considered a prominent dimension of timbre in sounds that are not pure tones (e.g., not comprising of a single frequency). The more energy that is present in the middle and high frequencies, the “brighter” the sound. Similarly, we could call a sound which has most of its energy in the lower frequencies a “darker” sound [1, 67, 68].

  • b. Signal Attribute(s): Brightness is a function of the amount of energy in the upper-middle and high frequencies in a sound wave [1, 67, 68].

  • c. Potential uses in sonified biofeedback: Brightness could be modulated in sound designs in a similar fashion as increasing or decreasing pitch. For example, sound synthesis parameters can reweight the energy levels across the frequency spectrum of a sound wave. However, it may be less obvious to perceive a re-weighting of the energy across a sound’s frequency spectrum as compared to a shift in the sound’s fundamental frequency to change the perception of pitch.

• 5. Duration

  • a. Perception: Duration is the length of time that a sound is perceived [1, 67, 68].

  • b. Signal Attribute(s): The duration can be specified in units of time, such as seconds or milliseconds. Or, if musical terminology is being used, the duration can be specified in note-durations (e.g., “whole-note”, “half-note”, “quarter-note”, etc.) with respect to the tempo being used [1, 67, 68].

  • c. Potential uses in sonified biofeedback: One potential use of duration is to convey whether some feature of the conveyed data is present or not. For instance, if the data feature values are within preset thresholds, a sound can be emitted by the sound synthesis system and held at a constant loudness and pitch for the duration that the biological data values lie within the thresholds. Sound synthesizer parameters can control the duration of an emitted sound with on/off step functions.

• 6. Tempo

  • a. Perception: Tempo refers to the speed, or the rate of repetition of some prominent aspect of the sound [1, 67, 68].

  • b. Signal attribute(s): Tempo is often measured in beats per minute (BPM) [1, 67, 68]. As an analog of this concept in gait, if a person is walking at a rate of 60 steps per minute their footstep tempo would be approximately 60 BPM.

  • c. Potential uses in sonified biofeedback: Tempo can be used to sonify any repeating aspect of the user’s biological signal. This may be useful when sonifying variations in a person’s stepping rate. For example, a drum beat sound can be emitted for each heel-strike and depending on the movement training instructions, they can try to walk at a speed that matches a desired drum tempo.

• 7. Rhythm

  • a. Perception: A rhythm is a perceptually prominent repeating pattern of sound events [1, 38, 67, 68].

  • b. Signal Attribute(s): Note that a rhythm may have a variety of attributes, such as the density (the number of sound events within one repetition of the rhythm), and whether the sound events are regularly spaced within the rhythm (isochronous) or irregularly spaced (syncopated) [1, 38, 67, 68].

  • c. Potential uses in sonified biofeedback: Rhythms are typically repetitive, so they can be especially useful for sonifying repetitive movements that are cyclic, such as gait. Variations in a rhythm might convey that the user’s coordination during the movement is changing. For example, the user may become fatigued and change the sequence of sound patterns triggered by different sequencing of joint motions, thereby changing the perceived rhythm emitted by the sonified biofeedback.

• 8. Spatial positioning (or spatialization)

  • a. Perception: The perception of the location from where the sound is originating. By using multiple loudspeakers and audio signal processing, sounds can be made to be perceived as originating from a particular direction or location. The ability to localize a sound is influenced by many factors such as the environment (e.g., echoes) as well as the positioning of the head of the system’s user (e.g., localizing sounds behind the head is most difficult due to the shape of human ears) [1, 67, 68].

  • b. Signal attribute(s): Sound synthesis parameters such as level (perceived as loudness) can be controlled between one sound source (e.g., a speaker) or many sound sources to induce the perception of the sound originating from a particular location [1, 67, 68].

  • c. Potential uses in sonified biofeedback: There are a variety of techniques for spatializing sound. The simplest would be to place loudspeakers at a specific location in a space and then send sounds to individual loudspeakers. However, the illusion of a sound originating at a particular location can also be created when there is in fact no physical sound source at the specified location. The most common example of this is with a stereo system (e.g., with two loudspeakers or two earphone speakers), where a sound can be “panned” so that it can be perceived to be originating from a location between the speakers, rather than the position of the actual sources of the sound waves. Similarly, sound systems with many loudspeakers can be used to generate the perception of sounds emanating from multiple locations.

Selecting the mapping between biological data features and sound dimensions benefits from user-centered design practices because sound dimensions are not often perceived as independent. Though these are distinct sound synthesis parameters controlling sound dimensions from a technical standpoint, some of them are not perceived independently or linearly to the physical characteristic [69]. For example, pitch and loudness are perceived to be coupled such that when the energy levels change but the frequency remains constant, people perceive a pitch increase instead of only a loudness increase. In fact, as explained by Grond and Berger in their chapter in The Sonification Handbook [1], some sound designers overcome the coupled perception between pitch and loudness by using “proactive corrections” to adapt the amplitude of a specific frequency to support the perception of equivalent loudness, even though sound synthesis levels will be different.

Design considerations relevant for movement training

Thoughtfully considering which and how biological signals are mapped to specific sounds dimensions is paramount to the efficacy of sonified biofeedback in motor learning paradigms. Sonified biofeedback presents broad possibilities to map rich movement data with many different sounds that can be layered. However, this wide range of possibilities can concurrently challenge the field’s ability to systematically select parameter mappings that are easily understood and reproducible by future researchers.

Depending on the user and movement training context, if multiple signals are concurrently sonified, each signal may be able to be distinguished, partly because the human auditory system enables attending simultaneously to multiple sounds. For example, the “cocktail party effect” demonstrates this ability, as one might be attending to an interlocutor, and then hear one’s name mentioned nearby [70]. In this way, humans can distinguish multiple aspects of a complex sound environment. This ability to distinguish sounds can be leveraged in neurorehabilitation if it is helpful to make more engaging biofeedback that is musical, but still allows for readily detecting the change of one single, but important signal. For example, an error signal to indicate that the user strays from the desired motor pattern is likely still perceivable as the user is moving with other sonified signals, just as one’s name can be distinguished in a loud and crowded room. However, as mentioned, leveraging this ability for humans to distinguish multiple aspects in a sound environment within a sonified biofeedback design depends on the user and use-case. For instance, end-users with attentional issues, sensory sensitivities, or cognitive impairments are not likely to benefit from complex sound designs.

There are other design choices regarding how to provide augmented biofeedback to best support motor learning. Biofeedback can be provided concurrently with the movement (e.g., real-time) or after the movement, as terminal biofeedback (e.g., instant replay of video). This review article focuses on concurrent sonified biofeedback to explore the potential efficacy of leveraging auditory-motor coupling (as described in Sect. "Sonified biofeedback in balance and gait training"). Additionally, cues can focus on improving the knowledge of performance of the movement (e.g., joint angles) or knowledge of results of the movement (e.g., reaching the target or not). Each have their strengths and disadvantages, but knowledge of performance designs are beneficial to ensure that users find “good” solutions through providing adequate guidance [16]. In continuous state biofeedback, the sensory cues related to performance or results can be provided to the user, regardless of the signal’s value (e.g., conveying the current state of the signal). Alternatively, sensory cues can direct attention to specific targets to attain and errors to avoid through cues that are only provided intermittently, if the data feature’s value rises above or declines below specific thresholds, (e.g., “error biofeedback”). For example, biofeedback cues about a data feature like the knee joint angle can be provided continuously through time, or provided only when it is within a threshold range of values that the user is targeting during training. Note, a design can include a combination of continuous and error biofeedback through many-to-many mappings, which may be helpful to avoid startling the user with unexpected error biofeedback. Similarly, progressing users from continuous to error biofeedback may also be a strategic decision in the intervention design. Further discussion about the advantages and disadvantages of these and related design choices is provided in Sect. "Perspectives about sound design considerations".

During user-centered design processes for sonified biofeedback, it is also important to allow users to try prototype designs of the sonified biofeedback themselves. Viewing videos of the use of sonified biofeedback is distinctly different than first-person experience using it; First-person experience is vastly different than observed experiences. For instance, the sound design may be perceived as more responsive, enjoyable, and helpful for the user than it is for the bystander, or vise-versa [1].

Methods

The search keywords, number of results, and screening rules are included in Fig. 2. First, Scopus was used to identify articles using sonification for balance and gait biofeedback. Then, Web of Science was used to find unique results relative to the first Scopus search. Finally, Pubmed was used to find additional unique results, relative to prior Scopus and Web of Science searches. The most recent database search for Scopus, Web of Science (Core Collection), and Pubmed was completed, compiled on October 21, 2024. The searches, tabulation, and assessment of the results were performed by author AZ and reviewed and verified by co-authors.

Fig. 2

Literature search terms, number of results, and stages of selecting the included papers

Scopus was selected as the primary database because it included the largest journal holdings across engineering, design/art, and medical sources. There were two Scopus search queries performed. The first Scopus search yielded 203 results by querying the following keywords (including Boolean and wildcard terms) within the title, abstract, or keyword list: “(audio OR sonification) AND *feedback AND (balance OR posture)”, where wildcard “*” indicated that any prefix can be used prior to “feedback”, including, but not limited to “biofeedback” or “feedback”. The second Scopus search used the following search terms, looking within the title only: “(audio* OR auditory OR sonification) AND (*feedback OR rehabilitation) AND (balance OR postur* OR gait OR “motor learning”))”, where the search query “wildcard” (denoted by an asterisk *) was used so that any prefix or suffix prior to or after the wildcard can be used (i.e., “audio*” allowed for “audio-biofeedback” or “audiobiofeedback” to be valid results, just as “*feedback” allowed for “biofeedback” or “bio-feedback” or “audiofeedback”, etc. to be valid results, and “postur*” allowed for “postural” or “posture” to be valid results). The second Scopus search yielded 151 results (including 22 duplicates from the first Scopus search). 354 unique results were yielded by the Scopus search.

Two equivalent search queries were performed with Web of Science (Core Collection) after Scopus. The first search yielded 201 results by querying the following search terms: “(audio OR sonification) AND feedback AND (balance OR posture)(All Fields)”. The second search used the following search terms within the title only: “(audio* OR auditory OR sonification) AND (*feedback OR rehabilitation) AND (balance OR postur* OR gait OR "motor learning") (Title)”. The second search yielded 104 results (including seven duplicates of the first Web of Science search). Of the total 305 unique results of both Web of Science searches, 191 were duplicates from the prior Scopus search, and were removed.

Finally, two equivalent search queries were performed with Pubmed after Web of Science (Core Collection). The first search yielded 71 results by querying the following search terms within the title and abstract only: “(audio[Title/Abstract] OR sonification[Title/Abstract]) AND feedback[Title/Abstract] AND (balance[Title/Abstract] OR posture[Title/Abstract])”. The second search used the following search terms, looking within the title only: “(audio*[Title] OR auditory[Title] OR sonification[Title]) AND (feedback[Title] OR biofeedback[Title] OR rehabilitation[Title]) AND (balance[Title] OR postur*[Title] OR gait[Title] OR "motor learning"[Title])”. The second search yielded 50 results (including three duplicates). Of the 118 unique results from both Pubmed searches, 114 were duplicates from the prior Scopus and Web of Science searches and were removed.

Full results of the papers and conference proceedings that were identified and screened are included in a supplemental file (see Supplemental Document). Note, these database search results may not include relevant older, non-digitized articles that are not included in journals' electronic collections. Additionally, 35 articles that were referenced by others or previously known to the authors that were not results of the database searches are also indicated in the Supplemental Document and included in the “Previous Studies” section, when applicable.

After the unique search results were tabulated, they were screened through three phases: abstract screening to initially remove non-relevant articles, full-text screening to confirm the fit of the topic -using sonified biofeedback to improve upright balance, postural control, or gait–and that it was written in English, and a third screening phase to include papers with statistical analyses of biomechanical effects or clinical outcome measures from human user experimental tests. The goal was to select papers that used unimodal sonified biofeedback to improve unassisted upright activities of standing, walking, or running during at least one experimental condition. Thus, experimental papers that studied sonified biofeedback in combination with other feedback or assistive devices (e.g., multimodal feedback, audio-visual biofeedback, use of crutches, exoskeletons, etc.) were excluded for this review (for example, [64, 71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89]). Finally, prior methods/design papers that do not include statistical analyses of biomechanical or clinical outcome measures are referenced throughout this review paper [14, 30, 40,41,42, 46,47,48,49,50,51,52,53,54,55, 57,58,59,60,61,62,63,64, 86, 89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110], but are not included in the analysis of prior experimental studies in Sect. "Previous studies".

Previous studies

Placement of papers across fields and subdivision into categories

Due to the nature of this research topic, papers and conference proceedings were spread across technical and practice-based fields and communities (computer science, biomechanics, sonic arts, etc.). In this review, previous studies testing the effects of sonified biofeedback designs have been split between their focus on understanding how sonified biofeedback affects either feet-in-place balance activities (such as standing still on a foam surface) or dynamic gait activities (such as walking or running). This division may be helpful in comparing rehabilitation practices and analyses that prioritize different aspects of upright balance and gait skills (e.g., stationary vs. dynamic balance).

The search results were parsed into separate feet-in-place and gait tasks sections to facilitate comparisons within and across the two movement task types [30]. Feet-in-place balance tasks and training goals are distinctly different that gait tasks from a clinical perspective. Additionally, from a design perspective, there may be features in the biological data signal specific to feet-in-place balance or gait tasks that could prompt specific sound mapping design choices. For example, during gait, there are cyclically changing data signal features that are not typically present during feet-in-place balance tasks. Further, the feet-in-place studies tended to be published earlier than the gait tasks, potentially due to advancements in biomechanical sensors, processors, and/or computerized sound technology. Of the 20 feet-in-place designs, nearly half (nine; 45%) were published before 2011 as compared to about a quarter (seven; 24.1%) of the 29 gait studies. This trend towards gait studies is promising because during gait, sonified biofeedback may have distinct advantages when facing a visual display could distract from visual processing required to navigate an environment (as postulated in Sect. "Sonified biofeedback in balance and gait training").

The following Sound Designs section (Sect. "Sound designs") summarizes the sensors used and sound mapping decisions for feet-in-place balance (Sect. "Feet-in-place balance training") and dynamic gait (Sect. "Gait training") training tasks. Then, the following Previous Experimental Designs and Findings section, (Sect. "Previous experimental designs and findings") summarizes the experimental designs employed, populations of interest, and outcomes of these papers.

Studies varied greatly in methods, research questions, and populations studied, which limits the ability for systematic reviews or meta-analysis reviews to capture the state of this rehabilitation practice in a meaningful or inclusive way. Despite this variety, overarching summaries of each of these categories (feet-in-place balance and gait training tasks) are provided to demonstrate the promise of sonified biofeedback in rehabilitation practice and to support the perspectives discussion section that follows (Sect. "Perspectives for future opportunities for sonified biofeedback").

Sound designs

Sound designs for feet-in-place balance training

The sound designs used in 20 papers studying feet-in-place balance are summarized in Table 1 [111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130]. This section provides a brief overview to contextualize experimental outcomes from these papers that are presented in Sect. "Feet-in-place balance training".

Table 1 Summary of the sound designs used in 20 experimental studies for feet-in-place tasks

Sound dimensions used in designs: Most (19 of 20) designs used pitch and/or loudness mappings to convey the biomechanical measure of interest [111,112,113,114,115,116,117,118,119,120, 122,123,124,125,126,127,128,129,130]. Only one design did not employ pitch and loudness sound mappings and instead, used brightness, pulse speed, and dissonance of a synthesizer [121]. Most (12 of 19) of the mapping designs using loudness used it to convey right-left body sway with right-left spatialization in earphones or spatially-oriented speakers surrounding the participant [111, 114,115,116,117,118, 122, 123, 127,128,129,130]. Loudness and pitch were often mapped together, possibly to emphasize the importance of the feedback as the body sway increased [111, 114,115,116,117,118, 128, 129].

Ten of the 20 papers in this category used designs similar to or exactly the same as a design first described by Dozza, Chiari, and Horak in 2004 [116]. This design mapped the direction of body sway measured by an inertial measurement unit with a sigmoid mapping to pitch and/or volume. Eight papers used the same design [111, 112, 114,115,116,117,118, 128, 129] and another paper by Dozza et al. [113] compared a few related designs (though, it is difficult to determine if this paper [113] used the same exact design as the others). The design first described by Dozza, Chiari, and Horak in 2004 [116] used a person-specific threshold or “target zone” [111, 112, 114,115,116,117,118, 128, 129] to personalize the design, as did one other paper [126] in the feet-in-place balance training category.

Sound design specifications: Only one paper reported estimates of system latency [118], and eleven papers did not include enough sound details to fully describe the design [111, 113, 119,120,121,122,123, 125,126,127, 130] For example, the word “sound” was used to describe the sound, rather than something more specific, like “pure sine tone”. Lack of these technical and design details can hinder the ability to replicate these designs or fully contextualize the results from their use in movement training. In contrast, Dozza, Chiari, and Horak [116] provided the equations for the sound design to allow a full understanding of the design’s sound synthesis parameters and facilitate replication in future research.

Sound designs for gait training

29 papers focused on evaluating how sonified biofeedback can improve different aspects of gait: 11 designs were intended to improve balance and symmetry at the whole-body level (Table 2) [131,132,133,134,135,136,137,138,139,140,141], eight designs were intended to improve spatiotemporal factors of walking or running (e.g., cadence, footfall biomechanics; Table 3) [142,143,144,145,146,147,148,149], and ten designs were intended to improve body segment-specific (e.g., foot) or joint-specific (e.g., ankle) biomechanics (Table 4) [150,151,152,153,154,155,156,157,158,159]. This section provides a brief overview to contextualize experimental outcomes from these papers in Sect. "Gait training".

Table 2 Summary of the sound designs used in 11 experimental studies to improve balance and/or symmetry in gait tasks

Table 3 Summary of the sound designs used in eight experimental studies to improve spatiotemporal factors of gait tasks

Table 4 Summary of the sound designs used in ten experimental studies for gait tasks to improve specific details of the movement at the body-segment or joint level

Sound dimensions used in designs: Among gait studies, there was a wider variety of sound designs, as compared to the feet-in-place balance designs. The mixture of sound designs ranged from simple beeps and pitch-loudness mappings to tonal chord progressions (e.g., “chromatic glissando” [144]), naturalistic sounds (like walking in the snow [160]), and even designs that incorporated preferred music soundtracks (e.g., [137, 149, 156]). In gait studies, six designs included personalizing the design to the participant and/or collaborating clinician [137,138,139,140, 156], using participant-specific settings, including using baseline cadence measurements [158, 160] or adapting to the participant cadence during training [146, 147, 156]. Notably, Gomez-Andres et al. [142] amplified the natural sound of walking in one of the training conditions, to compare it to sonified biofeedback designs that change the brightness by emphasizing low frequency or high frequency content of the sound [142].

Sound design specifications: In gait studies, fewer than half of the studies (13 of 29) included full musical notation details, sound-mapping equations/models, and specific sound synthesis settings [132,133,134, 142, 144,145,146,147, 149,150,151, 156, 158] which allowed a full description of the sound design. Additionally, four of 29 studies provided latency estimates [132, 134, 142, 145]. Three papers included sound or video samples as supplementary materials [145, 151, 156]. Thus, only a small percent of the papers shared technical details that are helpful for building upon in future research.

Previous experimental designs and findings

Feet-in-place balance training experimental designs and findings

The experimental designs and findings of 20 papers using sonified biofeedback to improve feet-in-place balance are summarized in Tables 5 and 6.

Table 5 Summary of the 11 experimental studies for feet-in-place tasks to improve balance of healthy young adult participants

Table 6 Summary of the nine experimental studies for feet-in-place tasks to improve balance of healthy older adults, people with ataxia, people with bilateral vestibular loss, or users of bilateral cochlear implants with cochleovestibular dysfunction

Participant characteristics: Sixteen of 20 papers focused on healthy individuals (young or older adults). Table 5 papers included only healthy young adults [111,112,113, 119,120,121,122,123,124,125,126] and Table 6 papers included older adults [114,115,116, 127, 128], people with ataxia [129], people with bilateral vestibular loss [117, 118], or users of cochlear implants with cochleovestibular dysfunction [130]. All but one [129] of these papers included healthy young or older participants (sometimes included as a control group). Twelve of these studies included fewer than 20 participants.

Goals of feet-in-place balance training studies: Predominantly, these studies sought to reduce the amount of body sway during feet-in-place balance tasks with varied sensorimotor or stance contexts. Two papers strayed from this goal [121, 126]. For example, V dos Anjos et al. [126] aimed to understand if sonified biofeedback can facilitate selectively reducing calf muscle activation levels [126]. Additionally, Pirini et al. [128] also compared cortical activations during sonified biofeedback and fake sonified biofeedback [128]. Some studies compared across biofeedback modalities or movement training procedures, with comparison/control groups [117, 118, 122, 124, 129, 130] or comparison conditions within participant group [119, 120, 125] including no biofeedback, visual biofeedback, background music/sounds, or dose-equivalent training.

Participant instructions and familiarization with sonified biofeedback: Experimental details like how participants were cued to balance while using sonified biofeedback or how they became familiarized with the sonified biofeedback were provided for most feet-in-place balance studies. Thirteen papers included some description of a familiarization protocol, with varied levels of detail [112, 113, 116,117,118,119,120,121, 126,127,128,129,130]. Dozza et al. [118] provided the most details about the familiarization process used: (1) providing practice time with voluntary postural swaying while listening to the sound biofeedback, (2) evaluating if the participants could achieve a constant 400 Hz tone with their postural control, and (3) providing three practice trials for each experimental condition (e.g., standing with eyes open/closed, etc.) before the experiment began [118].

Experimental Designs: All but two feet-in-place studies [119, 120], used cross-sectional (within one session) experimental designs and all papers compared balance measures during or after sonified biofeedback training to person-specific balance baseline measures before sonified biofeedback. One longitudinal study in this category by Hasegawa et al. [119] randomized participants into groups (visual or sonified biofeedback) and evaluated their balance across multiple days. This included a first and second day with pre-biofeedback (baseline), training (during biofeedback), and post-biofeedback and a fourth day that only included a retention test without biofeedback [119]. The other longitudinal paper by Hasegawa et al. [120] also used a separate day for a retention test during a lab visit on the third day, but the training period was shorter [120]. None of these feet-in-place studies evaluated balance using a different task to see if balance improvements transferred to a different task.

Reported Outcomes and their Task Contexts: Most of the feet-in-place studies (19 of 20) reported at least one positive effect of sonified biofeedback on balance measures in one or more stance or perturbation condition relative to each participant’s baseline performance (“pre”) or across groups with and without sonified biofeedback. Six studies included a control group of participants [119, 120, 122, 124, 125, 129], that completed a different training that did not include sonified biofeedback. In these studies, either training doses were equivalent [129], or background music was used instead of sonified biofeedback [124], or visual biofeedback was used instead of sonified biofeedback [119, 120, 125]. Five of these six studies with control groups reported greater improvements in outcome measures with sonified biofeedback than the control group [119, 120, 122, 124, 129]. One additional study included a within-subject visual biofeedback condition and found better improvements in center of pressure sway outcomes using (a sigmoid design) sonified biofeedback versus visual biofeedback [111]. Of the 13 studies without a control group or control condition, 13 reported an improvement in at least one balance measurement with respect to a person-specific baseline [111,112,113,114,115,116,117,118, 123, 126,127,128, 130].

Sixteen of these 20 studies challenged balance more than standing in place with eyes open during the use of sonified biofeedback [111,112,113,114,115,116,117,118, 121,122,123,124, 127,128,129,130]. These challenges included sensory modifications such as standing on foam with the eyes open or closed, or support surface perturbations. Others challenged users to stand in configurations that are more challenging, such as single support or “tandem stance” with one foot directly in front of the other. Only one of these 16 papers reported a decline in an outcome measure of interest during large perturbations in one direction. In this one paper, Benjamin et al. [130] included a group of children and young adults with cochleovestibular dysfunction and a typically developing healthy control group. They found mixed results: the sonified biofeedback improved their stability during some anteroposterior treadmill perturbation types during biofeedback versus without biofeedback. However, they also reported decreased stability during some mediolateral and anterior treadmill perturbations, depending on the perturbation size, in those with cochleovestibular function during biofeedback versus without biofeedback [130]. Additionally, during these challenging sensory contexts (eyes closed, standing on foam, etc.), sonified biofeedback was thought to provide pertinent information about the balance state (as in “sensory substitution” [118]) which is useful for future applications with persons with sensory issues. In cases that the stance was challenged in end-users with intact sensory systems, the positive results may support the interpretation that the information provided by sonified biofeedback was helpful during more challenging balance conditions.

Few studies provided intermittent biofeedback, which may limit the potential adverse effect of depending on the presence of biofeedback (the guidance effect). One study by Sánchez-Tormo et al. [122] compared outcomes between one group receiving 100% of trials with sonified biofeedback and another group receiving two of three trials with sonified biofeedback while balancing on a seesaw that was unstable in the anteroposterior direction [122]. It was thought that this design would mimic prior findings of “faded” biofeedback, which may help retain improvements in the absence of the sonified biofeedback. In this study, they found retained improvements in the median frequency of the power spectral density in both sonified biofeedback groups compared to the control group with no biofeedback, but no improvements were measured in the time to stability balance measure [122]. Also, pertinent to minimizing the chances of the guidance effect, seven of these 20 feet-in-place balance studies provided intermittent feedback only when the biomechanical measure exceeded a target value, thus providing an “error” sound signal [119, 120, 122, 123, 125, 127, 130].

Perspectives regarding the outcomes of feet-in-place balance training studies: The prior studies that used sonified biofeedback to improve balance during feet-in-place tasks provide promising evidence of balance improvements facilitated by training with sonified biofeedback. Positive outcomes were demonstrated by adolescents, healthy younger adults, healthy older adults, people with bilateral vestibular loss, and people with cerebral ataxia. Further, studies that varied task difficulty found that the sonified biofeedback condition facilitated the largest improvements in balance when sensory conditions or body configurations were more challenging (e.g., eyes closed, standing on foam, etc.). This is sensible because sensory feedback is augmented in sonified biofeedback, which may become more useful as tasks become more challenging (i.e., sensory substitution [118], or reweighting sensory signals by placing more weight on the augmented biofeedback in sensorimotor integration). Overall, it is encouraging to know that using sonified biofeedback improved outcome measures in most of these studies.

Unfortunately, no prior feet-in-place balance studies evaluated the effects of balance training with sonified biofeedback during tasks other than the training task. Thus, the ability for the feet-in-place sonified biofeedback training to transfer to other tasks is still unknown. In other words, without more longitudinal studies or use of evaluation tasks that are different than the training tasks, it is difficult to contextualize the clinical impact of these otherwise promising findings. Finally, it was notable that some studies provided commentary on the person-specific responses to sonified biofeedback relative to an individual’s sensory organization [118] or attitudes towards the sonified biofeedback [121]. This personalized lens may assist in future clinical translation [161].

Gait training experimental designs and findings

The experimental design and outcomes of 29 papers studying the effects of sonified biofeedback on different aspects of gait [131,132,133,134,135,136, 142,143,144,145,146,147, 150,151,152,153,154,155, 157, 159, 160], running [149, 156, 158], or overall balance control [137,138,139,140,141] are included in Tables 7, 8 and 9. Table 7 summarizes studies that sought to improve balance or symmetry during gait, Table 8 summarizes studies that sought to improve spatiotemporal aspects of gait (e.g., cadence), and Table 9 summarizes studies that sought to improve segment or joint-specific biomechanics during gait.

Table 7 Summary of the 11 experimental studies of gait tasks to improve balance and symmetry

Table 8 Summary of the eight experimental studies of gait tasks to improve spatiotemporal factors of gait or running

Table 9 Summary of the ten experimental studies of gait tasks to improve movement details at the body-segment or joint-level

Participant characteristics: In contrast to the prior feet-in-place balance studies (which predominantly included healthy participants), only six of these 29 gait studies focused on healthy participants [145, 149, 150, 156, 158, 159]. In addition to healthy adults, participants for gait studies ranged from people living with or recovering from: Parkinson’s disease [137, 139, 144, 146,147,148, 151], Progressive Supranuclear Palsy [140], Multiple Sclerosis [143], Stroke [131, 135, 136, 142], spinal cord injury [153, 154], bilateral vestibular loss [134], joint osteoarthritis or lower extremity joint replacement [132, 133], ankle joint issues [155, 157], mild traumatic brain injury [138], posttraumatic otolith disorders [141], and children with dynamic equinus (toe walking) [152]. Similar to the feet-in-place studies, gait studies included as few as one participant and fifteen of 29 papers included fewer than 20 participants. However, the paper with the largest sample size included 240 participants [132], which far exceeds the sample size of the feet-in-place studies.

Goals of gait training studies: Gait training studies focused on a broader set of research questions about sonified biofeedback, fitting with the wider variety of participants relative to the feet-in-place balance training studies. These 29 gait training studies ranged from studying the effects of sonified biofeedback on walking or running speed or cadence [142, 143, 145, 149], gait symmetry [132, 133, 136, 160], step length variation [144, 159], balance [134,135,136,137,138,139,140], kinematic patterns [151,152,153,154, 159], lower extremity kinetic patterns [150, 155,156,157,158], and clinical outcome measures, such as the Six-Minute Walk Test distance [148]. A handful of papers focused on the ability to transfer improvements from the training task to some other tasks to better understand motor learning effects [131, 136,137,138,139,140, 148]. Similarly, some of the sonified biofeedback training progressions included transitioning from feet-in-place balance tasks towards gait tasks [134, 139, 140].

Experimental Designs: In contrast with the feet-in-place studies, a larger portion of these gait studies included longitudinal experimental designs (14 of 29) [131, 133, 135,136,137,138,139,140, 146, 148, 152, 154, 155, 157], including one “preliminary randomized control trial” [131] and three randomized controlled trials [132, 135, 155].

Reported Outcomes and their Task Contexts: 28 of 29 of the gait studies reported a positive effect of sonified biofeedback on one or more measure relative to baseline performance or compared to a control group without sonified biofeedback. Of the 14 studies with a control group [131, 132, 133, 135, 136, 138, 141, 143, 146,147,148, 153, 155, 157], 13 demonstrated improvements in outcomes greater than those observed in the control group [131, 132, 135, 136, 138, 141, 143, 146,147,148, 153, 155, 157]. None of the 14 studies with a control group reported declines in outcome measures. Additionally, two of two studies that compared sonified biofeedback to traditional or verbal instruction demonstrated improved outcomes from sonified biofeedback when compared to the alternative [149, 151]. Despite the overall positive effects reported, five of 29 studies reported a negative effect on at least one measured outcome [133, 140, 142, 145, 159]. Only one of these five studies with negative effects did not report any other improvements [145]. Two of these four reported a negative effect on gait speed and/or cadence [145, 159]. Overall, four studies of 29 reported improved psychosocial measures [101, 139, 140, 160], such as positive affect [101] and mental state [160] from training with sonified biofeedback.

Fifteen of 29 studies used intermittent biofeedback either to express a desired or undesired biomechanical state [131, 143,144,145,146,147,148,149, 152, 153, 155,156,157,158,159]. The use of intermittent biofeedback may have supported the ability for the observed improvements to transfer or be retained in evaluation tasks that differed from training tasks. Outcome measures varied greatly across these 29 gait studies. Six studies used validated clinical assessment tools to evaluate balance, gait, and psychosocial factors [136,137,138,139,140, 148]. Relatedly, seven studies included evaluating effects of biofeedback during a task other than the task(s) used for training [131, 136,137,138,139,140, 148].

Notable research by Uchitomi et al. [146, 147] compared the “fractal scaling” of gait timing fluctuations between using sonified biofeedback and rhythmic auditory stimulation, finding that the adaptive rhythmic cueing provided by sonified biofeedback allowed people with Parkinson’s disease to restore healthy levels of fractal variations in their gait [146, 147]. One randomized controlled trial by Owaki et al. [135] sonified heel-to-toe plantar pressure with a musical scale and found the 2-week intervention decreased whole body angular momentum in the frontal plane in the sonified biofeedback group. This study demonstrated the utility of sonifying a data feature that is distinctly different than the desired outcome measure [135]. Like the feet-in-place balance studies, few (four) gait studies evaluated both biomechanical effects of sonified biofeedback and quantitative rankings of personal affect or mental state when using the sonified biofeedback to analyze perceptual outcomes [137, 142, 145, 160].

Many longitudinal gait studies reported positive effects on validated clinical measures. For example, Nicolai et al. [140] included eight people with supranuclear palsy in a 6-week trial in which participants worked with physical therapists to personalize their progression through standard balance training activities while using a sonified biofeedback device during sessions that lasted about 45 min and took place three times per week. Clinical assessments were completed within 1 week before participation in this 6-week trial (T1), within 1 week after the trial (short-term retention, T2), and 4 weeks after (longer-term retention, T3). Notable findings included that the Berg Balance Score improved significantly from T1 to T2 and T1 to T3, despite a significant decline between T2 and T3, though no significant changes were detected across these timepoints in other clinical assessments such as the Timed Up and Go test [140]. Similarly, Campbell et al. [138] used biweekly sessions for 6 weeks of vestibular rehabilitation physical therapy for people with mild traumatic brain injury in two groups: one with sonified biofeedback and a control group without sonified biofeedback. Both groups improved clinical measures of balance and sensory organization, but the statistical effect sizes were greater in the sonified biofeedback group for specific central sensorimotor integration scores including sub-scores that indicate improvements in “motor activation” and decreased “time delay” [138].

Perspectives regarding the outcomes of gait training studies: The 29 gait training studies supported the potential of positively influencing gait training with sonified biofeedback across diverse end users, many of whom had movement disorders. It is encouraging that many studies used clinical outcome measures to evaluate effects of sonified biofeedback training and that all studies with a control group [133, 134, 136, 138, 141, 143, 146,147,148, 151, 153, 155, 157] reported positive outcomes and all, except one [133] found positive outcomes exceeding those observed in the control group. More gait studies than feet-in-place studies were longitudinal (including a few randomized controlled trials), and more end-users in the gait studies had movement disorders than those in the feet-in-place studies. The gait training study outcomes support an expanded future use of sonified biofeedback to facilitate positive clinical outcomes and health impacts.

Perspectives for future opportunities for sonified biofeedback

Building from the evidence shared in the previous sections about the positive outcomes from using sonified biofeedback for balance and gait training, in this section, we offer additional perspectives about prior research and ideas for future research. First, we provide our perspectives about the outcomes of the prior research reviewed. Then, we discuss opportunities for sonified biofeedback, including considerations for sound and experimental designs, towards advancing this promising research. Finally, we guide readers towards supplemental reading of related reviews, opinion, and methods papers that may be of interest.

Feet-in-place balance training studies were more homogenous in their sound designs and training goals. Most focused on reducing body sway by using designs that conveyed sway through sounds of varied pitch and loudness. It is important to acknowledge that all feet-in-place studies did not assess the influence of sonified biofeedback on the performance of tasks other than the training tasks, so this contextualizes the early-stage nature of these outcomes with respect to possible clinical translation. Additionally, most of these studies used sonified biofeedback in training tasks that challenged the sensory system (e.g., eyes closed, standing on foam, etc.), so sonified biofeedback could be used to substitute for the less reliable or less available physiological sensory signals. In contrast to feet-in-place balance training studies, gait training studies used heterogeneous and more musical sound designs. They used these varied sonified biofeedback designs to train a larger variety of movement qualities (e.g., gait symmetry, improved balance, etc.) and often evaluated the effects from training by using evaluation tasks that differed from the training tasks, including typical clinical outcome measures (e.g., Six-Minute Walk Test, Timed Up and Go, etc.).

This review provided an overwhelmingly positive account of the effects of sonified biofeedback on balance and gait training. 47 of 49 studies shared improvements in at least one outcome measure, with only seven sharing any declines in outcome measures during or after sonified biofeedback with respect to prior sonified biofeedback. Of the seven papers with at least one negative result [125, 130, 133, 140, 142, 159], five also reported positive results [130, 133, 140, 142, 159], and the negative results shared were not surprising or overly adverse when discussed in context with sound design or study design.

Here, we offer possible interpretations to contextualize the reported negative effects of training with sonified biofeedback. (1) The negative result reported by Takeya et al. [125] was that the audiovisual group outperformed the visual biofeedback, sonified biofeedback, and control groups in a study that used a buzzer as the sonified feedback to indicate departure from a target zone of postural alignment. Only the audiovisual group demonstrated significant improvements in performance relative to the control group [125]. (2) Benjamin et al. [130] shared that during medium and large perturbations of the instrumented treadmill in specific directions (mediolateral and posterior), there was increased area under the curve for average head, torso, and feet rotation and displacement timeseries during biofeedback relative to baseline perturbations without biofeedback in persons with cochleovestibular dysfunction. They also shared positive results for anterior perturbations. As described by Benjamin et al. [130], one possible explanation for the negative results in some conditions could be related to the sound design. The sonified biofeedback increased loudness for mediolateral head tilt in the direction of the tilt, decreased pitch for forward head tilt, and increased pitch for posterior head tilt. It could be likely that the most perceptible and usable context for the biofeedback was during anterior perturbations, when the head would have a relative posterior tilt, triggering the higher pitched biofeedback to avoid [130]. (3) Nicolai et al. [140] shared mostly positive results after a 6-week intervention using sonified biofeedback within physical therapy sessions for persons with progressive supranuclear palsy (post- vs. pre- intervention demonstrated improved Berg Balance Scale, Parkinson’s disease questionnaire (PDQ-39) communication sub-score and follow-up vs. pre- demonstrated improved Berg Balance Scale, PDQ-39 summary index, cognition, and communication sub-scores). However, there was a significant decline in the Activities-specific Balance Confidence scale at post versus pre intervention and at follow-up and there were no significant differences in this balance confidence scale relative to pre-intervention. Nicolai et al. [140] offered a description that the increased awareness of balance deficits brought about during the training could have been a factor in this unexpected decline in balance confidence [140]. (4) Gomez-Andrez et al.  [142] reported a reversal in the gait asymmetry of persons with chronic stroke during high frequency sonified biofeedback use, but also reported improved symmetry and improved (increased) heal contact forces during natural and low frequency sonified biofeedback relative to baseline. Thus, this negative result may be specific to the high frequency design [142]. In healthy young adults, (5) Horsak et al. [145] reported decreased gait speed and cadence during sonified biofeedback when they asked participants walk with a constant speed [145] and (6) Tomita et al. [159] reported decreased cadence post sonified biofeedback training, amidst other desired improvements like increased ankle dorsiflexion, step length, etc. [159]. Decreased cadence and or gait speed during or after biofeedback training is an understandable (and possibly temporary) adaptation when attempting to learn to improve the data feature that is conveyed by biofeedback. Finally, (7) Reh et al. [133] reported increase stride length variability and stride time variability during week two of sonified biofeedback use versus the first week during sonified biofeedback use, despite also observing desired increases in stride length and stride symmetry [133].

Perspectives about sound design considerations

Overall, the studies included in this review varied greatly in sound designs, particularly in the gait training studies. Of the feet-in-place balance studies, all but one study used pitch and loudness sound dimensions to convey estimates of body sway. In the gait studies, designs tended to incorporate more musical elements, and were described more often using musical notations. Additionally, it was striking how many prior studies included in this review designed the sonified biofeedback in ways that could facilitate translation from the lab to the clinic or gym settings. Though we did not discuss details about the sensor systems used, a vast majority of studies used wearable sensors, or sensor data features that could readily translate to wearable systems (Tables 1–4). This is an exciting prospect, given that most studies described positive outcomes from training with sonified biofeedback.

Though sonified biofeedback offers the ability to convey multiple biological measures concurrently, or to make biofeedback musically interesting, decisions about sound mapping complexity are critical. In this review, the gait sound designs were generally more complex and musical than the feet-in-place designs, which primarily used simple pitch and loudness mappings to convey body sway. The gait sound designs could have been designed to be more complex to match the more complex and drastically time-varying nature of biomechanical measurements during gait. For example, many biomechanical measures oscillate greatly from step to step during gait. However, if the user is allocating cognitive resources to understand more complex designs while performing more complex movements, this could be a strength or a risk. On one hand, challenging both cognition and motor behavior concurrently during balance interventions could yield even better preparation for ecological conditions [162, 163] than if cognitive loads are lower during training than during real-world mobility. However, if cognitive resources are overtaxed when using a complex design, there is a risk for confusion or other adverse responses.

Most studies included in this review used a combination of intermittent and continuous cues. Future studies that compare across intermittent error versus continuous state feedback designs can also advance our understanding about whether positive or negative reinforcement or punishment is most appropriate across individuals or as people progress from novice to more skilled. Positive reinforcement has been identified as more effective [164, 165]. However, even though error biofeedback can likely align with negative punishment, error biofeedback may importantly reduce dependence on biofeedback [166]. This is attributed to error biofeedback’s intermittent nature, in that measures only exceeding an error threshold are sonified, mitigating the guidance effect [167]. Progressing from continuous state biofeedback towards intermittent error biofeedback or fading the number of practice trials with sonified biofeedback [168], are likely the best approaches to support skill acquisition. Only two of 49 papers discussed “fading” biofeedback [122, 132], which prompts future research to evaluate the effects of fading the biofeedback if more longitudinal studies are conducted.

There are exciting avenues to improve sonified biofeedback approaches by providing “instructional” sounds as well as embedding sonified biofeedback in games. Instead of sonified biofeedback, Young et al. (2013) provided instructional sounds so users can “[perceive and reenact] spatiotemporal characteristics of walking sounds” [90, 133]. These instructional sounds can provide what the sonified biofeedback should sound like if the user achieves the movement training goals. Relatedly, Reh et al. [133] alternated between practicing gait with playing instructional sounds and practicing with sonified biofeedback. This example of instructional sound opens another avenue for movement training paradigms to blend instructional sound cues trials with sonified biofeedback concurrently. Future research can explore if instructional and sonified biofeedback can blend in the same trial. For instance, if a physical therapist wants participants to keep walking at a specific speed while working on their posture, there could be a non-sonified drum beat to “instruct” a specific walking cadence while conveying the posture angle through sonified biofeedback. An adjacent example of this proposed technique was included in this review: in a case-study, Szydlowski et al. [137] provided sonified biofeedback of each step taken in addition to optional background music, which was added depending on the preference of the clinician guiding the movement practice if they believed it would provide more “encouragement” during practice [137].

Building upon successful “exergaming” (exercise games) approaches in rehabilitation [169,170,171], it would be exciting for the field to incorporate sonified biofeedback into game constructs. By embedding sonified biofeedback in games, Avissar et al. [93] explored the possibility to embed a postural control training system within a “limbo” music game and made initial (non-statistical) comparisons between this limbo game and using a pitch and loudness-based design described by Chiari et al. [115]. Avissar et al. [93] discussed that designing the interaction to be motivational and enjoyable is critical when considering the types of sounds used (e.g., musical vs. pure sine tones) and the structure of the interaction (i.e., within the motivational construct of a game vs. repetitions without reward systems in place).

A few papers included in this review shared design frameworks that may be helpful to converge to a design customized to end-users. Lorenzoni et al. [149] used a phased design approach by including perceptual studies with end-users while they were performing the task of interest [149, 172]. Similarly, Kantan et al. [51] used ecologically valid sounds and iterative rounds of qualitative feedback through focus groups and discussions with physical therapists and end-users to converge on designs that were positively received by a sample of people with hemiparesis following stroke [110]. By using ecological sounds and sound mappings, interacting with the designs may be more intuitive and informative [14, 173]. Kantan et al. [51] also provided specific quotes from users as part of their design process, which allows readers direct access to their design process. Since user-centered design is often unpredictable, our overall design process suggestion is to customize and thoroughly document the design process with the end-users, facilitators (e.g., physical therapists, caregivers, health aides, etc.), and movement(s) of interest [174]. What is of the utmost importance is to share the quantitative and qualitative design decisions along with the publication (perhaps in supplemental files, as necessary), so that future researchers can build upon prior work.

We have several suggestions about including sound design details in future publications to improve the ability for the field to build upon previous work. In our review, few papers included sound or video samples or supplemental material to adequately describe the sound design [142, 144, 145, 145,146,147, 151, 156]. In the future, if more groups share sound design details, including a description of design choices (as in [51, 103, 110, 149, 172]), we can advance the field more cohesively. Another important measurement is the overall system latency and how that may affect specific movement training applications (only six of the 49 reviewed papers included latency estimates). It is our position that latency should be evaluated empirically while using conditions the same as the experimental conditions. For example, rather than only adding the reported values of latency for each measurement, processing, and sound generation components (e.g., those reported by the manufacturers), our group also measures latency empirically. Empirical estimates are suggested because manufacturers share latency estimates from specific hardware and software configurations, but experimental conditions may increase latency due to differences in hardware, communication, and software settings (e.g., data transmission across systems, differing computer specifications, Wi-Fi environments, etc.). More specific design descriptions will also facilitate more accurate interpretation and contextualization of responses to the sonified biofeedback.

Perspectives about experimental designs

The majority of papers included in this review demonstrated positive outcomes of training with sonified biofeedback. Positive effects from using sonified biofeedback were shared in 47 of 49 prior papers, across end-users with a wide spectrum of movement skills and motor control issues. Despite these encouraging findings, for feet-in-place balance training studies, most of these positive outcomes were measured during training or during the same training task used, without evaluating if skills learned can transfer to other meaningful tasks. For the feet-in-place balance training studies, balance with sonified biofeedback improved most during more challenging balance tasks that altered sensory contexts or made the stance configurations that are typically more difficult to maintain. This provides some support for the use of sonified biofeedback in people with sensory issues (e.g., low vision, issues with plantar sensations, etc.). In the gait training studies, more studies used validated clinical assessments and longitudinal experimental designs than were used in the feet-in-place studies. Thus, the gait training studies included practical and translational clinical trial designs that allowed personalization of the therapy provided with the sonified biofeedback as a movement training assistive technology.

Most studies reviewed did not include randomization of tasks or control groups, which is generally considered a gold standard in clinical experimental designs. We believe that as the field progresses beyond pilot and feasibility tests, more studies will include larger participant groups, control groups, or other comparative groups. However, in some sonified biofeedback studies, there may be valid reasons not to randomize trials or conditions. For example, Pirini et al. [128] provided justification for why randomizing the condition order would have negatively impacted the study’s scientific rigor because they included a “fake” sonified biofeedback condition that, if provided after the real sonified biofeedback, could distract participants into assigning meaning to the sounds based upon learning the meaning during the real sonified biofeedback condition [128]. Additionally, Mirelman et al. [139] and Nicolai et al. [140] provided clinical justifications for pre-determined progressions of practice balance tasks from easier to more challenging, as well as to support the personalization of prescribing these tasks in order to present a safe, yet appropriate challenge for the participant throughout their 6-week interventions [139, 140].

In fact, there is some support for avoiding the rigidity of randomized controlled trials in complex technology-based interventions. For example, in Craig et al. [175] provided context for the updated guidance from the Medical Research Council’s evaluation framework that was originally provided by Campbell et al. [176]. Both resources provide guidance to outline key benchmarks at different phases of intervention development that are specific to “complex interventions”. Complex interventions are defined as those that include several components, including interventions directed at changing the behavior of health professionals or individual patients [175, 176]. In Wang et al. [174], several limiting factors in randomized controlled trials are identified specific to technology-based interventions in rehabilitation. For example, personalizing the fit between the technology and end-user is usually iterative, which would be limited by a rigid structure of a randomized controlled trial [174]. N-of-1 trials could also be useful to overcome these issues [174, 177, 178], but are not yet widely accepted. Wang et al. [174] proposed a guideline that emphasizes user-centered design and iterative cycles of design through development, progressive usability and feasibility tests, and finally, scaled evaluation and implementation [174]. These alternative approaches in study design are likely what will best support wider translational uses of sonified biofeedback in clinical practice. However, if the use of trial designs that may not fit complex-technological interventions continues to be the gold standard for producing “quality results”, it may slow the growth and acceptance of sonified biofeedback in clinical practice.

Additionally, we hope that future research publications include information about the familiarization period and instructions provided to the participants, as well as information about participant self-reported or perceptual measurements. More studies should provide specific familiarization goals, detailing verbal instructions (if any were provided). It would be beneficial if more studies followed the level of detail provided by Dozza et al. [118], by explaining explicit familiarization procedures. Dozza et al. [118] included a competency test within the familiarization to ensure the participant had a baseline level of understanding about how to control/affect the sound [118]. A protocol paper by Fino et al. [63] was also able to provide sufficient details about planning a longitudinal intervention with a control group, which provides ample details for others to learn from or be able to build upon [63].

Most studies did not employ mixed methods that include both quantitative and qualitative data about the interaction with sonified biofeedback. In the future, including qualitative feedback can inform future sonified biofeedback designs, experimental designs, or provide context for the performance effects observed. For example, Pirini et al. [128] interviewed a participant to better understand that participant’s divergent balance measurement behavior. After this interview, the authors suspected that the participant attempted to move their body to try to silence the sonified biofeedback, instead of trying to achieve the goal of hearing a constant tone sonified biofeedback. In this study, adding an interview was important in providing context to better understand how sonified biofeedback affected balance performance at a participant-specific level [128].

A final detailed methodological suggestion is for studies to include more details about participants and to include more female participants. Most studies reviewed included more male than female participants and some papers did not specify the biological sex or gender identity of the participants. Relatedly, it is unclear if the participants of these studies represent the socioeconomic, racial, or ethnic diversity of the communities or populations studied.

Suggested future research priorities

In this exciting field, there are many open research questions and areas for advancement of knowledge, including deliberately focusing on how to support lasting motor improvements and better understanding of neurological pathways and cognitive load when interacting with sonified biofeedback.

Research should advance from evaluating improvements in the balance or gait variable sonified during the training task, towards investigating whether improvements transfer to real-world mobility tasks and meaningful quality of life changes (e.g., improvements in balance during outdoor gait, participation in society, etc.). Some of the studies included have already measured improvements in clinical balance scores during tasks other than the training tasks retained after the sonified biofeedback intervention [138,139,140]. As the field grows, we hope others will prioritize quantifying meaningful impacts on human health, beyond measuring immediate effects.

It would be transformative to better understand the underlying neural mechanisms of motor control used when a rhythmic cue is present, and how sonified biofeedback relates to rhythmic cues. Rhythm may play a key role in providing timing guidance for the motor system to plan and execute movement via neural entrainment [35, 179,180,181]. Using externally driven fixed-interval rhythmic cues in RAS has been successful in improving temporal patterns in gait tasks for certain clinical populations [39, 55, 182, 183]. Relatedly, in addition to fixed-interval (isochronous) rhythmic auditory stimulation that provides instructional sounds, complex fractal [23], harmonic [184], and a non-linear co-entrainment [185] rhythmic cueing designs have been explored. These non-isochronous cueing approaches are thought to be more compatible as instructional sound cues in movement training because people do not often move with fixed-interval rhythms in the real-world. Adjacently, in older adults with and without Parkinson’s Disease, walking while singing was found to improve gait more than listening to the same song while walking [44]. This finding was attributed to the notion that signing provides internally sourced rhythmic cues [44]. Singing is not only internally sourced, but it allows for adaptive rhythmic cues. This adaptive cueing stands in contrast to the isochronous rhythms in music playback or RAS. Sonified biofeedback can provide rhythmic cueing that is adaptive and arguably, internally sourced, as it follows the user’s movement. Therefore, it is unknown if sonified biofeedback can leverage the benefits of both adaptive and internally sourced rhythmic cueing. Sonified biofeedback may opportunely balance the sensorimotor benefits of rhythmic entrainment via RAS and self-generated music, all while providing additional task-relevant cues during motor learning.

Related reviews, opinion, design, and methods papers

In addition to our review, there are previous review, methodologies, design, perspective, and opinion papers that span from simply alluding to or specifically focusing on designing and using sonified biofeedback [14, 30, 40,41,42, 46,47,48,49,50,51,52,53,54,55,56,57,58,59, 61,62,63,64, 86, 89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110]. Harmonizing with previous reviews of sonified biofeedback, this review also found evidence of the positive impact of using sonified biofeedback to train movement, in our case, specifically balance and gait tasks. While this review focused on upright posture and gait activities, it is worth noting that there are likely fundamental design features that can successfully transfer across movement tasks (e.g., upper extremity rehabilitation tasks).

Notable features of prior articles include providing information about the neurological basis for sound-based biofeedback [55, 92], providing fundamentals of perception–action coupling [14, 64], and discussing the benefits and use of rhythm in sound-based biofeedback or movement cueing [40,41,42, 53, 54, 56, 62]. Some review articles focused on the experimental results of using sonified biofeedback, rhythmic cueing, or other music interventions to help specific clinical populations. For example, Ghai and colleagues completed a series of meta-analyses and reviews with varied coauthors about the use of rhythmic cueing to improve the gait patterns of people with Parkinson’s disease [41], people with multiple sclerosis [53], older adults [40], people post-stroke [42, 56], people with effects from neurotoxic cancer therapy [54]. Despite the predominant focus in these reviews on rhythmic cueing (mainly, isochronous cueing, as in RAS), Ghai and colleagues allude to the promise of using sonified biofeedback and other means to provide variable or adaptive tempo cues [23, 185, 186] for timing-related feedback, and to extend beyond rhythmic adjustments to improve motor behavior.

Prior review, theory, and design articles have provided pertinent experimental and sound design suggestions and details to unify and advance the field [30, 48, 52, 64]. In a scoping review, Guerra et al. [52] discussed that only a third of the papers they reviewed included clinical populations and urges the use of standard outcome measurements in randomized control trials to allow comparisons with traditional clinical approaches [52]. In a theoretical review, Sigrist et al. [30] reviewed sonified biofeedback and audiovisual feedback in motor performance studies of healthy participants and provided overarching design suggestions ranging from how to select specific sound timbres and how to design sound when more than one parameter is mapped to sound [30]. Ludovico and Presti [48] proposed a “sonification space” to qualify the sound design with respect to two axes: time granularity and abstraction of sound. In other words, sonified biofeedback would be characterized across one axis with extremes from continuous movement sonification to discrete audio alerts and the other axis with extremes from low-level “audification” with 1:1 measurement-to-sound mappings to more complex sound designs [48].

In addition, there are methodology, theory, opinion, and perspective papers about sonified biofeedback that have shared interesting and pertinent perspectives [14, 46, 47, 64]. Dyer, Stapleton, and Rodger [14] provided opinions on the role that understanding perception–action coupling can play in the design process. For example, one pertinent design suggestion includes carefully considering if the motor variable that is tracked for performance improvement is the most useful or valued to the end-user through extensive pilot testing [14]. Finally, in a methodology paper that provides a substantial review of all modalities of augmented sensory biofeedback, Lehrer et al. [64] proposed “feature spaces” to categorize and document how the feedback aligns with sensory modalities (3D map of audio, visual, or tactile), time structures (concurrent intermittent to continuous or offline aggregate to terminal), information processing spectra (i.e., explicit to extracted), and whether the feedback is in the format of “online control” for continuous motor adjustments or “feedforward” to plan future motor actions [64].

Conclusions

This review shares the positive potential impact of using sonified biofeedback in balance and gait training. Sonified biofeedback can leverage auditory-motor coupling and the benefits of music in therapeutic approaches while providing task-relevant augmented feedback to facilitate motor learning. Impressively, positive effects from experiments using sonified biofeedback were shared in 47 of 49 prior papers, including end-users who had a wide spectrum of movement skills and motor control issues. Only seven papers reported negative effects from sonified biofeedback, with five of these papers also reporting other positive effects. Despite these encouraging findings, this review highlights areas for improvement for future research. For the experimental design, there is a particular need for strategic selection of control groups and longitudinal studies that evaluate motor learning benefits beyond immediate effects. From a design perspective, researchers are urged to share their design process, as it is pertinent to customize the biofeedback to the user and movement context. Overall, this review highlights the evidence of existing research and points towards the potential future positive impacts of using sonified biofeedback in balance and gait training to help diverse populations with a spectrum of balance and mobility challenges and goals.