Presentation Method of Walking Sensation Based on Walking Behavior Measurement with Inertial Sensors and Pressure Sensors

Abstract

In this paper, we give the way to make the sitting user feel the walking sensation, by using the walking sensation presenting device in which we put the data we got from measuring walking behavior. We measured walking behavior from the inertial sensors which we put on the insteps and waist of the pedestrian and pressure sensors on the bottoms of their feet. We get the data of back-&-forth movements of feet from the data of acceleration and directions from inertial sensors on insteps. In the same way, we get back-&-forth and left-and-right movements of the whole body from the sensor on back. Walking sensation presenting device has a chair for user to ride, which moves back-&-forth and right-&-left to presents walking body’s swing to user and the device which presents walking sensation to user’s lower limb. The device which presents walking sensation to user’s lower limb consists two parts. One part are the lower limb movement presenting board that moves back-&-forth to make user’s lower limbs do the same movement as the real walking. The other is the landing vibration presenting device. About 10 % of amount of the whole body movement in real walking presents the best walking sensation to the user. About 20 % of amount of the lower limb movement in real walking presents the best walking sensation to the user.

1 Introduction

In these days, with the development of information and communication technology, it is expected to construct display about five senses and give new space experiences. In this research, we tried to show users’ physical feeling of walking by precise control of information through five senses. When we succeed in presenting users quasi-real walking sensation, they can experience travels and sports in the system. This is useful in study through experience, entertainment industry, and enhancing the QOL of the people who have trouble moving their limbs freely.

In early study, Helig gave the users the senses other than visual and auditory in The Sensorama Machine [1]. This presented users the wind, smell and vibrations experienced through riding a motorcycle.

Recently Five Senses Theater [2] by Ikei et al. is well known. In this research, they experimentally made smell and wind-touch displays and got knowledge and methods about how to mix smelling ingredients and control wind in the space. They researched presentation of human walking sensation, too. In their research, they collected various patterns of walking data through the measurement of real walking behaviors using motion-tracking markers. Based on these data, they studied vertical movements of lower limb. They said about 10 % of lower limbs’ movement in real walking fits best walking sensation by passive movement [3].

Figure 1 shows the outline of the system we made. In our research, we think main body movements in walking consist of back-&-forth and right-&-left slow movement of whole body and back-&-forth movements of lower limbs. And then we have constructed the system to show them to the users. Currently wearable computing has become more feasible due to technological progress that has enabled us to produce compact and light weight computers and sensors with lower battery consumption. Considering these situations, we have also constructed the walking movement measuring system that can process and present the walking data of a distant user wearing compact sensors and computers in real time.

Fig. 1.

Outline of the system

2 Walking Behavior Measurement with Inertial Sensors and Pressure Sensors

In this section we discuss the walking behavior measuring system that we’ve constructed in this research.

2.1 Constitution of the Walking Behavior Measuring System

In the measuring system, we attached inertial sensors to the user’s waist and insteps of both feet to detect and measure walking behavior. Figure 2 shows the footwear equipped with the inertial sensor units.

Fig. 2.

Appearance of the footwear with the sensor units

We used IMU (Inertial Measurement Unit) chip (MPU-9150 by InvenSense). This is composed of 3 axis accelerometer, gyroscope, and compass. We connect this to the lap-top computer using I2C communication through microcontroller (H8 by Renesas Electronics). When we put the microcontroller to the lap-top computer to measure walking behavior, we did USB converting and communicated through USB port. We set all the accelerometers so that their x axis plus shows forward, y axis plus shows right and z axis plus shows upward. The user walks in the footwear with the sensor units, a belt with the sensor unit around his/her waist and a lap-top computer on his back to get the walking behavior data. Figure 3 shows the user with these sensor units. In this research, we got the data every 1/200 s from the sensor units.

Fig. 3.

User with the measuring system

In the sensor units for the feet, we attached pressure sensors (FSR402 by Interlink Electronics Inc.) to the feet to detect the landing timing and amount of the shock. We connected them to the measuring lap-top computer through the microcontroller, too. We attached these pressure sensors to the inside and outside of the bottom of user’s feet near his toes, and to the heels.

2.2 Estimate of the Forward Vector and the Integration Calculation of Walking Behavior

In this section, we describe the method to estimate the forward acceleration vector which we need to drive the walking sensation presenting device by analyzing the data from inertial sensor units. Also, we describe the method to integrate the forward vector of the lower limbs and the waist of the pedestrian and calculate the coordinates which we finally put into the device. Figure 4 shows the outline of the calculation steps.

Fig. 4.

Outline of the calculation steps

We thought that lower limb mainly move back-&-forth and vertically in walking, and we divided legs’ walking motion into two direction levels, forward and vertical. So we measured how much x axis and z axis turn around y axis in horizontal and vertical level and determined each axis’ forward acceleration vectors. Then we estimated the acceleration vector of lower limb by summing them up.

First, we integrate the angular velocity vector gotten from the gyroscope to estimate the turning angle of the sensor units around y axis ($\theta_{y}$) at each time. At this time, we set the angle of sensor units at landing at 0 degree to correct errors brought by the drifts of the gyroscope. This is because by adjusting the sensor units on feet, we can make the angle of sensor units at landing at 0 degree. We use pressure sensors to know when the feet land. We determine feet landing time when all the pressure sensors on the bottom of the feet indicate over the threshold. We confirmed that we properly set the sensor unit at 0 degree in landing. Figure 5a shows $\theta_{y}$ at each time. $\theta_{y}$ reduces to 0 smoothly, not suddenly. That demonstrated that our process is appropriate.

Fig. 5a.

$\theta_{y}$ at each time

Then we estimate feet’s forward acceleration vector $a_{f} '$, using sensor units’ turning angle around y axis($\theta_{y}$) at each time. That is to say, adding forward acceleration vector on x axis $a_{x}$ and that on z axis $a_{z}$, we get feet’s forward acceleration vector $a_{f} '$.

$$ a_{f} '= a_{x} \cos \theta_{y} + a_{z} \sin \theta_{y} $$

Apropos, we ignored the rotation of the sensor unit of the waist, because it is very small.

The back-&-forth acceleration vector that we got above ($a_{f} '$) include the offset of the accelerometer. We use a high-pass filter to remove this offset. Figure 5b shows the back-&-forth acceleration vector of right foot ($a_{f}$) at each time that we got through this process.

Fig. 5b.

$a_{f}$ at each time

Then we get the velocity of the foot by integrate $a_{f}$. We use a high-pass filter again to remove the constant body movement in walking behavior. Figure 5c shows the back-&-forth velocity of right foot ($v_{f}$) at each time that we got through this process.

Fig. 5c.

$v_{f}$ at each time

Then we get the coordinate of the foot by integrate $v_{f}$. We use a high-pass filter again to set the center of the locomotion on the origin of the walking sensation presenting device. Figure 5d shows the back-&-forth coordinate of both feet at each time that we got through this process.

Fig. 5d.

Waveform of the feet locomotion

Also, Fig. 6 illustrates the movements of the waist.

Fig. 6.

State of motion of the waist

3 Walking Sensation Presenting Device

In this section, we describe the device which works based on the measured walking locomotion data and presents users walking sensation. First in 3.1 we explain the structure of it and then in 3.2 we state its operation.

3.1 Constitution of the Device

Figure 7 shows the appearance of the walking sensation presenting device. Walking sensation presenting device has a chair for user to ride, which moves back-&-forth and right-&-left to presents walking body’s swing to user and the device which presents walking sensation to user’s lower limb. The device which presents walking sensation to user’s lower limb consists two parts. One part are the lower limb movement presenting board that moves back-&-forth to make user’s lower limbs do the same movement as the real walking. The other is the landing vibration presenting device. Next we describe them in detail.

Fig. 7.

Appearance of the walking sensation presenting device

3.1.1 Walking Sensation Presenting Moving Floor

The walking sensation presenting moving floor presents users back-&-forth and right-&-left body movements. This system works corresponding to the coordinate measured by the accelerometers on the pedestrian’s waist. The floor can move both back or forth and right or left by 15 cm. We used a pulse motion controller (PEX-H741444V by Interface Co.). We attached this to the walking sensation presenting device and controlled the 2-axis motors’ drive.

3.1.2 Lower Limb Movement Presenting Boards

The lower limb movement boards presents users back-&-forth movement of lower limbs while walking. This system works corresponding to the coordinates measured by the inertial sensors on the pedestrian’s insteps. The boards can move back-&-forth by 15 cm. As is mentioned above, we used a pulse motion controller (PEX-H741444V by Interface Co.).

3.1.3 Landing Vibration Presenting Device

The landing vibration presenting device presents users the vibrations to the bottoms of the feet when the feet touch down the ground. This system presents users vibrations according to the timing and pressure of feet landing measured by pressure sensors on the pedestrian’s feet. We used an analogue output interface (PCI-3329 by Interface Co.). We attached this to the walking sensation presentation device and controlled vibrator at four points of the front and back of both feet. We used vibrators (vibro-transducer Vp604 by Acouve Laboratory, Inc.). We amplified the analogue signals output with a power amp (EPQ304 by Behringer) and input them into vibrators to produce vibrations.

3.2 Operation of the Device

Then we checked the operation of the walking sensation presenting device to know whether or not we can control it precisely. We compared the coordinate calculated through the data from accelerometers with that of motors’ indicating. We used the function included in the driver of the pulse motion controller PEX-H741444V to know the coordinate that the motor actually worked and indicated. In the actual control, measured amount are multiplied by the gain to reduce the amount of the movement, so that the amount of the movement is within the limits of the device. Figure 8 plots motors indicating coordinates divided by this gain and the coordinates calculated from the accelerometers’ data. Both coordinates corresponded and it was confirmed that the device operates accurately. Figure 8 illustrates the right foot’s lower limb movement presenting board. The same was confirmed with the left foot and with and the walking sensation presenting moving floor. And the gap between moving sensors and the device operation is 1 s.

Fig. 8.

Input position & actual position

4 Experiment

In this section, we describe the experiments, using the walking behavior measuring system and walking sensation presenting device we made.

4.1 Purpose of the Experiment

In the research of Ikei et al., they said that about 10 % of lower limbs movement in real walking presents the best walking sensation to the user. We made experiments to know what presents the best walking sensation to the user with our device.

4.2 Method of the Experiment

First, we took the subject’s walking data for 30 min through our walking behavior measuring system. We measured 60BPM (60Beat Per Minute 60 steps in 60 s), 86BPM and 100BPM. Ikei et al. experimented with 86BPM. To compare with this experiment, we did with a little faster beat and a little slower beat. Walking sensation presenting device worked on this data.

Second, we made the subject operate the walking sensation presenting device to match the best his/her real walking sensation and recorded it. It was recorded as the gain to the actual walking movement. To be specific, the subject operated the device by pressing the key of the wireless keyboard. Every time the subject presses the “↑” key, the gain of the walking sensation presenting moving floor(GainW) increases by 0.1 %, and the “↓” key, decreases by 0.1 %. And every time the subject presses the “;” key, the gain of the lower limb movement presenting boards(GainF) increases by 0.1 %, and the “-” key, decreases by 0.1 %. We presented the subject 60BPM, 86BPM, 100BP at random. Each subject operates on each BPM 6 times, so he/she operates 18 times in all. We repeated the date while the subject operated the device. Subjects are 5 men and 1 woman who have proper walking functions.

4.3 Result of the Experiment

Table 1 below shows the results of the experiment. The average of the gain of the walking sensation presenting moving floor (GainW) is about 10 % of all walking speed. Also, the average of the gain of the lower limb movement presenting boards (GainF) is about 20 % of all walking speed. And particularly in GainF, it seems that as the walking speed become faster, the gain become smaller.

Table 1. Results of the experiment

4.4 Experience of the Real-Time Transmission of the Walking Behavior

To investigate how much fine the walking behavior measuring system and the walking sensation presenting device we made can present the walking sensation, we tried to do different patterns of walking behavior and present them to users by using the real-time transmission of the walking behavior. As a result, we found that users can know how they walk to some extent in some patterns, for example turn 90 degree, back, and who are walking actually with the measuring system.

5 Conclusion

In this paper we discussed the walking behavior measuring system and the walking sensation presenting device that we’ve constructed. We proposed a method of measuring the human walking locomotion by using inertial sensors and pressure sensors. Also, it was confirmed that the walking sensation presenting device operates accurately, based on the measured walking locomotion data. About 10 % of amount of the whole body movement in real walking presents the best walking sensation to the user. About 20 % of amount of the lower limb movement in real walking presents the best walking sensation to the user. We will research the factors which influence the walking sensation user feels in more detail through more experiments.