UltraMouse 3D

Ultramouse 3D is a spatial human interface device that lets the user interact with computer
programs in three dimensions.

Introduction
Ultramouse 3D times the delay of high-frequency sound waves from the unit held by the user to
each of three receivers and passes this information along a serial cable to the computer. The
accompanying open-source API provides easy functions that let any Win32/C++ application start
reading 3d data in less than 5 minutes.

This is our unique final project for ECE 4760 at Cornell University.

Karl Gluck
MCU software implementation, C++ API and 3d demo code
David DeTomaso
Hardware design, refinement and analysis

Design

Details of the software and hardware setup for Ultramouse3D

Conclusions

Results of our project and ideas for future improvements

Appendix, Videos & Downloads

Everything from schematics and a bill of materials to the C++ API and a DirectX demo project.

Jump to a Section

 High-Level Design
 Design Considerations
 Analog Design

 MCU Software
 PC Software (API)
 Software Downloads

 Results
  Ethical Considerations
 Legal Considerations

High-Level Design
Sound travels through air at approximately 345 m/s. From this, the time it takes for sound to
travel between a transmitter and receiver can be described as:

From this, a reasonably fast counter could keep track of a time delay between transmission and
reception and back calculate a distance.

If the time of transmission and the time of reception on each of three receivers is known, R1, R2,
and R3 can be calculated. Then, with known values of w and h, the 3d coordinates of the
transmitter can be calculated by solving a system of equations of the form:

A high-level diagram of the entire system is pictured here:

Pulses from the transmitter are initiated by the MCU. The MCU then checks input pins from
each of three receivers every 6.25 µs while keeping track of a counter variable. When an input is
detected, the current counter value is saved. After all three inputs have been detected, the three
saved counter times are sent to the PC for processing and update of the mouse position.

The only patent we could find is US Patent # 4862152 which details a similar system of
ultrasonic 3d positioning and was filed in 1985. However, beause the patent does not seem to be
extended and because our design differs in the specific way pulses are detected and position data
is filtered, we do not think that our project is in violation of this patent.

Design Considerations

Placement of the ultrasonic sound receivers will affect the accuracy of the measurements. In
order for sufficient resolution, the distances between receives should be on the same scale as the
distance between the transmitter and the receivers. Receivers very far apart are not practical for
the user and receivers close together will have very small time offsets. For the application space
we intended, it made sense to mount the three receivers on three corners of a computer monitor.

In order for the MCU to accurately resolve the three distances, a fast counter is needed.
However, because the MCU’s onboard Timers cannot be interrupted with three separate external
inputs we must emulate a timer by calling an interrupt at a fixed rate. Increasing the rate of this
interrupt will allow the system to resolve distances more accurately but also limit the amount of
code that can be executed within the interrupt. We chose to run this interrupt at a rate of 160 kHz
so that accuracies as fine as 2mm could be resolved and the interrupt could use up to 100
processor cycles to handle the receiver signals.

Analog Design
Receiver Circuit
The main purpose of the receiver circuit is to amplify the small 40 kHz signal coming from the
ultrasonic receiver so that a digital input port on the MCU will read a logic high when a signal is
being transmitted and a logic low at all other times. Additionally, the delay through the circuit
should be minimal to allow for accurate positioned based on the sounds propagation delay in air.
To accomplish this, we have cascaded three operational amplifiers configured for an overall gain
of around 80 dB. For each amplifier, we required the following specifications:

 An input range that includes ground

 Low input offset voltage
 30-40 dB of gain at 40 kHz
 High slew rate (at least 1 V/µS)

We found the following specifications well satisfied by the Japan Radio Co. NJU7031D
operational amplifier. The resulting circuit worked well within the required specifications and
was able to amplify input signals to small to be discerned on the oscilloscope to levels detectable
by the digital input pin.

Transmitter Circuit

Our transmitter circuit is modeled after the fan driver circuit found in lab 5. To drive the
ultrasonic transmitter an alternating voltage at 40 kHz must supplied across the transmitters two
leads. To accomplish this, we used a digital output pin the MCU to drive the gate of a power
transistor. By placing the transmitter, in parallel with a resistor, between the transistors drain and
the supply voltage, a 40 kHz signal supplied by the MCU can drive a 40 kHz signal across the
transmitter ranging from supply to ground. To transmit as strong a signal as possible, we use the
voltage coming directly out of the AC/DC transformer which tends to range from 9 – 13 volts.

MCU Software Design (AVRStudio / GCC)

The most critical section of software design is the TIMER0_COMPA ISR. When enabled this
interrupt runs at 160 kHz by setting Timer 0 at the full 16 MHz speed, activating interrupt-on-
overflow, and setting the overflow value at 99. Each pass, the interrupt updates
timebase_160kHz, a counter variable, generates the 40 kHz transmit signal if transmission is
active, and polls each of 3 receiver inputs, copying over the timebase_160kHz value if a trigger
is detected. Because of its high frequency of occurrence, the interrupt must run in under 100
cycles. To achieve this specification, various optimizations have been made. Several cycles were
saved by outputting timebase_160kHz on port B and using B.1 as the 40 kHz transmit signal.
Redundant polling of receiver pins is avoided by checking first to see if the pin had already been
triggered and recorded.

The main method of the MCU program is used to initialize required values and to take over once
all three receivers are triggered. After variables are initialized, an infinite while loop
continuously checks to see whether three receiver triggers have been detected or if the timeout
value has been reached. If either of these is the case, TCCR0B is set to 0 to disable the 160 kHz
interrupt so that transmission can be handled efficiently. If three triggers have been detected, the
loop waits as ISR USART0_UDRE is called to send time delays to the PC. In either case, delay
and counter variables are re-initialized and the 160 kHz interrupt is enabled and set to transmit
for its first millisecond.

PC Software Design (Visual C++)

The API for UltraMouse 3D exposes three main functions:

 UM3DAPI_Create

Initializes the serial port and Ultramouse3D structure

 UM3DAPI_Update

Called every loop of the main program to read data from the device and calculate its
coordinates

 UM3DAPI_Destroy

Cleans up the structure when the program exits

The functionality of these methods is clearly explained in the source file, but a few design points
are worth mentioning.

First, the incoming data from the receiver is very noisy--for every hundred good samples, there
are a few huge spikes in the delays. Standard linear filters we attempted (low-pass, mean,
moving average, exponential weighting) were unable to remove these effectively without
creating such large delays in the data that the interface felt sluggish. Instead, the implementation
uses a median filter on the incoming data points. This effectively eliminates all large jumps in
the data; however, it leaves smaller ones. In the figure below, data was taken for the user
drawing a square (more data here). The blue stars are the source data, and the red line is the
median-filtered data.
The second important point is how we removed the smaller jitters in the data. By smoothing the
triangulated spatial coordinate from the output of the median filter into the "current" position by
an exponential weight, the smaller jumps are hardly noticeable. Unfortunately, this means that
the filter's ideal exponential constant has to be tuned to the rate at which UM3DAPI_Update is
called.

Finally, the Z-axis is different than one would expect. Although the math derived to calculate a
true z-coordinate works fine in theory and MATLAB simulation with ideal data, in practice it is
unusable. Instead, the UM3D API simply takes the sum of square distances to each receiver and
calls that the "Z" coordinate. As a result, "Z" is more sensitive on the top and left side of the
screen (where there are two transmitters close together); however, the slight distortion isn't
nearly as bad as using the extreme distortion experienced with a true mathematical solution. A
second effect is that Z is positive coming out of the screen, making this a left-handed coordinate
system.

What Went Wrong?

Initially we tried using a much cheaper op-amp for signal amplification. Problems arose because
of a large offset voltage on the inputs (limiting gain) and a slew rate too low to support 5V
output swings at 40 kHz. Ordering the NJR NJU7031D op-amps solved both of these problems.
We were originally going to implement this receiver as a plug-and-play mouse replacement by
using standard USB mouse drivers for communication with a PC. However, implementing USB
proved to be difficult and require an additional MCU. Because USB output was not an
interesting or vital aspect of our project we decided to simplify and use serial output instead.
This proved to be a good decision because it allowed us to implement the mouse in full 3d
(mouse input only handles 2d) and to handle filtering of the pulse delays in the C++ program on
the PC.

We also found that the serial interface on the computer is very slow. The MCU has to delay 30
milliseconds (enough time for 480,000 instructions!) every loop to avoid filling up the PC's
serial-port buffer. The symptom of transmitting too quickly is that motion becomes delayed by
several seconds as the buffer is slowly emptied by the receiving program.

One final "gotcha" was that if you start up the MCU before starting the client program, there is a
chance that the program will start picking up bytes in the middle of a transmission. The incoming
data will look like junk (delays should be less than 400 most of the time, but junk will often be
tens of thousands), or it may seem like receivers are placed in the wrong order.

Results
Our device samples position data at a rate of 50 Hz allowing for quick updating of the receivers
position in a real-time feel. The range of operation is about a meter. This is because we are using
a transmitter to act as one of our receivers in order to save cost and its lowered sensitivity to the
ultrasound signal limits our range.

To test the accuracy of each receiver, we recorded while holding the transmitter still for 12
seconds.
 Receiver Mean Distance Maximum Variation Standard Deviation
#1: Bottom Left 54.3 cm 7.1 cm 1.5 cm
#2: Top Left 45.8 cm 2.6 cm 0.5 cm
#3: Top Right 45.8 cm 3.4 cm 0.6 cm

Because our design emits audio frequencies at 40 kHz, it will interfere with other designs that
operate on the same frequency. The transmitter is extremely easy to use: all that is needed is to
hold the device within range of the receivers and point it in their general direction.

Conclusions
Our device can function well in the way that we intended it to – as a 3-dimensional input device
to a computer. However, jitter in the time delays recorded by the MCU were higher than we
initially expected. Additionally, the range of approximately 1 meter was less than we expected.

If we were to complete an Ultra-Mouse v2.0, the following changes would be made:

 Increase the strength of the transmitted signal

 With increased transmission strength, decrease gain on receiver circuits to increase
selectivity
  Simplify receiver circuits to contain one instrumentation amplifier cascaded into a high
speed op amp.
 Run the sampling counter much faster by using a faster MCU so that more data is
available for signal processing.
 Use a battery and oscillator circuit to allow for a wireless transmitter.

If further development were considered to allow for the use multiple simultaneous inputs then we
propose the following design changes:

 Flip the circuit around so that the held units contain receivers and the mounted units
contain transmitters.
 Transmitters will pulse one at a time at a fixed rate
 Receivers will calculate how the rate they observe deviates from the known fixed rate
 Increase the number of transmitters from three to four so that precise synchrony between
transmitters and receivers is not needed
 Position information sent back to the input unit using the wireless Bluetooth protocol

In this way, a large number of 3d position units could be simultaneously used without interfering
with each other. This could be useful if the device were incorporated into a gaming system to
allow for multiplayer capability.

Ethical Considerations

The IEEE Code of Ethics can be found by following this link.

In the spirit of training ourselves to practice as ethical engineers, we have followed this code of
ethics throughout the design, assembly, testing, and reporting of our project. As per our
responsibility, we have taken precautions to ensure the safety of anyone using our device by
preventing accidental electrical contact. Since our principle interests in the design of this device
were to further our knowledge of design using microcontrollers and to complete the ECE 4760
course, no conflicts of interest exist. All of our claims on the devices performance were made
based on real, un-altered data collected while testing our device.

We believe our device is an appropriate application of technology using ultra-sound and in its
design we have improved our own and possibly a few friends understanding of what can created
using electrical devices. Because our project fits within the skill-set of ECE 4760 and because
both of us satisfy the requirements to enroll in this class, we felt qualified to undertake this
project. All work, unless otherwise credited, is our own. Issues such as bribery, fair-treatment,
and malicious action were not encountered throughout our completion of this project.

Legal Considerations

The safety of our users is both a legal and ethical consideration and so precautions have been
taken to ensure no skin-to-wire contact when using the device. The only significant transmitted
energy from this device is the 40 kHz sound wave emanating from the ultrasonic transmitter.
Because this is not an electromagnetic wave and 40 kHz is outside the range of human hearing,
its intensity is not regulated by the FCC. However, this noise will be detectable by some
household pets including mice, cats, and certain species of dog. Before commercial deployment
of such a device, research will have to be undertaken to rule out the possibility of induced
hearing damage in these household pets when near the transmitter of our device.

Intellectual Property Considerations

The only material we used not created by us is the Tiger_3d demo which is distributed for free
use by Microsoft. It would probably not be possible to patent our project in its current state
because 3d ultrasonic positioning has been patented before however a more specific
implementation (such as the suggested changes to include multiple receivers) would be
significantly different from previous ideas and might be able to receive a patent.