Wiimote Hacking

The wireless controllers sold with Nintendo's Wii gaming console, henceforth referred to as Wiimotes, are a cheap wireless input device with some 3D tracking abilities. However, all Wii games I know of only use the Wiimote as a pointing device (2D mouse-like interaction on the screen), or as a gesture-recognition device.

The motivation for this project was to investigate the actual capabilities of the Wiimote in a Virtual Reality context. What motion sensors are provided? How can those sensors be used, separately or together, to approximate a 6-DOF input device, i.e., a device that reports its own absolute position and orientation in space? Since the Wii console's release, several hacking sites have sprung up across the Internet (http://www.wiili.org, GlovePIE, http://wiihacks.blogspot.com), but all I found were descriptions of the low-level Bluetooth communications protocol, and the effects that can be achieved by that. To my knowledge, there are no projects that produce a full tracking solution, and I have found only very few applications that use the Wiimote to control 3D programs beyond simulating a mouse device. Two examples are a Wiimote input driver for the FlightGear flight simulator. Both of these work with raw accelerometer values directly, and neither do full 6-DOF tracking.

Figure 1: Still image from a video explaining how to use a Wiimote for full 6-DOF motion tracking. This still shows a "virtual light saber" drawn on the computer monitor using the Wiimote's position and orientation.

Wiimote Motion Sensors

The Wiimote contains two types of sensors that can be used for motion tracking:

Linear Accelerometer: The Wiimote contains three linear accelerometers that measure acceleration along three pairwise orthogonal axes. The accelerometers can directly be used to measure the Wiimote's pitch and roll angles, by measuring the direction of gravity, and also form the basis for the gesture recognition used in many Wii games. Unfortunately, the accelerometers alone cannot be used to track the (relative) position and orientation of the Wiimote in space. Since they only measure linear accelerations along three fixed axes, they are unable to distinguish between linear motions and rotations. Since the Wiimote does not contain gyroscopes (although many early reports state it does), 6-DOF tracking based on double integration of linear and angular acceleration values is impossible.
Newsflash: Nintendo just announced a new Wii controller add-on, the "MotionPlus." From the looks of it, and from early reports, it seems that the MotionPlus contains gyroscopes, which are the missing pieces to make 6-DOF motion tracking work. This could be very interesting. I'll try to get my hands on one of those, and see if I can adapt the driver software. Stay tuned for updates!
IR Camera: The Wiimote has a built-in IR camera mounted in the front of the case. This camera is connected to an integrated image analysis chip that can identify up to four IR light sources, and report their position on the camera CCD, and their (approximate) size and brightness. Since the camera is passive, it needs infrared light sources to work. The Wii gaming console comes with a so-called "sensor bar," which is a misnomer because the bar does not sense anything, but simply emits infrared lights from two clusters of IR LEDs mounted at either end of the bar. Together with the sensor bar, many Wii games, and the console operating system itself, use the IR camera to measure yaw, the horizontal angle of rotation of the Wiimote with respect to the sensor bar. Since the accelerometers can be used to measure pitch and roll, combining the two sources yields the full orientation of the Wiimote, as long as the camera sees the sensor bar. The Wii OS uses the Wiimote's pitch and yaw to simulate a 2D mouse for on-screen interaction, and some Wii games use it to "aim" at the screen, similar to the light guns that were available for older-generation gaming consoles.

Project Goals

The goal of this project was to build the hardware and software necessary to use a Wiimote as a 6-DOF input device for virtual reality applications. The software is implemented as an input device driver module for the Vrui VR toolkit to facilitate using the Wiimote with all existing Vrui VR applications.

Project Details

To connect the Wiimote to a PC, and use it for 6-DOF tracking, the following hardware is required:

Wiimote: Duh.
Bluetooth Adapter: The Wiimote communicates with the Wii gaming console via a Bluetooth protocol. The Wiimote VR driver software uses the same communications protocol, which requires a Bluetooth adapter compatible with the host OS. I am using a USB Bluetooth dongle, and the standard Linux Bluetooth protocol stack.
IR Beacon: The standard infrared beacon bundled with the Wii gaming console, the "sensor bar," only contains two LED clusters and is not sufficient for 6-DOF tracking. Instead, one needs to build a custom IR beacon that consists of four IR LEDs (or four LED clusters for longer range) arranged in a non-planar configuration. With such a beacon, the positions of the four LEDs on the Wiimote camera's imaging plane are sufficient to solve for the Wiimote's position and orientation relative to the beacon. My custom beacon uses a 9V battery, four Radio Shack high-output 940nm IR LEDs, and some resistors. My beacon is currently mounted on a customized empty candy box, but the next generation beacon will be based on a LEGO backbone.

Pages In This Section

Technology: Detailed description of the technology behind tracking a Wiimote in six degrees of freedom.
IR Tracking Beacon: Instructions on how to build the IR tracking beacon required to track a Wiimote in six degrees of freedom.
Pictures: Some pictures showing the Wiimote with VR applications.
Movies: Movies showing the Wiimote used as a 6-DOF input device.
Download: Contains the complete source code for the Wiimote input device driver based on the Linux Bluetooth protocol stack, released under the GNU General Public License.