SideBySide is a prototype system developed at Disney Research for ad-hoc multi-user interaction using a mobile camera and a hybrid visible/infrared light mobile projector. From completely separate devices, projected imagery can interact and respond to other imagery as if they share the same screen. The system tracks multiple projected images in relation to one another using invisible fiducial markers projected in infrared.

SideBySide allows new interaction scenarios where users are not tied to a fixed location. For example, exchanging digital content, such as personal contact information or media files, can be initiated using lightweight optical communication. Projector games can be played by multiple users with characters that interact together from separate devices. As the system does not require sensing hardware installed in the environment, interaction can take place almost anywhere.

Created by: Karl D.D. Willis, Ivan Poupyrev, Scott E. Hudson, & Moshe Mahler.

Research Paper: SideBySide: Ad-hoc Multi-user Interaction with Handheld Projectors

Awards: UIST 2011 "Best Paper", UIST 2011 "Best Demo"

Special Thanks: Christopher Ioffreda, Yu Willis, Cheng Xu, Jo Ana Parker, Sharon Hoosein, and Mark D Gross

Selected Press: Wired, New Scientist, Engadget, Gizmodo, Creative Applications, CNET, Discovery, PC World, Page Magazine, BBC Click

The SideBySide concept – a self-contained, full color, handheld projection device allowing multi-user interaction in almost any space.

The SideBySide system overview.

Example applications created with the SideBySide system.

The SideBySide prototype device.

Projecting visible and IR images in a single stream. The characters are visible to the user, and the markers are visible to the IR camera.

The SideBySide projection engine.