Major goals of the research have been to develop an imaging instrument of an exploratory, experimental nature, with the intent of providing an infrastructure for both engineering and artistic exploration. The system consisting of 3 cameras on individual rails each of which follows a specific rule of human photographic vision are translated into machine language. The instrument is being used in multiple ways, serving as an ongoing research test bed, bringing together specialists from different disciplines and ideally stimulating advances in engineering research and further arts-engineering collaboration.

As an engineering project, development of the instrument required collaboration of artists, controls engineers and computer vision researchers. Interviews with engineering researchers suggest that experimentation with the instrument may inspire new research directions in computer vision, machine learning, swarm robotics, remote collaboration and visualization. 

We have developed a project "Swarm Vision" by which to present the research in public venues either in engineering or art museum contexts.

Swarm Vision is a multimedia installation consisting of automated pan/tilt/zoom cameras driven by custom software. Each camera is programmed with a computer vision algorithm in order to "seek" a specific visual feature in the environment. Each camera also contains a memory of its visual environment, containing information on the presence of "interesting" features, and allowing the robot to keep an up-to-date database of the scene. The three robots in the system each communicate with each other: when one robot finds a particularly strong presence of the visual feature it is programmed to seek, it informs the other robots of the location of its gaze and they all turn to look at the same place, collaboratively imaging the scene. Two visualizations are presented to the viewer: a graphic showing the real-time computer vision analysis being performed by each camera, and a 3D rendered reconstruction of the environment showing image frames in their approximate spatial location.

The Swarm Vision project, after two years of presentations and installations around the world, has evolved into new spinoff projects using robotic cameras and 3D displays for interactive art.

A new direction for the Swarm Vision project has been to utilize a new 3D stereoscopic display (separate from the UCSB Allosphere) for visualizations. The Samsung display is notable for its high screen resolution, 3D display, and relatively low price.

Since we began the Swarm Vision project, visitors all over the world have asked about using the cameras to track people's faces. After all, humans take pictures of themselves and of other people more than any other subject. We finally decided to try to create a work using this idea.

Face Cloud uses the robotic cameras and virtual scene reconstruction of Swarm Vision, but differs in what drives the system. Rather than seeking geometric or color features, the cameras seek human faces. And rather than visualizing every moment, only images with faces in them are saved to the virtual cloud. The result is the creation of a virtual space defined by, and populated with, faces of all types looking back at the viewer (including the viewer's own face, eventually).

Publications