) movies by crossing your
eyes.
published in Videre: Journal of Computer Vision Research, 1(1):2-19, 1997.
Synopsis: The article advocates and demonstrates a new paradigm for the computational study of active vision and sensorimotor control. Animat vision eschews physical robots in favor of artificial animals - realistic, zoomimetic virtual agents situated in simulated physical worlds. The work draws from the fields of computer vision, computer graphics, and artificial life.
Notes: Click on any image in
this document to view a larger version. Try free-fusing the stereo
images and stereo MPEG () movies by crossing your
eyes.
Contents:
The artificial fishes in their physics-based
virtual world as it appears to an underwater observer. The 3 reddish
fish (center) are engaged in mating behavior, the greenish fish (upper
right) is a predator hunting for small prey, the remaining 3 fishes
are feeding on plankton (white dots). Seaweeds grow from the ocean
bed and sway in the current.
More images courtesy of Xiaoyuan Tu:
Here are (low-resolution) movie clips of artificial fishes:
movie clip from the
animation ``Go Fish!'' by X. Tu, D. Terzopoulos, E. Fiume, in
ACM SIGGRAPH Video Review Issue 91: SIGGRAPH'93 Electronic
Theater, Anaheim, CA, August, 1993;
QuickTime movie including clips from ``Go Fish!'' and from the animation ``A National Geo-Graphics Society Special: The Undersea World of Jack Cousto'' by X. Tu, D. Terzopoulos, R. Grzeszczuk, presented at the SIGGRAPH'95 Electronic Theater Evening Program, Los Angeles, CA, August, 1995.
The
fish animat is equipped with a pair of mobile eyes and is capable of
color, binocular vision. The stereoscopic view at the right is
acquired by the animat as it gazes at a red fish in the distance.
The animat has foveated vision. Its retinas
have a 64x64 high-acuity fovea surrounded by three annular peripheral
regions with progressively diminishing resolution (hence the
increasing image blockiness towards the periphery). The picture to
the left reveals the structure of a retinal image by demarcating the
peripheral regions with white borders.
The task of vision-guided navigation in a dynamic environment
serves to illustrate the active vision abilities of the fish animat.
Despite their limited quality, MPEG movies can better convey the
dynamic scenarios. Here is an
movie demonstrating the pursuit of the red target fish by the observer
animat (centered in the video frame) and an movie of the moving target as
seen by the eyes of the pursuer. The red fish tries to get away,
but the observer can maintain a hot pursuit by keeping the target
foveated and using controlled muscle actions to steer the body in
accordance with the gaze direction. These MPEGs give an idea of the
performance of early versions of the animat vision algorithms.
The image
at the right shows the format of the ensuing MPEG movies which are
more recent and better demonstrate the robustness of the animat vision
system (note: these large 640x480 MPEGs will run at less than the
normal frame rate on most computers). The top half of the frame shows
the left and right retinal images of the observer animat. At the lower
left is a side view of the action, with the observer at the center of
the frame. At the lower right is a top view of the action. The line
emanating from each eye of the observer indicates the direction of the
gaze. The line is colored white out to the estimated range of the
foveated target and yellow thereafter.
The next demonstration shows an observer animat that first detects
and foveates a reddish fish and then applies visually-guided
navigation to swim towards this target fish and follow it. Here is an
movie demonstrating
active-vision-based sensorimotor control in the animat.
The retinal image stabilization process of the animat vision
system implements an optokinetic reflex for the eye muscles that
maintains the stability of the visual world. To demonstrate the
difference between stabilized and unstabilized vision during
locomotion, let's make the observer animat strabismic by locking the
gaze angle of its left eye, while the right eye continually performs
saccadic eye movements under control of the optical flow based retinal
image stabilization process. Here is an movie demonstrating a fish animat with an inactive
left eye and a right eye that actively compensates for the pitching
and yawing of the swimming body.
We are extending the animat vision system to support other
behaviors in addition to the pursuit of moving targets. We have
implemented a module that integrates motion and color for selective
attention. For example, the next demonstration shows an observer
animat foveating and tracking a red target fish. The observer is also
alert to interesting colorful objects moving in its visual
periphery. Once it detects another target worth looking at, in this
case a bluish target fish that enters its peripheral vision, it
foveates the recognized object to obtain a high resolution view and
recognize the target. This perceptual processing enables the observer
to produce the appropriate behavioral response; for example, if the
detected object is food, the animat pursues, or if the object is a
predator, the animat attempts escape. Here is an movie demonstrating selective
attention. The observer shifts attention from the reddish target
to the bluish target at frame number 167 and pursues the new target.
The animat
vision system can now also support predation and feeding behavior.
The following movies show a large bluish predator fish employing
active vision to detect and foveate a small reddish prey fish and swim
towards the moving prey using visually-guided navigation. As the
picture on the right shows, when the predator manages to get
sufficiently close to the prey, it violently sucks the prey into a
gaping mouth and devours it. Here is an movie demonstrating the vision-guided predation
behavior, as well as an
movie showing what the predator sees during the predation behavior
sequence.