Russell M

Facial Image Access

Allen Klinger

Table of Contents and List of Tables and Figures.

ABSTRACT 2

1. Introduction 3

2. Technical Discussion 3

Detailed Statement of the Problem 3

2.1 Indexing 3

Proposed Work 5

2.2. Interfacing 5

2.3. Sequencing 6

3. Statement of Work 6

3. Conclusion and Relevance to MICRO 7

References 8

Appendix A 9

Appendix B 10

Appendix C 11

Appendix D 12

Personnel 13

Current and Pending Support: 14

List of Reviewers 15

Tables and Figures

Figure 1. Facial Descriptors in a Police Rating System 4

Facial Image Access

INVESTIGATOR Professor Allen Klinger

Department of Computer Science

University of California, Los Angeles

Los Angeles, California 90095-1596

COOPERATING COMPANY Facial Identification Digital Systems

ABSTRACT

This proposal presents software research needed to develop a system that combines existing technology with indexing, search, and display-modification facilities and features. The combination of new software and facial identification methods can have practical benefit for multi-site pictorial data sharing. This could impact diverse industries including entertainment production, i.e., movies, television, video games, digital libraries, and geographic mapping. The work is focussed on developing useful computer software to extend the bounds of knowledge. The research concerns software used in processing large sets of visual information. The work has immediate practical benefit in the domain of policing and potential to support efforts to counter terrorism.

Facial Image Access

Allen Klinger

1. Introduction

This effort concerns the varied indexing and human-assisted modifications of verbal descriptions of facial identity common in police work. The attached materials from the industrial sponsor consist of computer adaptations of systems developed for and by law enforcement agencies. Computers enable rapid variation of initial sketches, convergence of mask-like template sequences to accurate visual descriptions, and search on multiple keys. As the number of keys or index terms increases the victim of a crime and a police artist can more easily converge on a recognizable visual description. The synthesized image can be publicized ensuring that the public is alerted to dangers posed by perpetrators.

Commercial forces caused investment leading to handheld, palm-sized, and mobile or laptop computer platforms that have become more powerful, less costly, and lighter. This development coupled with cellular telephones has made careful examination of fundamental concepts of indexing useful. This is particularly true when comparing the issues regarding successive display of several visual images. Within a mobile environment the existence of five hundred descriptors for a face becomes a potential advantage: what is needed is programs taking advantage of the data structure built into the organization of visual features. The overall systems issue is how to create software that improves on the ability of stationary computer platforms to process files of digitized images organized in a database. The issue plays out against the background of rapid change in the input (scanning), communication (cell-phone), and data storage (database) domains.

2. Technical Discussion

Detailed Statement of the Problem

2.1 Indexing

A field officer or police artist captures the impressions of a crime victim through words, transforming them into a visual by assembling features. Choice of a corresponding template to represent the hair, eyes, mouth, distances (as from mouth to chin) all become indices to the composite. The anticipation that this visual conveys valuable information creates the incentive for both parties to communicate. The following describes first the technical context, then the software research need.

Figure 1 is J. McCall's representation of how this would be done using contemporary handheld computers. We assume the existence of a scanned database of two thousand facial images. The right side of Figure 1 shows some of the five hundred descriptors potentially in use. The left side there indicates how a current facial synthesis could be displayed alongside possible variations.

Display and Communication of Facial Descriptors (J. McCall)

Figure 1. Facial Descriptors in a Police Rating System with Associated Handheld Computer

There are many user experiences and descriptor confidences. They are varied and described in greater detail below (Section 2.3, Sequencing). We will select a subset of all descriptors to support a dynamic integration process [1]. As [1] indicates diverse reasoning modes can take a process to correct conclusions. We will examine descriptor group characteristics associated with sets likely to lead toward accurate facial classifications. This will be based on simulation studies conducted with real data we construct using a Hewlett-Packard 7400C scanner (Budget). The results of these studies will be research of general applicability to systems documentation, digital libraries, and other fields impacted by software research. Nevertheless, the principal mode of investigation will be to support system integration. The work focus involves creating new software. The function is adapting visual facial key or index elements (also known as descriptors) [3] to modern mobile computing technology. Accomplishments in software research here should be widely applicable within domains involving pictorial databases.

Proposed Work

2.2. Interfacing

The two references, [3, 4], both by the technical person at the industrial research cosponsor, J. McCall, detail practical needs for adapting visual indices to mobile computing. Both documents and a closely related research paper [5] that describes defining a measure for visual pattern complexity are present in this proposal as appendices (Appendices A, B, and C). A letter dated January 1, 2001 from the County of Los Angeles Sheriff's Department (Appendix D) is a clear call for the research to progress. It is especially noted that this letter is from a representative of those whose work takes them into the field. Beyond local governmental units' police functions, most citizens in the post September 11, 2001 environment are concerned with practical interfacing issues regarding facial descriptions because of heightened national and international awareness regarding terrorism.

The research in software that most closely impacts practical needs comes from the nature of mobility [6]. Networks depending on wireless data dissemination and field deployment must conserve expenditure of battery power. The implication reinforces our concern about indexing since locating key subsets of facial features can markedly reduce power needs to transmit visual information, while allowing human interaction with a recognizable, albeit stylized, individual.

The software for the interfacing function must take into account practical limits imposed by the field environment. To do that we plan to use machine language codes written to take advantage of double-linkage and similar devices for data structuring [7]. Finally, as we indicate below (Section 2.3, Sequencing) a model of human judgement about actual features seen is useful to focus the computation where they can have greatest impact.

Experimental computations will begin with acquiring two thousand physical facial images. We will scan these items to obtain a sample test bed for evaluating interactive processes based on approximate descriptions using the techniques described in [3, 4] and standard police manuals regarding the Identikit and Photofit products.

Student researchers will construct sample descriptions of faces. These will consist of descriptor sets that approximate visually the scanned images. We will exercise software to interact with a field officer who has an idea about a suspect's facial appearance. We will track convergence to or inability to find a close match. The experiments will rank software based on different uses of indices. Some possible approaches to the programs' use of facial feature descriptors are:

a. Use all.

b. Only those noted.

c. Subset based on highest certainty.

Finally, interaction with a central repository offers advantages (access to intelligence data, ability to sift through a large database of stored images) but they come at significant costs (transmission delays, mismatch between current data and past criteria). We will simulate stand-alone and interactive modes of processing to get cost benefit information as a function of number of indices describing a visual facial image.

2.3. Sequencing

When a police artist encounters a crime victim part of the skill set employed in obtaining a visual description involves sensitivity to the person's response to questions. Some items stand out in a person's memory. Other things may be vague or not recalled. The artist has to combine general impressions with specific recollections. Inevitably that is something the artist learns to do. Likewise often the artist's effectiveness depends on the rapport established with the victim. Hence it differs from case to case. Sequencing refers to designing software to have an inherent capability that mimics that capability. In order to do that we will describe three basic functions.

The primary function is distinguishing between near-certain recall and all others. By near certain we mean ninety, or ninety-five percent belief, or greater. We call near certain the class labeled 1, or using set notation with F for features, F₁ . Set F₁ will be described in words as bright.

All features where the individual has absolutely no recall, or up to at most a belief of ten percent, we call F₄. Set F₄ will be described in words as dark.

We distinguish two other situations, one F₂, which could be called light, and F₃(dull). In the case of F₂the recall is somewhat but uncertain, a belief not less than fifty percent but surely if more than that, no more than ninety. If the recollection is F₃that means the person has more than ten percent belief but less than fifty.

3. Statement of Work

We will acquire a Hewlett-Packard 7400C scanner. We will use it to digitize a set of facial images to develop a test bed for research. We will access representative facial images to develop a set of visual descriptors useful for suspect identification. These descriptors will be used as indices for new software programs we will develop. Specifically, we will build software that will allow experimental evaluation of modes of processing based on indices. I. e.,

We will develop new programs to allow use of: indices that are

a. All indices considered bright.

b. Bright plus light up to the highest one-tenth the number of bright.

c. Bright plus a selection of those that are light (e.g., highest quarter).

We will investigate the stability of these programs over varieties of digitized facial images and simulated visual descriptive features created by student researchers.

3. Conclusion and Relevance to MICRO

The proposed research seeks to develop useful computer software to extend the bounds of knowledge in processing large sets of visual information. A title used, [8] describes training as being within a domain. In contrast this research seeks to produce principles to govern creating software to approximate human skill. People often move forward within facial descriptions even when they have only partial knowledge. We seek to investigate dynamics of using limited and imperfect or approximate descriptions. We know from our own practical experience, as well as from that of police officers and artists, that there is value in partial visual knowledge.

This proposal presents a logical set of experiments and research developments to add to our knowledge when working outside, and accessing image information. It is research that is particularly timely (war on terrorism). It is also of direct interest to an industrial cosponsor, Facial Identification Digital Systems, and the Los Angeles County Sheriff's Department. This sponsor is working in the realm of mobile devices for police identification of facial images. The software research issues are of general interest impacting many other diverse realms, including digital libraries, entertainment (movie films, television show production), and mapping.

Finally the nature of mobile digital displays encourages communication that could lead to a display-modification sequence. This research proposes to create the software that could support system facilities involving user communications with visual databases. It involves investigating software algorithms using the features to increase effectiveness of such systems.

References

[1] Johnson, Jr., M. and Hayes-Roth, B., "Simultaneous Dynamic Integration of Diverse Reasoning Methods," Blackboard Architectures and Applications, Jagannathan, V., Dodhiawala, R. and Baum, L. eds., NY: Academic Press, 1989, pp. 57-73.

[2] Hallgren, C., "The hidden path," Proc. ACM ninth annual international conference on systems documentation, October 1991.

[3] McCall, J., "Police Artists Descriptor Rating System For In-Field Use," 1996; available here as Appendix A.

[4] McCall, J., "Digital Field System for Facial Identification," NASA/JPL Technology Transfer Program, 1998; available here as Appendix B.

[5] Klinger, A., and Salingaros, N., "A Pattern Measure," Environment and Planning B: Planning and Design, 27-4, July 2000, pp. 537-547; available here as Appendix C.

[6] Papadopouli, M. and Schulzrinne, H., "Sensor networks and energy management: Effects of power conservation, wireless coverage and cooperation on data dissemination among mobile devices," Proc. ACM International Symposium on Mobile Ad Hoc Networking & Computing, Oct. 2001.

[7] Klinger, A., "Data Structures," Encyclopedia of Physical Science and Technology 3rd Edition, Meyers, R. ed., NY: Academic Press, 4, 2002, pp. 263-276.

[8] Klinger, A., "Training and Thinking," Tau Beta Pi Bulletin, LXXV-3, Mar. 2002, pp. 3-5.

Appendix A

Appendix B

Appendix C

Appendix D

Personnel:

Allen Klinger is Professor of Engineering and Applied Science at University of California Los Angeles. He has held Visiting Professor positions at California Institute of Technology, Ben Gurion University of the Negev, Beersheva, Israel, and University of Hawaii. Before joining UCLA he was Researcher, Mathematics Department at the Rand Corporation. He received the Ph.D. degree from the University of California Berkeley, an M.S. from Caltech, and his B.E.E. from Cooper Union. He was a Fulbright Fellow to India. He also lectured in institutes and universities in Japan, England, France, Germany, Spain, Russia, Ukraine, Mexico, Chile, Vietnam and China.

He has published on image analysis, pattern recognition, optimization, control, computer applications, operations research, and applied mathematics and served on the Data Processing and Telecommunications Advisory Committee for the Los Angeles County Board of Supervisors. He has been a Rand and World Bank consultant. He has been a principal investigator on research projects for the Air Force Office of Scientific Research, National Science Foundation, and University of California.

Dr. Klinger is a Fellow of the Institute of Electrical and Electronics Engineers. He is a member, Chapter Advisor, and Division Director for the engineering honor society Tau Beta Pi.

Selected Publications

Edited books:

Klinger, A., Fu, K, and Kunii, T., Data Structures, Computer Graphics, and Pattern Recognition, NY: Academic Press, 1977.

Tanimoto, S., and Klinger, A., Structured Computer Vision, NY: Academic Press, 1980.

Klinger, A., Human Machine Interactive System, NY: Plenum Press, 1991.

Rhodes, M. L., and Klinger, A., "Conversational Text Input for Modifying Graphics Facial Images," Fuzzy Reasoning and its Applications, Mamdani, E., Gaines, B. eds., NY: Academic Press, 198l , 273-287; International Journal of Man-Machine Studies, 9: 653-667, 1977.

Klinger, A., "Data Structures," Encyclopedia of Physical Science and Technology 4, 3rd Edition, Meyers, R. ed., NY: Academic Press, 2002, pp. 263-276..

Current and Pending Support:

Allen Klinger is Principal Investigator on a $ 15, 000 Research Grant from the Verizon Foundation.

List of Reviewers