In this age of terrorism and identity theft, many organizations want a fast, secure way of verifying that you are who you say you are. Identification is a pressing problem all over the world, so it’s no wonder that an IEEE journal that concentrates on this field turned out to be the most-cited journal in 2004.

Passports, driver’s licenses, and other government-issued picture IDs no longer suffice, because they can be readily forged. Automated face recognition, fingerprints, voice identification, corneal or retinal scans, and DNA analysis can identify individuals effectively, but they are too slow for high-volume use, as when thousands of passengers are being screened by airport security. At its core, each of those biometric techniques relies on some form of pattern recognition. A person could be recognized, for example, by the shape and highlights of the face or the colors and shapes in a cornea.

Much R&D is under way to develop techniques for identifying individuals quickly, so it’s not surprising that a magazine that covers pattern recognition and analysis should prove popular. In fact, it can be argued that in 2004, the monthly IEEE Transactions on Pattern Analysis and Machine Intelligence (known as TPAMI) was the most popular of all. Of 209 electrical and electronics engineering publications, the magazine was ranked as the most-cited journal that year, according to the most recent science journal citation report, released in 2005 by Thomson Scientific, a book and journals publisher located in Philadelphia and London. TPAMI (pronounced tee-PAM-i) also ranked fifth in citations among 347 computer science journals. Thomson Scientific’s citation figures are widely valued because they are believed to be a measure of a journal’s impact in its field.

TPAMI introduces both fundamental and advanced techniques for solving problems in computer vision, shape, and texture analysis; artificial intelligence; medical image interpretation; document image processing; and biometrics, which is the study of automated methods for recognizing humans based upon one or more physical or behavioral traits.

“Face recognition is one of the hottest areas of research in biometrics,” says IEEE Senior Member David J. Kriegman, the volunteer editor of the 28-year-old monthly, which is sponsored by the IEEE Computer Society. Many researchers are interested in face recognition because of its potential for identifying people quickly. Applications range from video surveillance of crowds to the indexing of commercial or private libraries of photographs.

But the journal is just one way the IEEE disseminates information in the field of pattern recognition. Another way is to hold conferences of experts in the field. The IEEE Computer Society sponsors the annual IEEE Conference on Computer Vision and Pattern Recognition, which is scheduled for 17 to 23 June at the Kimmel Center at New York University, in New York City.

“I’M ME, HONEST!” “Facial images can be captured from a distance, and the images can be recorded, stored, sorted, and processed,” points out Shizuo Sakamoto, principal researcher at NEC Corp.’s Media and Information Research Laboratories, in Kanagawa, Japan. Because people’s faces are usually exposed, subjects don’t have to go through any special actions such as placing a hand or finger on a sensor or scraping the inside of a cheek for a DNA sample. (Sakamoto has written articles for the IEEE Systems, Man, and Cybernetics Society’s journals. His most recent article, “Development of Face Recognition Techniques at NEC Laboratories,” appeared in the society’s September eNewsletter.)

But there are innumerable complicating factors to automating the face-recognition chore. People’s faces change as they age, or they gain or lose weight. Cosmetics can transform a person’s appearance, as can illness or fatigue. “A person’s appearance can change significantly, even overnight,” says Harry Wechsler, an IEEE Fellow and director of the Distributed and Intelligent Computation Center at George Mason University, Fairfax, Va. Wechsler is also on TPAMI’s editorial board. “I would look very different tomorrow morning after a transatlantic flight, unshaven and exhausted, than I would today when I board.”

But even absent an actual facial alteration, “the appearance of a face changes drastically when pose and illumination are varied,” Sakamoto says. Sakamoto’s group at NEC is working on a face-recognition method that resists being fooled by variations in lighting or angle of pose. First they need to figure out how shadows and highlights vary for all angles of lighting and pose for a face; then they want to see whether that knowledge could help a computer identify someone from a two-dimensional image. To that end the group built a physical three-dimensional model of a specific individual’s face and set it opposite a single lamp that could be positioned at different angles around the model. With a range finder and reflectance meter that also could be moved around the model, they measured distances to features, plus the brightness of reflections, the darkness of shadows, the angles between visible features, and the length and shape of shadows and the features casting them.

Then, armed with all those biometric measurements as a basis for comparison, they presented a computer with one of 14 000 test images of 200 individuals, made under dramatically different illumination conditions and viewed from angles ranging up to 60 degrees sideward and 45 degrees upward from a frontal pose. Fully 94 percent of the time, no matter the angle of illumination or pose, the computer correctly identified the person, Sakamoto says. Pleased with that early success, he and his group are now exploring ways of applying their technique to a wider selection of human faces while minimizing the computation time required for comparing images.

“NO, IT’S NOT ME!” Getting a computer to recognize and track a single person in a crowd, another potential application for face recognition, is deceptively difficult. “First it has to detect or locate the face—which is not simple where a lot of people are moving, especially when the closed-circuit TV surveillance camera is positioned too high for a full view,” Wechsler explains. “Then the computer has to keep track of that face when it disappears behind other people’s heads, so it knows it is the same face when it reemerges.” For that reason, among others, it is becoming clear that traditional black-and-white closed-circuit TV surveillance systems must give way to color. “Color gives important additional information on the skin and its texture, and on distinguishing skin from cosmetics, fabrics, or other objects,” Wechsler says.

Even if a face in a crowd seems to bear a close resemblance to an image in a suspect watch-list database, “how close is close enough?” he asks. “Two pictures of the same person can be more different than pictures of two different people!”

That fact, he says, accounts for the high incidence of false-positive identifications by today’s surveillance security systems, alarming innocent citizens when they are detained because of their resemblance to a criminal suspect.

To combat false-positive identifications, Wechsler is investigating open-set recognition, which he and coauthor Fayin Li explain in “Open-Set Face Recognition Using Transduction,” which appeared in November’s TPAMI.

Positive identification of a person assumes that a subject has been enrolled, or has been seen before—such as when a person is photographed for a company ID card. That’s closed-set recognition. But in a surveillance system, the number of people who pass a security camera is far greater than the number of suspects whose images appear in a watch-list database. So, to avoid misidentifying innocent people, “you want a computer to be able to say, ‘I’ve never seen this face before.’ That is not at all the same thing as positive identification, which answers the question: ‘Who is this person?’” Wechsler explains. “The distinction is not trivial or straightforward.”

As much to avoid false positives as to ensure positive identification, Wechsler is intrigued by a new trend of combining methods for identifying people, pairings that “will be part of the next generation of passports,” he says. An obvious pairing might be combining faces with fingerprints. Another useful pairing might combine two-dimensional photographs with three-dimensional information, such as a holographic image, he says. But also useful would be a video or sequence of frames lasting a few seconds showing how a person speaks or moves the head or limbs. “People recognize each other at a distance by gestures as well as by appearance,” Wechsler says, referring to the way a person stands or walks and the way people gesture with their hands and make other expressive movements. Gestures might be especially helpful when searching for a suspect walking in a crowd, he says.

FOR GOOD GUYS, TOO Because changes in appearance, gestures, and coloring also reveal information about a person’s health, researchers are interested in potential medical applications of what’s coming to be called “face processing,” Wechsler says. “Suppose an elderly person has a stroke and is paralyzed and can’t speak but can understand conversation,” he says. “We would like to have the computer as an assistant, so face processing might help family and friends understand what the person is truly feeling and thinking.”

Some early commercial applications of face processing may help people in their everyday lives. “One company is testing face-recognition software to help organize libraries of photographs,” TPAMI’s Kriegman says, “so you could, for example, ask it to go through hundreds of digital images of your family and pick out all those of your favorite aunt.”

Without doubt, the software is improving. “A 2002 test of face-recognition software conducted by NIST [the U.S. National Institute of Standards and Technology] concluded that it is about as accurate as fingerprinting was in 1998—in other words, not too shabby,” Kriegman says. Right now, NIST is conducting another round of competitive testing and is scheduled to publish its results this year.