Visual search refers to the act of looking for targets in a visual world containing distractors. This could be a search for the cat in the living room, a typo in this paragraph, or a tumor in a chest x-ray. In all these cases and others, fundamental capacity limitations assure that we cannot fully process all visual input at one time. To deal with those limitations, humans (and any animals with a substantial nervous system) have mechanisms of selective attention that allow them to more extensively process some stimuli before moving on to others. Fortunately, attention is not deployed randomly but is under the systematic influence of bottom-up processes (driven by the stimulus) and top-down processes (driven by the conscious or unconscious desires of the searcher). Other factors, like observer experience and scene structure, also influence search. Together, these factors make most routine searches seem effortless, although the world abounds with search tasks that are more difficult, time-consuming, and error prone.

History

Visual search as a term and a topic of research is about a century old (e.g., Kingsley, 1932) with extensive work beginning after World War II (e.g., Mackworth, 1948). Of course, people have known forever that they must look for things: “Go bid my woman search for a jewel that too casually hath left mine arm,” says Shakespeare’s Imogen (Cymbeline, Act 2, Scene 3), and aspects of the search process have been part of scientific and philosophical inquiry since antiquity (Hatfield, 1998). For instance, Aristotle discusses the narrowing of attention in De Anima, and the steering of attention is found in Lucretius (De Rerum Natura) [see Attention].

By the 1950s and 60s, visual search was becoming an active topic of research, with basic laboratory work involving search through arrays of letters (Green & Anderson, 1956; Neisser et al., 1963) and more applied work like searching for tumors in x-rays (Tuddenham & Calvert, 1961) or mariners in the ocean (Koopman, 1956). The topic became central to the study of visual cognition in the 1970s and 80s with the work of researchers like Sperling, Shiffrin, Egeth, Townsend, Sternberg, and others (see Shiffrin, 1988 for a review) and, probably most importantly, with the publication of Anne Treisman’s feature integration theory (Treisman & Gelade, 1980). Her work crystalized many of the core concepts that have defined the field in the subsequent decades.

Core concepts

Figure 1 illustrates many core concepts in visual search. Find arrows pointing to the right. First, you must search. You cannot simply see the targets without search. If you swiftly detected a target, find the second one. The tendency to miss a second target is known as satisfaction of search, an important topic in medical image searches. Search for the target could proceed in two ways. There are overt movements of the eyes to look at one spot or another. There are also covert deployments of attention. Notice that, if you fixate your eyes at the center of the figure, you can find the blue target arrow without moving your eyes.

Some aspects of the image may be processed in parallel, across the entire image at once. This includes a limited set of basic features like color and orientation, known as preattentive features because they appear to be available before attention is directed to an object (Wolfe & Horowitz, 2017). These features can be used to guide attention. Thus, if asked to find tilted green arrows, you can guide your attention to arrows with tilted orientations (noticing that they are clustered on the right) or to green arrows (noticing that they are distributed across the image). Notice that it is easier to find a tilted arrow in the horizontal region than a horizontal arrow in the tilted region, a search asymmetry (Treisman & Gormican, 1988).

Guidance can take two main forms. Top-down guidance involves volitional control of the human “search engine”; for example, to look for “green” and “tilted” in the figure. The figure’s red item grabs attention in a bottom-up, stimulus-driven manner, largely independent of observer desires. An item having unique, salient features (here, color, shape, and size) seems to immediately grab attention and is said to pop out of the display. Calculating stimulus salience is important in many computational models.

Many methods are used to study visual search. The most typical behavioral technique is to present a search display and to ask the observer to respond as quickly and accurately as possible. Search efficiency is defined by the response time x set size function (often with correction for errors). Figure 2 shows idealized response time data. Pop-out tasks produce efficient search (e.g., finding that salient, red symbol). In contrast, a search for the letter T among Ls would be inefficient. Guided searches lie between efficient and inefficient. The eyes are deployed less often (3–4/sec) than for covert attention, so if each item must be fixated, the slope will be much steeper (as shown in Figure 2).

Additional factors become important in real-world search. To get a sense of this, search for people in the scene in Figure 3. This is quick, although they are not unique in color or size in this image. Here, scene guidance becomes important (Vo et al., 2019). You rapidly understand the gist of the scene; its meaning and layout (Oliva, 2005). Your history of searching similar scenes recently (e.g., on the last trial) and across your life also influences your search (Anderson et al., 2021). Thus, you guide search for humans to horizontal surfaces, not to the sky and probably not up a tree. The specific search history of a radiologist or a birder is an important part of their expertise in their search domains.

Questions, controversies, and new developments

Among current questions in the study of visual search, investigators want to know when and why attention can be captured by a stimulus despite a top-down desire to attend elsewhere (Luck et al., 2021). They try to understand how observers’ know when to end a search, especially an unsuccessful one (e.g., how many dogs are in the scene above?; Mazor & Fleming, 2022). Modelers debate if search is primarily serial, parallel, or both (Townsend & Wenger, 2004). They wonder if artificial aids (artificial intelligence, global positioning systems, etc.) will, with time, impair our normal search processes by delivering the answer without the search (Ying et al., 2024).

Broader connections

Behavioral studies of visual search contact the study of attention more generally. There is very considerable work on the neuroscience of visual search (e.g., on scene search, see Segraves, 2023) [see Visual Cognitive Neuroscience]. Various other tasks like foraging have an obvious relationship to search (Hills et al., 2008). The topic has significant real-world applications to tasks like driving, medical image perception, computer interface design, and a wide range of military and security concerns. Each of these applied domains raises questions about the role of expertise and the possibility that some people might have special aptitudes for specific tasks or for search in general. While there do appear to be reliable individual differences, it has proven difficult to determine in advance who might make a great airport screener or radiologist.

Acknowledgments

J. M. W. is supported by National Institutes of Health EY017001 and CA207490 and National Science Foundation 2146617.

Further reading 

• Hulleman, J., & Olivers, C. N. L. (2017). The impending demise of the item in visual search. Behavioral and Brain Sciences, 40, e132. https://doi.org/10.1017/S0140525X15002794

• Kristjansson, A., & Egeth, H. E. (2020). How feature integration theory integrated cognitive psychology, neurophysiology, and psychophysics. Attention, Perception & Psychophysics, 82(1), 7–23. https://doi.org/10.3758/s13414-019-01803-7

• Treisman, A., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97-136. https://doi.org/10.1016/0010-0285(80)90005-5

• Wolfe, J. M. (2023). Visual Search. In O. Braddick (Ed.), Oxford research encyclopedia of psychology. Oxford University Press.