My Research

The remarkable ability of humans to understand a real-world novel scene rapidly and accurately is widely known. Whether we are traveling to an unfamiliar place, quickly changing television channels, or simply trying to cross the road, our visual system is working with superb efficiency and accuracy in rapidly grasping the meaning of each scene. Examine the rapid sequence of scenes on the right. Is it fair to say that you can indeed grasp the gist of most?

Despite the ample work on scene understanding, the bulk of this visual process remains an open question, both behaviorally and computationally. In my research I conduct both behavioral and computational studies that aim to make further progress towards theoretical and computational scene recognition.


Perceptual Relations

A key issue in this context is perceptual relations between scene categories, a possibility that has been rarely considered either in the perceptual literature or the computational literature. However, even intuitively, when our visual system observes a bedroom scene for a fraction of a second and ``deliberates'' how to categorize it, what possibly comes to mind in addition to ``bedroom'' are perhaps classes like ``living room'' or ``kitchen''. It appears as if our visual system does not even consider possibilities such as ``coast'' or ``highway'', or more generally, scenes which are perceptually ``distant'' from the observable reference class. Put differently, prior knowledge about the perceptual relations between the different categories of scenes may help facilitate better, more efficient, and faster categorization.


Web-Based Experiment

The formal manifestation of the basic idea described above requires the extraction of the perceptual relations between scene classes. While we use controlled lab experiments to collect these data for small number of classes (see papers), doing the same with larger sets becomes infeasible without an enormous number of subjects, something that may only be possible by harnessing the power of the web. We therefore invite you to play our perceptual game and contribute to this data collection in a Pair-Matching Categorization (PMC) experiment. Please click on the image on the right to begin. Please note that the web-based experiment requires a web browser and just 4-5 short minutes of your time. (Unfortunately, at this point the software is supported on Windows platforms only). We greatly appreciate your participation and we thank you for helping the collection of these data.