A Need for Theory

With the expanding VA field, there is a need for a coherent theoretical framework to guide development of tools and evaluation of their effectiveness. This framework should be derived from the large number of disciplines that have been using VA to support human decision making across a broad range of application domains. From the field of human factors, the framework draws upon research on user interface design, information architecture, perceptual psychology, cognitive theory, and naturalistic decision making; other influential domains include research and practice in fields of graphics design, communication theory, systems evaluation, statistics, and data representations and theory of knowledge.

Framework
Effectiveness of VA tools depends on their ability to convey optimal representations (mental
models) of the problem.


A scientific foundation not only informs the design of VA systems but also facilitates establishment of metrics to evaluate their effectiveness. The evaluation focus of the Cognitive Foundations for Visual Analytics project complements other National Visualization and Analytics Center research efforts aiming to assess the benefits of VA technologies, including the Visual Analytics Science and Technology (VAST) Challenge competitions that address the efficacy of VA approaches in providing decision makers with effective visualizations for solving complex problems in a variety of domains.

Cognitive theory suggests that the impact of VA on decision making is a function of the extent to which VA tools contribute to correct decisions and increased understanding. In this view, the effectiveness of VA tools largely depends upon the consistency between the external representation they provide and the internal representations used by expert decision makers. Internal representations and problem-solving strategies may be characterized using mental models—highly-organized and dynamic knowledge structures, developed over time from experience, that enable experienced decision makers to correctly assess and interpret the current situation and to select an appropriate action. Mental models help operators rapidly integrate and assess the large amount of information that is inherent in complex operational environments.

Our Approach

§3425a2c08b024002f02c4e14b52112fc§
This research is examining scientific literature in cognitive science, human factors, and related fields to identify concepts and research results that inform the application of VA technologies to meet operational challenges. This work may also lead to a better understanding of VA tools’ contribution to insight formation. To provide a structure for the analysis, we are categorizing VA research to reveal where different cognitive science concepts and empirical results inform VA tool design and evaluation. By updating previous taxonomies for VA approaches and applications, we hope to provide a more comprehensive framework and benchmarks for this expanding field. More specifically, the focus of the research is on Scientific Foundations for visualization technology in problem solving, decision making, and insight research; Visualization Taxonomy to structure the research; and Experiment Designs and Scenarios for evaluating effects of visualization on decision making and insight. Follow-on research will define requirements for the establishment of a test bed laboratory for empirical research and validation of visualization tools.

Impact

This project will provide a cognitive-science-based foundation to guide research and evaluation of the impact of VA tools on critical decision-making tasks faced by the operational community. A set of comprehensive and consistent benchmarks will facilitate the application of rigorous evaluation techniques and enable deployment of more effective tools for the operational community.

We Want You to Get Involved...

Want to get involved and help form this research? We encourage you to join the discussion at the VACommunity Site. Simply surf over to http://www.VACommunity.org and select Cognitive Foundations from the Communities menu. Register and you’ll be ready to get involved. We’re excited to hear your thoughts and ideas. There is also a blog where we will be able to keep you informed of developments throughout the duration of the project.

Stakeholders

The research team is committed to working with a broad range of public and private entities, including governmental agencies, industry, and academia, to improve the production, quality, and evaluation of visualization research technologies and their impact on decision-making processes in an operational environment.

Team Members

Pacific Northwest National Laboratory: Frank L. Greitzer, Andrew J. Cowell, Olga A. Kuchar, Lyndsey Franklin, Christine F. Noonan, and Borik Zadeh.

Point of Contact

Frank L. Greitzer, Chief Scientist, Pacific Northwest National Laboratory, frank.greitzer at pnl.gov

Image

Share this article: Download this article as a PDF