“A picture is worth a thousands words.” There is certainly some truth in the saying, but is it true always? - No, also pictures need captions. Abstract facts can often be described much easier in a few words than a picture. Hence, images and text usually complement each other. This is particularly true if the image is a visualization: labels make sense of the abstract geometric shapes, a legend needs to explain the data mapping, and only a text connects the displayed data into a story. Yet, text and visualization are often treated as separate entities, reducing the accessibility of the presented data and hindering knowledge extraction. In this project, we want bring visualization and text closer together by interactively linking them in visually augmented digital text documents.
The project investigates two application scenarios. First, scientific publications aim at communicating insights gained from data to the readers. Diagrams and charts have become integral part of scientific publications, but are only loosely linked to the text by references. Making those links interactively explorable, we would like to strengthen the integration. For instance, clicking on a statement in the text, the diagram could adapt to a version that provides more explanations or evidence underlining the textual statement. Or, vice versa, selecting a data point in the visualization, related text sections become highlighted. Embedded additional texts and visualizations explain the relationship. Even more directly, word-sized graphics, known as sparklines, can augment the text with extra information. The second scenario will investigate similar approaches in context of software engineering. Tightly integrated visualization enrich software documentation, requirements, or bug reports with data retrieved from the software system.
The project is supported by the Baden-Württemberg Stiftung as part of the Postdoctoral Fellowship for Leading Early Career Researchers granted to VISUS researcher Fabian Beck.