Publications

Abstract

Recently, neural network for scene flow estimation show impressive results on automotive data such as the KITTI benchmark. However, despite of using sophisticated rigidity assumptions and parametrizations, such networks are typically limited to only two frame pairs which does not allow them to exploit temporal information. In our paper we address this shortcoming by proposing a novel multi-frame approach that considers an additional preceding stereo pair. To this end, we proceed in two steps: Firstly, building upon the recent RAFT-3D approach, we develop an improved two-frame baseline by incorporating an advanced stereo method. Secondly, and even more importantly, exploiting the specific modeling concepts of RAFT-3D, we propose a U-Net architecture that performs a fusion of forward and backward flow estimates and hence allows to integrate temporal information on demand. Experiments on the KITTI benchmark do not only show that the advantages of the improved baseline and the temporal fusion approach complement each other, they also demonstrate that the computed scene flow is highly accurate. More precisely, our approach ranks second overall and first for the even more challenging foreground objects, in total outperforming the original RAFT-3D method by more than 16%. Code is available at https://github.com/cv-stuttgart/M-FUSE.

Abstract

Our built world is one of the most important factors for a livable future, accounting for massive impact on resource and energy use, as well as climate change, but also the social and economic aspects that come with population growth. The architecture, engineering, and construction industry is facing the challenge that it needs to substantially increase its productivity, let alone the quality of buildings of the future. In this article, we discuss these challenges in more detail, focusing on how digitization can facilitate this transformation of the industry, and link them to opportunities for visualization and augmented reality research. We illustrate solution strategies for advanced building systems based on wood and fiber.

Abstract

This work investigates and compares the performance of node-link diagrams, adjacency matrices, and bipartite layouts for visualizing networks. In a crowd-sourced user study (n = 150), we measure the task accuracy and completion time of the three representations for different network classes and properties. In contrast to the literature, which covers mostly topology-based tasks (e.g., path finding) in small datasets, we mainly focus on overview tasks for large and directed networks. We consider three overview tasks on networks with 500 nodes: (T1) network class identification, (T2) cluster detection, and (T3) network density estimation, and two detailed tasks: (T4) node in-degree vs. out-degree and (T5) representation mapping, on networks with 50 and 20 nodes, respectively. Our results show that bipartite layouts are beneficial for revealing the overall network structure, while adjacency matrices are most reliable across the different tasks.

Abstract

Immersive analytics often takes place in virtual environments which promise the users immersion. To fulfill this promise, sensory feedback, such as haptics, is an important component, which is however not well supported yet. Existing haptic devices are often expensive, stationary, or occupy the user’s hand, preventing them from grasping objects or using a controller. We propose PropellerHand, an ungrounded hand-mounted haptic device with two rotatable propellers, that allows exerting forces on the hand without obstructing hand use. PropellerHand is able to simulate feedback such as weight and torque by generating thrust up to 11 N in 2-DOF and a torque of 1.87 Nm in 2-DOF. Its design builds on our experience from quantitative and qualitative experiments with different form factors and parts. We evaluated our prototype through a qualitative user study in various VR scenarios that required participants to manipulate virtual objects in different ways, while changing between torques and directional forces. Results show that PropellerHand improves users’ immersion in virtual reality. Additionally, we conducted a second user study in the field of immersive visualization to investigate the potential benefits of PropellerHand there.

Abstract

We present an exploratory study on the accessibility of images in publications when viewed with color vision deficiencies (CVDs). The study is based on 1,710 images sampled from a visualization dataset (VIS30K) over five years. We simulated four CVDs on each image. First, four researchers (one with a CVD) identified existing issues and helpful aspects in a subset of the images. Based on the resulting labels, 200 crowdworkers provided  30,000 ratings on present CVD issues in the simulated images. We analyzed this data for correlations, clusters, trends, and free text comments to gain a first overview of paper figure accessibility. Overall, about 60 % of the images were rated accessible. Furthermore, our study indicates that accessibility issues are subjective and hard to detect. On a meta-level, we reflect on our study experience to point out challenges and opportunities of large-scale accessibility studies for future research directions.

Abstract

Modern machines continuously log status reports over long periods of time, which are valuable data to optimize working routines. Data visualization is a commonly used tool to gain insights into these data, mostly in retrospective, e.g. to determine causal dependencies between faults of different machines. We present an approach to bring such visual analyses to the shop floor to support reacting to faults in real time. This approach combines combines spatio-temporal analyses of time series using a handheld touch device with augmented reality for live monitoring. Important information augments machines directly in their real-world context and detailed logs of current and historical events are displayed on the handheld device. In collaboration with an industry partner, we designed and tested our approach on a live production line to obtain feedback from operators. We compare our approach for monitoring and analysis with existing solutions that are currently deployed.

Abstract

Transient behavior occurs when a running software system changes from one steady-state to another. In microservice systems, such disruptions can, for example, be caused by continuous deployment, self-adaptation, and various failures. Although transient behavior could be captured in non-functional requirements, little is known of how that is handled in practice. Our objective was to study how architects and engineers approach runtime disruptions, which challenges they face, whether or not they specify transient behavior, and how currently employed tools and methods can be improved. To this end, we conducted semi-structured interviews with five experienced practitioners from major companies in Germany. We found that a big challenge in the industry is a lack of awareness of transient behavior by software stakeholders. Consequently, they often do not consider specifying it in non-functional requirements. Additionally, better tooling is needed to reduce the effort of analyzing transient behavior. We present two prototypes that we developed corresponding to these findings to improve the current situation. Beyond that, the insights we present can serve as pointers for interesting research directions for other researchers.

Abstract

The visualization of results while the simulation is running is increasingly common in extreme scale computing environments. We present a novel approach for in situ generation of image databases to achieve cost savings on supercomputers. Our approach, a hybrid between traditional inline and in transit techniques, dynamically distributes visualization tasks between simulation nodes and visualization nodes, using probing as a basis to estimate rendering cost. Our hybrid design differs from previous works in that it creates opportunities to minimize idle time from four fundamental types of inefficiency: variability, limited scalability, overhead, and rightsizing. We demonstrate our results by comparing our method against both inline and in transit methods for a variety of configurations, including two simulation codes and a scaling study that goes above 19K cores. Our findings show that our approach is superior in many configurations. As in situ visualization becomes increasingly ubiquitous, we believe our technique could lead to significant amounts of reclaimed cycles on supercomputers.

Abstract

Adaptive visualization and interfaces pervade our everyday tasks to improve interaction from the point of view of user performance and experience. This approach allows using several user inputs, whether physiological, behavioral, qualitative, or multimodal combinations, to enhance the interaction. Due to the multitude of approaches, we outline the current research trends of inputs used to adapt visualizations and user interfaces. Moreover, we discuss methodological approaches used in mixed reality, physiological computing, visual analytics, and proficiency-aware systems. With this work, we provide an overview of the current research in adaptive systems.

Abstract

Authoring situated visualizations in-situ is challenging due to the need of writing code in a mobile and highly dynamic fashion. To provide better support for that, we define requirements for text input methods that target situated visualization authoring. We identify wearable chorded keyboards as a potentially suitable method that fulfills some of these requirements. To further investigate this approach, we tailored a chorded keyboard device to visualization authoring, developed a learning application, and conducted a pilot user study. Our results confirm that learning a high number of chords is the main barrier for adoption, as in other application areas. Based on that, we discuss ideas on how chorded keyboards with a strongly reduced alphabet, hand gestures, and voice recognition might be used as a viable, multi-modal support for authoring situated visualizations in-situ.

Abstract

We present RagRug, an open-source toolkit for situated analytics. The abilities of RagRug go beyond previous immersive analytics toolkits by focusing on specific requirements emerging when using augmented reality (AR) rather than virtual reality. RagRug combines state of the art visual encoding capabilities with a comprehensive physical-virtual model, which lets application developers systematically describe the physical objects in the real world and their role in AR. We connect AR visualization with data streams from the Internet of Things using distributed dataflow. To this aim, we use reactive programming patterns so that visualizations become context-aware, i.e., they adapt to events coming in from the environment. The resulting authoring system is low-code; it emphasises describing the physical and the virtual world and the dataflow between the elements contained therein. We describe the technical design and implementation of RagRug, and report on five example applications illustrating the toolkit's abilities.

Abstract

Research data curation is the act of carefully preparing research data and artifacts for sharing and long-term preservation. Research data management is centrally implemented and formally defined in a data management plan to enable data curation. In tandem, data curation and management facilitate research repeatability. In contrast to other research fields, data curation and management in visualization are not yet part of the researcher’s compendium. In this position paper, we discuss the unique challenges visualization faces and propose how data curation can be practically realized. We share eight lessons learned in managing data in two large research consortia, outline the larger curation workflow, and define the typical roles. We complement our lessons with minimum criteria for selecting a suitable data repository and five challenging scenarios that occur in practice. We conclude with a vision of how the visualization research community can pave the way for new curation standards.

Abstract

We contribute MolecuSense, a virtual version of a physical molecule construction kit, based on visualization in Virtual Reality (VR) and interaction with force-feedback gloves. Targeting at chemistry education, our goal is to make virtual molecule structures more tangible. Results of an initial user study indicate that the VR molecular construction kit was positively received. Compared to a physical construction kit, the VR molecular construction kit is on the same level in terms of natural interaction. Besides, it fosters the typical digital advantages though, such as saving, exporting, and sharing of molecules. Feedback from the study participants has also revealed potential future avenues for tangible molecule visualizations.

Abstract

We propose a data-driven approach to music instrument practice that allows studying patterns and long-term trends through visualization. Inspired by life logging and fitness tracking, we imagine musicians to record their practice sessions over the span of months or years. The resulting data in the form of MIDI or audio recordings can then be analyzed sporadically to track progress and guide decisions. Toward this vision, we started exploring various visualization designs together with a group of nine guitarists, who provided us with data and feedback over the course of three months.

Abstract

We propose a 3D immersive visualization environment for analyzing the right hand movements of a cello player. To achieve this, we track the position and orientation of the cello bow and record audio. As movements mostly occur in a shallow volume and the motion is therefore mostly two-dimensional, we use the third dimension to encode time. Our concept further explores various mappings from motion and audio data to spatial and other visual attributes. We work in close cooperation with a cellist and plan to evaluate our prototype through a user study with a group of cellists in the near future.

Abstract

When visualizations are used to augment text, geographical information on a map, or on mobile devices, there often is not enough space to show complex data with approaches like juxtaposed visualizations or coordinated views. To alleviate this issue, we propose the use of animated transitions between several small-scale visualizations. We discuss design considerations for animated transitions in small-scale visualizations. We further present the results of a study on the effectiveness of those animated transitions in conveying information and attribute relations, and the mental load of following the animated transitions.

Abstract

We present an extension of multidimensional scaling (MDS) to uncertain data, facilitating uncertainty visualization of multidimensional data. Our approach uses local projection operators that map high-dimensional random vectors to low-dimensional space to formulate a generalized stress. In this way, our generic model supports arbitrary distributions and various stress types. We use our uncertainty-aware multidimensional scaling (UAMDS) concept to derive a formulation for the case of normally distributed random vectors and a squared stress. The resulting minimization problem is numerically solved via gradient descent. We complement UAMDS by additional visualization techniques that address the sensitivity and trustworthiness of dimensionality reduction under uncertainty. With several examples, we demonstrate the usefulness of our approach and the importance of uncertainty-aware techniques.

Abstract

This paper provides a brief overview of uncertainty visualization along with some fundamental considerations on uncertainty propagation and modeling. Starting from the visualization pipeline, we discuss how the different stages along this pipeline can be affected by uncertainty and how they can deal with this and propagate uncertainty information to subsequent processing steps. We illustrate recent advances in the field with a number of examples from a wide range of applications: uncertainty visualization of hierarchical data, multivariate time series, stochastic partial differential equations, and data from linguistic annotation.

Abstract

Many classical and learning-based optical flow methods rely on hierarchical concepts to improve both accuracy and robustness. However, one of the currently most successful approaches -- RAFT -- hardly exploits such concepts. In this work, we show that multi-scale ideas are still valuable. More precisely, using RAFT as a baseline, we propose a novel multi-scale neural network that combines several hierarchical concepts within a single estimation framework. These concepts include (i) a partially shared coarse-to-fine architecture, (ii) multi-scale features, (iii) a hierarchical cost volume and (iv) a multi-scale multi-iteration loss. Experiments on MPI Sintel and KITTI clearly demonstrate the benefits of our approach. They show not only substantial improvements compared to RAFT, but also state-of-the-art results -- in particular in non-occluded regions.

Abstract

Immersive Analytics is concerned with the systematic examination of the benefits and challenges of using immersive environments for data analysis, and the development of corresponding designs that improve the quality and efficiency of the analysis process. While immersive technologies are now broadly available, practical solutions haven’t received broad acceptance in real-world applications outside of several core areas, and proper guidelines on the design of such solutions are still under development. Both fundamental research and applications bring together topics and questions from several fields, and open a wide range of directions regarding underlying theory, evidence from user studies, and practical solutions tailored towards the requirements of application areas. We give an overview on the concepts, topics, research questions, and challenges.

Abstract

We propose a novel method for the computation of Jacobi sets in 2D domains. The Jacobi set is a topological descriptor based on Morse theory that captures gradient alignments among multiple scalar fields, which is useful for multi-field visualization. Previous Jacobi set computations use piecewise linear approximations on triangulations that result in discretization artifacts like zig-zag patterns. In this paper, we utilize a local bilinear method to obtain a more precise approximation of Jacobi sets by preserving the topology and improving the geometry. Consequently, zig-zag patterns on edges are avoided, resulting in a smoother Jacobi set representation. Our experiments show a better convergence with increasing resolution compared to the piecewise linear method. We utilize this advantage with an efficient local subdivision scheme. Finally, our approach is evaluated qualitatively and quantitatively in comparison with previous methods for different mesh resolutions and across a number of synthetic and real-world examples.

Abstract

We present a new topological connection method for the local bilinear computation of Jacobi sets that improves the visual representation while preserving the topological structure and geometric configuration. To this end, the topological structure of the local bilinear method is utilized, which is given by the nerve complex of the traditional piecewise linear method. Since the nerve complex consists of higher-dimensional simplices, the local bilinear method (visually represented by the 1-skeleton of the nerve complex) leads to clutter via crossings of line segments. Therefore, we propose a homotopy-equivalent representation that uses different collapses and edge contractions to remove such artifacts. Our new connectivity method is easy to implement, comes with only little overhead, and results in a less cluttered representation.

Abstract

Comparing mobile eye tracking data from multiple participants without information about areas of interest (AOIs) is challenging because of individual timing and coordinate systems. We present a technique, the gaze spiral, that visualizes individual recordings based on image content of the stimulus. The spiral layout of the slitscan visualization is used to create a compact representation of scanpaths. The visualization provides an overview of multiple recordings even for long time spans and helps identify and annotate recurring patterns within recordings. The gaze spirals can also serve as glyphs that can be projected to 2D space based on established scanpath metrics in order to interpret the metrics and identify groups of similar viewing behavior. We present examples based on two egocentric datasets to demonstrate the effectiveness of our approach for annotation and comparison tasks. Our examples show that the technique has the potential to let users compare even long-term recordings of pervasive scenarios without manual annotation.

Abstract

A large number of modern video background modeling algorithms deal with computational costly minimization problems that often need parameter adjustments. While in most cases spatial and temporal constraints are added artificially to the minimization process, our approach is to exploit Dynamic Mode Decomposition (DMD), a spectral decomposition technique that naturally extracts spatio-temporal patterns from data. Applied to video data, DMD can compute background models. However, the original DMD algorithm for background modeling is neither efficient nor robust. In this paper, we present an equivalent reformulation with constraints leading to a more suitable decomposition into fore- and background. Due to the reformulation, which uses sparse and low-dimensional structures, an efficient and robust algorithm is derived that computes accurate background models. Moreover, we show how our approach can be extended to RGB data, data with periodic parts, and streaming data enabling a versatile use.

Abstract

Redundancy matrices provide insights into the load carrying behavior of statically indeterminate structures. This information can be employed for the design and analysis of structures with regard to certain objectives, for example reliability, robustness, or adaptability. In this context, the structure is often iteratively examined with the help of slight adjustments. However, this procedure generally requires a high computational effort for the recalculation of the redundancy matrix due to the necessity of costly matrix operations. This paper addresses this problem by providing generic algebraic formulations for efficiently updating the redundancy matrix (and related matrices). The formulations include various modifications like adding, removing, and exchanging elements and are applicable to truss and frame structures. With several examples, we demonstrate the interaction between the formulas and their mechanical interpretation. Finally, a performance test for a scaleable structure is presented.

Abstract

Frequency-based decomposition of time series data is used in many visualization applications. Most of these decomposition methods (such as Fourier transform or singular spectrum analysis) only provide interaction via pre- and post-processing, but no means to influence the core algorithm. A method that also belongs to this class is Dynamic Mode Decomposition (DMD), a spectral decomposition method that extracts spatio-temporal patterns from data. In this paper, we incorporate frequency-based constraints into DMD for an adaptive decomposition that leads to user-controllable visualizations, allowing analysts to include their knowledge into the process. To accomplish this, we derive an equivalent reformulation of DMD that implicitly provides access to the eigenvalues (and therefore to the frequencies) identified by DMD. By utilizing a constrained minimization problem customized to DMD, we can guarantee the existence of desired frequencies by minimal changes to DMD. We complement this core approach by additional techniques for constrained DMD to facilitate explorative visualization and investigation of time series data. With several examples, we demonstrate the usefulness of constrained DMD and compare it to conventional frequency-based decomposition methods.

Abstract

Linking and visualizing multiple types of entities in a DH knowledge graph generates the need to deal with multiple types of data and media modalities both on the designer and the user side. The InTaVia project develops synoptic visual representations for a multimodal historical knowledge graph which draws together transnational data about cultural objects and historical actors. In this paper we reflect on the question how to integrate and mediate the informational and visual affordances of both kinds of cultural data with hybrid designs and show how a user-centered design process can help to ground the required selections and design choices in an empirical procedure.

Abstract

We introduce a novel visual-interactive approach for analyzing, understanding, and correcting neural machine translation. Our system supports users in automatically translating documents using neural machine translation and identifying and correcting possible erroneous translations. User corrections can then be used to fine-tune the neural machine translation model and automatically improve the whole document. While translation results of neural machine translation can be impressive, there are still many challenges such as over- and under-translation, domain-specific terminology, and handling long sentences, making it necessary for users to verify translation results. Our system aims at supporting users in this task. Our visual analytics approach combines several visualization techniques in an interactive system. A parallel coordinates plot with multiple metrics related to translation quality can be used to find, filter, and select translations that might contain errors. An interactive beam search visualization and graph- or matrix-based visualizations for attention weights can be used for post-editing and understanding machine-generated translations. The machine translation model is updated from user corrections to improve the translation quality of the whole document. We designed our approach for an LSTM-based translation model and extended it to also include the Transformer architecture. We show for representative examples possible mistranslations and how to use our system to deal with them. A user study revealed that many participants favor such a system over manual text-based translation, especially for translating large documents. Furthermore, we performed quantitative computer-based experiments that show that our system can be used to improve translation quality and reduce post-editing efforts for domain-specific documents.

Abstract

In this article, we discuss how Visualization (VIS) with Machine Learning (ML) could mutually benefit from each other. We do so through the lens of our own experience working at this intersection for the last decade. Particularly we focus on describing how VIS supports explaining ML models and aids ML-based Dimensionality Reduction techniques in solving tasks such as parameter space analysis. In the other direction, we discuss approaches showing how ML helps improve VIS, such as applying ML-based automation to improve visualization design. Based on the examples and our own perspective, we describe a number of open research challenges that we frequently encountered in our endeavors to combine ML and VIS.

Abstract

Recent approaches for surgical activity localization rely on motion features derived from the optical flow (OF). However, although they consider state-of-the-art CNNs when computing the OF, they typically resort to pre-trained implementations which are domain-unaware. We address this problem in two ways: (i) Using the pre-trained OF-CNN of recent localization approach, we analyze the impact of video properties such as reflections, motion and blur on the quality of the OF from neurosurgical data. (ii) Based on this analysis, we design a specifically tailored synthetic training dataset which allows us to customize the pre-trained OF-CNN for surgical activity localization. Our evaluation clearly shows the benefit of this customization approach. It not only leads to an improved accuracy of the OF itself but, even more importantly, also to an improved performance for the actual localization task.

Abstract

We propose a visual approach for interactive, AI-assisted composition that serves as a compromise between fully automatic and fully manual composition. Instead of generating a whole piece, the AI takes on the role of an assistant that generates short melodies for the composer to choose from and adapt. In an iterative process, the composer queries the AI for continuations or alternative fill-ins, chooses a suggestion, and adds it to the piece. As listening to many suggestions would take time, we explore different ways to visualize them, to allow the composer to focus on the most interesting-looking melodies. We also present the results of a qualitative evaluation with five composers.

Abstract

We introduce a conceptual model for scalability designed for visualization research. With this model, we systematically analyze over 120 visualization publications from 1990 to 2020 to characterize the different notions of scalability in these works. While many papers have addressed scalability issues, our survey identifies a lack of consistency in the use of the term in the visualization research community. We address this issue by introducing a consistent terminology meant to help visualization researchers better characterize the scalability aspects in their research. It also helps in providing multiple methods for supporting the claim that a work is “scalable.” Our model is centered around an effort function with inputs and outputs. The inputs are the problem size and resources, whereas the outputs are the actual efforts, for instance, in terms of computational run time or visual clutter. We select representative examples to illustrate different approaches and facets of what scalability can mean in visualization literature. Finally, targeting the diverse crowd of visualization researchers without a scalability tradition, we provide a set of recommendations for how scalability can be presented in a clear and consistent way to improve fair comparison between visualization techniques and systems and foster reproducibility.

Abstract

We introduce relaxed dot plots as an improvement of nonlinear dot plots for unit visualization. Our plots produce more faithful data representations and reduce moire´ effects. Their contour is based on a customized kernel frequency estimation to match the shape of the distribution of underlying data values. Previous nonlinear layouts introduce column-centric nonlinear scaling of dot diameters for visualization of high-dynamic-range data with high peaks. We provide a mathematical approach to convert that column-centric scaling to our smooth envelope shape. This formalism allows us to use linear, root, and logarithmic scaling to find ideal dot sizes. Our method iteratively relaxes the dot layout for more correct and aesthetically pleasing results. To achieve this, we modified Lloyd's algorithm with additional constraints and heuristics. We evaluate the layouts of relaxed dot plots against a previously existing nonlinear variant and show that our algorithm produces less error regarding the underlying data while establishing the blue noise property that works against moire´ effects. Further, we analyze the readability of our relaxed plots in three crowd-sourced experiments. The results indicate that our proposed technique surpasses traditional dot plots.

Abstract

We propose a three-step concept and visual design for supporting the visual exploration of high-dimensional data in scatterplots through eye-tracking. First, we extract subsets in the underlying data using existing classifications, automated clustering algorithms, or eye-tracking. For the latter, we map gaze to the underlying data dimensions in the scatterplot. Clusters of data points that have been the focus of the viewers’ gaze are marked as clusters of interest (eye-mind hypothesis). In a second step, our concept extracts various properties from statistics and scagnostics from the clusters. The third step uses these measures to compare the current data clusters from the main scatterplot to the same data in other dimensions. The results enable analysts to retrieve similar or dissimilar views as guidance to explore the entire data set. We provide a proof-of-concept implementation as a test bench and describe a use case to show a practical application and initial results.

Abstract

Blind inpainting algorithms based on deep learning architectures have shown a remarkable performance in recent years, typically outperforming model-based methods both in terms of image quality and run time. However, neural network strategies typically lack a theoretical explanation, which contrasts with the well-understood theory underlying model-based methods. In this work, we leverage the advantages of both approaches by integrating theoretically founded concepts from transform domain methods and sparse approximations into a CNN-based approach for blind image inpainting. To this end, we present a novel strategy to learn convolutional kernels that applies a specifically designed filter dictionary whose elements are linearly combined with trainable weights. Numerical experiments demonstrate the competitiveness of this approach. Our results show not only an improved inpainting quality compared to conventional CNNs but also significantly faster network convergence within a lightweight network design. Our code is available at https://github.com/cv-stuttgart/SDPF_Blind-Inpainting.

Abstract

Current adversarial attacks for motion estimation (optical flow) optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, we exploit a real-world weather phenomenon for a novel attack with adversarially optimized snow. At the core of our attack is a differentiable renderer that consistently integrates photorealistic snowflakes with realistic motion into the 3D scene. Through optimization we obtain adversarial snow that significantly impacts the optical flow while being indistinguishable from ordinary snow. Surprisingly, the impact of our novel attack is largest on methods that previously showed a high robustness to small L_p perturbations.

Abstract

Recent optical flow methods are almost exclusively judged in terms of accuracy, while their robustness is often neglected. Although adversarial attacks offer a useful tool to perform such an analysis, current attacks on optical flow methods focus on real-world attacking scenarios rather than a worst case robustness assessment. Hence, in this work, we propose a novel adversarial attack - the Perturbation-Constrained Flow Attack (PCFA) - that emphasizes destructivity over applicability as a real-world attack. PCFA is a global attack that optimizes adversarial perturbations to shift the predicted flow towards a specified target flow, while keeping the L2 norm of the perturbation below a chosen bound. Our experiments demonstrate PCFA's applicability in white- and black-box settings, and show it finds stronger adversarial samples than previous attacks. Based on these strong samples, we provide the first joint ranking of optical flow methods considering both prediction quality and adversarial robustness, which reveals state-of-the-art methods to be particularly vulnerable.

Abstract

We instrument Group Diagrams (GDs) to reduce clutter in sets of eye-tracking scanpaths. Group Diagrams consist of trajectory subsets that cover, or represent, the whole set of trajectories with respect to some distance measure and an adjustable distance threshold. The original GDs allow for an application of various distance measures. We implement the GD framework and evaluate it on scanpaths that were collected by a former user study on public transit maps. We find that the Fréchet distance is the most appropriate measure to get meaningful results, yet it is flexible enough to cover outliers.We discuss several implementation-specific challenges and improve the scalability of the algorithm.

Abstract

We introduce a system capable of generating interactive Augmented Reality guitar tutorials by parsing common digital guitar tablature and by capturing the performance of an expert using a multi-camera array. Instructions are presented to the user in an Augmented Reality application using either an abstract visualization, a 3D virtual hand, or a 3D video. To support individual users at different skill levels the system provides full control of the playback of a tutorial, including its speed and looping behavior, while delivering live feedback on the user’s performance.

Abstract

Augmented reality (AR) has a diverse range of applications, including language teaching. When studying a foreign language, one of the biggest challenges learners face is memorizing new vocabulary. While augmented holograms are a promising means of supporting this memorization process, few studies have explored their potential in the language learning context. We demonstrate the possibility of using flashcard along with an expressive holographic agent on vocabulary learning. Users scan a flashcard and play an animation that is connected with an emotion related to the word they are seeing. Our goal is to propose an alternative to the traditional use of flashcards, and also introduce another way of using AR in the association process.

Abstract

Line integration of stream-, streak-, and pathlines is widely used and popular for visualizing single-phase flow. In multiphase flow, i.e., where the fluid consists, e.g., of a liquid and a gaseous phase, these techniques could also provide valuable insights into the internal flow of droplets and ligaments and thus into their dynamics. However, since such structures tend to act as entities, high translational and rotational velocities often obfuscate their detail. As a remedy, we present a method for deriving a droplet-local velocity field, using a decomposition of the original velocity field removing translational and rotational velocity parts, and adapt path- and streaklines. Generally, the resulting integral lines are thus shorter and less tangled, which simplifies their analysis. We demonstrate and discuss the utility of our approach on droplets in two-phase flow data and visualize the removed velocity parts employing glyphs for context.

Abstract

Evaluation of rendering performance is crucial when selecting or developing algorithms, but challenging as performance can largely differ across a set of selected scenarios. Despite this, performance metrics are often reported and compared in a highly aggregated way. In this paper we suggest a more fine-grained approach for the evaluation of rendering performance, taking into account multiple perspectives on the scenario: camera position and orientation along different paths, rendering algorithms, image resolution, and hardware. The approach comprises a visual analysis system that shows and contrasts the data from these perspectives. The users can explore combinations of perspectives and gain insight into the performance characteristics of several rendering algorithms. A stylized representation of the camera path provides a base layout for arranging the multivariate performance data as radar charts, each comparing the same set of rendering algorithms while linking the performance data with the rendered images. To showcase our approach, we analyze two types of scientific visualization benchmarks.

Abstract

Despite its importance for assessing the effectiveness of communicating information visually, fine-grained recallability of information visualisations has not been studied quantitatively so far. In this work, we propose a question-answering paradigm to study visualisation recallability and present VisRecall - a novel dataset consisting of 200 visualisations that are annotated with crowd-sourced human (N = 305) recallability scores obtained from 1,000 questions of five question types. Furthermore, we present the first computational method to predict recallability of different visualisation elements, such as the title or specific data values. We report detailed analyses of our method on VisRecall and demonstrate that it outperforms several baselines in overall recallability and FE-, F-, RV-, and U-question recallability. Our work makes fundamental contributions towards a new generation of methods to assist designers in optimising visualisations.

Abstract

Gaze-based analysis of areas of interest (AOI) is widely used in information visualisation research to understand how people explore visualisations or assess the quality of visualisations concerning key characteristics such as memorability. However, nearby AOIs in visualisations amplify the uncertainty caused by the gaze estimation error, which strongly influences the mapping between gaze samples or fixations and different AOIs. We contribute a novel investigation into gaze uncertainty and quantify its impact on AOI-based analysis on visualisations using two novel metrics: the Flipping Candidate Rate (FCR) and Hit Any AOI Rate (HAAR). Our analysis of 40 real-world visualisations, including human gaze and AOI annotations, shows that uncertainty commonly appears in visualisations, which significantly impacts the analysis conducted in AOI-based studies. Moreover, we analysed four visualisation types and found that bar and scatter plots are commonly designed in a way that causes more uncertainty than line and pie plots in gaze-based analysis.

Abstract

We propose Unified Model of Saliency and Scanpaths (UMSS) -- a model that learns to predict visual saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect to several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5% for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6% for saliency prediction. These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.

Abstract

This paper provides an overview of uncertainty visualization in general, along with specific examples of applications in bioinformatics. Starting from a processing and interaction pipeline of visualization, components are discussed that are relevant for handling and visualizing uncertainty introduced with the original data and at later stages in the pipeline, which shows the importance of making the stages of the pipeline aware of uncertainty and allowing them to propagate uncertainty. We detail concepts and methods for visual mappings of uncertainty, distinguishing between explicit and implict representations of distributions, different ways to show summary statistics, and combined or hybrid visualizations. The basic concepts are illustrated for several examples of graph visualization under uncertainty. Finally, this review paper discusses implications for the visualization of biological data and future research directions.

Abstract

With the emergence of mainstream virtual reality (VR) platforms for social interactions, non-verbal communicative cues are increasingly being transmitted into the virtual environment. Since VR is primarily a visual medium, accessible VR solutions are required for people with visual impairments (PVI). However, existing propositions do not take into account social interactions, and therefore PVI are excluded from this type of experience. To address this issue, we conducted semi-structured interviews with eleven participants, seven of whom were PVI and four of whom were partners or close friends without visual impairments, to explore how non-verbal cues and joint attention are used and perceived in everyday social situations and conversations. Our goal was to provide guidelines for inclusive conversations in virtual environments for PVI. Our findings suggest that gaze, head direction, head movements, and facial expressions are important for both groups in conversations but often difficult to identify visually for PVI. From our findings, we provide concrete suggestions for the design of social VR spaces, inclusive to PVI.

Abstract

Visual quality measures (VQMs) are designed to support analysts by automatically detecting and quantifying patterns in visualizations. We propose a new data-driven technique called ClustRank that allows to rank scatterplots according to visible grouping patterns. Our model first encodes scatterplots in the parametric space of a Gaussian Mixture Model, and then uses a classifier trained on human judgment data to estimate the perceptual complexity of grouping patterns. The numbers of initial mixture components and final combined groups determine the rank of the scatterplot. ClustRank improves on existing VQM techniques by mimicking human judgments on two-Gaussian cluster patterns, and gives more accuracy when ranking general cluster patterns in scatterplots. We demonstrate its benefit by analyzing kinship data for genome-wide association studies, a domain in which experts rely on the visual analysis of large sets of scatterplots. We make the three benchmark datasets and the ClustRank VQM available for practical use and further improvements.

Abstract

Immersive analytics is a fast growing field that is often applied in virtual reality (VR). VR environments often lack immersion due to missing sensory feedback when interacting with data. Existing haptic devices are often expensive, stationary, or occupy the user’s hand, preventing them from grasping objects or using a controller. We propose PropellerHand, an ungrounded hand-mounted haptic device with two rotatable propellers, that allows exerting forces on the hand without obstructing hand use. PropellerHand is able to simulate feedback such as weight and torque by generating thrust up to 11 N in 2-DOF and a torque of 1.87 Nm in 2-DOF. Its design builds on our experience from quantitative and qualitative experiments with different form factors and parts. We evaluated our final version through a qualitative user study in various VR scenarios that required participants to manipulate virtual objects in different ways, while changing between torques and directional forces. Results show that PropellerHand improves users’ immersion in virtual reality.

Abstract

The large potential of force feedback devices for interacting in Virtual Reality (VR) has been illustrated in a plethora of research prototypes. Yet, these devices are still rarely used in practice and it remains an open challenge how to move this research into practice. To that end, we contribute a participatory design study on the use of haptic feedback devices in the automotive industry. Based on a 10-month observing process with 13 engineers, we developed STRIVE, a string-based haptic feedback device. In addition to the design of STRIVE, this process led to a set of requirements for introducing haptic devices into industrial settings, which center around a need for flexibility regarding forces, comfort, and mobility. We evaluated STRIVE with 16 engineers in five different day-to-day automotive VR use cases. The main results show an increased level of trust and perceived safety as well as further challenges towards moving haptics research into practice.

Abstract

The concept of biological age (BA) - although important in clinical practice - is hard to grasp mainly due to the lack of a clearly defined reference standard. For specific applications, especially in pediatrics, medical image data are used for BA estimation in a routine clinical context. Beyond this young age group, BA estimation is mostly restricted to whole-body assessment using non-imaging indicators such as blood biomarkers, genetic and cellular data. However, various organ systems may exhibit different aging characteristics due to lifestyle and genetic factors. Thus, a whole-body assessment of the BA does not reflect the deviations of aging behavior between organs. To this end, we propose a new imaging-based framework for organ-specific BA estimation. In this initial study we focus mainly on brain MRI. As a first step, we introduce a chronological age (CA) estimation framework using deep convolutional neural networks (Age-Net). We quantitatively assess the performance of this framework in comparison to existing state-of-the-art CA estimation approaches. Furthermore, we expand upon Age-Net with a novel iterative data-cleaning algorithm to segregate atypical-aging patients (BA ≉ CA) from the given population. We hypothesize that the remaining population should approximate the true BA behavior. We apply the proposed methodology on a brain magnetic resonance image (MRI) dataset containing healthy individuals as well as Alzheimer’s patients with different dementia ratings. We demonstrate the correlation between the predicted BAs and the expected cognitive deterioration in Alzheimer’s patients. A statistical and visualization-based analysis has provided evidence regarding the potential and current challenges of the proposed methodology.

Abstract

In a microservice system, runtime changes such as failures, deployments, or self-adaptation can trigger the system to transition from one steady state to another, i.e., exhibiting transient behavior. To assess a system's quality, it is imperative that this transient behavior is specified in non-functional requirements and that stakeholders can analyze whether these requirements are met. Yet, there is little support for either specifying transient behavior as a non-functional requirement or analyzing how such a requirement is met in production. We aim to make these two tasks more accessible by utilizing novel human-computer interaction methods. To this end, we developed TransVis, an approach for specifying and analyzing transient behavior based on chatbot interactions and visualizations of the systems' resilience. We examined the effectiveness of our approach by conducting an exploratory expert study on a prototypical implementation. The study revealed that the developed visualizations are effective for specifying and exploring transient behavior. Participants found especially helpful the feature to compare specifications with the actual behavior. However, the integration of a chatbot did not prove effective for our use cases. In conclusion, our approach is capable of supporting stakeholders in the exploration and specification of transient behavior.

Abstract

Class separation is an important concept in machine learning and visual analytics. We address the visual analysis of class separation measures for both high-dimensional data and its corresponding projections into 2D through dimensionality reduction (DR) methods. Although a plethora of separation measures have been proposed, it is difficult to compare class separation between multiple datasets with different characteristics, multiple separation measures, and multiple DR methods. We present ProSeCo, an interactive visualization approach to support comparison between up to 20 class separation measures and up to 4 DR methods, with respect to any of 7 dataset characteristics: dataset size, dataset dimensions, class counts, class size variability, class size skewness, outlieriness, and real-world vs. synthetically generated data. ProSeCo supports (1) comparing across measures, (2) comparing high-dimensional to dimensionally-reduced 2D data across measures, (3) comparing between different DR methods across measures, (4) partitioning with respect to a dataset characteristic, (5) comparing partitions for a selected characteristic across measures, and (6) inspecting individual datasets in detail. We demonstrate the utility of ProSeCo in two usage scenarios, using datasets 1 posted at https://osf.io/epcf9/.

Abstract

Strategies for selecting the next data instance to label, in service of generating labeled data for machine learning, have been considered separately in the machine learning literature on active learning and in the visual analytics literature on human-centered approaches. We propose a unified design space for instance selection strategies to support detailed and fine-grained analysis covering both of these perspectives. We identify a concise set of 15 properties, namely measureable characteristics of datasets or of machine learning models applied to them, that cover most of the strategies in these literatures. To quantify these properties, we introduce Property Measures (PM) as fine-grained building blocks that can be used to formalize instance selection strategies. In addition, we present a taxonomy of PMs to support the description, evaluation, and generation of PMs across four dimensions: machine learning (ML) Model Output, Instance Relations, Measure Functionality, and Measure Valence. We also create computational infrastructure to support qualitative visual data analysis: a visual analytics explainer for PMs built around an implementation of PMs using cascades of eight atomic functions. It supports eight analysis tasks, covering the analysis of datasets and ML models using visual comparison within and between PMs and groups of PMs, and over time during the interactive labeling process. We iteratively refined the PM taxonomy, the explainer, and the task abstraction in parallel with each other during a two-year formative process, and show evidence of their utility through a summative evaluation with the same infrastructure. This research builds a formal baseline for the better understanding of the commonalities and differences of instance selection strategies, which can serve as the stepping stone for the synthesis of novel strategies in future work.

Abstract

We propose a visualization method to understand the effect of multidimensional projection on local subspaces, using implicit function differentiation. Here, we understand the local subspace as the multidimensional local neighborhood of data points. Existing methods focus on the projection of multidimensional data points, and the neighborhood information is ignored. Our method is able to analyze the shape and directional information of the local subspace to gain more insights into the global structure of the data through the perception of local structures. Local subspaces are fitted by multidimensional ellipses that are spanned by basis vectors. An accurate and efficient vector transformation method is proposed based on analytical differentiation of multidimensional projections formulated as implicit functions. The results are visualized as glyphs and analyzed using a full set of specifically-designed interactions supported in our efficient web-based visualization tool. The usefulness of our method is demonstrated using various multi- and high-dimensional benchmark datasets. Our implicit differentiation vector transformation is evaluated through numerical comparisons; the overall method is evaluated through exploration examples and use cases.

Abstract

In this paper, we propose SineStream, a new variant of streamgraphs that improves their readability by minimizing sine illusion effects. Such effects reflect the tendency of humans to take the orthogonal rather than the vertical distance between two curves as their distance. In SineStream, we connect the readability of streamgraphs with minimizing sine illusions and by doing so provide a perceptual foundation for their design. As the geometry of a streamgraph is controlled by its baseline (the bottom-most curve) and the ordering of the layers, we re-interpret baseline computation and layer ordering algorithms in terms of reducing sine illusion effects. For baseline computation, we improve previous methods by introducing a Gaussian weight to penalize layers with large thickness changes. For layer ordering, three design requirements are proposed and implemented through a hierarchical clustering algorithm. Quantitative experiments and user studies demonstrate that SineStream improves the readability and aesthetics of streamgraphs compared to state-of-the-art methods.

Abstract

While graph drawing focuses more on the aesthetic representation of node-link diagrams, graph visualization takes into account other visual metaphors making them useful for graph exploration tasks in information visualization and visual analytics. Although there are aesthetic graph drawing criteria that describe how a graph should be presented to make it faster and more reliably explorable, many controlled and uncontrolled empirical user studies flourished over the past years. The goal of them is to uncover how well the human user performs graph-specific tasks, in many cases compared to previously designed graph visualizations. Due to the fact that many parameters in a graph dataset as well as the visual representation of them might be varied and many user studies have been conducted in this space, a state-of-the-art survey is needed to understand evaluation results and findings to inform the future design, research, and application of graph visualizations. In this article, we classify the present literature on the topmost level into graph interpretation, graph memorability, and graph creation where the users with their tasks stand in focus of the evaluation, not the computational aspects. As another outcome of this work, we identify the white spots in this field and sketch ideas for future research directions.

Abstract

We present the VIS30K dataset, a collection of 29,689 images that represents 30 years of figures and tables from each track of the IEEE Visualization conference series (Vis, SciVis, InfoVis, VAST). VIS30K's comprehensive coverage of the scientific literature in visualization not only reflects the progress of the field but also enables researchers to study the evolution of the state-of-the-art and to find relevant work based on graphical content. We describe the dataset and our semi-automatic collection process, which couples convolutional neural networks (CNN) with curation. Extracting figures and tables semi-automatically allows us to verify that no images are overlooked or extracted erroneously. To improve quality further, we engaged in a peer-search process for high-quality figures from early IEEE Visualization papers. With the resulting data, we also contribute VISImageNavigator (VIN, visimagenavigator.github.io ), a web-based tool that facilitates searching and exploring VIS30K by author names, paper keywords, title and abstract, and years.

Abstract

Dimensionality Reduction (DR) is a popular technique that is often used in Machine Learning and Visualization communities to analyze high-dimensional data. The approach is empirically proven to be powerful for uncovering previously unseen structures in the data. While observing the results of the intermediate optimization steps of DR algorithms, we coincidently discovered the artistic beauty of the DR process. With enthusiasm for the beauty, we decided to look at DR from a generative art lens rather than their technical application aspects and use DR techniques to create artwork. Particularly, we use the optimization process to generate images, by drawing each intermediate step of the optimization process with some opacity over the previous intermediate result. As another alternative input, we used a neural-network model for face-landmark detection, to apply DR to portraits, while maintaining some facial properties, resulting in abstracted facial avatars. In this work, we provide such a collection of such artwork.

Abstract

A common enhancement of scatterplots represents points as small multiples, glyphs, or thumbnail images. As this encoding often results in overlaps, a general strategy is to alter the position of the data points, for instance, to a grid-like structure. Previous approaches rely on solving expensive optimization problems or on dividing the space that alter the global structure of the scatterplot. To find a good balance between efficiency and neighborhood and layout preservation, we propose Hagrid, a technique that uses space-filling curves (SFCs) to “gridify” a scatterplot without employing expensive collision detection and handling mechanisms. Using SFCs ensures that the points are plotted close to their original position, retaining approximately the same global structure. The resulting scatterplot is mapped onto a rectangular or hexagonal grid, using Hilbert and Gosper curves. We discuss and evaluate the theoretic runtime of our approach and quantitatively compare our approach to three state-of-the-art gridifying approaches, DGrid, Small multiples with gaps SMWG, and CorrelatedMultiples CMDS, in an evaluation comprising 339 scatterplots. Here, we compute several quality measures for neighborhood preservation together with an analysis of the actual runtimes. The main results show that, compared to the best other technique, Hagrid is faster by a factor of four, while achieving similar or even better quality of the gridified layout. Due to its computational efficiency, our approach also allows novel applications of gridifying approaches in interactive settings, such as removing local overlap upon hovering over a scatterplot.

Abstract

In this design study, we present IRVINE, a Visual Analytics (VA) system, which facilitates the analysis of acoustic data to detect and understand previously unknown errors in the manufacturing of electrical engines. In serial manufacturing processes, signatures from acoustic data provide valuable information on how the relationship between multiple produced engines serves to detect and understand previously unknown errors. To analyze such signatures, IRVINE leverages interactive clustering and data labeling techniques, allowing users to analyze clusters of engines with similar signatures, drill down to groups of engines, and select an engine of interest. Furthermore, IRVINE allows to assign labels to engines and clusters and annotate the cause of an error in the acoustic raw measurement of an engine. Since labels and annotations represent valuable knowledge, they are conserved in a knowledge database to be available for other stakeholders. We contribute a design study, where we developed IRVINE in four main iterations with engineers from a company in the automotive sector. To validate IRVINE, we conducted a field study with six domain experts. Our results suggest a high usability and usefulness of IRVINE as part of the improvement of a real-world manufacturing process. Specifically, with IRVINE domain experts were able to label and annotate produced electrical engines more than 30% faster.

Abstract

It is crucial to visually extrapolate the characteristics of their evolution to understand critical spatio-temporal events such as earthquakes, fires, or the spreading of a disease. Animations embedded in the spatial context can be helpful for understanding details, but have proven to be less effective for overview and comparison tasks. We present an interactive approach for the exploration of spatio-temporal data, based on a set of neighborhood-preserving 1D projections which help identify patterns and support the comparison of numerous time steps and multivariate data. An important objective of the proposed approach is the visual description of local neighborhoods in the 1D projection to reveal patterns of similarity and propagation. As this locality cannot generally be guaranteed, we provide a selection of different projection techniques, as well as a hierarchical approach, to support the analysis of different data characteristics. In addition, we offer an interactive exploration technique to reorganize and improve the mapping locally to users' foci of interest. We demonstrate the usefulness of our approach with different real-world application scenarios and discuss the feedback we received from domain and visualization experts.

Abstract

Abstract We developed a new visualization approach to gain a better understanding of the displacement of one fluid phase by another in porous media. This is based on a recent experimental parameter study with varying capillary numbers and viscosity ratios. We analyse the temporal evolution of characteristic values in this two-phase flow scenario and discuss how to directly compare experiments across different temporal scales. To enable spatio-temporal analysis, we introduce a new abstract visual representation showing which paths through the porous medium were occupied and for how long. These transport networks allow to assess the impact of different acting forces and they are designed to yield expressive comparability and linking to the experimental parameter space both supported by additional visual cues. This joint work of porous media experts and visualization researchers yields new insights regarding two-phase flow on the microscale, and our visualization approach contributes towards the overarching goal of the domain scientists to characterize porous media flow based on capillary numbers and viscosity ratios.

Abstract

Both visual detail and a low-latency transfer of image data are required for collaborative exploration of scientific data sets across large high-resolution displays. In this work, we present an approach that reduces the resolution before the encoding and uses temporal upscaling to reconstruct the full resolution image, reducing the overall latency and the required bandwidth without significantly impacting the details perceived by observers. Our approach takes advantage of the fact that humans do not perceive the full details of moving objects by providing a perfect reconstruction for static parts of the image, while non-static parts are reconstructed with a lower quality. This strategy enables a substantial reduction of the encoding latency and the required bandwidth with barely noticeable changes in visual quality, which is crucial for collaborative analysis across display walls at different locations. Additionally, our approach can be combined with other techniques aiming to reduce the required bandwidth while keeping the quality as high as possible, such as foveated encoding. We demonstrate the reduced overall latency, the required bandwidth, as well as the high image quality using different visualisations.

Abstract

In this paper, we introduce a visual analytics approach aimed at helping machine learning experts analyze the hidden states of layers in recurrent neural networks. Our technique allows the user to interactively inspect how hidden states store and process information throughout the feeding of an input sequence into the network. The technique can help answer questions, such as which parts of the input data have a higher impact on the prediction and how the model correlates each hidden state configuration with a certain output. Our visual analytics approach comprises several components: First, our input visualization shows the input sequence and how it relates to the output (using color coding). In addition, hidden states are visualized through a nonlinear projection into a 2-D visualization space using t-distributed stochastic neighbor embedding to understand the shape of the space of the hidden states. Trajectories are also employed to show the details of the evolution of the hidden state configurations. Finally, a time-multi-class heatmap matrix visualizes the evolution of the expected predictions for multi-class classifiers, and a histogram indicates the distances between the hidden states within the original space. The different visualizations are shown simultaneously in multiple views and support brushing-and-linking to facilitate the analysis of the classifications and debugging for misclassified input sequences. To demonstrate the capability of our approach, we discuss two typical use cases for long short-term memory models applied to two widely used natural language processing datasets.

Abstract

In visual interactive labeling, users iteratively assign labels to data items until the machine model reaches an acceptable accuracy. A crucial step of this process is to inspect the model's accuracy and decide whether it is necessary to label additional elements. In scenarios with no or very little labeled data, visual inspection of the predictions is required. Similarity-preserving scatterplots created through a dimensionality reduction algorithm are a common visualization that is used in these cases. Previous studies investigated the effects of layout and image complexity on tasks like labeling. However, model evaluation has not been studied systematically. We present the results of an experiment studying the influence of image complexity and visual grouping of images on model accuracy estimation. We found that users outperform traditional automated approaches when estimating a model's accuracy. Furthermore, while the complexity of images impacts the overall performance, the layout of the items in the plot has little to no effect on estimations.

Abstract

We present a data-driven visual analysis approach for the in-depth exploration of large numbers of droplets. Understanding droplet dynamics in sprays is of interest across many scientific fields for both simulation scientists and engineers. In this paper, we analyze large-scale direct numerical simulation datasets of the two-phase flow of non-Newtonian jets. Our interactive visual analysis approach comprises various dedicated exploration modalities that are supplemented by directly linking to ParaView. This hybrid setup supports a detailed investigation of droplets, both in the spatial domain and in terms of physical quantities . Considering a large variety of extracted physical quantities for each droplet enables investigating different aspects of interest in our data. To get an overview of different types of characteristic behaviors, we cluster massive numbers of droplets to analyze different types of occurring behaviors via domain-specific pre-aggregation, as well as different methods and parameters. Extraordinary temporal patterns are of high interest, especially to investigate edge cases and detect potential simulation issues. For this, we use a neural network-based approach to predict the development of these physical quantities and identify irregularly advected droplets.

Abstract

Collaboration is essential in companies and often physical presence is required, thus, more and more Virtual Reality (VR) systems are used to work together remotely. To support social interaction, human representations in form of avatars are used in collaborative virtual environment (CVE) tools. However, up to now, the avatar representations often are limited in their design and functionality, which may hinder effective collaboration. In our interview study, we explored the status quo of VR collaboration in a large automotive company setting with a special focus on the role of avatars. We collected inter-view data from 21 participants, from which we identified challenges of current avatar representations used in our setting. Based on these findings, we discuss design suggestions for avatars in a company setting, which aim to improve social interaction. As opposed to state-of-the-art research, we found that users within the context of a large automotive company have an altered need with respect to avatar representations.

Abstract

In this paper, we explore the design space of word-sized visualizations—small graphics, usually the same size as a word, that visualize data in or related to a text—for displaying and exploring categories in social media feeds such as Twitter streams. Social media contributions are typically microposts, which allow us to attach word-sized visualizations to show category assignment, diversity, or development. We consider and combine word-sized visualizations made up of basic marks and visual variables, existing word-sized visualization concepts, as well as large text visualizations. In an application example we show how word-sized visualizations can evince context changes within a discussion on Twitter and reveal topic diversity.

Abstract

Nonlinear programming is a complex methodology where a problem is mathematically expressed in terms of optimality while imposing constraints on feasibility. Such problems are formulated by humans and solved by optimization algorithms. We support domain experts in their challenging tasks of understanding and troubleshooting optimization runs of intricate and high-dimensional nonlinear programs through a visual analytics system. The system was designed for our collaborators’ robot motion planning problems, but is domain agnostic in most parts of the visualizations. It allows for an exploration of the iterative solving process of a nonlinear program through several linked views of the computational process. We give insights into this design study, demonstrate our system for selected real-world cases, and discuss the extension of visualization and visual analytics methods for nonlinear programming.

Abstract

Spherical k-Means is frequently used to cluster document collections because it performs reasonably well in many settings and is computationally efficient. However, the time complexity increases linearly with the number of clusters k, which limits the suitability of the algorithm for larger values of k depending on the size of the collection. Optimizations targeted at the Euclidean k-Means algorithm largely do not apply because the cosine distance is not a metric. We therefore propose an efficient indexing structure to improve the scalability of Spherical k-Means with respect to k. Our approach exploits the sparsity of the input vectors and the convergence behavior of k-Means to reduce the number of comparisons on each iteration significantly.

Abstract

Dynamic Mode Decomposition (DMD) is a data-driven and model-free decomposition technique. It is suitable for revealing spatio-temporal features of both numerically and experimentally acquired data. Conceptually, DMD performs a low-dimensional spectral decomposition of the data into the following components: the modes, called DMD modes, encode the spatial contribution of the decomposition, whereas the DMD amplitudes specify their impact. Each associated eigenvalue, referred to as DMD eigenvalue, characterizes the frequency and growth rate of the DMD mode. In this paper, we demonstrate how the components of DMD can be utilized to obtain temporal and spatial information from time-dependent flow fields. We begin with the theoretical background of DMD and its application to unsteady flow. Next, we examine the conventional process with DMD mathematically and put it in relationship to the discrete Fourier transform. Our analysis shows that the current use of DMD components has several drawbacks. To resolve these problems we adjust the components and provide new and meaningful insights into the decomposition: we show that our improved components capture the spatio-temporal patterns of the flow better. Moreover, we remove redundancies in the decomposition and clarify the interplay between components, allowing users to understand the impact of components. These new representations, which respect the spatio-temporal character of DMD, enable two clustering methods that segment the flow into physically relevant sections and can therefore be used for the selection of DMD components. With a number of typical examples, we demonstrate that the combination of these techniques allows new insights with DMD for unsteady flow.

Abstract

In recent years, research on immersive environments has experienced a new wave of interest, and immersive analytics has been established as a new research field. Every year, a vast amount of different techniques, applications, and user studies are published that focus on employing immersive environments for visualizing and analyzing data. Nevertheless, immersive analytics is still a relatively unexplored field that needs more basic research in many aspects and is still viewed with skepticism. Rightly so, because in our opinion, many researchers do not fully exploit the possibilities offered by immersive environments and, on the contrary, sometimes even overestimate the power of immersive visualizations. Although a growing body of papers has demonstrated individual advantages of immersive analytics for specific tasks and problems, the general benefit of using immersive environments for effective analytic tasks remains controversial. In this article, we reflect on when and how immersion may be appropriate for the analysis and present four guiding scenarios. We report on our experiences, discuss the landscape of assessment strategies, and point out the directions where we believe immersive visualizations have the greatest potential.

Abstract

The world is still under the influence of the COVID-19 pandemic. Even though vaccines are deployed as rapidly as possible, it is still necessary to use other measures to reduce the spread of the virus. Measures such as social distancing or wearing a mask receive a lot of criticism. Therefore, we want to demonstrate a serious game to help the players understand these measures better and show them why they are still necessary. The player of the game has to avoid other agents to keep their risk of a COVID-19 infection low. The game uses Virtual Reality through a Head-Mounted-Display to deliver an immersive and enjoyable experience. Gamification elements are used to engage the user with the game while they explore various environments. We also implemented visualizations that help the user with social distancing.

Abstract

We present document domain randomization (DDR), the first successful transfer of convolutional neural networks (CNNs) trained only on graphically rendered pseudo-paper pages to real-world document segmentation. DDR renders pseudo-document pages by modeling randomized textual and non-textual contents of interest, with user-defined layout and font styles to support joint learning of fine-grained classes. We demonstrate competitive results using our DDR approach to extract nine document classes from the benchmark CS-150 and papers published in two domains, namely annual meetings of Association for Computational Linguistics (ACL) and IEEE Visualization (VIS). We compare DDR to conditions of style mismatch, fewer or more noisy samples that are more easily obtained in the real world. We show that high-fidelity semantic information is not necessary to label semantic classes but style mismatch between train and test can lower model accuracy. Using smaller training samples had a slightly detrimental effect. Finally, network models still achieved high test accuracy when correct labels are diluted towards confusing labels; this behavior hold across several classes.

Abstract

We present an integrated approach for creating and assigning color palettes to different visualizations such as multi-class scatterplots, line, and bar charts. While other methods separate the creation of colors from their assignment, our approach takes data characteristics into account to produce color palettes, which are then assigned in a way that fosters better visual discrimination of classes. To do so, we use a customized optimization based on simulated annealing to maximize the combination of three carefully designed color scoring functions: point distinctness, name difference, and color discrimination. We compare our approach to state-of-the-art palettes with a controlled user study for scatterplots and line charts, furthermore we performed a case study. Our results show that Palettailor, as a fully-automated approach, generates color palettes with a higher discrimination quality than existing approaches. The efficiency of our optimization allows us also to incorporate user modifications into the color selection process.

Abstract

Approaches based on order-adaptive regularisation belong to the most accurate variational methods for computing the optical flow. By locally deciding between first- and second-order regularisation, they are applicable to scenes with both fronto-parallel and ego-motion. So far, however, existing order-adaptive methods have a decisive drawback. While the involved first- and second-order smoothness terms already make use of anisotropic concepts, the underlying selection process itself is still isotropic in that sense that it locally chooses the same regularisation order for all directions. In our paper, we address this shortcoming. We propose a generalised order-adaptive approach that allows to select the local regularisation order for each direction individually. To this end, we split the order-adaptive regularisation across and along the locally dominant direction and perform an energy competition for each direction separately. This in turn offers another advantage. Since the parameters can be chosen differently for both directions, the approach allows for a better adaption to the underlying scene. Experiments for MPI Sintel and KITTI 2015 demonstrate the usefulness of our approach. They not only show improvements compared to an isotropic selection scheme. They also make explicit that our approach is able to improve the results from state-of-the-art learning-based approaches, if applied as a final refinement step – thereby achieving top results in both benchmarks.

Abstract

A plethora of dimensionality reduction techniques have emerged over the past decades, leaving researchers and analysts with a wide variety of choices for reducing their data, all the more so given some techniques come with additional parametrization (e.g. t-SNE, UMAP, etc.). Recent studies are showing that people often use dimensionality reduction as a black-box regardless of the specific properties the method itself preserves. Hence, evaluating and comparing 2D projections is usually qualitatively decided, by setting projections side-by-side and letting human judgment decide which projection is the best. In this work, we propose a quantitative way of evaluating projections, that nonetheless places human perception at the center. We run a comparative study, where we ask people to select 'good' and 'misleading' views between scatterplots of low-level projections of image datasets, simulating the way people usually select projections. We use the study data as labels for a set of quality metrics whose purpose is to discover and quantify what exactly people are looking for when deciding between projections. With this proxy for human judgments, we use it to rank projections on new datasets, explain why they are relevant, and quantify the degree of subjectivity in projections selected.

Abstract

Interactive visualization is a valuable tool for introductory or advanced courses in general relativity as well as for public outreach to provide a deeper understanding of the visual implications due to curved spacetime. In particular, the extreme case of a black hole where the curvature becomes so strong that even light cannot escape, benefits from an interactive visualization where students can investigate the distortion effects by moving objects around. However, the most commonly used technique of four-dimensional general-relativistic ray tracing is still too slow for interactive frame rates. Therefore, we propose an efficient and adaptive polygon rendering method that takes light deflection and light travel time into account. An additional advantage of this method is that it provides a natural demonstration of how multiple images occur and how light travel time affects them. Finally, we present our method using three example scenes: a triangle passing behind a black hole, a sphere orbiting a black hole and an accretion disk with different inclination angles.

Abstract

The logos of metal bands can be by turns gaudy, uncouth, or nearly illegible. Yet, these logos work: they communicate sophisticated notions of genre and emotional affect. In this paper we use the design considerations of metal logos to explore the space of "illegible semantics": the ways that text can communicate information at the cost of readability, which is not always the most important objective. In this work, drawing on formative visualization theory, professional design expertise, and empirical assessments of a corpus of metal band logos, we describe a design space of metal logos and present a tool through which logo characteristics can be explored through visualization. We investigate ways in which logo designers imbue their text with meaning and consider opportunities and implications for visualization more widely.

Abstract

Simulations of cosmic evolution are a means to explain the formation of the universe as we see it today. The resulting data of such simulations comprise numerous physical quantities, which turns their analysis into a complex task. Here, we analyze such high-dimensional and time-varying particle data using various visualization techniques from the fields of particle visualization, flow visualization, volume visualization, and information visualization. Our approach employs specialized filters to extract and highlight the development of so-called active galactic nuclei and filament structures formed by the particles. Additionally, we calculate X-ray emission of the evolving structures in a pre-processing step to complement visual analysis. Our approach is integrated into a single visual analytics framework to allow for analysis of star formation at interactive frame rates. Finally, we lay out the methodological aspects of our work that led to success at the 2019 IEEE SciVis Contest.

Abstract

When studying protein-ligand interactions, many different factors can influence the behaviour of the protein as well as the ligands. Molecular visualisation tools typically concentrate on the movement of single ligand molecules; however, viewing only one molecule can merely provide a hint of the overall behaviour of the system. To tackle this issue, we do not focus on the visualisation of the local actions of individual ligand molecules but on the influence of a protein and their overall movement. Since the simulations required to study these problems can have millions of time steps, our presented system decouples visualisation and data preprocessing: our preprocessing pipeline aggregates the movement of ligand molecules relative to a receptor protein. For data analysis, we present a web-based visualisation application that combines multiple linked 2D and 3D views that display the previously calculated data The central view, a novel enhanced sequence diagram that shows the calculated values, is linked to a traditional surface visualisation of the protein. This results in an interactive visualisation that is independent of the size of the underlying data, since the memory footprint of the aggregated data for visualisation is constant and very low, even if the raw input consisted of several terabytes.

Abstract

Many biochemical and biomedical applications such as protein engineering or drug design are concerned with finding functionally similar proteins, however, this remains to be a challenging task. We present a new image-based approach for identifying and visually comparing proteins with similar function that builds on the hierarchical clustering of Molecular Surface Maps. Such maps are two-dimensional representations of complex molecular surfaces and can be used to visualize the topology and different physico-chemical properties of proteins. Our method is based on the idea that visually similar maps also imply a similarity in the function of the mapped proteins. To determine map similarity, we compute descriptive feature vectors using image moments, color moments, or a Convolutional Neural Network and use them for a hierarchical clustering of the maps. We demonstrate the feasibility of our approach using two data sets: an ensemble of hand-selected proteins with known similarities used for verification and an ensemble of ketolase enzymes, where we analyzed the individual domains using our method. Our method is integrated in an interactive visualization application, which allows users to explore and analyze the results. It visualizes the hierarchical clustering and offers linked views that provide details for a comparative data analysis.

Abstract

We introduce inverted stippling, a method to mimic an inversion technique used by artists when performing stippling. To this end, we extend Linde-Buzo-Gray (LBG) stippling to multi-class LBG (MLBG) stippling with multiple layers. MLBG stippling couples the layers stochastically to optimize for per-layer and overall blue-noise properties. We propose a stipple-based filling method to generate solid color backgrounds for inverting areas. Our experiments demonstrate the effectiveness of MLBG in terms of reducing overlapping and intensity accuracy. In addition, we showcase MLBG with color stippling and dynamic multi-class blue-noise sampling, which is possible due to its support for temporal coherence.

Abstract

We present implicit visualization of 2D vector field topology, and show its utility for validating and guiding approaches for periodic orbit extraction. Instead of following the traditional approach by explicit extraction of the topological skeleton, we investigate its implicit visualization by approaches that label the regions that are separated by the skeleton. While such approaches perform well for gradient fields, they fail, in particular, to visualize periodic orbits. This motivates us to complement the label-based approach with a closely related distance-based metric. We show that our approach is able to reveal periodic orbits, also in configurations in which the state-of-the-art techniques for periodic orbit extraction fail, and demonstrate their utility for interactive extraction of all periodic orbits of a 2D vector field.

Abstract

We introduce an ML-driven approach that enables interactive example-based queries for similar behavior in ensembles of spatiotemporal scientific data. This addresses an important use case in the visual exploration of simulation and experimental data, where data is often large, unlabeled and has no meaningful similarity measures available. We exploit the fact that nearby locations often exhibit similar behavior and train a Siamese Neural Network in a self-supervised fashion, learning an expressive latent space for spatiotemporal behavior. This space can be used to find similar behavior with just a few user-provided examples. We evaluate this approach on several ensemble datasets and compare with multiple existing methods, showing both qualitative and quantitative results.

Abstract

Many professionals, like journalists, writers, or consultants, need to acquire information from various sources, make sense of this unstructured evidence, structure their observations, and finally create and deliver their product, such as a report or a presentation. In formative interviews, we found that tools allowing structuring of observations are often disconnected from the corresponding evidence. Therefore, we designed a sensemaking environment with a flexible observation graph that visually ties together evidence in unstructured documents with the user’s structured knowledge. This is achieved through bi-directional deep links between highlighted document portions and nodes in the observation graph. In a controlled study, we compared users’ sensemaking strategies using either the observation graph or a simple text editor on a large display. Results show that the observation graph represents a holistic, compact representation of users’ observations, which can be linked to unstructured evidence on demand. In contrast, users taking textual notes required much more display space to spatially organize source documents containing unstructured evidence. This implies that spatial organization is a powerful strategy to structure observations even if the available space is limited.

Abstract

We propose a data-driven space-filling curve method for 2D and 3D visualization. Our flexible curve traverses the data elements in the spatial domain in a way that the resulting linearization better preserves features in space compared to existing methods. We achieve such data coherency by calculating a Hamiltonian path that approximately minimizes an objective function that describes the similarity of data values and location coherency in a neighborhood. Our extended variant even supports multiscale data via quadtrees and octrees. Our method is useful in many areas of visualization including multivariate or comparative visualization ensemble visualization of 2D and 3D data on regular grids or multiscale visual analysis of particle simulations. The effectiveness of our method is evaluated with numerical comparisons to existing techniques and through examples of ensemble and multivariate datasets.

Abstract

We report on an interdisciplinary visual analytics project wherein automotive engineers analyze test drive videos. These videos are annotated with navigation-specific augmented reality (AR) con�tent, and the engineers need to identify issues and evaluate the behavior of the underlying AR navigation system. With the increas�ing amount of video data, traditional analysis approaches can no longer be conducted in an acceptable timeframe. To address this issue, we collaboratively developed Caarvida, a visual analytics tool that helps engineers to accomplish their tasks faster and handle an increased number of videos. Caarvida combines automatic video analysis with interactive and visual user interfaces. We conducted two case studies which show that Caarvida successfully supports domain experts and speeds up their task completion time.

Abstract

Among the many changes brought about by the COVID-19 pandemic, one of the most pressing for scientific research concerns user testing. For the researchers who conduct studies with human participants, the requirements for social distancing have created a need for reflecting on methodologies that previously seemed relatively straightforward. It has become clear from the emerging literature on the topic and from first-hand experiences of researchers that the restrictions due to the pandemic affect every aspect of the research pipeline. The current paper offers an initial reflection on user-based research, drawing on the authors' own experiences and on the results of a survey that was conducted among researchers in different disciplines, primarily psychology, human-computer interaction (HCI), and visualization communities. While this sampling of researchers is by no means comprehensive, the multi-disciplinary approach and the consideration of different aspects of the research pipeline allow us to examine current and future challenges for user-based research. Through an exploration of these issues, this paper also invites others in the VIS-as well as in the wider-research community, to reflect on and discuss the ways in which the current crisis might also present new and previously unexplored opportunities.

Abstract

Class separation is an important concept in machine learning and visual analytics. However, the comparison of class separation for datasets with varying dimensionality is non-trivial, given a) the various possible structural characteristics of datasets and b) the plethora of separation measures that exist. Building upon recent findings in visualization research about the qualitative and quantitative evaluation of class separation for 2D dimensionally reduced data using scatterplots, this research addresses the visual analysis of class separation measures for high-dimensional data. We present SepEx, an interactive visualization approach for the assessment and comparison of class separation measures for multiple datasets. SepEx supports analysts with the comparison of multiple separation measures over many high-dimensional datasets, the effect of dimensionality reduction on measure outputs by supporting nD to 2D comparison, and the comparison of the effect of different dimensionality reduction methods on measure outputs. We demonstrate SepEx in a scenario on 100 two-class 5D datasets with a linearly increasing amount of separation between the classes, illustrating both similarities and nonlinearities across 11 measures.

Abstract

In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems.

Abstract

As our field matures, evaluation of visualization techniques has extended from reporting runtime performance to studying user behavior. Consequently, many methodologies and best practices for user studies have evolved. While maintaining interactivity continues to be crucial for the exploration of large data sets, no similar methodological foundation for evaluating runtime performance has been developed. Our analysis of 50 recent visualization papers on new or improved techniques for rendering volumes or particles indicates that only a very limited set of parameters like different data sets, camera paths, viewport sizes, and GPUs are investigated, which make comparison with other techniques or generalization to other parameter ranges at least questionable. To derive a deeper understanding of qualitative runtime behavior and quantitative parameter dependencies, we developed a framework for the most exhaustive performance evaluation of volume and particle visualization techniques that we are aware of, including millions of measurements on ten different GPUs. This paper reports on our insights from statistical analysis of this data discussing independent and linear parameter behavior and non-obvious effects. We give recommendations for best practices when evaluating runtime performance of scientific visualization applications, which can serve as a starting point for more elaborate models of performance quantification.

Abstract

We report on the state-of-the-art of software visualization.To ensure reproducibility, we adopted the Systematic Literature Review methodology. That is, we analyzed 1440 entries from IEEE Xplore and ACM Digital Library databases. We selected 105 relevant full papers published in 2013–2019, which we classified based on the aspect of the software system that is supported (i.e., structure, behavior, and evolution). For each paper, we extracted main dimensions that characterize software visualizations, such as software engineering tasks, roles of users, information visualization techniques, and media used to display visualizations. We provide researchers in the field an overview of the state-of-the-art in software visualization and highlight research opportunities. We also help developers to identify suitable visualizations for their particular context by matching software visualizations to development concerns and concrete details to obtain available visualization tools.

Abstract

To characterize the building blocks of a legacy software system (e.g., structure, dependencies), programmers usually spend a long time navigating its source code. Yet, modern integrated development environments (IDEs) do not provide appropriate means to efficiently achieve complex software comprehension tasks. To deal with this unfulfilled need, we present Hunter, a tool for the visualization of JavaScript applications. Hunter visualizes source code through a set of coordinated views that include a node-link diagram that depicts the dependencies among the components of a system, and a treemap that helps programmers to orientate when navigating its structure. In this paper, we report on a controlled experiment that evaluates Hunter. We asked 16 participants to solve a set of software comprehension tasks, and assessed their effectiveness in terms of (i) user performance (i.e., completion time, accuracy, and attention), and (ii) user experience (i.e., emotions, usability). We found that when using Hunter programmers required significantly less time to complete various software comprehension tasks and achieved a significantly higher accuracy. We also found that the node-link diagram panel of Hunter gets most of the attention of programmers, whereas the source code panel does so in Visual Studio Code. Moreover, programmers considered that Hunter exhibits a good user experience.

Abstract

Collaborative exploration of scientific data sets across large high-resolution displays requires both high visual detail as wellas low-latency transfer of image data (oftentimes inducing the need to trade one for the other). In this work, we present a system thatdynamically adapts the encoding quality in such systems in a way that reduces the required bandwidth without impacting the detailsperceived by one or more observers. Humans perceive sharp, colourful details, in the small foveal region around the centre of the fieldof view, while information in the periphery is perceived blurred and colourless. We account for this by tracking the gaze of observers,and respectively adapting the quality parameter of each macroblock used by the H.264 encoder, considering the so-called visual acuityfall-off. This allows to substantially reduce the required bandwidth with barely noticeable changes in visual quality, which is crucial forcollaborative analysis across display walls at different locations. We demonstrate the reduced overall required bandwidth and the highquality inside the foveated regions using particle rendering and parallel coordinate

Abstract

While visualisation often strives for abstraction, the interactive exploration of large scientific data sets like densely sampled 3Dfields or massive particle data sets still benefits from rendering their graphical representation in large detail on high-resolutiondisplays such as Powerwalls or tiled display walls driven by multiple GPUs or even GPU clusters. Such visualisation systemsare typically rather unique in their setup of hardware and software which makes transferring a visualisation application fromone high-resolution system to another one a complicated task. As more and more such visualisation systems get installed,collaboration becomes desirable in the sense of sharing such a visualisation running on one site in real time with another high-resolution display on a remote site while at the same time communicating via video and audio. Since typical video conferencesolutions or web-based collaboration tools often cannot deal with resolutions exceeding 4K, with stereo displays or with multi-GPU setups, we designed and implemented a new system based on state-of-the-art hardware and software technologies totransmit high-resolution visualisations including video and audio streams via the internet to remote large displays and back.Our system architecture is built on efficient capturing, encoding and transmission of pixel streams and thus supports a multitudeof configurations combining audio and video streams in a generic approac

Abstract

In this paper, we introduce a visualization technique aimed to help machine learning experts to analyze the hidden states of layers in recurrent neural networks (RNNs). Our technique allows the user to visually inspect how hidden states store and process information throughout the feeding of an input sequence into the network. It can answer questions such as which parts of the input data had a higher impact on the prediction and how the model correlates each hidden state configuration with a certain output. Our visualization comprises several components: our input visualization shows the input sequence and how it relates to the output (using color coding); hidden states are visualized by nonlinear projection to 2-D visualization space via t-SNE in order to understand the shape of the space of hidden states; time curves are employed to show the details of the evolution of hidden state configurations; and a time-multi-class heatmap matrix visualizes the evolution of expected predictions for multi-class classifiers. To demonstrate the capability of our approach, we discuss two typical use cases for long short-term memory (LSTM) models applied to two widely used natural language processing (NLP) datasets.

Abstract

We propose ClaVis, a visual analytics system for comparative analysis of classification models. ClaVis allows users to visually compare the performance and behavior of tens to hundreds of classifiers trained with different hyperparameter configurations. Our approach is plugin-based and classifier-agnostic and allows users to add their own datasets and classifier implementations. It provides multiple visualizations, including a multivariate ranking, a similarity map, a scatterplot that reveals correlations between parameters and scores, and a training history chart. We demonstrate the effectivity of our approach in multiple case studies for training classification models in the domain of natural language processing.

Abstract

We present a user study comparing a pre-evaluated mapping approach with a state-of-the-art direct mapping method of facial expressions for emotion judgment in an immersive setting. At its heart, the pre-evaluated approach leverages semiotics, a theory used in linguistic. In doing so, we want to compare pre-evaluation with an approach that seeks to directly map real facial expressions onto their virtual counterparts. To evaluate both approaches, we conduct a controlled lab study with 22 participants. The results show that users are significantly more accurate in judging virtual facial expressions with pre-evaluated mapping. Additionally, participants were slightly more confident when deciding on a presented emotion. We could not find any differences regarding potential Uncanny Valley effects. However, the pre-evaluated mapping shows potential to be more convenient in a conversational scenario.

Abstract

Nonlinear programming targets nonlinear optimization with constraints, which is a generic yet complex methodology involving humans for problem modeling and algorithms for problem solving. We address the particularly hard challenge of supporting domain experts in handling, understanding, and trouble-shooting high-dimensional optimization with a large number of constraints. Leveraging visual analytics, users are supported in exploring the computation process of nonlinear constraint optimization. Our system was designed for robot motion planning problems and developed in tight collaboration with domain experts in nonlinear programming and robotics. We report on the experiences from this design study, illustrate the usefulness for relevant example cases, and discuss the extension to visual analytics for nonlinear programming in general.

Abstract

Movement-compensating interactions like teleportation are commonly deployed techniques in virtual reality environments. Although practical, they tend to cause disorientation while navigating. Previous studies show the effectiveness of orientation-supporting tools, such as trails, in reducing such disorientation and reveal different strengths and weaknesses of individual tools. However, to date, there is a lack of a systematic comparison of those tools when teleportation is used as a movement-compensating technique, in particular under consideration of different tasks. In this paper, we compare the effects of three orientation-supporting tools, namely minimap, trail, and heatmap. We conducted a quantitative user study with 48 participants to investigate the accuracy and efficiency when executing four exploration and search tasks. As dependent variables, task performance, completion time, space coverage, amount of revisiting, retracing time, and memorability were measured. Overall, our results indicate that orientation-supporting tools improve task completion times and revisiting behavior. The trail and heatmap tools were particularly useful for speed-focused tasks, minimal revisiting, and space coverage. The minimap increased memorability and especially supported retracing tasks. These results suggest that virtual reality systems should provide orientation aid tailored to the specific tasks of the users.

Abstract

Heatmaps are a popular visualization technique that encode 2D density distributions using color or brightness. Experimental studies have shown though that both of these visual variables are inaccurate when reading and comparing numeric data values. A potential remedy might be to use 3D heatmaps by introducing height as a third dimension to encode the data. Encoding abstract data in 3D, however, poses many problems, too. To better understand this tradeoff, we conducted an empirical study (N=48) to evaluate the user performance of 2D and 3D heatmaps for comparative analysis tasks. We test our conditions on a conventional 2D screen, but also in a virtual reality environment to allow for real stereoscopic vision. Our main results show that 3D heatmaps are superior in terms of error rate when reading and comparing single data items. However, for overview tasks, the well-established 2D heatmap performs better.

Abstract

Eye movement data analysis plays an important role in examining human cognitive processes and perceptions. Such analysis at times needs data recording from additional sources too during experiments. In this paper, we study a pair programming based collaboration using two eye trackers, stimulus recording, and an external camera recording. To analyze the collected data, we introduce the EyeSAC system that synchronizes the data from different sources and that removes the noisy and missing gazes from eye tracking data with the help of visual feedback from the external recording. The synchronized and cleaned data is further annotated using our system and then exported for further analysis.

Abstract

Many applications in eye tracking have been increasingly employing neural networks to solve machine learning tasks. In general, neural networks have achieved impressive results in many problems over the past few years, but they still suffer from the lack of interpretability due to their black-box behavior. While previous research on explainable AI has been able to provide high levels of interpretability for models in image classification and natural language processing tasks, little effort has been put into interpreting and understanding networks trained with eye movement datasets. This paper discusses the importance of developing interpretability methods specifically for these models. We characterize the main problems for interpreting neural networks with this type of data, how they differ from the problems faced in other domains, and why existing techniques are not sufficient to address all of these issues. We present preliminary experiments showing the limitations that current techniques have and how we can improve upon them. Finally, based on the evaluation of our experiments, we suggest future research directions that might lead to more interpretable and explainable neural networks for eye tracking.

Abstract

Technical progress in hardware and software enables us to record gaze data in everyday situations and over long time spans. Among a multitude of research opportunities, this technology enables visualization researchers to catch a glimpse behind performance measures and into the perceptual and cognitive processes of people using visualization techniques. The majority of eye tracking studies performed for visualization research is limited to the analysis of gaze distributions and aggregated statistics, thus only covering a small portion of insights that can be derived from gaze data. We argue that incorporating theories and methodology from psychology and cognitive science will benefit the design and evaluation of eye tracking experiments for visualization. This position paper outlines our experiences with eye tracking in visualization and states the benefits that an interdisciplinary research field on visualization psychology might bring for better understanding how people interpret visualizations.

Abstract

Subtitles play a crucial role in cross-lingual distribution of multimedia content and help communicate information where auditory content is not feasible (loud environments, hearing impairments, unknown languages). Established methods utilize text at the bottom of the screen, which may distract from the video. Alternative techniques place captions closer to related content (e.g., faces) but are not applicable to arbitrary videos such as documentations. Hence, we propose to leverage live gaze as indirect input method to adapt captions to individual viewing behavior. We implemented two gaze-adaptive methods and compared them in a user study (n=54) to traditional captions and audio-only videos. The results show that viewers with less experience with captions prefer our gaze-adaptive methods as they assist them in reading. Furthermore, gaze distributions resulting from our methods are closer to natural viewing behavior compared to the traditional approach. Based on these results, we provide design implications for gaze-adaptive captions.

Abstract

We propose a new technique for visual analytics and annotation of long-term pervasive eye tracking data for which a combined analysis of gaze and egocentric video is necessary. Our approach enables two important tasks for such data for hour-long videos from individual participants: (1) efficient annotation and (2) direct interpretation of the results. Exemplary time spans can be selected by the user and are then used as a query that initiates a fuzzy search of similar time spans based on gaze and video features. In an iterative refinement loop, the query interface then provides suggestions for the importance of individual features to improve the search results. A multi-layered timeline visualization shows an overview of annotated time spans. We demonstrate the efficiency of our approach for analyzing activities in about seven hours of video in a case study and discuss feedback on our approach from novices and experts performing the annotation task.

Abstract

IoT devices no longer affect single users only because others like visitors or family members - denoted as bystanders - might be in the device’s vicinity. Thus, data about bystanders can be collected by IoT devices and bystanders can observe what IoT devices output. To better understand how this affects the privacy of IoT device owners and bystanders and how their privacy can be protected better, we interviewed 42 young adults. Our results include that owners of IoT devices wish to adjust the device output when visitors are present. Visitors wish to be made aware of the data collected about them, to express their privacy needs, and to take measures. Based on our results, we show demand for scalable solutions that address the tension that arises between the increasing discreetness of IoT devices, their increase in numbers and the requirement to preserve the self-determination of owners and bystanders at the same time.

Abstract

Professional video editing tools can generate slow-motion video by interpolating frames from video recorded at astandard frame rate. Thereby the perceptual quality of such in-terpolated slow-motion videos strongly depends on the underlyinginterpolation techniques. We built a novel benchmark databasethat is specifically tailored for interpolated slow-motion videos(KoSMo-1k). It consists of 1,350 interpolated video sequences,from 30 different content sources, along with their subjectivequality ratings from up to ten subjective comparisons per videopair. Moreover, we evaluated the performance of twelve exist-ing full-reference (FR) image/video quality assessment (I/VQA)methods on the benchmark. In this way, we are able to show thatspecifically tailored quality assessment methods for interpolatedslow-motion videos are needed, since the evaluated methods –despite their good performance on real-time video databases – donot give satisfying results when it comes to frame interpolation.

Abstract

Current benchmarks for optical flow algorithms evaluate the estimation either directly by comparing the predicted flow fields with the ground truth or indirectly by using the predicted flow fields for frame interpolation and then comparing the interpolated frames with the actual frames. In the latter case, objective quality measures such as the mean squared error are typically employed. However, it is well known that for image quality assessment, the actual quality experienced by the user cannot be fully deduced from such simple measures. Hence, we conducted a subjective quality assessment crowdscouring study for the interpolated frames provided by one of the optical flow benchmarks, the Middlebury benchmark. It contains interpolated frames from 155 methods applied to each of 8 contents. For this purpose, we collected forced-choice paired comparisons between interpolated images and corresponding ground truth. To increase the sensitivity of observers when judging minute difference in paired comparisons we introduced a new method to the field of full-reference quality assessment, called artefact amplification. From the crowdsourcing data (3720 comparisons of 20 votes each) we reconstructed absolute quality scale values according to Thurstone’s model. As a result, we obtained a re-ranking of the 155 participating algorithms w.r.t. the visual quality of the interpolated frames. This re-ranking not only shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks, the results also provide the ground truth for designing novel image quality assessment (IQA) methods dedicated to perceptual quality of interpolated images. As a first step, we proposed such a new full-reference method, called WAE-IQA, which weights the local differences between an interpolated image and its ground truth.

Abstract

In immersive augmented reality (IAR), users can wear a head-mounted display to see computer-generated images superimposed to their view of the world. IAR was shown to be beneficial across several domains, e.g., automotive, medicine, gaming and engineering, with positive impacts on, e.g., collaboration and communication. We think that IAR bears a great potential for software engineering but, as of yet, this research area has been neglected. In this vision paper, we elicit potentials and obstacles for the use of IAR in software engineering. We identify possible areas that can be supported with IAR technology by relating commonly discussed IAR improvements to typical software engineering tasks. We further demonstrate how innovative use of IAR technology may fundamentally improve typical activities of a software engineer through a comprehensive series of usage scenarios outlining practical application. Finally, we reflect on current limitations of IAR technology based on our scenarios and sketch research activities necessary to make our vision a reality. We consider this paper to be relevant to academia and industry alike in guiding the steps to innovative research and applications for IAR in software engineering.

Abstract

We introduce AVAR, a prototypical implementation of an agile situated visualization (SV) toolkit targeting liveness, integration, and expressiveness. We report on results of an exploratory study with AVAR and seven expert users. In it, participants wore a Microsoft HoloLens device and used a Bluetooth keyboard to program a visualization script for a given dataset. To support our analysis, we (i) video recorded sessions, (ii) tracked users' interactions, and (iii) collected data of participants' impressions. Our prototype confirms that agile SV is feasible. That is, liveness boosted participants' engagement when programming an SV, and so, the sessions were highly interactive and participants were willing to spend much time using our toolkit (i.e., median ≥ 1.5 hours). Participants used our integrated toolkit to deal with data transformations, visual mappings, and view transformations without leaving the immersive environment. Finally, participants benefited from our expressive toolkit and employed multiple of the available features when programming an SV.

Abstract

We present a systematic review of 45S papers that report on evaluations in mixed and augmented reality (MR/AR) published in ISMAR, CHI, IEEE VR, and UIST over a span of 11 years (2009-2019). Our goal is to provide guidance for future evaluations of MR/AR approaches. To this end, we characterize publications by paper type (e.g., technique, design study), research topic (e.g., tracking, rendering), evaluation scenario (e.g., algorithm performance, user performance), cognitive aspects (e.g., perception, emotion), and the context in which evaluations were conducted (e.g., lab vs. in-thewild). We found a strong coupling of types, topics, and scenarios. We observe two groups: (a) technology-centric performance evaluations of algorithms that focus on improving tracking, displays, reconstruction, rendering, and calibration, and (b) human-centric studies that analyze implications of applications and design, human factors on perception, usability, decision making, emotion, and attention. Amongst the 458 papers, we identified 248 user studies that involved 5,761 participants in total, of whom only 1,619 were identified as female. We identified 43 data collection methods used to analyze 10 cognitive aspects. We found nine objective methods, and eight methods that support qualitative analysis. A majority (216/248) of user studies are conducted in a laboratory setting. Often (138/248), such studies involve participants in a static way. However, we also found a fair number (30/248) of in-the-wild studies that involve participants in a mobile fashion. We consider this paper to be relevant to academia and industry alike in presenting the state-of-the-art and guiding the steps to designing, conducting, and analyzing results of evaluations in MR/AR.

Abstract

Even though load testing is an established technique to assess load-related quality properties of software systems, it is applied only seldom and with questionable results. Indeed, configuring, executing, and interpreting results of a load test require high effort and expertise. Since chatbots have shown promising results for interactively supporting complex tasks in various domains (including software engineering), we hypothesize that chatbots can provide developers suitable support for load testing. In this paper, we present PerformoBot, our chatbot for configuring and running load tests. In a natural language conversation, PerformoBot guides developers through the process of properly specifying the parameters of a load test, which is then automatically executed by PerformoBot using a state-of-the-art load testing tool. After the execution, PerformoBot provides developers a report that answers the respective concern. We report on results of a user study that involved 47 participants, in which we assessed our tool's acceptance and effectiveness. We found that participants in the study, particularly those with a lower level of expertise in performance engineering, had a mostly positive view of PerformoBot.

Abstract

Visualization in virtual 3D environments can provide a natural way for users to explore data. Often, arm and short head movements are required for interaction in augmented reality, which can be tiring and strenuous though. In an effort toward more user-friendly interaction, we developed a prototype that allows users to manipulate virtual objects using a combination of eye gaze and an external clicker device. Using this prototype, we performed a user study comparing four different input methods of which head gaze plus clicker was preferred by most participants.

Abstract

Often, requirements engineering (RE) activities demand project stakeholders to communicate and collaborate with each other towards building a common software product vision. We conjecture that augmented reality (AR) can be a good fit to support such communication and collaboration. In this vision paper, we report on state-of-the-art research at the intersection of AR and RE. We found that requirements elicitation and analysis have been supported by the ability of AR to provision on-the-fly information such as augmented prototypes. We discuss and map the existing challenges in RE to the aspects of AR that can boost the productivity and user experience of existing RE techniques. Finally, we elaborate on various envisioned usage scenarios in which we highlight concrete benefits and challenges of adopting immersive AR to assist project stakeholders in RE activities.

Abstract

We present a novel variant of parallel coordinates plots (PCPs) in which we show clusters in 2D subspaces of multivariate data and emphasize flow between them. We achieve this by duplicating and stacking individual axes vertically. On a high level, our cluster-flow layout shows how data points move from one cluster to another in different subspaces. We achieve cluster-based bundling and limit plot growth through the reduction of available vertical space for each duplicated axis. Although we introduce space between clusters, we preserve the readability of intra-cluster correlations by starting and ending with the original slopes from regular PCPs and drawing Hermite spline segments in between. Moreover, our rendering technique enables the visualization of small and large data sets alike. Cluster-flow PCPs can even propagate the uncertainty inherent to fuzzy clustering through the layout and rendering stages of our pipeline. Our layout algorithm is based on A*. It achieves an optimal result with regard to a novel set of cost functions that allow us to arrange axes horizontally (dimension ordering) and vertically (cluster ordering).

Abstract

Rectangular treemaps visualize hierarchical numerical data by recursively partitioning an input rectangle into smaller rectangles whose areas match the data. Numerical data often has uncertainty associated with it. To visualize uncertainty in a rectangular treemap, we identify two conflicting key requirements: (i) to assess the data value of a node in the hierarchy, the area of its rectangle should directly match its data value, and (ii) to facilitate comparison between data and uncertainty, uncertainty should be encoded using the same visual variable as the data, that is, area. We present Uncertainty Treemaps, which meet both requirements simultaneously by introducing the concept of hierarchical uncertainty masks. First, we define a new cost function that measures the quality of Uncertainty Treemaps. Then, we show how to adapt existing treemapping algorithms to support uncertainty masks. Finally, we demonstrate the usefulness and quality of our technique through an expert review and a computational experiment on real-world datasets.

Abstract

The analysis of large multiphase flow simulation data poses an interesting and complex research question, which can be addressed with interactive visualization techniques, as well as semi-automated analysis processes. In this project, the focus lies on the investigation of forces governing droplet evolution. Therefore, our proposed methods visualize and allow the analysis of droplet deformation and breakup, droplet behavior and evolution, and droplet-internal flow. By deriving quantities for interface stretching and bending, we visualize and analyze the influence of surface tension force on breakup dynamics, and forces induced by Marangoni convection. Using machine learning to train a simple model for the prediction of physical droplet properties, we provide a visual analysis framework that can be used to analyze large simulation data. Computing droplet-local velocity fields where every droplet is observed separately in its own frame of reference, we create local, interpretable visualizations of flow within droplets, allowing for the investigation of the influence of flow dynamics on droplet evolution.

Abstract

With the growing interest in Immersive Analytics, there is also a need for novel and suitable input modalities for such applications. We explore eye tracking, head tracking, hand motion tracking, and data gloves as input methods for a 2D tracing task and compare them to touch input as a baseline in an exploratory user study (N=20). We compare these methods in terms of user experience, workload, accuracy, and time required for input. The results show that the input method has a significant influence on these measured variables. While touch input surpasses all other input methods in terms of user experience, workload, and accuracy, eye tracking shows promise in respect of the input time. The results form a starting point for future research investigating input methods.

Abstract

Smart speakers become increasingly ubiquitous in our homes. Consequently, we need to study how smart speakers affect the members of a household. Understanding the adoption of a smart speaker can assure it does not negatively influence the social dynamics within a household and create opportunities for further assistance. We deployed an Amazon Echo dot in nine households with 20 inhabitants who were new smart speaker users. We conducted multiple interviews, inquiring how a smart speaker was integrated into a household from day one. We investigated the development of social rules around using the device and how the smart speaker was appropriated. Users developed different strategies of using the device which altered social behaviours in some households. Further, we identified barriers and unmet requirements in introducing smart speakers to home environments. Our work contributes to an understanding of ubiquitous assistance for user groups at home.

Abstract

With the rise of the Internet of Things, home appliances become connected and they can proactively provide status information to users. Facing a steadily increasing number of notification sources, it is unclear how information from smart home devices should be provided without overloading the users’ attention. In this paper, we investigate the design of non-urgent smart home notifications using a smart plant system. Based on feedback from focus groups, we designed four notification types and compared them in an eight-week in-situ study. We show that notifications displayed on smart home devices are preferred to those received on smartphones. Event-based notifications are unobtrusive, actionable and are preferred to persistent notifications. We derive guidelines that address the need of being in control, opportune locations for notification delivery at opportune moments, notification blindness, the importance of discretizing continuous information, and combining related notifications.

Abstract

Appropriate evaluation is a key component in visualization research. It is typically based on empirical studies that assess visualization components or complete systems. While such studies often include the user of the visualization, empirical research is not necessarily restricted to user studies but may also address the technical performance of a visualization system such as its computational speed or memory consumption. Any such empirical experiment faces the issue that the underlying visualization is becoming increasingly sophisticated, leading to an increasingly difficult evaluation in complex environments. Therefore, many of the established methods of empirical studies can no longer capture the full complexity of the evaluation. One promising solution is the use of data-rich observations that we can acquire during studies to obtain more reliable interpretations of empirical research. For example, we have been witnessing an increasing availability and use of physiological sensor information from eye tracking, electrodermal activity sensors, electroencephalography, etc. Other examples are various kinds of logs of user activities such as mouse, keyboard, or touch interaction. Such data-rich empirical studies promise to be especially useful for studies in the wild and similar scenarios outside of the controlled laboratory environment. However, with the growing availability of large, complex, time-dependent, heterogeneous, and unstructured observational data, we are facing the new challenge of how we can analyze such data. This challenge can be addressed by establishing the subfield of visualization for visualization (Vis4Vis): visualization as a means of analyzing and communicating data from empirical studies to advance visualization research.

Abstract

Ubiquitous, situated, and physical visualizations create entirely new possibilities for tasks contextualized in the real world,such as doctors inserting needles. During the development of situated visualizations, evaluating visualizations is a core requirement.However, performing such evaluations is intrinsically hard as the real scenarios are safety-critical or expensive to test. To overcomethese issues, researchers and practitioners adapt classical approaches from ubiquitous computing and use surrogate empiricalmethods such as Augmented Reality (AR), Virtual Reality (VR) prototypes, or merely online demonstrations. This approach’s primaryassumption is that meaningful insights can also be gained from different, usually cheaper and less cumbersome empirical methods.Nevertheless, recent efforts in the Human-Computer Interaction (HCI) community have found evidence against this assumption, whichwould impede the use of surrogate empirical methods. Currently, these insights rely on a single investigation of four interactive objects.The goal of this work is to investigate if these prior findings also hold for situated visualizations. Therefore, we first created a scenariowhere situated visualizations support users in do-it-yourself (DIY) tasks such as crafting and assembly. We then set up five empiricalstudy methods to evaluate the four tasks using an online survey, as well as VR, AR, laboratory, and in-situ studies. Using this studydesign, we conducted a new study with 60 participants. Our results show that the situated visualizations we investigated in this studyare not prone to the same dependency on the empirical method, as found in previous work. Our study provides the first evidence thatanalyzing situated visualizations through different empirical (surrogate) methods might lead to comparable result

Abstract

We propose a photographic method to show scalar values of high dynamic range (HDR) by color mapping for 2D visualization. We combine (1) tone-mapping operators that transform the data to the display range of the monitor while preserving perceptually important features, based on a systematic evaluation, and (2) simulated glares that highlight high-value regions. Simulated glares are effective for highlighting small areas (of a few pixels) that may not be visible with conventional visualizations; through a controlled perception study, we confirm that glare is preattentive. The usefulness of our overall photographic HDR visualization is validated through the feedback of expert users.

Abstract

Gaze tracking in 3D has the potential to improve interaction with objects and visualizations in augmented reality. However, previous research showed that subjective perception of distance varies between real and virtual surroundings. We wanted to determine whether objectively measured 3D gaze depth through eye tracking also exhibits differences between entirely real and augmented environments. To this end, we conducted an experiment (N = 25) in which we used Microsoft HoloLens with a binocular eye tracking add-on from Pupil Labs. Participants performed a task that required them to look at stationary real and virtual objects while wearing a HoloLens device. We were not able to find significant differences in the gaze depth measured by eye tracking. Finally, we discuss our findings and their implications for gaze interaction in immersive analytics, and the quality of the collected gaze data.

Abstract

Automatic clustering techniques play a central role in Visual Analytics by helping analysts to discover interesting patterns in high-dimensional data. Evaluating these clustering techniques, however, is difficult due to the lack of universal ground truth. Instead, clustering approaches are usually evaluated based on a subjective visual judgment of low-dimensional scatterplots of different datasets. As clustering is an inherent human-in-the-loop task, we propose a more systematic way of evaluating clustering algorithms based on quantification of human perception of clusters in 2D scatterplots. The core question we are asking is in how far existing clustering techniques align with clusters perceived by humans. To do so, we build on a dataset from a previous study 1, in which 34 human subjects la-beled 1000 synthetic scatterplots in terms of whether they could see one or more than one cluster. Here, we use this dataset to benchmark state-of-the-art clustering techniques in terms of how far they agree with these human judgments. More specifically, we assess 1437 variants of K-means, Gaussian Mixture Models, CLIQUE, DBSCAN, and Agglomerative Clustering techniques on these benchmarks data. We get unexpected results. For instance, CLIQUE and DBSCAN are at best in slight agreement on this basic cluster counting task, while model-agnostic Agglomerative clustering can be up to a substantial agreement with human subjects depending on the variants. We discuss how to extend this perception-based clustering benchmark approach, and how it could lead to the design of perception-based clustering techniques that would better support more trustworthy and explainable models of cluster patterns.

Abstract

Performance analysis is an integral part of developing and optimizing parallel applications for high performance computing (HPC) platforms. Hierarchical data from different sources is typically available to identify performance issues or anomalies. Some hierarchical data such as the calling context can be very large in terms of breadth and depth of the hierarchy. Classic tree visualizations quickly reach their limits in analyzing such hierarchies with the abundance of information to display. In this position paper, we identify the challenges commonly faced by the HPC community in visualizing hierarchical performance data, with a focus on calling context trees. Furthermore, we motivate and lay out the bases of a visualization that addresses some of these challenges.

Abstract

Manually labeling data sets is a time-consuming and expensive task that can be accelerated by interactive machine learning andvisual analytics approaches. At the core of these approaches are strategies for the selection of candidate instances to label. Weintroduce degree-of-interest (DOI) functions as atomic building blocks to formalize candidate selection strategies. We introducea taxonomy of DOI functions and an approach for the visual analysis of DOI functions, which provide novel complementaryviews on labeling strategies and DOIs, support their in-depth analysis and facilitate their interpretation. Our method shallsupport the generation of novel and better explanation of existing labeling strategies in futu

Abstract

Data visualization is the art and science of mapping data to graphical variables. In this context, networks give rise to unique difficulties because of inherent dependencies among their elements. We provide a high-level overview of the main challenges and common techniques to address them. They are illustrated with examples from two application domains, social networks and automotive engineering. The chapter concludes with opportunities for future work in network visualization.

Abstract

Foveal vision is located in the center of the field of view with a rich impression of detail and color, whereas peripheral visionoccurs on the side with more fuzzy and colorless perception. This visual acuity fall-off can be used to achieve higher frame ratesby adapting rendering quality to the human visual system. Volume raycasting has unique characteristics, preventing a directtransfer of many traditional foveated rendering techniques. We present an approach that utilizes the visual acuity fall-off toaccelerate volume rendering based on Linde-Buzo-Gray sampling and natural neighbor interpolation. First, we measure gazeusing a stationary 1200 Hz eye-tracking system. Then, we adapt our sampling and reconstruction strategy to that gaze. Finally,we apply a temporal smoothing filter to attenuate undersampling artifacts since peripheral vision is particularly sensitive tocontrast changes and movement. Our approach substantially improves rendering performance with barely perceptible changes invisual quality. We demonstrate the usefulness of our approach through performance measurements on various data se

Abstract

We present a method for the spatio-temporal analysis of gaze data from multiple participants in the context of a video stimulus. For such data, an overview of the recorded patterns is important to identify common viewing behavior (such as attentional synchrony) and outliers. We adopt the approach of space-time cube visualization, which extends the spatial dimensions of the stimulus by time as the third dimension. Previous work mainly handled eye tracking data in the space-time cube as point cloud, providing no information about the stimulus context. This paper presents a novel visualization technique that combines gaze data, a dynamic stimulus, and optical flow with volume rendering to derive an overview of the data with contextual information. With specifically designed transfer functions, we emphasize different data aspects, making the visualization suitable for explorative analysis and for illustrative support of statistical findings alike.

Abstract

We present an approach for the visualization and interactive analysis of dynamic graphs that contain a large number of time steps. A specific focus is put on the support of analyzing temporal aspects in the data. Central to our approach is a static, volumetric representation of the dynamic graph based on the concept of space-time cubes that we create by stacking the adjacency matrices of all time steps. The use of GPU-accelerated volume rendering techniques allows us to render this representation interactively. We identified four classes of analytics methods as being important for the analysis of large and complex graph data, which we discuss in detail: data views, aggregation and filtering, comparison, and evolution provenance. Implementations of the respective methods are presented in an integrated application, enabling interactive exploration and analysis of large graphs. We demonstrate the applicability, usefulness, and scalability of our approach by presenting two examples for analyzing dynamic graphs. Furthermore, we let visualization experts evaluate our analytics approach.

Abstract

AgraphG=(V, E)isasupportof a hypergraphH=(V, S)if every hyperedge induces a connected subgraph inG. Supports are usedfor certain types of hypergraph visualizations. In this paper we considervisualizingspatialhypergraphs, where each vertex has a fixed location inthe plane. This is the case, e.g., when modeling set systems of geospatiallocations as hypergraphs. By applying established aesthetic quality cri-teria we are interested in finding supports that yield plane straight-linedrawings with minimum total edge length on the input point setV.Wefirst show, from a theoretical point of view, that the problem isNP-hardalready under rather mild conditions as well as a negative approxima-bility results. Therefore, the main focus of the paper lies on practicalheuristic algorithms as well as an exact, ILP-based approach for comput-ing short plane supports. We report results from computational exper-iments that investigate the effect of requiring planarity and acyclicityon the resulting support length. Further, we evaluate the performanceand trade-offs between solution quality and speed of several heuristicsrelative to each other and compared to optimal solutions.

Abstract

We present MegaMol, a low-overhead prototyping framework for interactive visualization of large scientific data sets. We give a brief summary of related work for context and then focus on a comprehensive overview of the core architecture of the framework. This is followed by the existing and novel features and techniques in MegaMol that define its current functionality. MegaMol has originally been developed to support the visualization and analysis of particle-based data sets that, for instance, come from molecular dynamics simulations. Meanwhile, the software has evolved beyond that. New algorithms and techniques have been implemented to handle many diverse tasks, including information visualization. Additionally, improvements have been made on the software engineering side to make MegaMol more accessible for domain scientists, like an easy-to-handle scripting interface.

Abstract

We propose a technique to represent two-dimensional data using stipples. While stippling is often regarded as an illustrative method, we argue that it is worth investigating its suitability for the visualization domain. For this purpose, we generalize the Linde-Buzo-Gray stippling algorithm for information visualization purposes to encode continuous and discrete 2D data. Our proposed modifications provide more control over the resulting distribution of stipples for encoding additional information into the representation, such as contours. We show different approaches to depict contours in stipple drawings based on locally adjusting the stipple distribution. Combining stipple-based gradients and contours allows for simultaneous assessment of the overall structure of the data while preserving important local details. We discuss the applicability of our technique using datasets from different domains and conduct observation-validating studies to assess the perception of stippled representations

Abstract

Current benchmarks for optical flow algorithms evaluate the estimation quality by comparing their predicted flow field with the ground truth, and additionally may compare interpolated frames, based on these predictions, with the correct frames from the actual image sequences. For the latter comparisons, objective measures such as mean square errors are applied. However, for applications like image interpolation, the expected user's quality of experience cannot be fully deduced from such simple quality measures. Therefore, we conducted a subjective quality assessment study by crowdsourcing for the interpolated images provided in one of the optical flow benchmarks, the Middlebury benchmark. We used paired comparisons with forced choice and reconstructed absolute quality scale values according to Thurstone's model using the classical least squares method. The results give rise to a re-ranking of 141 participating algorithms w.r.t. visual quality of interpolated frames mostly based on optical flow estimation. Our re-ranking result shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks.

Abstract

Generalized Pythagoras trees were developed for visualizing hierarchical data, producing organic, fractal-like representations. However, the drawback of the original layout algorithm is visual overlap of tree branches. To avoid such overlap, we introduce an adapted drawing algorithm using ellipses instead of circles to recursively place tree nodes representing the subhierarchies. Our technique is demonstrated by resolving overlap in diverse real-world and generated datasets, while comparing the results to the original approach.

Abstract

The advent of modern and affordable augmented reality head sets like Microsoft HoloLens has sparked new interest in using virtual and augmented reality technology in the analysis of molecular data. For all visualisation in immersive, mixed-reality scenarios, a sufficiently high rendering speed is an important factor, which leads to the issue of limited processing power available on fully untethered devices facing the situation of handling computationally expensive visualisations. Recent research shows that the space-filling model of even small data sets from the Protein Data Bank (PDB) cannot be rendered at desirable frame rates on the HoloLens. In this work, we report on how to improve the rendering speed of atom-based visualisation of proteins and how the rendering of more abstract representations of the molecules compares against it. We complement our findings with in-depth GPU and CPU performance numbers.

Abstract

In this manuscript, we review the evolution of atomistic visualization over the last 12 years and put the development of the community in context with our own efforts within the DFG collaborative research center 716. The goal is to provide a comprehensive summary of all relevant work that has been conducted under the auspices of project D.3. In this project, we focused on the visualization and analysis of particle-based data sets, and on how to bring these visualizations onto the workstation of domain scientists without the need for a large rendering infrastructure. We discuss how our decisions and goals evolved over time and show the success stories and publications. Finally, we give an outlook on the challenges that still require additional research and to which extent the requirements and constraints of current research have changed the way visualization works after these 12 years.

Abstract

We present a novel technique to correct errors introduced by the discretization of a fluid body when animating it with smoothed particle hydrodynamics (SPH). Our approach is based on the Shepard correction, which reduces the interpolation errors from irregularly spaced data. With Shepard correction, the smoothing kernel function is normalized using the weighted sum of the kernel function values in the neighborhood. To compute the correction factor, densities of neighboring particles are needed, which themselves are computed with the uncorrected kernel. This results in an inconsistent formulation and an error-prone correction of the kernel. As a consequence, the density computation may be inaccurate, thus the pressure forces are erroneous and may cause instabilities in the simulation process.We present a consistent formulation by using the corrected densities to compute the exact kernel correction factor and, thereby, increase the accuracy of the simulation. Employing our method, a smooth density distribution is achieved, i.e., the noise in the density field is reduced by orders of magnitude. To show that our method is independent of the SPH variant, we evaluate our technique on weakly compressible SPH and on divergence-free SPH. Incorporating the corrected density into the correction process, the problem cannot be stated explicitly anymore. We propose an efficient and easy-to-implement algorithm to solve the implicit problem by applying the power method. Additionally, we demonstrate how our model can be applied to improve the density distribution on rigid bodies when using a well-known rigid-fluid coupling approach.

Abstract

The IEEE SciVis 2019 Contest targets the visual analysis of structure formation in the cosmic evolution of the universe from when the universe was five million years old up to now. In our submission, we analyze high-dimensional data to get an overview, then investigate the impact of Active Galactic Nuclei (AGNs) using various visualization techniques, for instance, an adapted filament filtering method for detailed analysis and particle flow in the vicinity of filaments. Based on feedback from domain scientists on these initial visualizations, we also analyzed X-ray emissions and star formation areas. The conversion of star-forming gas to stars and the resulting increasing molecular weight of the particles could be observed.

Abstract

We present a novel abstract visualization for the binding sites of proteins. Binding sites play an essential role in enzymatic reactions and are, thus, often investigated in structural biology. They are typically located within cavities. The shape and properties of the cavity influence whether and how easily a substrate can reach the active site where the reaction is triggered. Molecular surface visualizations can help to analyze the accessibility of binding sites, but are typically prone to visual clutter. Our novel abstract visualization shows the cavity containing the binding site as well as the surface region directly surrounding the cavity entrance in a simplified manner. The resulting visualization resembles a hat, where the brim depicts the surrounding surface region and the crown the cavity. Hence, we dubbed our abstraction Molecular Sombrero, using the Spanish term for 'hat'. Our abstraction is less cluttered than traditional molecular surface visualizations. It highlights important parameters, like cavity diameter, by mapping them to the shape of the sombrero. The visual abstraction also facilitates an easy side-by-side comparison of different data sets. We show the applicability of our Molecular Sombreros to different real-world use cases.

Abstract

In this paper, we review the advances in molecular visualization over the last 12 years and put the development of the community in context with our own efforts in the DFG Collaborative Research Center (CRC) 716. This includes advances in the field of molecular surface computation and rendering, interactive extraction of protein cavities, and comparative visualization for biomolecules. Our main focus was on the development of methods that assist the interactive and explorative visual analysis of large, dynamic molecular data sets on single desktop computers. To meet this goal, we often developed GPU-accelerated algorithms, which is in line with the general research direction of the field. Over the last years, we made seminal contributions to the field of molecular visualization, which partially still constitute the state of the art development or provided the basis for follow-up works

Abstract

We present a framework for rapid prototyping of pervasive visual deficiency simulation in the context of graphical interfaces, virtual reality, and augmented reality. Our framework facilitates the emulation of various visual deficiencies for a wide range of applications, which allows users with normal vision to experience combinations of conditions such as myopia, hyperopia, presbyopia, cataract, nyctalopia, protanopia, deuteranopia, tritanopia, and achromatopsia. Our framework provides an infrastructure to encourage researchers to evaluate visualization and other display techniques regarding visual deficiencies, and opens up the field of visual disease simulation to a broader audience. The benefits of our framework are easy integration, configuration, fast prototyping, and portability to new emerging hardware. To demonstrate the applicability of our framework, we showcase a desktop application and an Android application that transform commodity hardware into glasses for visual deficiency simulation. We expect that this work promotes a greater understanding of visual impairments, leads to better product design for the visually impaired, and forms a basis for research to compensate for these impairments as everyday help

Abstract

Visual analytics (VA) research provides helpful solutions for interactive visual data analysis when exploring large and complex datasets. Due to recent advances in eye tracking technology, promising opportunities arise to extend these traditional VA approaches. Therefore, we discuss foundations for eye tracking support in VA systems. We first review and discuss the structure and range of typical VA systems. Based on a widely used VA model, we present five comprehensive examples that cover a wide range of usage scenarios. Then, we demonstrate that the VA model can be used to systematically explore how concrete VA systems could be extended with eye tracking, to create supportive and adaptive analytics systems. This allows us to identify general research and application opportunities, and classify them into research themes. In a call for action, we map the road for future research to broaden the use of eye tracking and advance visual analytics.

Abstract

Knowledge workers, such as scientists, journalists, or consultants, adaptively seek, gather, and consume information. These processes are often inefficient as existing user interfaces provide limited possibilities to combine information from various sources and different formats into a common knowledge representation. In this paper, we present the concept of an information collage (IC) -- a web browser extension combining manual spatial organization of gathered information fragments and automatic text analysis for interactive content exploration and expressive visual summaries. We used IC for case studies with knowledge workers from different domains and longer-term field studies over a period of one month. We identified three different ways how users collect and structure information and provide design recommendations how to support these observed usage strategies.

Abstract

We propose the concept of Speculative Execution for Visual Analytics and discuss its effectiveness for model exploration and optimization. Speculative Execution enables the automatic generation of alternative, competing model configurations that do not alter the current model state unless explicitly confirmed by the user. These alternatives are computed based on either user interactions or model quality measures and can be explored using delta-visualizations. By automatically proposing modeling alternatives, systems employing Speculative Execution can shorten the gap between users and models, reduce the confirmation bias and speed up optimization processes. In this paper, we have assembled five application scenarios showcasing the potential of Speculative Execution, as well as a potential for further research.

Abstract

When investigating multiphase flow, important aspects to consider are free surfaces, their influence on coherence and breakup of droplets, and the influence of droplet-local flow on droplet shape and topology. To this end, we want to provide visualization techniques and the means for visual analysis of complex multiphase flow data. This data is obtained from direct numerical simulations (DNS), in this case from the solver Free Surface 3D (FS3D), which solves the Navier-Stokes equations for incompressible multiphase flow. The data consists of a velocity field and a volume of fluid (VOF) field, whereas the latter can be used to reconstruct the fluid interface. In the following, we will present three projects, which all aim at using visualization for gaining understanding of phenomena related to flow in droplets and forces acting on the free surfaces.

Abstract

Local digital image correlation is a popular method for accurate full field displacement measurements. However, the technique struggles at autonomously tracking emerging and propagating cracks. We proposed a hybrid approach which utilizes image processing techniques in combination with local digital image correlation to autonomously monitor cracks in a mechanically loaded specimen. Our approach can extract and track crack surfaces and provide a volume-based visualization of the crack growth. This approach was applied to single-fiber composite experimental results with interfacial debonding from the literature. Results quantitatively show that strong interfacial fiber/matrix bonding leads to slower interfacial crack growth, delays interfacial crack growth in the matrix, requires higher loadings for crack growth and shows a specific crack path distinct from the one obtained for weak interfaces. The approach was also validated against a manual approach where a domain scientist extracts a crack using a polygon extraction tool. The method can be used on any local digital image correlation results involving damage observations.

Abstract

We present a machine learning-based approach for detecting and visualizing complex behavior in spatiotemporal volumes. For this, we train models to predict future data values at a given position based on the past values in its neighborhood, capturing common temporal behavior in the data. We then evaluate the model's prediction on the same data. High prediction error means that the local behavior was too complex, unique or uncertain to be accurately captured during training, indicating spatiotemporal regions with interesting behavior. By training several models of varying capacity, we are able to detect spatiotemporal regions of various complexities. We aggregate the obtained prediction errors into a time series or spatial volumes and visualize them together to highlight regions of unpredictable behavior and how they differ between the models. We demonstrate two further volumetric applications: adaptive timestep selection and analysis of ensemble dissimilarity. We apply our technique to datasets from multiple application domains and demonstrate that we are able to produce meaningful results while making minimal assumptions about the underlying data.

Abstract

Selecting a good aspect ratio is crucial for effective 2D diagrams. There are several aspect ratio selection methods for function plots and line charts, but only few can handle general, discrete diagrams such as 2D scatter plots. However, these methods either lack a perceptual foundation or heavily rely on intermediate isoline representations, which depend on choosing the right isovalues and are time-consuming to compute. This paper introduces a general image-based approach for selecting aspect ratios for a wide variety of 2D diagrams, ranging from scatter plots and density function plots to line charts. Our approach is derived from Federer's co-area formula and a line integral representation that enable us to directly construct image-based versions of existing selection methods using density fields. In contrast to previous methods, our approach bypasses isoline computation, so it is faster to compute, while following the perceptual foundation to select aspect ratios. Furthermore, this approach is complemented by an anisotropic kernel density estimation to construct density fields, allowing us to more faithfully characterize data patterns, such as the subgroups in scatterplots or dense regions in time series. We demonstrate the effectiveness of our approach by quantitatively comparing to previous methods and revisiting a prior user study. Finally, we present extensions for ROI banking, multi-scale banking, and the application to image data.

Abstract

In this paper, we examine the robustness of scagnostics through a series of theoretical and empirical studies. First, we investigate the sensitivity of scagnostics by employing perturbing operations on more than 60M synthetic and real-world scatterplots. We found that two scagnostic measures, Outlying and Clumpy, are overly sensitive to data binning. To understand how these measures align with human judgments of visual features, we conducted a study with 24 participants, which reveals that i) humans are not sensitive to small perturbations of the data that cause large changes in both measures, and ii) the perception of clumpiness heavily depends on per-cluster topologies and structures. Motivated by these results, we propose Robust Scagnostics (RScag) by combining adaptive binning with a hierarchy-based form of scagnostics. An analysis shows that RScag improves on the robustness of original scagnostics, aligns better with human judgments, and is equally fast as the traditional scagnostic measures.

Abstract

We present a visualization approach for the analysis of CO 2 bubble-induced attenuation in porous rock formations. As a basis for this, we introduce customized techniques to extract CO 2 bubbles and their surrounding porous structure from X-ray computed tomography data (XCT) measurements. To understand how the structure of porous media influences the occurrence and the shape of formed bubbles, we automatically classify and relate them in terms of morphology and geometric features, and further directly support searching for promising porous structures. To allow for the meaningful direct visual comparison of bubbles and their structures, we propose a customized registration technique considering the bubble shape as well as its points of contact with the porous media surface. With our quantitative extraction of geometric bubble features, we further support the analysis as well as the creation of a physical model. We demonstrate that our approach was successfully used to answer several research questions in the domain, and discuss its high practical relevance to identify critical seismic characteristics of fluid-saturated rock that govern its capability to store CO 2.

Abstract

In this paper, we propose a perceptually-guided visualization sharpening technique. We analyze the spectral behavior of an established comprehensive perceptual model to arrive at our approximated model based on an adapted weighting of the bandpass images from a Gaussian pyramid. The main benefit of this approximated model is its controllability and predictability for sharpening color-mapped visualizations. Our method can be integrated into any visualization tool as it adopts generic image-based post-processing, and it is intuitive and easy to use as viewing distance is the only parameter. Using highly diverse datasets, we show the usefulness of our method across a wide range of typical visualizations.

Abstract

As dementia will most likely become an impactful challenge for our future society, it is imperative to maintain the well-being of the diverse group of people with dementia (PwD). Thus, appropriate interventions that effectively trigger identity-stabilizing memories, and at the same time encourage sensorimotor activities, have to be designed and implemented. To that end, we present a novel natural user interface (NUI) system combined with a reminiscence-provoking virtual 3D environment (VE). With it, PwD can delve into memories while interacting with the VE over dementia-fitted gestures. The results of the preliminary evaluations are promising, as they show that most PwD get immersed and cheerfully engage in gesture interactions after a short settling-in period.

Abstract

Tools for visual data exploration allow users to visually browse through and analyze datasets to possibly reveal interesting infor- mation hidden in the data that users are a priori unaware of. Such tools rely on both query recommendations to select data to be visualized and visualization recommendations for these data to best support users in their visual data exploration process. EVLIN ( e xploring v isually with lin eage) is a system that assists users in visually exploring relational data stored in a data ware- house. EVLIN implements novel techniques for recommending both queries and their result visualization in an integrated and interactive way 3 . Recommendations rely on provenance (aka lineage) that describes the production process of displayed data . The demonstration of EVLIN includes an introduction to its features and functionality through sample exploration sessions. Conference attendees will then have the opportunity to gain hands- on experience of provenance-based visual data exploration by performing their own exploration sessions. These sessions will explore real-world data from several domains. While exploration sessions use a Web-based visual interface, the demonstration also features a researcher console, where attendees may have a look behind the scenes to get a more in-depth understanding of the underlying recommendation algorithms. Fachgebiet (DDC): 004 Informatik

Abstract

We present an approach for interactively analyzing large dynamic graphs consisting of several thousand time steps with a particular focus on temporal aspects. we employ a static representation of the time-varying graph based on the concept of space-time cubes, i.e., we create a volumetric representation of the graph by stacking the adjacency matrices of each of its time steps. To achieve an efficient analysis of complex data, we discuss three classes of analytics methods of particular importance in this context: data views, aggregation and filtering, and comparison. For these classes, we present a GPU-based implementation of respective analysis methods that enable the interactive analysis of large graphs. We demonstrate the utility as well as the scalability of our approach by presenting application examples for analyzing different time-varying data sets.

Abstract

We present LTMA, a Layered Topic Matching approach for the unsupervised comparative analysis of topic modeling results. Due to the vast number of available modeling algorithms, an efficient and effective comparison of their results is detrimental to a data- and task-driven selection of a model. LTMA automates this comparative analysis by providing topic matching based on two layers (document-overlap and keyword-similarity), creating a novel topic-match data structure. This data structure builds a basis for model exploration and optimization, thus, allowing for an efficient evaluation of their performance in the context of a given type of text data and task. This is especially important for text types where an annotated gold standard dataset is not readily available and, therefore, quantitative evaluation methods are not applicable. We confirm the usefulness of our technique based on three use cases, namely: (1) the automatic comparative evaluation of topic models, (2) the visual exploration of topic modeling differences, and (3) the optimization of topic modeling results through combining matches.

Abstract

We visualize contours for spatio‐temporal processes to indicate where and when non‐continuous changes occur or spatial bounds are encountered. All time steps are comprised densely in one visualization, with contours allowing to efficiently analyze processes in the data even in case of spatial or temporal overlap. Contours are determined on the basis of deep raycasting that collects samples across time and depth along each ray. For each sample along a ray, its closest neighbors from adjacent rays are identified, considering time, depth, and value in the process. Large distances are represented as contours in image space, using color to indicate temporal occurrence. This contour representation can easily be combined with volume rendering‐based techniques, providing both full spatial detail for individual time steps and an outline of the whole time series in one view. Our view‐dependent technique supports efficient progressive computation, and requires no prior assumptions regarding the shape or nature of processes in the data. We discuss and demonstrate the performance and utility of our approach via a variety of data sets, comparison and combination with an alternative technique, and feedback by a domain scientist.

Abstract

We present an approach that dynamically adapts encoder settings for image tiles to yield the best possible quality for a given bandwidth. This reduces the overall size of the image while preserving details. Our application determines the encoding settings in two steps. In the first step, we predict the quality and size of the tiles for different encoding settings using a convolutional neural network. In the second step, we assign the optimal encoder setting to each tile, so that the overall size of the image is lower than a predetermined threshold. Commonly, for tiles that contain complicated structures, a high quality setting is used in order to prevent major information loss, while quality settings are lowered for others to keep the size below the threshold. We demonstrate that we can reduce the overall size of the image while preserving the details in areas of interest using the example of both particle and volume visualisation applications.

Abstract

We present a novel type of circular treemap, where we intentionally allocate extra space for additional visual variables. With this extended visual design space, we encode hierarchically structured data along with their uncertainties in a combined diagram. We introduce a hierarchical and force-based circle-packing algorithm to compute Bubble Treemaps, where each node is visualized using nested contour arcs. Bubble Treemaps do not require any color or shading, which offers additional design choices. We explore uncertainty visualization as an application of our treemaps using standard error and Monte Carlo-based statistical models. To this end, we discuss how uncertainty propagates within hierarchies. Furthermore, we show the effectiveness of our visualization using three different examples: the package structure of Flare, the S&P 500 index, and the US consumer expenditure survey

Abstract

Pipeline approaches that interpolate and refine an initial set of point correspondenceshave recently shown a good performance in the field of optical flow estimation. However,so far, these methods are typically restricted to two frames which makes exploiting tem-poral information difficult. In this paper, we show how such pipeline approaches can beextended to the temporal domain and how directional constraints can be incorporated tofurther improve the estimation. To this end, we not only suggest to exploit temporal infor-mation in the prefiltering step, we also propose a trajectorial refinement method that liftssuccessful concepts of recent variational two-frame methods to the multi-frame domain.Experiments demonstrate the usefulness of our pipeline approach. They do not only showgood results in general, they also demonstrate the clear benefits of using multiple framesand of imposing directional constraints on the prefiltering step and the refinement.

Abstract

Temporal coherence is a valuable source of information in the context of optical flow estimation. However, finding a suitable motion model to leverage this information is a non-trivial task. In this paper we propose an unsupervised online learning approach based on a convolutional neural network (CNN) that estimates such a motion model individually for each frame. By relating forward and backward motion these learned models not only allow to infer valuable motion information based on the backward flow, they also help to improve the performance at occlusions, where a reliable prediction is particularly useful. Moreover, our learned models are spatially variant and hence allow to estimate non-rigid motion per construction. This, in turns, allows to overcome the major limitation of recent rigidity-based approaches that seek to improve the estimation by incorporating additional stereo/SfM constraints. Experiments demonstrate the usefulness of our new approach. They not only show a consistent improvement of up to 27% for all major benchmarks (KITTI 2012, KITTI 2015, MPI Sintel) compared to a baseline without prediction, they also show top results for the MPI Sintel benchmark -- the one of the three benchmarks that contains the largest amount of non-rigid motion.

Abstract

Many recent energy-based methods for optical flow estimation rely on a good initialization that is typically provided by some kind of feature matching. So far, however, these initial matching approaches are rather general: They do not incorporate any additional information that could help to improve the accuracy or the robustness of the estimation. In particular, they do not exploit potential cues on the camera poses and the thereby induced rigid motion of the scene. In the present paper, we tackle this problem. To this end, we propose a novel structure-from-motion-aware PatchMatch approach that, in contrast to existing matching techniques, combines two hierarchical feature matching methods: a recent two-frame PatchMatch approach for optical flow estimation (general motion) and a specifically tailored three-frame PatchMatch approach for rigid scene reconstruction (SfM). While the motion PatchMatch serves as baseline with good accuracy, the SfM counterpart takes over at occlusions and other regions with insufficient information. Experiments with our novel SfM-aware PatchMatch approach demonstrate its usefulness. They not only show excellent results for all major benchmarks (KITTI 2012/2015, MPI Sintel), but also improvements up to 50% compared to a PatchMatch approach without structure information.

Abstract

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. This difference in approaching the reconstruction problem yields complementary advantages that are worthwhile being combined. So far, however, most “joint” approaches are based on an initial stereo mesh that is subsequently refined using shading information. In this paper we follow a completely different approach. We propose a joint variational method that combines both cues within a single minimisation framework. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the resulting model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images demonstrate the benefits of our combined approach: They not only show that our method is capable of generating very detailed reconstructions, but also that joint approaches are feasible in practice.

Abstract

Data provenance can support the understanding and debugging of complex data processing pipelines, which are for instance common in data integration scenarios. One task in data integration is entity resolution (ER), i.e., the identification of multiple representations of a same real world entity. This paper focuses of provenance modeling and capture for typical ER tasks. While our definition of ER provenance is independent of the actual language or technology used to define an ER task, the method we implement as a proof of concept instruments ER rules specified in HIL, a high-level data integration language.

Abstract

Conventional dot plots use a constant dot size and are typically applied to show the frequency distribution of small data sets. Unfortunately, they are not designed for a high dynamic range of frequencies. We address this problem by introducing nonlinear dot plots. Adopting the idea of nonlinear scaling from logarithmic bar charts, our plots allow for dots of varying size so that columns with a large number of samples are reduced in height. For the construction of these diagrams, we introduce an efficient two-way sweep algorithm that leads to a dense and symmetrical layout. We compensate aliasing artifacts at high dot densities by a specifically designed low-pass filtering method. Examples of nonlinear dot plots are compared to conventional dot plots as well as linear and logarithmic histograms. Finally, we include feedback from an expert review.

Abstract

The analysis of eye-tracking data can be very useful when evaluating controlled user studies. To support the analysis in a fast and easy fashion, we have developed a web-based framework for a visual inspection of eye-tracking data and a comparison of scanpaths based on filtering of fixations and similarity measures. Concerning the first part, we introduce a multiscale aggregation of fixations and saccades based on a spatial partitioning that reduces visual clutter of overlaid scanpaths without changing the overall impression of large-scale eye movements. The multiscale technique abstracts the individual scanpaths and allows an analyst to visually identify clusters or patterns inherent to the gaze data without the need for lengthy precomputations. For the second part, we introduce an approach where analysts can remove fixations from a pair of scanpaths in order to increase the similarity between them. This can be useful to discover and understand reasons for dissimilarity between scanpaths, data cleansing, and outlier detection. Our implementation uses the MultiMatch algorithm to predict similarities after the removal of individual fixations. Finally, we demonstrate the usefulness of our techniques in a use case with scanpaths that were recorded in a study with metro maps.

Abstract

We present a technique that conveys the uncertainty in the secondary structure of proteins-an abstraction model based on atomic coordinates. While protein data inherently contains uncertainty due to the acquisition method or the simulation algorithm, we argue that it is also worth investigating uncertainty induced by analysis algorithms that precede visualization. Our technique helps researchers investigate differences between multiple secondary structure assignment methods. We modify established algorithms for fuzzy classification and introduce a discrepancy-based approach to project an ensemble of sequences to a single importance-weighted sequence. In 2D, we depict the aggregated secondary structure assignments based on the per-residue deviation in a collapsible sequence diagram. In 3D, we extend the ribbon diagram using visual variables such as transparency, wave form, frequency, or amplitude to facilitate qualitative analysis of uncertainty. We evaluated the effectiveness and acceptance of our technique through expert reviews using two example applications: the combined assignment against established algorithms and time-dependent structural changes originating from simulated protein dynamics.

Abstract

We propose a tree visualization technique for comparison of structures and attributes across multiple hierarchies. Many software systems are structured hierarchically by design. For example, developers subdivide source code into libraries, modules, and functions. This design propagates to software configuration and business processes, rendering software hierarchies even more important. Often these structural elements are attributed with reference counts, code quality metrics, and the like. Throughout the entire software life cycle, these hierarchies are reviewed, integrated, debugged, and changed many times by different people so that the identity of a structural element and its attributes is not clearly traceable. We argue that pairwise comparison of similar trees is a tedious task due to the lack of overview, especially when applied to a large number of hierarchies. Therefore, we strive to visualize multiple similar trees as a whole by merging them into one supertree. To merge structures and combine attributes from different trees, we leverage the Jaccard similarity and solve a matching problem while keeping track of the origin of a structure element and its attributes. Our visualization approach allows users to inspect these supertrees using node-link diagrams and indented tree plots. The nodes in these plots depict aggregated attributes and, using word-sized line plots, detailed data. We demonstrate the usefulness of our method by exploring the evolution of software repositories and debugging data processing pipelines using provenance data.

Abstract

This project aims at investigating droplet-related phenomena, especially the influence of forces acting on the surface. To this end, we visualize interface deformation using two quantities: interface stretching and interface bending. This allows the visual analysis of droplet behavior and interface-related forces, such as the surface tension force and forces induced by Marangoni convection. For the latter, we show an application case for debugging and aiding the implementation of the Marangoni term in the solver Free Surface 3D (FS3D). Furthermore, we demonstrate the usefulness of our approach for prediction of droplet breakup.

Abstract

We present an empirical study that illustrates how individual users' decision making preferences and biases influence visualization design choices. Twenty-three participants, in a lab study, were shown two interactive financial portfolio optimization interfaces which allowed them to adjust the return for the portfolio and view how the risk changes. One interface showed the sensitivity of the risk to changes in the return and one did not have this feature. Our study highlights two classes of users. One which preferred the interface with the sensitivity feature and one group that does not prefer the sensitivity feature. We named these two groups the "risk fixers" and the "sweet spotters" due to the analysis method they used. The "risk fixers" selected a level of risk which they were comfortable with while the "sweet spotters" tried to find a point right before the risk increased greatly. Our study shows that exposing the sensitivity of investment parameters will impact the investment decision process and increase confidence for these "sweet spotters." We also discuss the implications for design.

Abstract

We investigate priming and anchoring effects on perceptual tasks in visualization. Priming or anchoring effects depict the phenomena that a stimulus might influence subsequent human judgments on a perceptual level, or on a cognitive level by providing a frame of reference. Using visual class separability in scatterplots as an example task, we performed a set of five studies to investigate the potential existence of priming and anchoring effects. Our findings show that - under certain circumstances - such effects indeed exist. In other words, humans judge class separability of the same scatterplot differently depending on the scatterplot(s) they have seen before. These findings inform future work on better understanding and more accurately modeling human perception of visual patterns.

Abstract

Dimensionality reduction (DR) is a common strategy for visual analysis of labeled high-dimensional data. Low-dimensional representations of the data help, for instance, to explore the class separability and the spatial distribution of the data. Widely-used unsupervised DR methods like PCA do not aim to maximize the class separation, while supervised DR methods like LDA often assume certain spatial distributions and do not take perceptual capabilities of humans into account. These issues make them ineffective for complicated class structures. Towards filling this gap, we present a perception-driven linear dimensionality reduction approach that maximizes the perceived class separation in projections. Our approach builds on recent developments in perception-based separation measures that have achieved good results in imitating human perception. We extend these measures to be density-aware and incorporate them into a customized simulated annealing algorithm, which can rapidly generate a near optimal DR projection. We demonstrate the effectiveness of our approach by comparing it to state-of-the-art DR methods on 93 datasets, using both quantitative measure and human judgments. We also provide case studies with class-imbalanced and unlabeled data.

Abstract

We present an improved stress majorization method that incorporates various constraints, including directional constraints without the necessity of solving a constraint optimization problem. This is achieved by reformulating the stress function to impose constraints on both the edge vectors and lengths instead of just on the edge lengths (node distances). This is a unified framework for both constrained and unconstrained graph visualizations, where we can model most existing layout constraints, as well as develop new ones such as the star shapes and cluster separation constraints within stress majorization. This improvement also allows us to parallelize computation with an efficient GPU conjugant gradient solver, which yields fast and stable solutions, even for large graphs. As a result, we allow the constraint-based exploration of large graphs with 10K nodes - an approach which previous methods cannot support.

Abstract

Personal networks formed within scientific communities and the collaborations they yield are one of the driving forces behind innovation and new discoveries. Luckily, successful collaboration produces analyzable data points in the form of publications that allow us to learn and understand some of the connections and collaborative structures in a scientific community. Co-author information is one important aspect of this, and various solutions to the fundamental visualization problems of co-author graphs exist. In this work, we introduce ColTop, a multi-level, interactive graph visualization system that allows users to effectively analyze publication data. It combines coauthor information with other meta-data and information extracted from textual content to support comprehensive analyses. ColTop includes a novel, heuristics-based approach to create a meaningful abstraction of co-author networks, and enriches them with topic information. To demonstrate the applicability of our approach, we discuss an example analysis scenario based on a practical data set.