Epistemic Interactions Lab

Characterizing Photorealism and Artifacts in Diffusion Model-Generated Images

Negar Kamali, Karyn Nakamura, Aakriti Kumar, Angelos Chatzimparmpas, Jessica Hullman, Matthew Groh ACM Human Factors in Computing Systems (CHI) 2025 | PDF

Underspecified Human Decision Experiments Considered Harmful

Jessica Hullman, Alex Kale, Jason Hartline ACM Human Factors in Computing Systems (CHI) 2025 | PDF

Seeing Eye to AI? Applying Deep-Feature-Based Similarity Metrics to Information Visualization

Sheng Long, Angelos Chatzimparmpas, Emma Alexander, Matthew Kay, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2025 | PDF | Supplementary Material

Improving out-of-population prediction: The complementary effects of model assistance and judgmental bootstrapping

Mathew D. Hardy, Sam Zhang, Jessica Hullman, Jake M. Hofman, Daniel G. Goldstein International Journal of Forecasting | PDF

Automating the Practice of Science: Opportunities, Challenges, and Implications

Sebastian Musslick, Laura K Bartlett, Suyog H Chandramouli, Marina Dubova, Fernand Gobet, Thomas L Griffiths, Jessica Hullman, Ross D King, J Nathan Kutz, Christopher G Lucas, Suhas Mahesh, Franco Pestilli, Sabina J Sloman, William R Holmes Proceedings of the National Academy of Science (PNAS) | PDF

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling

Dongping Zhang, Angelos Chatzimparmpas, Negar Kamali, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2024 | Best Paper Honorable Mention | PDF | GitHub

Designing Shared Information Displays for Agents of Varying Strategic Sophistication

Dongping Zhang, Jason Hartline, Jessica Hullman ACM Computer-Supported Cooperative Work (CSCW) 2024 | PDF | GitHub

A Decision Theoretic Framework for Measuring AI Reliance

Ziyang Guo, Yifan Wu, Jason Hartline, Jessica Hullman ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT '24) | PDF

VMC: A Grammar for Visualizing Statistical Model Checks

Ziyang Guo, Alex Kale, Matthew Kay, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. VIS) 2024 | Package | Documentation

Milliways: Taming Multiverses through Principled Evaluation of Data Analysis Paths

Abhraneel Sarma, Kyle Hwang, Jessica Hullman, Matthew Kay ACM Human Factors in Computing Systems (CHI) 2024 | PDF

Erie: A Declarative Grammar for Data Sonification

Hyeok Kim, Yea-Seul Kim, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2024 | PDF | Package | Documentation | Online Editor

Measure-Observe-Remeasure: An Interactive Paradigm for Differentially-Private Exploratory Analysis

Priyanka Nanayakkara, Hyeok Kim, Yifan Wu, Ali Sarvghad, Narges Mahyar, Gerome Miklau, Jessica Hullman IEEE Symposium on Security & Privacy (SP) | PDF

Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms

Fumeng Yang, Mandi Cai, Chloe Mortenson, Hoda Fakhari, Ayse D. Lokmanoglu, Jessica Hullman, Steven Franconeri, Nicholas Diakopoulos, Erik C. Nisbet, Matthew Kay IEEE Trans. Vis. Compt. Graph. (Proc. VIS) 2023 | Best Paper | PDF | Supplementary Material

Dupo: A Mixed-Initiative Authoring Tool for Responsive Visualization

Hyeok Kim, Ryan Rossi, Jessica Hullman, Jane Hoffswell IEEE Trans. Vis. Compt. Graph. (Proc. VIS) 2023 | PDF | Supplementary Material

The Rational Agent Benchmark for Data Visualization

Yifan Wu, Ziyang Guo, Michails Mamakos, Jason Hartline, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. VIS) 2023 | PDF

EVM: Incorporating Model Checking into Exploratory Visual Analysis

Alex Kale, Ziyang Guo, Xiao Li Qiao, Jeffrey Heer, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. VIS) 2023 | PDF

Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research

Hariharan Subramonyam, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. VIS) 2023 | PDF

Causal Quartets: Different Ways to Obtain the Same Average Treatment Effect

Andrew Gelman, Jessica Hullman, Lauren Kennedy American Statistician | PDF

multiverse: Multiplexing Alternative Data Analyses in R Notebooks

Abhraneel Sarma, Alex Kale, Michael Moon, Nathan Taback, Fanny Chevalier, Jessica Hullman, Matthew Kay ACM Human Factors in Computing Systems (CHI) 2023 | Best Paper Honorable Mention | PDF

MetaExplorer: Facilitating Reasoning with Epistemic Uncertainty in Meta-analysis

Alex Kale, Sarah Lee, Terrance Goan, Elizabeth Tipton, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2023 | PDF | GitHub

What's Driving Conflicts Around Differential Privacy for the US Census

Priyanka Nanayakkara, Jessica Hullman IEEE Security & Privacy Magazine | PDF

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning

Jessica Hullman, Sayash Kapoor, Priyanka Nanayakkara, Andrew Gelman, Arvind Narayanan AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES '22) | PDF

Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews

David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES '22) | PDF

Cicero: A Declarative Grammar for Responsive Visualization

Hyeok Kim, Ryan Rossi, Fan Du, Eunyee Koh, Shunan Guo, Jessica Hullman, Jane Hoffswell ACM Human Factors in Computing Systems (CHI) 2022 | PDF | Supplementary Material | Gallery

Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releases

Priyanka Nanayakkara, Johes Bater, Xi He, Jessica Hullman, Jennie Rogers Proceedings on Privacy Enhancing Technologies 2022 | PDF

Designing for interactive exploratory data analysis requires theories of graphical inference

Jessica Hullman, Andrew Gelman Harvard Data Science Review 2021 | PDF | Open Access

Visualization Equilibrium

Paula Kayongo, Glenn Sun, Jason Hartline, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2021 | PDF

Visualizing Uncertainty in Probabilistic Graphs with Network Hypothetical Outcome Plots (NetHOPs)

Dongping Zhang, Eytan Adar, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2021 | PDF | GitHub

Causal Support: Modeling Causal Inferences with Visualizations

Alex Kale, Yifan Wu, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2021 | PDF | Supplementary Material | GitHub

An Automated Approach to Reasoning About Task-Oriented Insights in Responsive Visualization

Hyeok Kim, Ryan Rossi, Abhraneel Sarma, Dominik Moritz, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2021 | PDF | Supplementary Material

Unpacking the Expressed Consequences of AI Research in Broader Impact Statements

Priyanka Nanayakkara, Jessica Hullman, Nicholas Diakopoulos AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society 2021 (AIES '21) | PDF | Blog post

Design Patterns and Trade-Offs in Responsive Visualization for Communication

Hyeok Kim, Dominik Moritz, Jessica Hullman Computer Graphics Forum (Proc. EUROVIS 2021) | PDF | Supplementary Material | Gallery

Information, incentives, and goals in election forecasts

Andrew Gelman, Jessica Hullman, Christopher Wlezien, George Elliott Morris Judgment and Decision Making 2020 | PDF

Bayesian-Assisted Inference from Visualized Data

Yea-Seul Kim, Paula Kayongo, Madeleine Grunde-McLaughlin, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2020 | PDF

Visual Reasoning Strategies for Effect Size Judgments and Decisions

Alex Kale, Matthew Kay, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2020 | Best Paper | PDF | GitHub

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs

Sungsoo Ray Hong, Jessica Hullman, Enrico Bertini ACM Computer Supported Coopoerative Work (CSCW) 2020 | PDF

How Visualizing Inferential Uncertainty Can Mislead Readers About Treatment Effects in Scientific Results

Jake M. Hofman, Daniel G. Goldstein, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2020 | Best Paper Honorable Mention | PDF

Exploring the Effects of Aggregation Choices on Untrained Visualization Users’ Generalizations from Data

Francis Nguyen, Xiaoli Qiao, Jeffrey Heer, Jessica Hullman Computer Graphics Forum 2020 | PDF | GitHub

Why Authors Don't Visualize Uncertainty

Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2019 | PDF

Illusion of Causality in Visualized Data

Cindy Xiong, Joel Shapiro, Jessica Hullman, Steve Franconeri IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2019 | PDF

A Bayesian Cognition Approach to Improve Data Visualization

Yea-Seul Kim, Logan A Walls, Peter Krafft, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2019 | PDF | GitHub

Decision-Making Under Uncertainty in Research Synthesis: Designing for the Garden of Forking Paths

Alex Kale, Matthew Kay, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2019 | PDF | GitHub

Vocal Shortcuts for Creative Experts

Yea-Seul Kim, Mira Dontcheva, Eytan Adar, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2019 | PDF

Some Prior(s) Experience Necessary: Templates for Getting Started With Bayesian Analysis

Chanda Phelan, Jessica Hullman, Matthew Kay, Paul Resnick ACM Human Factors in Computing Systems (CHI) 2019 | PDF

Belief-Driven Data Journalism

Francis Nguyen, Samana Shrestha, Joe Germuska, Yea-Seul Kim, Jessica Hullman Computation + Journalism Symposium 2019 | PDF | Medium post | Code

Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data

Alex Kale, Francis Nguyen, Matthew Kay, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2018 | PDF | GitHub

In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation

Jessica Hullman, Xiaoli Qiao, Michael Correll, Alex Kale, Matthew Kay IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2018 | PDF | GitHub | Website

Improving Comprehension of Measurements Using Concrete Re-expression Strategies

Jessica Hullman, Yea-Seul Kim, Francis Nguyen, Lauren Speers, Maneesh Agrawala ACM Human Factors in Computing Systems (CHI) 2018 | PDF | GitHub | Website

Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making

Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, Matthew Kay ACM Human Factors in Computing Systems (CHI) 2018 | Best Paper Honorable Mention | PDF

Keeping Multiple Views Consistent: Constraints, Validations, and Exceptions in Visualization Authoring

Zening Qu, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2017 | Best Paper Honorable Mention | PDF

Data Through Others' Eyes: The Impact of Visualizing Others' Expectations on Visualization Interpretation

Yea-Seul Kim, Katharina Reinecke, Jessica Hullman IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2017 | PDF

Imagining Replications: Graphical Prediction & Discrete Visualizations Improve Recall & Estimation of Effect Uncertainty

Jessica Hullman, Matthew Kay, Yea-Seul Kim, Samana Shrestha IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2017 | PDF | GitHub

Finding a Clear Path: Structuring Strategies for Visualization Sequences

Jessica Hullman, Robert Kosara, Heidi Lam Computer Graphics Forum (Proc. EUROVIS) 2017 | PDF

GraphScape: A Model for Automated Reasoning about Visualization Similarity and Sequencing

Younghoon Kim, Kanit Wongsuphasawat, Jessica Hullman, Jeffrey Heer ACM Human Factors in Computing Systems (CHI) 2017 | Best Paper Honorable Mention | PDF | GitHub

Explaining the Gap: Visualizing One's Predictions Improves Recall and Comprehension of Data

Yea-Seul Kim, Katharina Reinecke, Jessica Hullman ACM Human Factors in Computing Systems (CHI) 2017 | Best Paper | PDF

Why Evaluating Uncertainty Visualization is Error Prone

Jessica Hullman BELIV 2016 | PDF

Evaluating Visualization Sets: Trade-offs Between Local Effectiveness and Global Consistency

Zening Qu, Jessica Hullman BELIV 2016 | PDF

Generating Personalized Spatial Analogies for Distances and Areas

Yea-Seul Kim, Jessica Hullman, Maneesh Agrawala ACM Human Factors in Computing Systems (CHI) 2016 | PDF | GitHub

When(ish) is My Bus? User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems

Matthew Kay, Tara Kola, Jessica Hullman, Sean Munson ACM Human Factors in Computing Systems (CHI) 2016 | PDF | GitHub

Content, Context, and Critique: Commenting on a Data Visualization Blog

Jessica Hullman, Nicholas Diakopoulos, Elaheh Momeni Roochi, Eytan Adar ACM Computer-Supported Cooperative Work (CSCW) 2015 | PDF

Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering

Jessica Hullman, Paul Resnick, Eytan Adar PLOS ONE 2015 | PDF

NewsViews: An Automated Pipeline for Creating Custom Geovisualizations for News

Tong Gao, Jessica Hullman, Eytan Adar, Brent Hecht, Nicholas Diakopoulos ACM Human Factors in Computing Systems (CHI) 2014 | PDF

A Deeper Understanding of Sequence in Narrative Visualization

Jessica Hullman, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher, Eytan Adar IEEE Trans. Vis. Compt. Graph. (Proc. INFOVIS) 2013 | PDF