Skip to feedback

Award Abstract # 1910264
SHF: Small: Declaratively Creating Semantics-driven Visualizations

NSF Org: CCF
Division of Computing and Communication Foundations
Recipient: CARNEGIE MELLON UNIVERSITY
Initial Amendment Date: May 31, 2019
Latest Amendment Date: March 3, 2023
Award Number: 1910264
Award Instrument: Standard Grant
Program Manager: Anindya Banerjee
abanerje@nsf.gov
 (703)292-7885
CCF
 Division of Computing and Communication Foundations
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2019
End Date: September 30, 2023 (Estimated)
Total Intended Award Amount: $449,680.00
Total Awarded Amount to Date: $513,680.00
Funds Obligated to Date: FY 2019 = $449,680.00
FY 2020 = $16,000.00

FY 2021 = $16,000.00

FY 2022 = $16,000.00

FY 2023 = $16,000.00
History of Investigator:
  • Jonathan Aldrich (Principal Investigator)
    jonathan.aldrich@cs.cmu.edu
  • Keenan Crane (Co-Principal Investigator)
  • Joshua Sunshine (Co-Principal Investigator)
Recipient Sponsored Research Office: Carnegie-Mellon University
5000 FORBES AVE
PITTSBURGH
PA  US  15213-3815
(412)268-8746
Sponsor Congressional District: 12
Primary Place of Performance: Carnegie Mellon University
5000 Forbes Ave, WQED Building
Pittsburgh
PA  US  15213-3815
Primary Place of Performance
Congressional District:
12
Unique Entity Identifier (UEI): U3NKNFLNQ613
Parent UEI: U3NKNFLNQ613
NSF Program(s): Software & Hardware Foundation
Primary Program Source: 01002223DB NSF RESEARCH & RELATED ACTIVIT
01002324DB NSF RESEARCH & RELATED ACTIVIT

01001920DB NSF RESEARCH & RELATED ACTIVIT

01002021DB NSF RESEARCH & RELATED ACTIVIT

01002122DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7923, 7943, 9251
Program Element Code(s): 779800
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Visual diagrams are essential for communicating difficult technical concepts; for instance, there is ample evidence that students learn more effectively and can solve problems more efficiently when using diagrams rather than plain text. Effective communication of scientific concepts is limited by the fact that good diagrams remain hard to create: abstract logical or mathematical concepts are difficult for non-experts to translate into compelling graphical figures, and traditional software tools produce static diagrams that do not easily facilitate interaction or exploration. This project develops a next-generation framework called Penrose, where high-quality diagrams can be created by simply typing expressions in domain-specific languages that reflect the way students and scientists already talk about mathematics. The project's novelties are to systematically encode the relationships between abstract mathematical statements and their visual representations, and to automatically generate diagrams satisfying these relationships via constrained nonlinear optimization. The project's impacts are to significantly lower the barrier to creating effective diagrams, and to accelerate the rate at which complex technical ideas are communicated---especially in the domain of scientific education and research.

The project takes an interdisciplinary approach between programming languages and computer graphics. At the language level, Penrose enforces a clean separation between mathematical content and its visual representation via two extensible specification languages, Substance and Style, akin to HTML and CSS. On the graphics side, programs are compiled into a constrained optimization program whose solutions describe a family of possible diagrams. Automatic or user-assisted tools can then be used to select or interactively explore specific diagrams. The systematic encodings provided by Penrose can also be used to intelligently search for useful examples, special cases, or counterexamples. A free web-based Penrose interface encourages community development of new user-designed modules; integration of these modules into an interactive mathematical tutor system helps to enrich and accelerate the training of the next generation of scientists and engineers in the US.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Ma'ayan, Dor and Ni, Wode and Ye, Katherine and Kulkarni, Chinmay and Sunshine, Joshua "How Domain Experts Create Conceptual Diagrams and Implications for Tool Design" CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , 2020 10.1145/3313831.3376253 Citation Details
Ye, Katherine and Ni, Wode and Krieger, Max and Ma'ayan, Dor and Wise, Jenna and Aldrich, Jonathan and Sunshine, Joshua and Crane, Keenan "Penrose: from mathematical notation to beautiful diagrams" ACM Transactions on Graphics , v.39 , 2020 10.1145/3386569.3392375 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

The primary outcome of this award is a free and open-source mathematical diagramming tool, called Penrose. Penrose does not to develop “canned” visualization tools for specific types of logical relationships, but rather it is an extensible language-based framework that enables users to rapidly develop their own custom visualizations for structured information. Moreover, rather than generate single static images, Penrose produces rich families of diagrams that support interactive exploration, animation, and dynamic retargeting. Our focus is primarily on mathematical expressions, since this domain has a well-established syntax, and can have immediate impact on both education and scientific communication. 

Unlike tools for data visualization, the core value of Penrose is that it handles visualization of non-quantitative logical relationships. Complex logical relationships can be difficult to understand---and easy to misinterpret.  Penrose helps users explor a large space of possibilities, clarifying and enriching understanding.

Ultimately, diagrams can be incorporated into course notes, online discussions, slide presentations, etc., to improve and accelerate scientific communication and education. Just as raw HTML can be displayed in different ways and re-targeted for different platforms via swappable CSS files, Penrose content can easily be “re-styled” in many different ways.  Different styles enable easy delivery to alternate platforms (e.g., web vs. mobile), make content accessible to broader audiences (e.g., users with nonstandard color vision or who speak a different language)—or fundamentally change the visual representation.  Viewing the same idea from multiple perspectives gives students a deeper understanding, and helps ease the transition from one concept to another. Penrose is increasingly widely used by technical communicators, researchers, and mathematicians. Penrose has been integrated into other important research projects including the Lean theorem prover and the Alloy model finder.


Last Modified: 12/27/2023
Modified by: Joshua S Sunshine

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page