Award Abstract # 2236233
CAREER: Interactive Program Synthesis for Web Automation

NSF Org: CCF
Division of Computing and Communication Foundations
Recipient: REGENTS OF THE UNIVERSITY OF MICHIGAN
Initial Amendment Date: January 25, 2023
Latest Amendment Date: April 7, 2025
Award Number: 2236233
Award Instrument: Continuing Grant
Program Manager: Anindya Banerjee
abanerje@nsf.gov
 (703)292-7885
CCF
 Division of Computing and Communication Foundations
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: March 1, 2023
End Date: February 29, 2028 (Estimated)
Total Intended Award Amount: $524,926.00
Total Awarded Amount to Date: $308,808.00
Funds Obligated to Date: FY 2023 = $108,468.00
FY 2024 = $107,391.00

FY 2025 = $92,949.00
History of Investigator:
  • Xinyu Wang (Principal Investigator)
    xwangsd@umich.edu
Recipient Sponsored Research Office: Regents of the University of Michigan - Ann Arbor
1109 GEDDES AVE STE 3300
ANN ARBOR
MI  US  48109-1015
(734)763-6438
Sponsor Congressional District: 06
Primary Place of Performance: Regents of the University of Michigan - Ann Arbor
2260 Hayward
ANN ARBOR
MI  US  48109-2121
Primary Place of Performance
Congressional District:
06
Unique Entity Identifier (UEI): GNJ7BBP73WE9
Parent UEI:
NSF Program(s): Software & Hardware Foundation
Primary Program Source: 01002425DB NSF RESEARCH & RELATED ACTIVIT
01002526DB NSF RESEARCH & RELATED ACTIVIT

01002324DB NSF RESEARCH & RELATED ACTIVIT

01002728DB NSF RESEARCH & RELATED ACTIVIT

01002627DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 1045, 8206
Program Element Code(s): 779800
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Process Automation continues to be a main driver of digital transformation. Yet, it is still technically very demanding to create such automation programs. This project aims to significantly lower the technical barrier of creating web automation programs. The project?s novelties are a suite of new rewrite-based program synthesis algorithms that can automatically generate programs from user demonstrations. The project?s impacts are to enable non-experts in need of performing a tedious but programmatic web-related task to create a program to automate the work, even if they have little or no background in web programming.

At the core, this project is developing algorithms that take as input a user demonstration ? in the form of a trace A of user actions (e.g., clicking buttons, scraping data) ? and synthesize a parameterized program P with control-flow structures by rewriting A to P. The approach is based on finite tree automata and involves both neural and symbolic elements in the underlying web automation language, with the goal of being able to effectively reason about the webpage contents while still leveraging the underlying webpage structure. The project is curating a new suite of web automation tasks that are independently useful for future research beyond this project. The investigator is collaborating with partners at Michigan (e.g., Women in Science and Engineering (WISE), Michigan Louis Stokes Alliance for Minority Participation (MI-LSAMP) and Engineering Center for Academic Success (ECAS)) to increase participation of students from underrepresented groups in his research.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Chen, Weihao and Liu, Xiaoyu and Zhang, Jiacheng and Lam, Ian Iong and Huang, Zhicheng and Dong, Rui and Wang, Xinyu and Zhang, Tianyi "MIWA: Mixed-Initiative Web Automation for Better User Control and Confidence" , 2023 https://doi.org/10.1145/3586183.3606720 Citation Details
Li, Xiang and Zhou, Xiangyu and Dong, Rui and Zhang, Yihong and Wang, Xinyu "Efficient Bottom-Up Synthesis for Programs with Local Variables" Proceedings of the ACM on Programming Languages , v.8 , 2024 https://doi.org/10.1145/3632894 Citation Details
Pu, Kevin and Yang, Jim and Yuan, Angel and Ma, Minyi and Dong, Rui and Wang, Xinyu and Chen, Yan and Grossman, Tovi "DiLogics: Creating Web Automation Programs with Diverse Logics" , 2023 https://doi.org/10.1145/3586183.3606822 Citation Details

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page