Award Abstract # 1922713
Prediction of Solar Eruptions with Machine-Learning Algorithms Combining Physical Models and Observations

NSF Org: AGS
Division of Atmospheric and Geospace Sciences
Recipient: THE LELAND STANFORD JUNIOR UNIVERSITY
Initial Amendment Date: June 24, 2019
Latest Amendment Date: November 8, 2022
Award Number: 1922713
Award Instrument: Standard Grant
Program Manager: Lisa Winter
AGS
 Division of Atmospheric and Geospace Sciences
GEO
 Directorate for Geosciences
Start Date: July 1, 2019
End Date: June 30, 2024 (Estimated)
Total Intended Award Amount: $494,705.00
Total Awarded Amount to Date: $494,705.00
Funds Obligated to Date: FY 2019 = $494,705.00
History of Investigator:
  • Jon Hoeksema (Principal Investigator)
    todd@sun.stanford.edu
  • Philip Scherrer (Former Principal Investigator)
Recipient Sponsored Research Office: Stanford University
450 JANE STANFORD WAY
STANFORD
CA  US  94305-2004
(650)723-2300
Sponsor Congressional District: 16
Primary Place of Performance: Stanford University
452 Lomita Mall
Stanford
CA  US  94305-4008
Primary Place of Performance
Congressional District:
16
Unique Entity Identifier (UEI): HJD6G4D6TJY5
Parent UEI:
NSF Program(s): SOLAR-TERRESTRIAL
Primary Program Source: 01001920DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 0000, OTHR
Program Element Code(s): 152300
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.050

ABSTRACT

Space-weather prediction has been around for many decades but is reaching new heights due to the advent of more data, sophisticated data processing techniques, and computing power. Modern satellites and ground-based telescopes designed to study space weather take more data than ever before. The Solar Dynamics Observatory (SDO), for example, acquires ~1.5 terabytes of data a day. As such, the time may be right to give an affirmative answer to the question: Can we harness the data revolution to effectively predict the onset of a major solar flare? The main purpose of this three-year project is to answer this question by bringing together an interdisciplinary team of computer scientists and solar physicists to analyze data taken by the SDO, Global Oscillation Network Group (GONG), and Geostationary Operational Environmental Satellite (GOES) observatories, as well as data products derived from numerical models, using machine-learning algorithms to characterize and understand which signatures indicate the imminent eruption of a solar flare. Previous research studies have attempted to predict solar flares using a subset of these components, but not all of them. Few space-weather studies have harnessed the data revolution to predict space-weather using machine learning algorithms despite the vast amount of data available.

This three-year project addresses open questions in solar flare physics. First, the project aims to determine which features contribute to local and global pre-flare signatures. While interaction between active regions can trigger flaring behavior, hyper-local phenomena can also trigger flaring behavior. Second, the project team will explore how the rate of change of any given feature influences eruptive activity on the Sun. Finally, the project aims to identify the timescales on which each individual feature best predicts future flaring activity. To accomplish these goals, the project team will use the relevant features identified in these three tasks, along with interpretable machine learning algorithms, such as state-space models, to predict solar activity.


Space weather prediction is a national priority, listed as a key goal of the most recent decadal survey and a prime focus of the National Space-Weather Action Plan. This three-year project aims to build an open-source, well-documented, unified, reproducible, and operational feature dataset and machine-learning model. The project team will use this dataset and machine-learning model to teach students both through summer internships and via their book entitled Statistics, Data Mining, and Machine Learning in Heliophysics. The research and EPO agenda of this project supports the Strategic Goals of the AGS Division in discovery, learning, diversity, and interdisciplinary research.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Bobra, Monica G. and Wright, Paul J. and Sun, Xudong and Turmon, Michael J. "SMARPs and SHARPs: Two Solar Cycles of Active Region Data" The Astrophysical Journal Supplement Series , v.256 , 2021 https://doi.org/10.3847/1538-4365/ac1f1d Citation Details
Pauker, Lucas A. and Bobra, Monica G. and Jonas, Eric "Predicting Solar Flares Using Time Series Analysis" Research Notes of the AAS , v.3 , 2019 10.3847/2515-5172/ab4db0 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

We investigated the physical conditions present in solar active regions that lead to solar eruptions. Machine Learning methods require large databases. Quantitative data about active region development were already available from 2010 onward based on observations from the Helioseismic and Magnetic Imager (HMI) in the Space-weather HMI Active Region Patches (SHARPs) data series. We extended that record back in time to 1996 using measurements from its predecessor instrument, the Michelson Doppler Imager (MDI). We published the Space-weather MDI Active Region Patch (SMARP) catalog, making it available to the community, and we used it to investigate sympathetic flares (flares that occur in association with others).

The earlier MDI instrument did not provide measurements of the vector magnetic field, which is available for the SHARPs after 2010. Quantities based on the vector magnetic field are highly useful for predicting solar eruptions. To mitigate this limitation, we then trained a ML model to first synthesize coronal extreme ultraviolet (EUV) images and photospheric magnetograms by rendering results of a non-linear force-free field (NLFFF) model computed using the vector magnetic field measurements. That allowed us to train another ML model to solve the difficult inverse problem of deducing the vector coronal magnetic field from the synthesized extreme ultraviolet (EUV) images. We then began the more challenging step of using actual EUV images from the Atmospheric Imaging Assembly (AIA) along with HMI magnetograms to derive information about the vector magnetic field in the corona. The goal was to learn how to recover additional information during the MDI era from the available line-of-sight magnetograms and EUV images. While significant progress was made, COVID-related delays and personnel changes prevented us from completing this task.

The third outcome was the development of a greatly enhanced dataset (SHARP-E) including all significant active regions observed since May 2010. This comprehensive database complements the SHARPs by adding many additional physical parameters computed using the electric field determined from time series of HMI magnetic field and velocity observations. The electric field was derived as part of the Coronal Global Evolutionary Model (CGEM) project by considering temporal and spatial variations of both the vector magnetic and velocity fields measured in the photosphere. More than 232 quantities are computed at each 12-minute time step for each region. We then completed a preliminary analysis of the usefulness of the quantities for flare forecasting, comparing them with results based on the magnetic-only parameters typically used for prediction. Ranking their effectiveness based on the True Skill Score (considering only one quantity at a time), we found that eight of the top ten ranked parameters were from the newly derived parameters (see figures for a list of the top ten and their relative TSSs). The computed parameters have recently been made publicly available at the Stanford Data Repository (https://purl.stanford.edu/jz065wt1927) and will be presented at meetings and in publications in the near future.

The SMARP and SHARP-E data sets greatly increase the time range and capability of active region catalogs useful for machine learning applications for prediction of solar eruptions. Interpretation of these data opens the door to improved understanding of the conditions leading up to important space weather events.


Last Modified: 10/26/2024
Modified by: Jon T Hoeksema

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page