High performance computing (HPC) systems, more commonly known as supercomputers, play a pivotal rule in society today, including the U.S. economy.  They are essential tools in a diverse range of areas including finance, oil and gas exploration, pharmaceutical drug design, medical and basic research, computer animation, aeronautics, and automotive design to name a few.  Today’s supercomputers are a complex combination of computer hardware (servers, network switches, storage) and software, and it is important that system support personnel have at their disposal tools to ensure that this complex infrastructure is running with optimal efficiency as well as the ability to proactively identify underperforming hardware and software.  In addition, most HPC systems are overloaded, with many jobs queued waiting to run, and accordingly system support personnel desire the capability to monitor and analyze all end-user jobs to determine how efficiently they are running and what resources they are consuming (computer memory, processing, storage, networking, etc.) in order to optimize the number of computations run as well as plan for future needs. 

Given the important role that high performance computers play in research and the economy, it is somewhat surprising that, prior to this project, no open source tools were available that provided for the comprehensive management of HPC systems. With this deficiency in mind, XDMoD was developed to provide a comprehensive management framework for the NSF’s high performance computing systems that are managed through the XSEDE program.  In addition, the closely related Open XDMoD, an open source tool, provides similar functionality for HPC systems in general including government, industrial and academic HPC centers as well as Blue Waters – the largest supercomputer in the NSF portfolio.  XDMoD for XSEDE and Open XDMoD were designed to meet the following objectives:

(1)    provide the end-user community with a tool to optimize their use of HPC resources,

(2)    provide operational staff with the ability to monitor, diagnose, and tune system performance as well as measure the performance of all applications running on the HPC systems they manage,

(3)    provide software developers with the ability to easily obtain detailed analysis of application performance to aid in optimizing code performance,

(4)    provide stakeholders with a diagnostic tool to facilitate HPC planning and analysis, and

(5)    provide metrics to help measure return on investment.

XDMoD  provides a rich set of features accessible through an intuitive graphical interface, which is tailored to the role of the user, from scientists and engineers running computations to HPC facility and funding agency managers. Metrics provided by XDMoD include comprehensive statistics on: number and type of computational jobs run, resources (computation, memory, disk, network, etc.) consumed, job wait and wall time, scientific impact, and quality of service.  The web interface is intuitive, allowing one to chart various metrics and interactively drill down to access additional related information.

The XDMoD framework is also designed to help ensure that the HPC infrastructure is delivering a high quality of service to its end-users by continuously monitoring system performance and reliability through the deployment of a series of programs specifically designed to monitor overall system performance. System managers are therefore able to proactively monitor the HPC infrastructure as opposed to having to rely on users to report failures or underperforming hardware and software.

An important capability of XDMoD is centered around monitoring the performance of all user jobs running on a given HPC resour...

Please report errors in award information by writing to: awardsearch@nsf.gov.