The SPEC High-Performance Group (HPG) develops, publishes, and curates a number of benchmarks of relevance to the high-performance computing community. ORNL is an Associate Member of SPEC HPG, and a number of ORNL staff are active participants.

Getting and Using the Benchmarks

Most SPEC benchmarks are proprietary, licensed software. As an HPG member, we have a license for all benchmarks published by HPG (but not other groups within SPEC):

ACCEL
HPC
MPI
OMP

Getting an Account

To get the benchmarks, you will first need to obtain an account to access the members-only portion of the SPEC site. Contact the ORNL Primary Representative (see below) to obtain an account.

Note: in 2018, SPEC HPG started making its benchmarks available at no charge to qualified non-profit organizations who are not already SPEC HPG members. Because we are members, SPEC will redirect all requests from ORNL staff through this mechanism to the ORNL Primary Representative.

Finding the Benchmarks

The easy place to get the benchmarks is at https://pro.spec.org/private/hpg/benchmarks/. This location has the ISO images of the most recent couple of releases of each benchmark.

If you want other formats or versions, you will need to go to the HPG private wiki https://pro.spec.org/private/wiki/HPG/WebHome. In the left-hand navigation bar, below the committee information, you will see headings for the different benchmarks. The Kits (pro) links will take you to the repositories for each benchmark. These include tarballs of the released versions (in addition to the ISO images), and the "kits" which are periodic intermediate releases representing progress in development.

Rules for Using the Benchmarks

The actual benchmark scores can only be used in public settings if they have been submitted to SPEC HPG and passed its peer review process.

Timings obtained from running the benchmarks (these are not the scores) can be used in an academic context, as long as it is clear that they are not official SPEC results.

For further information, see http://spec.org/fairuse.html and/or talk to one of the contacts listed below.

Contributing Results

If you have access to a platform which does not already have official benchmark results, consider running them and submitting the results for review and publication. See http://spec.org/hpg/submitting_results.html, and contact the people below for more information.

Providing Feedback on the Benchmarks

We welcome feedback to help improve SPEC HPG's offerings. If you have feedback on the benchmarks, gaps in them, etc. please contact any of the people below.

Getting Involved in Developing Benchmarks

We welcome anyone who wants to get more deeply involved in SPEC HPG. We're also often looking for kernels that represent workloads important to ORNL for possible inclusion in the benchmarks. Contact any of the people below.

Key Contacts

ORNL Primary Representative: David Bernholdt
Lead technical contact: Oscar Hernandez

Browser not supported