COL729 Lab 0 : Benchmarking

Due date - 23:55, 14 January 2019

In this assignment, we will benchmark a few common C/C++ applications and examine the effect of different compilers and different compiler optimization levels on program performance.

Instructions

Install an Ubuntu 18.04 system
Install GCC and Clang
```
 sudo apt-get install gcc clang 
```
Install ICC by obtaining the academic license: Follow these instructions.
Install SPEC CPU 2017 benchmark suite (please note that this is an IITD-internal link only; this benchmark suite is proprietary so please do not share it outside IIT Delhi)
- Download http://poorvi.cse.iitd.ac.in/~sbansal/isos/cpu2017-1_0_5.iso
- Follow instructions at https://www.spec.org/cpu2017/Docs/install-guide-unix.html to install and run the CPU 2017 benchmark suite
- Optional: you can use the commands in this file to run any benchmark individually: cpu2017_cmdlines.sh.
Learn about benchmarking: look at the video on "Benchmarking C++ Code" by Bryce Adelstein-Lelbach posted on the references page.
Go through the SPEC-CPU overview
Build SPEC CPU 2017 SPEC-RATE benchmarks for Integers with GCC, Clang, ICC
- Build with the unoptimised (may not be default) versions of GCC/Clang/ICC (on Ubuntu 18.04) for 32-bit and 64-bit x86 architectures.
- Build with different optimization options: -O1, -O2, -O3, etc.
- Plot your runtime results, make conclusions
  - Plot Scores or Runtimes vs Optimisation levels
  - Make separate plots for each benchmark
  - A total of 10 graphs are expected, one per benchmark, containing plots for each compiler (GCC/ICC/Clang 32 and 64-bits)

Prepare a report summarizing your findings: Your report should be succinct yet insightful. It should talk about the most consequential optimizations (to runtime) and the least consequential optimizations.

Which compilers are better in what aspects?
Which are faster: 32-bit or 64-bit executables? Why?
How are the various optimisation levels different? How do these differ accross compilers?
Does the kind of benchmark influence the results between compilers? Why?

Submissions:

anz188387.pdf cs1150460.pdf cs1150600.pdf cs5150287.pdf csz188011.pdf mcs182840.pdf siy187504.pdf