# Benchmarking suite 
AI4HPC comes with a benchmarking suite to test the limits of an HPC system with the help of [JUBE](https://apps.fz-juelich.de/jsc/jube/jube2/docu/)! Check [`./Bench`](https://gitlab.jsc.fz-juelich.de/CoE-RAISE/FZJ/ai4hpc/ai4hpc/-/blob/main/Bench/) folder for details!

### Setup
To start, initial step is to source the created environment by AI4HPC and setup the benchmarking suite.
```bash
$ source ../envAI_<sys_name>/bin/activate
$ python setup_bench.py
```
Here, location of the dataset and the account name of that particular HPC system will be prompt. Dataset location can be skipped, but, account name should be given.

After the installation, the main folder should consist of 
1. `ai4hpc.py` benchmarked case copied from parent folder 
2. `src` folder from parent folder
3. `datadir` folder 
4. `general_jobsys.xml` file, to be used for JUBE's basis configuration  

One can modify `general_jobsys.xml` for their needs. If satisfied, simply run the benchmark with
```bash
$ jube run general_jobsys.xml
```

One can also run benchmark on development partitions via
```bash
$ jube run general_jobsys.xml --tag devel
```

To check if benchmarking is finalised
```bash
$ jube continue bench_run --id last
```

Finally, the results can be shown with  
```bash
$ jube result -a bench_run --id last
```

This will create results file `result-csv.dat` in `results` folder. One can use `plotbench.ipynb` notebook to visualise the results (as done below in Results section)!

### Results
AI4HPC's benchmarking results on [JUWELS-BOOSTER](https://apps.fz-juelich.de/jsc/hps/juwels/booster-overview.html):

1. Speedup
![](../images/bench1.png)

2. Efficiency
![](../images/bench2.png)