# Instructions to reproduce the examples from the paper


## Installation

The following three subsections describe how to install the requirements for running Benchpress.


### MacOS/Linux/Windows (Docker):

1. Install Docker (<https://www.docker.com/>)

The snakemake commands below should be run inside the container.


### Linux (Non-Docker):

1. Install Snakemake (<https://snakemake.readthedocs.io/en/stable/getting_started/installation.html>)
2. Install Apptainer (<https://apptainer.org/docs/admin/main/installation.html>)


### Windows (WSL)

1. Install Windows Subsystems for Linux (WSL), see (<https://learn.microsoft.com/en-us/windows/wsl/install>)
2. In the WSL Linux installation, follow the instructions above for "Linux (Non-Docker)"


## Replicate studies

The following steps show how to replicate the studies from Sections 5.1-5.7 in the paper.

1. Navigate to the root directory of the Benchpress as:

   ```bash
   cd benchpress_2_13_0
   ```


### Docker user:

1. Run the following command to enter the container:

   ```bash
   $ docker run -it -w /mnt --privileged -v $(pwd):/mnt bpimages/snakemake:v9.7.1
   ```

   Depending on your system, you may need to run the command with sudo, i.e.

   ```bash
   $ sudo docker run -it -w /mnt --privileged -v $(pwd):/mnt bpimages/snakemake:v9.7.1
   ```

### Non-docker Linux or WSL user:

1. If Snakemake was installed in a conda environment called snakemake, you should activate it by running:

   ```bash
   $ conda activate snakemake
   ```

# Producing the examples in Section 5

The commands below should be run inside the folder `benchpress_2_13_0`.

---

**NOTE**

Sections 5.4-5.7 are large studies that takes about 1-2 hours each to run on 80 cores. Scaled-down versions of these studies can be obtained by changing the seed range in the corresponding configuration files.


**NOTE**

Note that the gobnilp module currently not included in the config files as it only runs on AMD64 architectures and not ARM64. If your machine has the AMD64 achitecture, please add back the id gobnilp to the relevant places in the JSON files.

---


## 5.1. Comparative study between the PC and the dual PC algorithm

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_pc_vs_dualpc.json
```

Copy the results to the figures folder

```bash
cp -r results/output/paper_pc_vs_dualpc figures/section_5_1
```

The figures in the paper are found in the following directories:

- Figure 2a: `figures/section_5_1/benchmarks/paper_pc_vs_dualpc/pc_vs_dualpc`
- Figure 2b: `figures/section_5_1/graph_true_stats`
- Figure 2c: `figures/section_5_1/benchmarks/paper_pc_vs_dualpc/f1_skel_joint`
- Figure 2d: `figures/section_5_1/benchmarks/paper_pc_vs_dualpc/elapsed_time_joint`


## 5.2. Biological dataset with fixed DAG

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_sachs.json
``` 

Copy the results to the figures folder

```bash
cp -r results/output/paper_sachs figures/section_5_2
```

- Figure 3a: `figures/section_5_2/ggally_ggpairs`
- Figure 4a: `figures/section_5_2/benchmarks/paper_sachs/SHD_cpdag_joint`
- Figure 5a: `figures/section_5_2/graph_plots/adjmats`
- Figure 5b: `figures/section_5_2/graph_plots/graphs` (find the same filename as the corresponding adjmat file)
- Figure 6ab: `figures/section_5_2/graph_plots/graphvizcompare`


## 5.3. Linear Gaussian SEM with random weights and random DAGs (small-scale study)

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_er_sem_small.json
``` 

Copy the results to the figures folder

```bash
cp -r results/output/paper_er_sem_small figures/section_5_3
```

The figures in the paper are found in the following directories:

- Figure 7a: `figures/section_5_3/benchmarks/paper_er_sem_small/SHD_cpdag_joint`
- Figure 7b: `figures/section_5_3/benchmarks/paper_er_sem_small/f1_skel_joint`


## 5.4. Binary-valued Bayesian network with random parameters and random DAG

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_er_bin.json
``` 

Copy the results to the figures folder

```bash
cp -r results/output/paper_er_bin figures/section_5_4
```

The figure in the paper is found in the following directory:

- Figure 8a: `figures/section_5_4/benchmarks/paper_er_bin/FPR_TPR_pattern`


## 5.5. Linear Gaussian SEM with random weights and random DAG

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_er_sem.json
``` 

Copy the results to the figures folder

```bash
cp -r results/output/paper_er_sem figures/section_5_5
```

The figure in the paper is found in the following directory:

- Figure 8b: `figures/section_5_5/benchmarks/paper_er_sem/FPR_TPR_pattern`


## 5.6. Bayesian network with random parameters and fixed DAG

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_hepar2_bin.json
``` 

Copy the results to the figures folder

```bash
cp -r results/output/paper_hepar2_bin figures/section_5_6
```

The figure in the paper is found in the following directory:

- Figure 8c: `figures/section_5_6/benchmarks/paper_hepar2_bin/FPR_TPR_pattern`


## 5.7. Linear Gaussian SEM with random weights and fixed DAG

```bash
snakemake --cores all --sdm apptainer --configfile config/paper_hepar2_sem.json
``` 

Copy the results to the figures folder

```bash
cp -r results/output/paper_hepar2_sem figures/section_5_7
```

The figure in the paper is found in the following directory:

- Figure 8d: `figures/section_5_7/benchmarks/paper_hepar2_sem/FPR_TPR_pattern`
