Instructions to reproduce the examples from the paper

Installation

The following three subsections describe how to install the requirements for running Benchpress.

MacOS/Linux/Windows (Docker):

1. Install Docker (https://www.docker.com/)

The snakemake commands below should be run inside the container.

Linux (Non-Docker):

1. Install Snakemake (https://snakemake.readthedocs.io/en/stable/getting_started/installation.html)
2. Install Apptainer (https://apptainer.org/docs/admin/main/installation.html)

Windows (WSL)

1. Install Windows Subsystems for Linux (WSL), see (https://learn.microsoft.com/en-us/windows/wsl/install)
2. In the WSL Linux installation, follow the instructions above for “Linux (Non-Docker)”

Replicate studies

The following steps show how to replicate the studies from Sections 5.1-5.7 in the paper.

1. Navigate to the root directory of the Benchpress as:

   cd benchpress_2_13_0

Docker user:

1. Run the following command to enter the container:

   $ docker run -it -w /mnt --privileged -v $(pwd):/mnt bpimages/snakemake:v9.7.1

   Depending on your system, you may need to run the command with sudo, i.e.

   $ sudo docker run -it -w /mnt --privileged -v $(pwd):/mnt bpimages/snakemake:v9.7.1

Non-docker Linux or WSL user:

1. If Snakemake was installed in a conda environment called snakemake, you should activate it by running:

   $ conda activate snakemake

Producing the examples in Section 5

The commands below should be run inside the folder benchpress_2_13_0.

NOTE

Sections 5.4-5.7 are large studies that takes about 1-2 hours each to run on 80 cores. Scaled-down versions of these studies can be obtained by changing the seed range in the corresponding configuration files.

NOTE

Note that the gobnilp module currently not included in the config files as it only runs on AMD64 architectures and not ARM64. If your machine has the AMD64 achitecture, please add back the id gobnilp to the relevant places in the JSON files.

5.1. Comparative study between the PC and the dual PC algorithm

snakemake --cores all --sdm apptainer --configfile config/paper_pc_vs_dualpc.json

Copy the results to the figures folder

cp -r results/output/paper_pc_vs_dualpc figures/section_5_1

The figures in the paper are found in the following directories:

• Figure 2a: figures/section_5_1/benchmarks/paper_pc_vs_dualpc/pc_vs_dualpc
• Figure 2b: figures/section_5_1/graph_true_stats
• Figure 2c: figures/section_5_1/benchmarks/paper_pc_vs_dualpc/f1_skel_joint
• Figure 2d: figures/section_5_1/benchmarks/paper_pc_vs_dualpc/elapsed_time_joint

5.2. Biological dataset with fixed DAG

snakemake --cores all --sdm apptainer --configfile config/paper_sachs.json

Copy the results to the figures folder

cp -r results/output/paper_sachs figures/section_5_2

• Figure 3a: figures/section_5_2/ggally_ggpairs
• Figure 4a: figures/section_5_2/benchmarks/paper_sachs/SHD_cpdag_joint
• Figure 5a: figures/section_5_2/graph_plots/adjmats
• Figure 5b: figures/section_5_2/graph_plots/graphs (find the same filename as the corresponding adjmat file)
• Figure 6ab: figures/section_5_2/graph_plots/graphvizcompare

5.3. Linear Gaussian SEM with random weights and random DAGs (small-scale study)

snakemake --cores all --sdm apptainer --configfile config/paper_er_sem_small.json

Copy the results to the figures folder

cp -r results/output/paper_er_sem_small figures/section_5_3

The figures in the paper are found in the following directories:

• Figure 7a: figures/section_5_3/benchmarks/paper_er_sem_small/SHD_cpdag_joint
• Figure 7b: figures/section_5_3/benchmarks/paper_er_sem_small/f1_skel_joint

5.4. Binary-valued Bayesian network with random parameters and random DAG

snakemake --cores all --sdm apptainer --configfile config/paper_er_bin.json

Copy the results to the figures folder

cp -r results/output/paper_er_bin figures/section_5_4

The figure in the paper is found in the following directory:

• Figure 8a: figures/section_5_4/benchmarks/paper_er_bin/FPR_TPR_pattern

5.5. Linear Gaussian SEM with random weights and random DAG

snakemake --cores all --sdm apptainer --configfile config/paper_er_sem.json

Copy the results to the figures folder

cp -r results/output/paper_er_sem figures/section_5_5

The figure in the paper is found in the following directory:

• Figure 8b: figures/section_5_5/benchmarks/paper_er_sem/FPR_TPR_pattern

5.6. Bayesian network with random parameters and fixed DAG

snakemake --cores all --sdm apptainer --configfile config/paper_hepar2_bin.json

Copy the results to the figures folder

cp -r results/output/paper_hepar2_bin figures/section_5_6

The figure in the paper is found in the following directory:

• Figure 8c: figures/section_5_6/benchmarks/paper_hepar2_bin/FPR_TPR_pattern

5.7. Linear Gaussian SEM with random weights and fixed DAG

snakemake --cores all --sdm apptainer --configfile config/paper_hepar2_sem.json

Copy the results to the figures folder

cp -r results/output/paper_hepar2_sem figures/section_5_7

The figure in the paper is found in the following directory:

• Figure 8d: figures/section_5_7/benchmarks/paper_hepar2_sem/FPR_TPR_pattern