VizAly is a general framework for Analysis and Visualization of simulation data. As supercomputing resources increase, cosmological scientists are able to run more detailed and larger simulations generating massive amounts of data. Analyzing these simulations with an available open-source toolkit is important for collaborative Department of Energy scientific discovery across labs, universities, and other partners. Developed software as a part of this collection include: comparing data with other existing simulations, verifying and validating results with observation databases, new halo finder algorithms, and using analytical tools to get insights into the physics of the cosmological universe. The goal of this software project is to provide a set of open-source libraries, tools, and packages for large-scale cosmology that allows scientists to visualize, analyze, and compare large-scale simulation and observational data sets. Developed software will provide a variety of methods for processing, visualization, and analysis of astronomical observation and cosmological simulation data. These tools are intended for deployment on multiple scientific computing platforms, including but not limited to personal computers, cloud computing, experimental sites (telescopes) and high-performance supercomputers.
Foresight has three components:
- CBench: the compression benchmark arm of Foresight desighned to run at scale on supercomputers
- PAT: Python Analysis Toolkit, which contains a bunch of untilities to speed up analysis and plotting of results
- Cinema: The tool used to visualize the results of this project
- Link to the live cinema database with results : https://lanl.github.io/VizAly-Foresight/
- Cinema comparison is at: https://lanl.github.io/VizAly-Foresight/cinema_compare/
The default master branch should always point to the latest working version. However, for more stable releases, you should checkout the latest tag release.
CBench
- CMake 3.8.1 or higher
- GCC 6.4 or higher (C++14 minimum)
- OpenMPI 2 or higher
PAT
- Python 3.6 or higher with (+matplotlib=3.0.2, +apsw=3.9.2, +numpy=1.15.4)
- SLURM (for job launching)
Cinema
- Mozilla Firefox Recommended
The folder scripts contains scripts to load modules on Cooley @ ANL, Cori @ NERSC, and Darwin @ LANL . These build a base version of Foresight:
git clone https://github.com/lanl/VizAly-Foresight.git
cd VizAly-Foresight
source scripts/<Name of the environment> # that sets up the environment
source buildDependencies.sh
source build.sh
This will set up and run the full analysis workflow on SLURM and generate a cinema database, the command is as follows:
cd Analysis
python3 -m <name_of_analysis> --input-file <path to input JSON file>
For example, to run the NYX analysis, the command is:
python3 -m pat.nyx.workflow --input-files inputs/nyx/nyx_darwin_test.json
The outputs/logs will be generated in a folder using "project-home" and "wflow-path" in the input JSON file. The folder cinemaDB.cdb will contain the graphed outputs.
mpirun -np 2 build/CBench ../inputs/hacc/hacc_cbench_test.json
cat metrics_hacc_test_.csv
Alternatively, CBench can also be run throughj foresight as follows:
cd Analysis
python3 -m <name_of_analysis> --input-file <path to input JSON file> --cbench
Note: The above will only run a toy dataset meant for testing if the code runs. The results should NOT be used as an indicator for the performance of the compressors!
The folder in cinemaDB.cdb will also contain a wflow.json that will point to a git-hash tag that the code was ran on. To replicate the run, git checkout to get the exact same code.
Foresight takes as input a json file (examples of input json files are in the inputs folder) that specifies the input parameters.
- GenericIO
- NYX(HDF5 version hdf5-1_10_3)
- Binary
- Lossless BLOSC - version 1.10.2
- SZ - version 2.1.4.2
- zfp - version 0.5.5
- fpzip - version 1.2.0
- ISABELA - version 0.2.1
The following experimental GPU compressors are also supported:
- SZ - custom version
- zfp - version 0.5.4
- Absolute Error
- Relative Error
- Mean Square Error (MSE)
- Peak Signal-to-Noise Ratio (PSNR)
- Data Ranges (Data distribution)
- Memory Usage
- Compute times
Below instructions illustrate how to run Foresight toolkit on the HACC and Nyx dataset using the PantaRhei cluster at the University of Alabama in order to reproduce the experimental reults shown in the paper "Understanding GPU-Based Lossy Compression forExtreme-Scale Cosmological Simulations" submitted to IPDPS'20.
- Download HACC dataset at http://dx.doi.org/10.21227/zg3m-8j73.
- Download Nyx dataset at http://dx.doi.org/10.21227/k8gb-vq78.
- The JSON files used in the experiment for Nyx and HACC are included in inputs/hacc and inputs/hacc, separately.
- Based on the locations of your downloaded datasets, please modify the JSON files accordingly.
- When build Foresight, please use the command:
source build.sh -all
- Follow the instructions shown in Analysis/README.md to use the JSON files. The corresponding JSON files are pantarhei_hacc_sz.json and pantarhei_nyx_sz.json.
- Please run on a node equipped with more than 20 CPU cores.
- If logarithmic transformation is desired, please uncomment the related code in CBench/main.cpp from line 281-298 and 328-341.
- When build Foresight, please use the command:
source build.sh -gpu
- Follow the instructions shown in Analysis/README.md to use the JSON files. The corresponding JSON files are pantarhei_hacc_zfp.json and pantarhei_nyx_zfp.json.
- Please run the code on a node equipped with GPU.
For information on how to add new compressors and/or metrics, look at the readme in CBench/compressors and CBench/metrics respectively. To add new analysis routines, look at the readme in Analysis/ folder.
For information on Travis CI and Docker image, look at the travis folder
- Chris Biwer
- Pascal Grosset
- Sian Jin
- Jesus Pulido
- Hoby Rakotoarivelo
This software is open source software available under the BSD-3 license.
Copyright (c) 2017, Triad National Security, LLC. All rights reserved.
This program was produced under U.S. Government contract 89233218CNA000001 for Los Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC for the U.S. Department of Energy/National Nuclear Security Administration. The U.S. Government has rights to use, reproduce, and distribute this software. NEITHER THE GOVERNMENT NOR LOS ALAMOS NATIONAL SECURITY, LLC MAKES ANY WARRANTY, EXPRESS OR IMPLIED, OR ASSUMES ANY LIABILITY FOR THE USE OF THIS SOFTWARE. If software is modified to produce derivative works, such modified software should be clearly marked, so as not to confuse it with the version available from LANL.
All rights in the program are reserved by Triad National Security, LLC, and the U.S. Department of Energy/National Nuclear Security Administration. The Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in this material to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.