Counting 3,567 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Last Commit
Feb. 20, 2019
Jan. 17, 2019


Read the documentation

Read the paper preprint


Table of contents

  1. Repository overview
  2. Getting Started
    1. With Docker
    2. With Docker
  3. Usage
  4. Reproduce paper
  5. Pipeline Description
  6. Running tests
  7. Build the docs

Repository overview

├── supernnova              --> main module
    ├──data                 --> scripts to create the processed database
    ├──visualization        --> data plotting scripts
    ├──training             --> training scripts
    ├──validation           --> validation scripts
    ├──utils                --> utilities used throughout the module
├── tests                   --> unit tests to check data processing
├── sandbox                 --> WIP scripts

Getting started

With Docker

cd env

# Build docker images
make cpu  # cpu image
make gpu  # gpu image (requires NVIDIA Drivers + nvidia-docker)

# Launch docker container
python (--use_gpu to run GPU based container)

With Conda

cd env

# Create conda environment
conda create --name <env> --file <conda_file_of_your_choice>

# Activate conda environment
source activate <env>

For more detailed instructions, check the full setup instructions


# Create data
python --data  --dump_dir tests/dump

# Train a baseline RNN
python --train_rnn --dump_dir tests/dump

# Train a variational dropout RNN
python --train_rnn --model variational --dump_dir tests/dump

# Train a Bayes By Backprop RNN
python --train_rnn --model bayesian --dump_dir tests/dump

# Train a RandomForest
python --train_rf --dump_dir tests/dump

Reproduce (soon to be published) results


General pipeline description

  • Parse raw data in FITS format
  • Create processed database in HDF5 format
  • Train Recurrent Neural Networks (RNN) or Random Forests (RF) to classify photometric lightcurves
  • Validate on test set

Running tests with py.test

PYTHONPATH=$PWD:$PYTHONPATH pytest -W ignore --cov supernnova tests

Build docs

cd docs && make clean && make html && cd ..
firefox docs/_build/html/index.html