Counting 3,742 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1


This repo contains the implementation of Multi-gate Mixture-of-Experts model in Keras.

Here's the video explanation of the paper by the authors.

The repository includes:

  • A Python 3.6 implementation of the model in Keras with TensorFlow backend
    • The code is also compatible with Python 2.7
  • Example demo of running the model with the census-income dataset from UCI
    • This dataset is the same one in Section 6.3 of the paper

The code is documented and designed to be extended relatively easy. If you plan on using this in your work, please consider citing this repository (BibTeX is included below) and also the paper.

Getting Started


  • Python 3.6
  • Other libraries such as TensorFlow and Keras listed in requirements.txt


  1. Clone the repository
  2. Install dependencies
pip install -r requirements.txt
  1. Run the example code


  • Due to ambiguity in the paper and time and resource constraints, we unfortunately can't reproduce the exact results in the paper


Contributions to this repository are welcome. Examples of things you can contribute:

  • Performance improvements by re-writing the model in TensorFlow, PyTorch, or MXNet
  • Improve the census income benchmark to be as close as the one from the paper
  • Improve the synthetic benchmark to be as close as the one from the paper
  • Training on other public datasets for different benchmarks
  • Accuracy improvements
  • Visualizations


Use this BibTeX to cite the repository:

  title={Multi-gate Mixture-of-Experts model in Keras and TensorFlow},
  author={Deng, Alvin},
  journal={GitHub repository},


The code is built upon the work by Emin Orhan.