Counting 2,870 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Contributors
Last Commit
Jun. 16, 2018
Created
Feb. 9, 2015

Gosl – Go scientific library

GoDoc Build Status Go Report Card Awesome

Gosl is a Go library to develop Artificial Intelligence and High-Performance Scientific Computations.

The library tries to be as general and easy as possible. Gosl considers the use of both Go concurrency routines and parallel computing using the message passing interface (MPI). Gosl has several modules (sub-packages) for a variety of tasks in scientific computing, image analysis, and data post-processing.

Gosl includes high-performant linear algebra functions (wrapping MKL, OpenBLAS, LAPACK, SuiteSparse, UMFPACK...), fast Fourier transform algorithms (wrapping FFTW), numerical integration (wrapping QUADPACK), functions and structures for geometry calculations (e.g. 3D transfinite interpolation, grid search, octree...), random numbers generation (SFMT and DSFMT) and probability distributions, optimisation and graph algorithms, plotting and visualisation using the VTK, and much more.

Gosl has also solvers to (stiff or not) ordinary differential equations and several tools for 2D/3D mesh generation to assist on the development of solvers for partial differential equations.

We now give focus to Machine Learning (see ml package) and Big Data (see h5 package). We are also planning wrappers to powerful tools such as CNTK, TensorFlow, and Hadoop. We have a wrapper to OpenCV in the works as well.

Resources

  1. Examples and benchmarks
  2. White papers
  3. Documentation
  4. Contributing and TODO

Installation

Gosl works on Windows, macOS, and Linux.

Click on Installation on Windows Installation on macOS Installation on Linux/Debian/Ubuntu for installation instructions.

Sub-packages

Gosl includes the following sub-packages:

  1. chk – Check code and unit test tools
  2. io – Input/output, read/write files, and print commands
  3. io/h5 – Read/write HDF5 (Big Data) files
  4. utl – Utilities. Lists. Dictionaries. Simple Numerics
  5. utl/al – Utilities. (Naive) Implementation of Classic algorithms and structures (e.g. Linked Lists)
  6. plt – Plotting and drawing (png and eps)
  7. mpi – Message Passing Interface for parallel computing
  8. la – Linear Algebra: vector, matrix, efficient sparse solvers, eigenvalues, decompositions, etc.
  9. la/mkl – Lower level linear algebra using Intel MKL
  10. la/oblas – Lower level linear algebra using OpenBLAS
  11. num/qpck – Go wrapper to QUADPACK for numerical integration
  12. num – Fundamental numerical methods such as root solvers, non-linear solvers, numerical derivatives and quadrature
  13. fun – Special functions, DFT, FFT, Bessel, elliptical integrals, orthogonal polynomials, interpolators
  14. fun/dbf – Database of functions of a scalar and a vector like f(t,{x}) (e.g. time-space)
  15. fun/fftw – Go wrapper to FFTW for fast Fourier Transforms
  16. gm – Geometry algorithms and structures
  17. gm/msh – Mesh structures and interpolation functions for FEA, including quadrature over polyhedra
  18. gm/tri – Mesh generation: triangles and Delaunay triangulation (wrapping Triangle)
  19. gm/rw – Mesh generation: read/write routines
  20. graph – Graph theory structures and algorithms
  21. opt – Numerical optimization: Interior Point, Conjugate Gradients, Powell, Grad Descent, more
  22. rnd – Random numbers and probability distributions
  23. rnd/dsfmt – Go wrapper to dSIMD-oriented Fast Mersenne Twister
  24. rnd/sfmt – Go wrapper to SIMD-oriented Fast Mersenne Twister
  25. vtk – 3D Visualisation with the VTK tool kit
  26. ode – Solvers for ordinary differential equations
  27. ml – Machine learning algorithms
  28. ml/imgd – Machine learning. Auxiliary functions for handling images
  29. pde – Solvers for partial differential equations (FDM, Spectral, FEM)
  30. tsr – Tensors, continuum mechanics, and tensor algebra (e.g. eigendyads)

We are currently working on the following additional packages:

  1. img - Image and machine learning algorithms for images
  2. img/ocv - Wrapper to OpenCV

About the filenames

  1. t_something_test.go is a unit test. We have several of them! Some usage information can be learned from these files.
  2. t_something_main.go is a test with a main function to be run with go run ... or mpirun -np ? go run ... (replace ? with the number of cpus).
  3. t_b_something_test.go is a benchmark test. Run benchmarks with go test -run=XXX -bench=.

Design strategies

Here, we call structure any user-defined type. These are simply Go types defined as struct. One may think of these structures as classes. Gosl has several global functions as well and tries to avoid complicated constructions.

An allocated structure (instance) is called an object and functions attached to this object are called methods. In Gosl, the variable holding the pointer to an object is always named o (lower case "o"). This variable is similar to the self or this keywords in other languages (Python, C++, respectively).

Functions that allocate a pointer to a structure are prefixed with New; for instance: NewIsoSurf. Some structures require an explicit call to another function to release allocated memory. Be aware of this requirement! In this case, the function is named Free and appears in a few sub-packages that use CGO. Also, some objects may need to be initialized before use. In this case, functions named Init have to be called.

The directories corresponding to each package have a README.md file that should help with understanding the library. Also, there are links to godoc.org where all functions, structures, and variables are well explained.

Test coverage

We aim for a 100% test coverage! Despite trying our best to accomplish this goal, full coverage is difficult, in particular with (sub)packages that involve Panic or figure generation. Nonetheless, critical algorithms are completely tested.

We use the following bash macro frequently to check our test coverage:

gocov() {
    go test -coverprofile=/tmp/cv.out
    go tool cover -html=/tmp/cv.out
}

Some results from cover.run

Bibliography

The following works take advantage of Gosl:

  1. Pedroso DM, Bonyadi MR, Gallagher M (2017) Parallel evolutionary algorithm for single and multi-objective optimisation: differential evolution and constraints handling, Applied Soft Computing http://dx.doi.org/10.1016/j.asoc.2017.09.006 paper available here
  2. Pedroso DM (2017) FORM reliability analysis using a parallel evolutionary algorithm, Structural Safety 65:84-99 http://dx.doi.org/10.1016/j.strusafe.2017.01.001
  3. Pedroso DM, Zhang YP, Ehlers W (2017) Solution of liquid-gas-solid coupled equations for porous media considering dynamics and hysteretic retention behaviour, Journal of Engineering Mechanics 04017021 http://dx.doi.org/10.1061/(ASCE)EM.1943-7889.0001208
  4. Pedroso DM (2015) A solution to transient seepage in unsaturated porous media. Computer Methods in Applied Mechanics and Engineering, 285:791-816 http://dx.doi.org/10.1016/j.cma.2014.12.009
  5. Pedroso DM (2015) A consistent u-p formulation for porous media with hysteresis. Int. Journal for Numerical Methods in Engineering, 101(8):606-634 http://dx.doi.org/10.1002/nme.4808

Authors and license

See the AUTHORS file.

Unless otherwise noted, the Gosl source files are distributed under the BSD-style license found in the LICENSE file.