Counting 3,146 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Last Commit
Sep. 24, 2018
Created
Nov. 1, 2017

TensorFlow as a Service (TFaaS)

Build Status Go Report Card License:MIT DOI Tweet

A general purpose framework (written in Go) to serve TensorFlow models. It provides reach and flexible set of APIs to efficiently access your favorite TF models via HTTP interface. The TFaaS supports JSON and ProtoBuffer data-formats.

The following set of APIs is provided:

  • #c5f015 /upload to push your favorite TF model to TFaaS server
  • #f03c15 /delete to delete your TF model from TFaaS server
  • #1589F0 /models to view existing TF models on TFaaS server
  • #521b92 /json to serve TF model predictions in JSON data-format
  • #9437ff /proto to serve TF model predictions in ProtoBuffer data-format

From deployment to production

➀ install docker image (TFaaS port is 8083)

docker run --rm -h `hostname -f` -p 8083:8083 -i -t veknet/tfaas

➁ upload your TF model to TFaaS server

curl -X POST http://localhost:8083/upload
-F 'name=ImageModel' -F '[email protected]/path/params.json'
-F '[email protected]/path/tf_model.pb' -F '[email protected]/path/labels.txt'

➂ get your predictions

curl https://localhost:8083/image -F '[email protected]/path/file.png' -F 'model=ImageModel'

Fore more information please visit curl client page.

TFaaS interface

Clients communicate with TFaaS via HTTP protocol. See examples for Curl, Python and C++ clients.

TFaaS benchmarks

Benchmark results on CentOS, 24 cores, 32GB of RAM serving DL NN with 42x128x128x128x64x64x1x1 architecture (JSON and ProtoBuffer formats show similar performance):

  • 400 req/sec for 100 concurrent clients, 1000 requests in total
  • 480 req/sec for 200 concurrent clients, 5000 requests in total

For more information please visit bencmarks page.

More information

Latest Releases
v01.00.08
 Sep. 12 2018
v01.00.07
 Jul. 12 2018
First public version
 Jul. 9 2018
v01.00.05
 Jun. 27 2018
v01.00.04
 Jun. 17 2018