Counting 3,146 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Last Commit
Sep. 25, 2018
Created
Sep. 21, 2017

uTensor - Test Release

CircleCI

Release Note

  • Updated uTensor ReadMe
  • Updated uTensor-CLI ReadMe
  • ROM Tensor support

Introduction

What is it?

uTensor is an extremely light-weight machine learning inference framework built on Mbed and Tensorflow. It consists of a runtime library and an offline tool. The total size of graph definition and algorithm implementation of a 3-layer MLP produced by uTensor is less than 32kB in the resulting binary (excluding the weights).

How does it work?

A model is constructed and trained in Tensorflow. uTensor takes the model and produces a .cpp and .hpp file. These files contains the generated C++11 code needed for inferencing. Working with uTensor on the embedded side is as easy as copy-and-paste. The function interface looks like this:

#include "models/deep_mlp.hpp"
...
Context ctx;  //creating a context
...
//preparing for the input tensor
...
get_deep_mlp_ctx(Context& ctx, Tensor* input_0);  //perform inference
ctx.eval();
S_TENSOR prediction = ctx.get({"y_pred:0"});  //getting the result

The .hpp and .cpp files can be generated given a model (protocal buffer) file, for example:

$ utensor-cli deep_mlp.pb --output-nodes=y_pred
...
... Generate weight file: models/deep_mlp_weight.hpp
... Generate header file: models/deep_mlp.hpp
... Generate source file: models/deep_mlp.cpp

What's supported?

The project is work-in-progress. Here are the operators, of their quantized versions, that are currently avaliable:

  • Add
  • ArgMax
  • Dropout
  • MatMal
  • Max
  • Min
  • Placeholder
  • Quantization Ops
  • ReLu
  • Reshape

Quick Start

Hardware

uTensor should support any Mbed enabled board that has sufficient memory (128+ kB RAM and 512kB+ flash recommended). However, these two boards are popular among the core developers:

The Environment

There are two flows to get started with uTensor. For Windows users, please choose the Cloud9 flow as shown below.

  • Local Installation
  • The Cloud9 Flow
    • Requires Amazon Cloud9 Account
    • Does not support runtime debugging

Getting Started

Creating a New Project

Step-by-step instructions on how to create your own uTensor project from scratch. You will need a K66F board a micro-SD card (32GB max) for this guide.

MNIST Touch Screen

The example uses a 3-layer MLP trained on the MNIST dataset. The touch screen input is fed into the neural network for processing and the result is printed on the screen.

The Activity of Daily Living (ADL)

This example shows how to buffer time-series data into batches of snapshots. These snapshots are then fed into the neural network for inferencing. The model a small multi-layer MLP trained on the ADL dataset.

Development

uTensor is young and under going rapid development. Many exciting features are on the way:

  • Convolution
  • Pooling
  • CMSIS-NN integration
  • Smaller binary
  • More efficient Run-time

You can also check the project page for the latest progress. If you'd like to take part in this project, please have a look at our contributor guide and feel free to reach out to us.

Further Reading

Latest Releases
ROM Tensor
 Sep. 17 2018
Test Release
 Jul. 2 2018