What is this?
- Basic database backed multi user annotation
- Integrated data preparation tools (ie creating tfrecords)
- Optional cloud machine learning training manager
- Rapid prototyping
- Collaboration on new dataset creation
- Rough starting point for someone looking to build a service like microsoft vision service or clarifai
- This does object detection where as at time of writing microsoft's only does classification
What does this use?
- Vue JS for annotation UI
- Sample object detection code is tensorflow object detection
- Database is PSQL with SQL Alchemy
- Core web app is flask
These instructions are specific to Google's cloud implementation. With various modifications this code should work with other providers like Microsoft or your own server.
This is a prototype system with known bugs. Use at your own risk.
- helpers/ (backend logic)
- methods/ (backend logic)
- static/ (JS / HTML / CSS)
- main.py (Main entry)
- database_setup.py (PSQL classes)
- local files
Quick start instructions
Create storage bucket.
- Suggest regional, us-central1 for speed with machine learning file transfer.
Create postgres sql instance
- Wait to enable apis
- Save password and id you choose
- Go to config options -> machine type and storage
- Slide cores left for development machine
- Wait for various running tasks things to finish
- Create a new database. While you should be able to use the default one this lets you choose an appropriate name, and appears to avoid some strange issues with google's psql implementation (specifically connection limits).
Transfer object detection files into cloud storage bucket
- You can use any python tensorflow file but this demo is optimized for the object detection research. If you do use your own you will need to package your code.
Change variables and paths as needed in settings/settings.py
- In settings/settings.py
- In app.yaml
- Change secret keys
- Set create tables to true
Various google setup stuff
- Download and install google cloud sdk
- Enable machine learning engine api
- In google api browser you may need to search for, Enable cloud SQL API (there are two of them, other is enabled automatically)
- Create pubsub topic
You need to deploy your app once for the option to create an app engine service account to come up
- Once you have done this you can create a service account
- Go to API -> Credentials -> Create -> Service account -> JSON
- Service account -> App engine default service account
- Place service account file in helpers folder
- Check name == service account name in settings
- for local
pip install -r requirements.txt
- set default deploy
cd your_working_directory && gcloud app deploy
- database scheme is
- **Download YAML files from cloud storage and use as you wish
- For example, update paths in local_files/create_tf_records_local.py and run
- Download config file, modify as needed
- Then run training locally using tensorflow object detection API
- Or use annotations with custom setup
- For video / prediction
- If using object detection API, download model file
- Then create frozen weights (By default online system creates frozen weights in different format than is preferred by local system)
- Update paths and run predict.py
- Run images to video if you wish
Contributions welcome, for example:
- Polygon capture and ground truth creation for semantic segmentation
- Add other object detection configurations
- More functions around adding users to the same project & permissions
- Better error handling
- Test cases and test coverage
- UI work (vue js), ie display status of training from pub/sub
- More training options, ability to tune hyper parameters from UI
- Integration with annotation services such as Amazon Mechanical Turk & scale API
- Improved label options, ie choose which labels get used in specific training version
A few known issues and notes
Upload speed and connections
Despite calling session.close() the session will sometimes not close fully. I don't know why so if anyone does please tell me.
There is a max connection limit set by google of 20 (for their lowest level psql db). This will mean sometimes you will hit a max connection limit, especially with more users. Upgrading to at least 1 full vCPU (instead of 1 shared) seems to help with this.
Related to this, for uploading, there are a max number of threads, set in settings.py, each thread spawns a new connection. The logs have updates on zip processor status and/or adjust number of threads / timeout in the settings page.
Most status updates are published to standard error, the UI needs work.
Adding users to projects
The fastest way is to add a code at signup. ie if project "roses" than assign to project id == 15 Clearly this is an area that could use some more work.