Counting 3,834 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Contributors
Last Commit
May. 19, 2019
Created
Jan. 24, 2019


Get started: InstallTutorialDemo VideoDocsExamples

Learn more: WebsiteFAQSubscribeBlogContact


Machine learning infrastructure for developers: build and deploy scalable TensorFlow applications on AWS without worrying about setting up infrastructure, managing dependencies, or orchestrating data pipelines.

Cortex is actively maintained by Cortex Labs. We're a venture-backed team of infrastructure engineers and we're hiring.


How it works

Data validation: validate data to prevent data quality issues early

- kind: raw_column
  name: col1
  type: INT_COLUMN
  min: 0
  max: 10

Data ingestion: connect to your data warehouse and ingest data at scale

- kind: environment
  name: dev
  data:
    type: csv
    path: s3a://my-bucket/data.csv
    schema: [@col1, @col2, ...]

Data transformation: use custom Python and PySpark code to transform data at scale

- kind: transformed_column
  name: col1_normalized
  transformer_path: normalize.py  # Python / PySpark code
  input: @col1

Model training: train models with custom TensorFlow code

- kind: model
  name: my_model
  estimator_path: dnn.py  # TensorFlow code
  target_column: @label_col
  input: [@col1_normalized, @col2_indexed, ...]
  hparams:
    hidden_units: [16, 8]
  training:
    batch_size: 32
    num_steps: 10000

Prediction serving: deploy models as prediction APIs that scale horizontally

- kind: api
  name: my-api
  model: @my_model
  compute:
    replicas: 3

Deploying to AWS: deploy your pipeline to AWS and make prediction requests

$ cortex deploy
Ingesting data ...
Transforming data ...
Training models ...
Deploying API ...
Ready! https://abc.amazonaws.com/my-api

Key features

  • End-to-end machine learning workflow: Cortex spans the machine learning workflow from feature management to model training to prediction serving.

  • Machine learning pipelines as code: Cortex applications are defined using a simple declarative syntax that enables flexibility and reusability.

  • TensorFlow and PySpark support: Cortex supports custom TensorFlow code for model training and custom PySpark code for data processing.

  • Built for the cloud: Cortex can handle production workloads and can be deployed in any AWS account in minutes.

Latest Releases
v0.3.0
 Apr. 15 2019
v0.2.0
 Mar. 12 2019
v0.1.0
 Feb. 18 2019
v0.1.0
 Feb. 13 2019