Counting 3,384 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Last Commit
Dec. 15, 2018
Created
Oct. 5, 2017

YugaByte DB


License Documentation Status Ask in forum Gitter chat Analytics

YugaByte Database is a transactional, high-performance database for planet-scale cloud applications. This repository contains the Community Edition of the YugaByte Database.

Table of Contents

About YugaByte DB

YugaByte DB offers both NoSQL and SQL in a single, unified database. It is meant to be a system-of-record/authoritative database that applications can rely on for correctness and availability. It allows applications to easily scale up and scale down in the cloud, on-premises or across hybrid environments without creating operational complexity or increasing the risk of outages.

Supported APIs

In terms of data model and APIs, YugaByte DB supports the following on top of a common core data platform:

  • Cassandra Query Language (CQL) - with strong consistency, distributed ACID transactions, low latency secondary indexes and a native JSONB data type
  • Redis - as a full database with automatic sharding, clustering, elasticity
  • PostgreSQL (in progress) - with linear scalability, high availability and fault tolerance

Note: You can run your Apache Spark applications on YugaByte DB

YugaByte DB is driver compatible with Apache Cassandra CQL and Redis - you can run existing applications written using existing open-source client drivers.

The distributed transactions feature is supported in the core data platform. The work to expose this as strongly consistent secondary indexes, multi-table/row ACID operations and SQL support is actively in progress. You can follow the progress of these features in our community forum.

Getting Started

Here are a few resources for getting started with YugaByte DB:

  • Quick start guide - install, create a local cluster and read/write from YugaByte DB.
  • Explore core features - automatic sharding & rebalancing, linear scalability, fault tolerance, tunable reads etc.
  • Real world apps - how real-world, end-to-end applications can be built using YugaByte DB.
  • Architecture docs - to understand how YugaByte was designed and how it works

Cannot find what you are looking for? Have a question? We love to hear from you - please post your questions or comments to our community forum.

Developing Apps

Here is a tutorial on implementing a simple Hello World application for YugaByte DB's Cassandra-compatible YCQL and Redis-compatible YEDIS APIs in different languages:

We are constantly adding documentation on how to build apps using the client drivers in various languages, as well as the ecosystem integrations we support. Please see our app-development docs for the latest information.

Once again, please post your questions or comments to our community forum if you need something.

Building YugaByte code

Prerequisites for CentOS 7

CentOS 7 is the main recommended development and production platform for YugaByte.

Update packages on your system, install development tools and additional packages:

sudo yum update
sudo yum groupinstall -y 'Development Tools'
sudo yum install -y ruby perl-Digest epel-release ccache git python2-pip
sudo yum install -y cmake3 ctest3

Also we expect cmake / ctest binaries to be at least version 3. On CentOS one way to achive this is to symlink them into /usr/local/bin.

sudo ln -s /usr/bin/cmake3 /usr/local/bin/cmake
sudo ln -s /usr/bin/ctest3 /usr/local/bin/ctest

You could also symlink them into another directory that is on your PATH.

We also use Linuxbrew to provide some of the third-party dependencies on CentOS. We install Linuxbrew in a separate directory, ~/.linuxbrew-yb-build, so that it does not conflict with any other Linuxbrew installation on your workstation, and does not contain any unnecessary packages that would interfere with the build.

git clone https://github.com/Linuxbrew/brew.git ~/.linuxbrew-yb-build
~/.linuxbrew-yb-build/bin/brew install autoconf automake flex gcc libtool maven readline openssl \
libuuid bzip2

We don't need to add ~/.linuxbrew-yb-build/bin to PATH. The build scripts will automatically discover this Linuxbrew installation.

Prerequisites for Mac OS X

Install Homebrew:

/usr/bin/ruby -e "$(
  curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install the following packages using Homebrew:

brew install autoconf automake bash bison ccache cmake coreutils flex gnu-tar icu4c libtool maven \
             ninja pkg-config pstree wget zlib

Also YugaByte DB build scripts rely on Bash 4. Make sure that which bash outputs /usr/local/bin/bash before proceeding. You may need to put /usr/local/bin as the first directory on PATH in your ~/.bashrc to achieve that.

Prerequisites for drivers and sample apps

YugaByte DB core is written in C++, but the repository contains Java code needed to run sample applications. To build the Java part, you need:

Also make sure Maven's bin directory is added to your PATH, e.g. by adding to your ~/.bashrc

export PATH=$HOME/tools/apache-maven-3.5.0/bin:$PATH

if you've installed Maven into ~/tools/apache-maven-3.5.0.

For building YugaByte DB Java code, you'll need to install Java and Apache Maven.

Java driver

YugaByte DB and Apache Cassandra use different approaches to split data between nodes. In order to route client requests to the right server without extra hops, we provide a custom load balancing policy in our modified version of Datastax's Apache Cassandra Java driver.

The latest version of our driver is available on Maven Central. You can build your application using our driver by adding the following Maven dependency to your application:

<dependency>
  <groupId>com.yugabyte</groupId>
  <artifactId>cassandra-driver-core</artifactId>
  <version>3.2.0-yb-18</version>
</dependency>

Building the code

Assuming this repository is checked out in ~/code/yugabyte-db, do the following:

cd ~/code/yugabyte-db
./yb_build.sh release --with-assembly

The above command will build the release configuration, put the C++ binaries in build/release-gcc-dynamic-community, and will also create the build/latest symlink to that directory. Then it will build the Java code as well. The --with-assembly flag tells the build script to build the yb-sample-apps.jar file containing sample Java apps.

Running the C++ tests

To run all the C++ tests you can use following command:

./yb_build.sh release --ctest

If you omit release argument, it will run java tests against debug YugaByte build.

To run specific test:

./yb_build.sh release --cxx-test util_monotime-test

Also you can run specific sub-test:

./yb_build.sh release --cxx-test util_monotime-test --gtest_filter TestMonoTime.TestCondition

Building Java code alone

You can skip building C++ code, this can be useful when you only need to rebuild Java code:

cd ~/code/yugabyte-db
./yb_build.sh --scb

Running the Java tests

Given that you've already built C++ and Java code you can run Java tests using following command:

./yb_build.sh release --scb --sj --java-tests

If you omit release argument, it will run java tests against debug YugaByte build, so you should then either build debug binaries with ./yb_build.sh or omit --scb and then it will build debug binaries automatically.

Alternatively, to run specific test:

./yb_build.sh release --scb --sj --java-test org.yb.client.TestYBClient

To run a specific Java sub-test within a test file use the # syntax, for example:

./yb_build.sh release --scb --sj --java-test org.yb.client.TestYBClient#testClientCreateDestroy

Viewing log outputs of Java tests

You can find Java tests output in corresponding directory (you might need to change yb-client to respective Java tests module):

$ ls -1 java/yb-client/target/surefire-reports/
TEST-org.yb.client.TestYBClient.xml
org.yb.client.TestYBClient-output.txt
org.yb.client.TestYBClient.testAffinitizedLeaders.stderr.txt
org.yb.client.TestYBClient.testAffinitizedLeaders.stdout.txt
…
org.yb.client.TestYBClient.testWaitForLoadBalance.stderr.txt
org.yb.client.TestYBClient.testWaitForLoadBalance.stdout.txt
org.yb.client.TestYBClient.txt

Note that the YB logs are contained in the output file now.

Reporting Issues

Please use GitHub issues to report issues. Also feel free to post on the YugaByte Community Forum.

Contributing

We accept contributions as GitHub pull requests. Our code style is available here (mostly based on Google C++ Style Guide).

License

YugaByte DB Community Edition is distributed under an Apache 2.0 license. See the LICENSE.txt file for details.

Latest Releases
v1.1.0 (Sep 17, 2018)
 Sep. 19 2018
v1.1.0
 Sep. 17 2018
v1.0.5 (Jul 27, 2018)
 Aug. 16 2018
v1.0.5
 Jul. 27 2018
v1.0.4 (Jun 20, 2018)
 Jul. 17 2018