Counting 2,285 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Last Commit
Jan. 19, 2018
Dec. 2, 2017

An Introduction to DataFrames

Bogumił Kamiński, Dec 25, 2017

A brief introduction to basic usage of DataFrames. Tested under master from 2017-12-05 and Julia 0.6.2.

I will try to keep it up to date as the package evolves. This tutorial covers DataFrames, CSV, Missings, and CategoricalArrays, as they constitute the core of DataFrames.

In the last extras part mentions selected functionalities of selected useful packages that I find useful for data manipulation, currently those are: FreqTables, DataFramesMeta.


File Topic
01_constructors.ipynb Creating DataFrame and conversion
02_basicinfo.ipynb Getting summary information
03_missingvalues.ipynb Handling missing values
04_loadsave.ipynb Loading and saving DataFrames
05_columns.ipynb Working with columns of DataFrame
06_rows.ipynb Working with row of DataFrame
07_factors.ipynb Working with categorical data
08_joins.ipynb Joining DataFrames
09_reshaping.ipynb Reshaping DataFrames
10_transforms.ipynb Transforming DataFrames
11_performance.ipynb Performance tips
12_pitfalls.ipynb Possible pitfalls
13_extras.ipynb Additional interesting packages


Date Changes
2017-12-05 Initial release
2017-12-06 Added description of insert!, merge!, empty!, categorical!, delete!, DataFrames.index
2017-12-09 Added performance tips
2017-12-10 Added pitfalls
2017-12-18 Added additional worthwhile packages: FreqTables and DataFramesMeta
2017-12-29 Added description of filter and filter!
2017-12-31 Added description of conversion to Matrix

Core functions summary

  1. Constructors: DataFrame
  2. Getting summary: size, nrow, ncol, length, describe, showcols, names, eltypes, head, tail
  3. Handling missing: missing (singleton instance of Missing), ismissing, Missings.T, skipmissing, Missings.replace, allowmissing, disallowmissing, allowmissing!, completecases, dropmissing, dropmissing!
  4. Loading and saving: CSV (package), JLD (package),, CSV.write, save (from JLD), load (from JLD)
  5. Working with columns: rename, rename!, names!, hcat, insert!, DataFrames.hcat!, merge!, delete!, empty!, categorical!, DataFrames.index
  6. Working with rows: sort!, sort, append!, vcat, push!, view, filter, filter!, deleterows!, unique, nonunique, unique!
  7. Working with categorical: categorical, cut, isordered, ordered!, levels, unique, levels!, droplevels!, get, recode, recode!
  8. Joining: join
  9. Reshaping: stack, melt, stackdf, meltdf, unstack
  10. Transforming: groupby, vcat, by, aggregate, eachcol, eachrow, colwise
  11. Extras: