Counting 3,663 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Last Commit
Mar. 21, 2019
Created
Aug. 29, 2017

Apache Daffodil


Apache Daffodil (incubating) is the open source implementation of the Data Format Description Language (DFDL), a specification created by the Open Grid Forum. DFDL is capable of describing many data formats, including textual and binary, commercial record-oriented, scientific and numeric, modern and legacy, and many industry standards. It leverages XML technology and concepts, using a subset of W3C XML schema type system and annotations to describe such data. Daffodil uses this description to parse data into an infoset represented as XML or JSON, easily capable of ingestion, validation, and transformation.

For more information about Daffodil, see https://daffodil.apache.org/.

Build Requirements

  • JDK 8 or higher
  • SBT 0.13.8 or higher

Getting Started

SBT is the officially supported tool to build Daffodil, run all tests, create packages, and more. Below are some of the more common commands used for Daffodil development.

❗️ SBT Memory Requirements

Compiling and running all Daffodil tests works best with at least 6GB of memory, more than is usually provided by default. We recommended that you supply -mem 6144 either as an sbt option (e.g. sbt -mem 6144 test) or edit /etc/sbt/sbtopts to increase the available memory when running the below commands.

Compile

$ sbt compile

Tests

Run all unit tests:

$ sbt test 

Run all command line interface tests:

$ sbt it:test

Command Line Interface

Create Linux and Windows shell scripts in daffodil-cli/target/universal/stage/bin/. See the Command Line Interface documentation for details on its usage:

$ sbt daffodil-cli/stage

License Check

Generate an Apache RAT license check report located in target/rat.txt and error if any unapproved licenses are found:

$ sbt ratCheck

Test Coverage Report

Generate an sbt-scoverage test coverage report located in target/scala-ver/scoverage-report/:

$ sbt clean coverage test it:test
$ sbt coverageReport
$ sbt coverageAggregate

Getting Help

For questions, we can be reached at the dev@daffodil.apache.org or users@daffodil.apache.org mailing lists. Bugs can be reported via the Daffodil JIRA.

License

Apache Daffodil is licensed under the Apache License, v2.0.

Disclaimer

Apache Daffodil is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

Latest Releases
v2.3.0-rc1
 Feb. 14 2019
v2.2.0-rc2
 Aug. 22 2018
v2.2.0-rc1
 Aug. 16 2018
rel/v2.1.0
 Apr. 5 2018
v2.1.0-rc3
 Apr. 5 2018