Today the team at Elementl is proud to announce an early release of Dagster, an open-source library for building systems like ETL processes and ML pipelines.

What is Dagster for?

Dagster is a data orchestrator for machine learning, analytics, and ETL. It lets you define pipelines in terms of the data flow between reusable, logical components, then test locally and run anywhere.

How do I run Dagster locally?

To run dagit locally, first ensure that you are running a recent Python version. Typically, you'll want to run Dagit inside a virtualenv. Then, you can install Dagit and any additional libraries you might need. In this configuration, Dagit will write execution logs to $DAGSTER_HOME/logs and listen on 0.0.