Welcome to Kedro-Dagster's documentation

Kedro-Dagster is a Kedro plugin that enables seamless integration between Kedro, a framework for creating reproducible and maintainable data science code, and Dagster, a data orchestrator for data (and machine learning) pipelines. This plugin enables the use of Dagster's orchestration capabilities to deploy, automate, and monitor Kedro pipelines effectively.

Example of a Dagster Asset Lineage Graph generated from a Kedro project.

Table of Contents

Introduction

Overview of Kedro-Dagster, its purpose, and key concepts.

• What Is Kedro?
• What Is Dagster?
• Why Kedro‑Dagster?
• Key features
• Limitations & Considerations
• Contributing & Community

Getting Started

Step-by-step guide to installing and setting up Kedro-Dagster in your project.

• 1. Create a Kedro Project (Optional)
• 2. Installation
• 3. Initialize Dagster Integration
• 4. Configure Jobs, Executors, and Schedules
• 5. Browse the Dagster UI

Example

A practical example demonstrating how to use Kedro-Dagster in a real project.

• Project Overview
• Quick Start

Technical Documentation

In-depth documentation on the design, architecture, and core concepts of the plugin.

• Project Configuration
• Kedro-Dagster Concept Mapping

API Reference

Detailed reference for the Kedro-Dagster API, including classes, functions, and configuration options.

• Command Line Interface
• Configuration
• Translation Modules
• Utilities

License

Kedro-Dagster is open source and licensed under the Apache 2.0 License. You are free to use, modify, and distribute this software under the terms of this license.