Tool
Visit website →
Dagster
Dagster is a data orchestration platform that builds, runs, and observes ETL/ELT and ML pipelines, integrating dbt, Databricks, Python, SaaS sources and warehouses, with scheduling, lineage, observability, data cataloging, governance, and enterprise security.
Use Cases
- 🟢 Orchestrate end-to-end ETL/ELT workflows with Dagster, integrating dbt, Databricks, Python and SaaS sources to load and transform data into your warehouse, schedule jobs, monitor runs and dataset lineage, and enforce governance and enterprise security.
- 🟢 Automate AI/ML pipeline orchestration using Dagster to coordinate data ingestion, feature engineering, distributed model training on Databricks, model lineage and versioning, and continuous retraining with real-time observability and alerts.
- 🟢 Centralize dataset cataloging and compliance by using Dagster to capture metadata and lineage across pipelines, provide searchable data catalogs for analysts, apply access controls and governance policies, and monitor data quality and provenance in production.