- Introduction
-
Welcome! 0 hr 1 min
- The Basics
-
Why Dynamic DAGs? 0 hr 1 min
-
Dynamic DAGs vs Dynamic Task Mapping 0 hr 1 min
- The Methods
-
The single-file method 0 hr 2 min
-
Drawbacks of the single-file method 0 hr 1 min
-
The best method! 0 hr 3 min
- Finishing up...
-
Quiz!
-
Summary
-
Resources
-
How was it?
Airflow: Dynamic DAGs
Learn how to avoid duplicating DAGs with Dynamic DAGs.
Welcome! We're so glad you're here 😍
Do you have multiple partners or maybe data sources where your DAG is similar across each of them.
And you are not a fan of writing duplicate code as it is more likely to be prone to errors.
Also for a minor change it is very unfeasible to make the same changes across each DAG that are similar.
Well Dynamic DAGs helps you keep the same DAG and generates as many instances of it as you need according to inputs
🎯Objectives
At the end of this course, you'll be able to:
- Define what is a Dynamic DAG
- What are the different methods
- Select the right method
👥 Audience
Who should take this course:
- Data Engineers
- Data Analysts
- Software Engineers
Set aside 13 minutes to complete the course.
💻 Setup Requirements
You need to have the following:
- Docker and Docker compose on your computer (cf: get Docker)
- The Astro CLI
- Access to a web browser