- Introduction
-
Welcome! 0 hr 1 min
-
What is a DAG Run 0 hr 2 min
-
How DAGs are scheduled 0 hr 2 min
- The 2 DAG scheduling parameters
-
The start_date 0 hr 2 min
-
The schedule_interval 0 hr 2 min
-
Practice: Different Ways of writing a Schedule
- Schedule in the past
-
The catchup parameter 0 hr 1 min
-
Backfill your DAGs 0 hr 1 min
- Finishing up...
-
Quiz!
-
Summary
-
How was it?
Airflow: DAG Scheduling
Learn about how DAGs are scheduled.
Welcome! We're so glad you're here 😍
Want to know how exactly your DAGs are scheduled in airflow. Each DAG may or may not have a schedule, which informs how DAGRuns are created.
Hence want to know what exactly is a DAGRun and how the start date and the schedule interval work together in order to schedule your data pipelines.
Knowing about the concept of back filling and catching up for your DAGs is also necessary to not overburden you airflow environment.
🎯Objectives
At the end of this course, you'll be able to:
- Define what is a DAG run
- Understand how DAGs are scheduled
- How to define the start date and the schedule interval
- Use the catchup/backfill mechanism to run/rerun DAG Runs
👥 Audience
Who should take this course:
- Data Engineers
- Data Analysts
- Software Engineers
Set aside 15 minutes to complete the course.
💻 Setup Requirements
You need to have the following:
- Docker and Docker compose on your computer (cf: get Docker)
- The Astro CLI
- Access to a web browser