- Introduction
-
Welcome 0 hr 1 min
- Debugging Airflow Settings
-
Basic Checks
-
Airflow Settings for DAGs 0 hr 4 min
- DAG Parsing Errors
-
Module Error Checks 0 hr 4 min
-
Scheduler Error Checks 0 hr 3 min
-
Module Management 0 hr 4 min
-
Practice: Organizing your Airflow instance
- Improper Behavior
-
Improper DAG behavior - Common Issues
-
DAGs not running properly - Validating Connections 0 hr 5 min
-
Finding Connection Conflicts 0 hr 5 min
-
Resolve a Dependency Conflict
- Finishing Up
-
Quiz
-
Summary
-
How was it?
Airflow: Debug DAGs
Discover how to effectively debug DAGs.
Welcome! We're so glad you're here 😍
Do you find yourself spending endless hours attempting to debug your Airflow DAGs?
Or do you keep searching for a more straightforward approach to recognize and resolve the most typical problems in a particular DAG?
If you are ready to Debug your DAGs and discover ways to resolve your DAG issues. Let's get started.
🎯Objectives
At the end of this course, you'll be able to:
- Detect the most prevalent problem that causes errors in your tasks, which could have arisen due to various factors such as missing Airflow dependencies or incorrect configurations.
- Explore other common Airflow issues, such as connection problems with external systems.
- Identify when a lack of understanding of Airflow's configuration might lead you to believe that there are problems in your DAG while there aren't any, and the solution is to have a better understanding of Airflow's behavior.
👥 Audience
Who should take this course:
- Data Engineers
- Data Analysts
- Software Engineers
Set aside 35 minutes to complete the course.
💻 Setup Requirements
You need to have the following:
- Docker and Docker compose on your computer (cf: get Docker)
- The Astro CLI
🧠 Prior Knowledge:
- Configuring an Airflow Connection
- Reading logs of Airflow components
- Astro CLI or Airflow CLI
- Scheduler logic - how does the start date and schedule interval affect a DAG?
- Airflow DAG authoring