- Introduction
-
Welcome! 0 hr 1 min
-
Why pools? 0 hr 1 min
- Pools in action!
-
What is a pool? 0 hr 2 min
-
Define a pool to tasks 0 hr 1 min
-
Pools with SubDags 0 hr 1 min
-
Practice: Manage resources per team with Pools
-
Define the number of slots 0 hr 1 min
- Finishing up...
-
Quiz!
-
Summary
-
How was it?
Airflow: Pools
Learn how to define how many tasks run in parallel for a set of tasks with Airflow pools.
Welcome! We're so glad you're here 😍
Want to know why the pools are great? Yea swimming pools are great but here we will find out about the Airflow pools.
What they are, and how to better manage your resources with them as some systems can get overwhelmed when too many processes hit them at the same time.
Therefore Airflow pools can be used to limit the execution parallelism on arbitrary sets of tasks.
🎯Objectives
At the end of this course, you'll be able to:
- why pools are great?
- what they are
- better manage your resources
👥 Audience
Who should take this course:
- Data Engineers
- Data Analysts
- Software Engineers
Set aside 12 minutes to complete the course.
💻 Setup Requirements
You need to have the following:
- Docker and Docker compose on your computer (cf: get Docker)
- The Astro CLI
- Access to a web browser