- 👋 Welcome to Astro!
-
Your onboarding
-
Get the most out of the Onboarding
- 🧩 Discover Astro
-
Log in to Astro 00:00:30
-
Quick tour of the UI 00:02:01
-
What's a Deployment? 00:03:19
-
What's a Workspace? 00:01:36
-
Workspace overview 00:02:31
-
Set up your Workspace
-
Deployment overview 00:01:49
-
Manage your resources 00:02:06
-
Set up your Deployment
- 🧩 Set up your local environment
-
What is it? 00:03:23
-
Install the CLI 00:00:37
-
Create your project 00:01:53
-
Run your project 00:00:49
- 🧩 Start with the basics
-
Why Airflow? 00:01:15
-
What is Airflow? 00:01:32
-
Core components 00:02:04
-
Core Concepts 00:02:56
-
Airflow is not... 00:01:06
-
How does it work? 00:01:40
- 🧩 Create your first DAG
-
DAG minimum requirements 00:01:35
-
Create a task 00:01:54
-
Define dependencies 00:01:19
-
Start coding your first DAG...
-
Quiz!
- 🧩 Manage your DAG on the Airflow UI
-
DAG's overview 00:02:22
-
Monitor historical DAG Runs and TIs 00:01:13
-
Check DAG dependencies 00:00:57
-
Spot common failing patterns 00:00:55
-
Optimize your tasks 00:01:02
-
Any bottleneck in your DAG? 00:01:08
-
Check your modifications 00:00:25
-
Debug your tasks 00:01:12
-
Manage DAG Runs and TIs 00:01:22
-
Play with your DAG
-
Quiz!
- 🧩 Properly schedule your DAG
-
What's a DAG Run? 00:01:50
-
How DAGs get scheduled? 00:01:43
-
The start date 00:01:42
-
The schedule interval 00:01:49
-
Catchup 00:01:27
-
Quiz!
- 🧩 Interact with external services
-
What's a connection? 00:00:40
-
Create a connection 00:01:59
-
Install a Provider and add a Connection
-
Find the parameters you need 00:01:05
-
Interact with an External Service
- 🧩 Superpower your DAG
-
What's a variable? 00:01:22
-
Create a variable 00:01:33
-
Access a variable from your DAG 00:01:42
-
Adding a Variable
-
What's templating?
-
Add data at runtime with Templating
-
Use variables to your DAG
- 🧩 Share data between your tasks
-
What's an XCOM? 00:01:29
-
An XCOM example 00:01:32
-
Methods for sharing data 00:01:38
-
Pulling multiple values 00:01:51
-
The limitations 00:01:56
-
Data sharing in your DAG
-
Quiz!
- 🧩 Deploy!
-
Authenticate to Astro 00:00:43
-
Select your Workspace 00:00:44
-
Select your Deployment 00:02:00
-
How to deploy to Astro 00:02:37
-
Deploy your DAG to Astro!
- 🚀 Well done!
-
How was it?
-
What's next?
Astro: Onboarding
Discover Astro and Airflow by building and deploying a data pipeline!
Welcome to the Astro Onboarding course! 🚀
Astro, a fully managed service for Apache Airflow, helps effectively manage and streamline data pipelines for optimal efficiency and productivity.
This course will explore all the basics you need to set you up for success with Astro and Airflow.
You'll learn (or brush up) on how to set up a local Airflow development environment; and create, test, and deploy data pipelines on Astro to start making data-driven decisions.
At the end of this course, you'll be able to:
- Use the Astro CLI to set up and build data pipelines with Airflow in a local development environment.
- Deploy a data pipeline to Astro
- Use Astro to manage data pipelines at scale
Prerequisites:
- Set aside 1.5 hours to complete the course.
- Have access to an environment with Docker installed.
- Have an Astro account. (All new accounts have a 14-day trial and $300 in credits.)