Changelog

Recent Changes and Updates to Astronomer Academy

April 3rd, 2024 - Introducing Learning Bytes 🧠

Happy April! We're excited to announce a new learning format on Astronomer Academy: Learning Bytes.

Learning Bytes are short-form (less than 10 minutes) modules that provide an overview of key Astro or Airflow features. They provide a brief run-through of the topic broadly as well as a demo of a use-case of the feature.

In our first release, we cover new Astro features that were released as part of the Astronomer Q1 release. More specifically, we go through the Reporting, Custom Deployment Roles, and Customer Managed Workload Identity Astro platform features. All Learning Bytes can be found by visiting the All Modules page and filtering by Learning Bytes on the left hand side.

We expect to leverage this format this year and in the future to make learning more accessible for learners looking for a quick way to pick up knowledge about Astro or Airflow. We hope you will check them out and we are looking forward to hearing your feedback!



March 11th, 2024 - Updates to Our Airflow 101 Learning Path
🎉

Great news! We're updating the Airflow 101 Learning Path. In this first major update to the path, we are making improvements to the first three modules in the path to help decrease the steep learning curve of getting started with Apache Airflow and keep up with the changes that have happened to Airflow since the original release of the Learning Path.

Why?

Back in late 2022, we released our Airflow 101 Learning Path. Since then, we have seen Apache Airflow evolve substantially. To keep up with the new features of Airflow and the feedback we have collected from Astronomer Academy learners, we are taking the first step to provide the best possible learning experience for individuals new to Airflow.

What are the specific changes?

While the core of the Learning Path remains the same, we’ve taken some time to update and restructure the first three modules. The specific changes are as follows:

  1. Two new modules have been added to the path and replace the Airflow: Concepts module currently in the Learning Path. The following modules have been added:

    • Introduction to Orchestration and Airflow: This new module will provide a comprehensive understanding of the history and significance of data orchestration and introduce Airflow as a data orchestrator. This will help technical and non-technical learners grasp what Airflow is and why it is useful.
    • Airflow Basics: This new module closely represents the current Airflow: Concepts module but with some improvements. Specifically, we have shortened the module by focusing only on the basics, such as Airflow core components, how Airflow runs a DAG, and the components of a DAG file.

  2. We’ve improved the following modules:

    • Local Development Environment:
      • We have added a new lesson showing the different ways of running Airflow.
      • We’ve shortened the module quite drastically to focus only on the essentials for getting your Airflow local environment up and ready.
      • We’ve added a new lesson for installing and setting up the Astro CLI with Windows.
      • We’ve added a new lesson for configuring Airflow with VSCode.
      • We removed the lesson about configuring Airflow with PyCharm, as we saw learners lean towards using VSCode. We still have an available resource about setting up PyCharm on our documentation site.

  3. Improvements to the overall path:

    • The path used to start with the Local Development Environment, followed by Airflow Concepts Modules. With the update, the path  will be ordered as follows:
      1. Introduction to Orchestration & Airflow
      2. Airflow Basics
      3. Local Development Environment
    • We have added quizzes to every lesson rather than just the end of the module. The end of the module quiz will now be used as a final assessment, testing your knowledge of the module as a whole, while quizzes in each lesson will test your understanding of the content from the lesson video.

When?

The changes went live on April 1, 2024.

How does this impact you?

We will apply updates directly to our existing content. You'll automatically get the latest updates if you’re enrolled in any affected modules. However, since we are adding lessons and modules to the Airflow 101 learning path, it will affect your progress. That means:

  • If you’ve already completed the path, it will still be marked as complete in our system, but your progress bar on the learning path will not be set to 100% anymore. You will need to complete the new modules to reach 100% completion.
  • If you are currently enrolled but have not completed the path, you will see your progress decrease and will need to complete the new modules to reach 100% completion.
  • If you have passed the Airflow Fundamentals certification, there is no impact. The certification stays valid, and you will not need to pass it again as we haven’t changed any existing questions, and we still cover the same content in the Learning Path.

Where?

Applicable changes will appear in the following modules:

  • Introduction to Orchestration and Airflow (New)
  • Airflow: Basics (Replaces the existing Airflow: Concepts module)
  • Airflow: Local Development Environment (Existing)

And the following Learning Path:

Airflow: Dynamic DAGs
Learn how to avoid duplicating DAGs with Dynamic DAGs.
Free
0 hr 8 min
Data Engineer Astro Runtime Advanced Airflow Module
Airflow: Pools
Learn how to define how many tasks run in parallel for a set of tasks with Airflow pools.
Free
0 hr 7 min
Data Engineer Astro Runtime Intermediate Airflow Module
Airflow: Deferrable Operators
Learn how to efficiently manage resources by using Deferrable Operators.
Free
0 hr 9 min
Data Engineer Astro Runtime Intermediate Airflow Module
Astronomer Software 101
Get started with Software and get the most out of the platform for your workflows
Free
5 Modules
Astronomer Platform Beginner Astronomer Enterprise Admin 101
Airflow: Datasets
Learn how to take advantage datasets to create data-aware scheduled DAGs.
Free
0 hr 16 min
Beginner Data Engineer Astro Runtime Intermediate Airflow Module
Airflow 101
The Airflow 101 learning path guides you through the foundational skills and knowledge you need to start with Apache Airflow. This includes the core concepts, the Airflow UI, creating your first data pipeline following best practices, how to schedule this data pipeline efficiently and more!
Free
12 Modules
Learning Path Airflow
Airflow: Basics
Explore how Apache Airflow works under the hood and the basics of how a DAG is built!
Free
Beginner 101 Airflow
Airflow: XComs 101
Learn the basics of sharing data between tasks using XComs
Free
0 hr 10 min
Beginner 101 Astro Runtime Airflow Module
Astro 101
Get started with Astro quickly and learn how to get the most out of it!
Free
8 Modules
Beginner 101 Astro
Learning Bytes: Customer Managed Workload Identity
An overview of the Customer Managed Workload Identity feature on Astro!
Free
5 min
Astro Runtime Learning Byte
Airflow: DAGs 101
Learn the basics of how to create a data pipeline in Airflow.
Free
0 hr 17 min
Beginner 101 Astro Runtime Airflow Module
Airflow: Local Development Environment
Set up a local development environment for Airflow!
Free
Beginner Airflow
Learning Bytes: Reporting
An overview of the reporting feature on Astro!
Free
7 min
Astro Astro Runtime Learning Byte
Learning Bytes: Custom Deployment Roles
An overview of the custom deployment roles feature on Astro!
Free
8 min
Astro Astro Runtime Learning Byte
Introduction to Orchestration and Airflow
An overview of the world of data orchestration and Apache Airflow!
Free
Beginner Airflow Module
Astro: User permissions
Define the right permissions to the right users in Astro
Free
0 hr 12 min
Beginner 101 Astro permissions Module
Astro: CI/CD
Make reliable and resilient deployments with CI/CD
Free
0 hr 27 min
Beginner 101 Astro cicd Module
Airflow: Templating
Learn how to pass dynamic data to tasks at runtime.
Free
0 hr 12 min
Beginner Data Engineer Astro Runtime Intermediate Templating Airflow Module
Astro: Cloud IDE
Learn about the Cloud IDE which makes it simple for someone new to Airflow and Astronomer users to write a data pipeline.
Free
0 hr 37 min
Beginner 101 Astro Module
Software: Metrics
What are the key metrics to know, what to do if something goes wrong, let's find out!
Free
0 hr 17 min
Astronomer Platform Beginner Astronomer Enterprise Admin Monitoring Software Module
Astro: SDK
Speed up your DAG development with the Astro SDK
Free
0 hr 50 min
Astro Data Engineer Astro Runtime Intermediate Module
Airflow: Branching
Learn how to branch in order to tell the DAGs to not to run all dependent tasks, but instead to pick and choose one or more paths to go down.
Free
0 hr 26 min
Beginner 101 Intermediate Airflow Module
Airflow: Task Groups
Learn how to optimize DAGs and clean up the Airflow UI with Task Groups.
Free
0 hr 18 min
Beginner Astro Runtime task groups Airflow Module
Airflow: Connections 101
Learn how to interact with systems from data pipelines with connections.
Free
0 hr 7 min
Beginner 101 Airflow Module
Astro: Local Development Environment
Set up your local development environment with the Astro CLI and speed up your DAGs development
Free
0 hr 9 min
Beginner 101 Astro Module
Software: User Permissions
Setting the proper permissions and roles for your users can be a pain. Let us help you make it more enjoyable and straightforward.
Free
0 hr 15 min
Beginner Astronomer Enterprise Admin Software Module
Software: Alerting
Be notified when something goes wrong and know what to do
Free
0 hr 12 min
Beginner Astronomer Enterprise Admin Software Module
Astro: Environment Variables
The fundamentals of how to apply Environment Variables in Astro
Free
0 hr 18 min
Beginner 101 Astro environment variables Module
Software: Workspaces
Learn how to better organize your Airflow deployments and Users with Workspaces.
Free
0 hr 16 min
Astronomer Platform Beginner Astronomer Enterprise Admin Workspace Software Module
Astro: Onboarding
Discover Astro and Airflow by building and deploying a data pipeline!
Free
01:17:53
101 Astro Module
Astro: Metrics
Monitor your deployments and quickly spot failures
Free
0 hr 10 min
Beginner 101 Astro Module
Software: Deployment Resources
Don't get your tasks killed anymore. Learn how to define your resources according to your needs.
Free
0 hr 17 min
Astronomer Platform Beginner Astronomer Enterprise Admin resources Software Module
Astro: Deployment Analytics
Learn how to interpret powerful insights relating to your deployment
Free
0 hr 9 min
Beginner Monitoring 101 Astro analytics Module
Astro: Secrets Backend
The fundamentals of how to configure an external secrets backend in Astro
Free
0 hr 13 min
Beginner modules 101 Astro AWS secrets backend secrets Module
Airflow: Command Line Interface (CLI)
Learn how to use the Airflow CLI to perform a variety of operations.
Free
0 hr 19 min
Beginner Airflow Module
Airflow: Variables 101
Learn how to store data with variables and make DAGs more dynamic.
Free
0 hr 6 min
Beginner 101 Astro Runtime Airflow Module
Airflow: DAG Scheduling
Learn about how DAGs are scheduled.
Free
0 hr 10 min
Beginner Data Engineer Astro Runtime Scheduling Airflow Module
Airflow: Taskflow API
Learn how to use the Taskflow API to create DAGs faster.
Free
0 hr 28 min
Data Engineer Astro Runtime Airflow Module
Airflow: Sensors
Learn how to use Airflow sensors to wait for events before triggering tasks.
Free
0 hr 15 min
Beginner 101 Airflow Module
Airflow Project: Dynamic Task Mapping with S3
Build a DAG that interacts with S3 buckets using Dynamic Task Mapping.
Free
0 hr 25 min
Beginner AWS Data Engineer Astro Runtime Projects Airflow Module
Airflow: Debug DAGs
Discover how to effectively debug DAGs.
Free
0 hr 27 min
Beginner Airflow Module
Airflow: Dynamic Task Mapping
Learn to create tasks based on current data using Dynamic Task Mapping.
Free
0 hr 12 min
Astro Runtime Advanced Airflow Module
Airflow: The ExternalPythonOperator
Learn how to run python tasks in a safe virtual environment.
Free
0 hr 21 min
Beginner 101 Data Engineer Astro Runtime Airflow Module
Airflow: UI
Learn how to efficiently monitor DAGs and Tasks in the Airflow UI.
Free
Beginner 101 Astro Runtime Airflow Module
Airflow: SubDAGs
Learn how to use SubDAGs to write cleaner and more efficient DAGs.
Free
0 hr 28 min
Beginner Data Engineer Astro Runtime Grouping Intermediate Airflow Module
[separator] Airflow Fundamentals
Test your skills and make sure you've mastered the Airflow fundamentals
Free
separator
[separator] Airflow DAG Authoring
Show your expertise to the world and level-up your data pipelines with DAG Authoring
Free
separator
Certification Exam: DAG Authoring
Challenge your DAG authoring skills and show to the world your expertise in creating amazing DAGs!
$150
certification
Prep Course: DAG Authoring
The course is an on-demand class which dives into topics covered on the Astronomer Certification: Apache Airflow DAG Authoring
Free
preparation course certification
Certification Exam: Airflow Fundamentals
Demonstrate your mastery of fundamental Apache Airflow concepts to the world, highlighting your proficiency in designing, scheduling, and overseeing data pipelines.
$150
certification
[separator] Popular Astro Modules
Free
separator
Astro: Deployments
Organize and configure your deployments to better run your workflows
Free
0 hr 14 min
Beginner 101 Astro Module
Airflow: Trigger Rules
Learn how to effectively control the task execution flow using trigger rules.
Free
0 hr 6 min
Data Engineer Astro Runtime Intermediate Airflow Module
Astro: HTTP Provider
Interact with HTTP Resources by making Web Requests to an API.
Free
0 hr 35 min
Beginner Astro Providers Module
Airflow: The KubernetesPodOperator
Master the KubernetesPodOperator (KPO) to better manage resources and dependencies of DAG tasks.
Free
0 hr 13 min
Data Engineer Astro Runtime Advanced Kubernetes Airflow Module
[separator] Popular Airflow Modules
Free
separator
Airflow: Basics
Explore how Apache Airflow works under the hood and the basics of how a DAG is built!
Free
Beginner 101 Airflow
Introduction to Orchestration and Airflow
An overview of the world of data orchestration and Apache Airflow!
Free
Beginner Airflow Module
Airflow: Sensors
Learn how to use Airflow sensors to wait for events before triggering tasks.
Free
0 hr 15 min
Beginner 101 Airflow Module
Airflow: Debug DAGs
Discover how to effectively debug DAGs.
Free
0 hr 27 min
Beginner Airflow Module
Airflow: Command Line Interface (CLI)
Learn how to use the Airflow CLI to perform a variety of operations.
Free
0 hr 19 min
Beginner Airflow Module
Airflow: DAG Scheduling
Learn about how DAGs are scheduled.
Free
0 hr 10 min
Beginner Data Engineer Astro Runtime Scheduling Airflow Module
Airflow: XComs 101
Learn the basics of sharing data between tasks using XComs
Free
0 hr 10 min
Beginner 101 Astro Runtime Airflow Module
Airflow: UI
Learn how to efficiently monitor DAGs and Tasks in the Airflow UI.
Free
Beginner 101 Astro Runtime Airflow Module
Airflow: DAGs 101
Learn the basics of how to create a data pipeline in Airflow.
Free
0 hr 17 min
Beginner 101 Astro Runtime Airflow Module
Airflow: Local Development Environment
Set up a local development environment for Airflow!
Free
Beginner Airflow
Airflow: Connections 101
Learn how to interact with systems from data pipelines with connections.
Free
0 hr 7 min
Beginner 101 Airflow Module
Airflow: Variables 101
Learn how to store data with variables and make DAGs more dynamic.
Free
0 hr 6 min
Beginner 101 Astro Runtime Airflow Module
Astro: Deployments
Organize and configure your deployments to better run your workflows
Free
0 hr 14 min
Beginner 101 Astro Module
Astro: Local Development Environment
Set up your local development environment with the Astro CLI and speed up your DAGs development
Free
0 hr 9 min
Beginner 101 Astro Module
Astro: User permissions
Define the right permissions to the right users in Astro
Free
0 hr 12 min
Beginner 101 Astro permissions Module
Astro: CI/CD
Make reliable and resilient deployments with CI/CD
Free
0 hr 27 min
Beginner 101 Astro cicd Module
Astro: Secrets Backend
The fundamentals of how to configure an external secrets backend in Astro
Free
0 hr 13 min
Beginner modules 101 Astro AWS secrets backend secrets Module
Astro: Environment Variables
The fundamentals of how to apply Environment Variables in Astro
Free
0 hr 18 min
Beginner 101 Astro environment variables Module
Astro: Deployment Analytics
Learn how to interpret powerful insights relating to your deployment
Free
0 hr 9 min
Beginner Monitoring 101 Astro analytics Module
Astro: Metrics
Monitor your deployments and quickly spot failures
Free
0 hr 10 min
Beginner 101 Astro Module
Software: Metrics
What are the key metrics to know, what to do if something goes wrong, let's find out!
Free
0 hr 17 min
Astronomer Platform Beginner Astronomer Enterprise Admin Monitoring Software Module
Software: Deployment Resources
Don't get your tasks killed anymore. Learn how to define your resources according to your needs.
Free
0 hr 17 min
Astronomer Platform Beginner Astronomer Enterprise Admin resources Software Module
Software: User Permissions
Setting the proper permissions and roles for your users can be a pain. Let us help you make it more enjoyable and straightforward.
Free
0 hr 15 min
Beginner Astronomer Enterprise Admin Software Module
Software: Alerting
Be notified when something goes wrong and know what to do
Free
0 hr 12 min
Beginner Astronomer Enterprise Admin Software Module
Software: Workspaces
Learn how to better organize your Airflow deployments and Users with Workspaces.
Free
0 hr 16 min
Astronomer Platform Beginner Astronomer Enterprise Admin Workspace Software Module