Airflow How-To Guides
101 videos • 2,028 views • by The Data Guy
1
How to Create Data Quality Checks in Snowflake Using Airflow!
The Data Guy
Download
2
How to Use Airflow's Dynamic Task Mapping with S3 Buckets!
The Data Guy
Download
3
How to Execute Predictions from a Machine Learning Model in AWS Sagemaker using Airflow!
The Data Guy
Download
4
How to Visualize DBT workflows with Airflow and the Astro Cosmos Framework!
The Data Guy
Download
5
How to Build a Salesforce ELT Pipeline with Fivetran, dbt, and Census using Airflow!
The Data Guy
Download
6
How to Implement Data Quality Checks with Redshift using Airflow!
The Data Guy
Download
7
How to Chain Multiple DAG's Together in Airflow!
The Data Guy
Download
8
How to Use AI to Write Airflow DAG's for You Using ChatGPT!
The Data Guy
Download
9
How t to Dynamically Create Tasks in Airflow!
The Data Guy
Download
10
How to Incorporate AI into Your DataPipeline using the ChatGPT Operator
The Data Guy
Download
11
How to Create an Automated Email Marketing Pipeline Using Airflow!
The Data Guy
Download
12
How to Build an ETL Analytics Pipeline using Airflow, DuckDB and Streamlit!
The Data Guy
Download
13
How to Schedule and Run Snowflake Snowpark Queries Using Airflow and the Python VENV Operator
The Data Guy
Download
14
2.6 Airflow Features in Practice!
The Data Guy
Download
15
How to Schedule and Run Snowflake Snowpark Jobs Using Airflow and a PythonVirtualEnvironment!
The Data Guy
Download
16
How to Use AI to Create Airflow Operators with Google Bard!
The Data Guy
Download
17
How to Add Slack Notifications to Your Airflow DAG's with Airflow Notifiers!
The Data Guy
Download
18
How to Run ML Models In Snowflake Using Airflow!
The Data Guy
Download
19
How to Create a Custom Sensor in Airflow!
The Data Guy
Download
20
How to Do HuggingFace Sentiment Analysis Using Airflow!
The Data Guy
Download
21
How to Create Datasets in Airflow!
The Data Guy
Download
22
How to Monitor a Kafka Topic Using Airflow!
The Data Guy
Download
23
How to Manage DBT Workflows Using Airflow!
The Data Guy
Download
24
How to Do Data-Driven Scheduling in Airflow with the Astro SDK and Snowflake!
The Data Guy
Download
25
How to Use DBeaver to Interact with a Local Airflow Postgres Meta-Database!
The Data Guy
Download
26
How to Use Airflow Variables Programmatically!
The Data Guy
Download
27
How to Use Scikit for Machine Learning with Airflow!
The Data Guy
Download
28
How to Transfer Every File from an S3 Bucket Using Airflow!
The Data Guy
Download
29
How to Build an ETL Analytics Pipeline With Databricks Using Airflow!
The Data Guy
Download
30
How to Track Ethereum Transactions Using the Etherscan API and Airflow!
The Data Guy
Download
31
31 MORE Things You STILL Didn't Know In Zelda Tears Of The Kingdom
GameSpot
Download
32
How to Do Data Quality Checks on Pandas Dataframes Using GreatExpectations!
The Data Guy
Download
33
How to Produce and Consume Messages to/from a Kafka Cluster Using Airflow!
The Data Guy
Download
34
How to Connect to an MSSQL Database Using Airflow!
The Data Guy
Download
35
How to Set Up CICD Pipelines Using Github Actions for Airflow!
The Data Guy
Download
36
How to Upload Pandas Dataframes to S3 Using Airflow!
The Data Guy
Download
37
How to Create Cross Dag Dependencies Using Airflow Datasets!
The Data Guy
Download
38
How to Transform Task Outputs in Airflow!
The Data Guy
Download
39
How to Do Model Prediction Using MLFlow and Airflow!
The Data Guy
Download
40
How to Run Airflow Locally without Docker!
The Data Guy
Download
41
How to Build an End-to-End ETL Pipeline with Airflow, Snowflake, S3, and Salesforce!
The Data Guy
Download
42
How to Prepare for an Airflow Upgrade
The Data Guy
Download
43
How to Do Efficient Data Quality Checks Using Airflow!
The Data Guy
Download
44
How to Manage Azure Data Factory Pipelines Using Airflow!
The Data Guy
Download
45
How to Use DuckDB, MotherDuck and Airflow Together!
The Data Guy
Download
46
How to Debug Your Airflow DAG's!
The Data Guy
Download
47
How to Create an End-to-End ETL Analytics Pipeline with Airflow and Databricks
The Data Guy
Download
48
How to Do Data Integrity Checks with Airflow!
The Data Guy
Download
49
How to Make Multiple API Requests at Once Using Airflow!
The Data Guy
Download
50
How to Use Soda for Data Quality Checks with Airflow!
The Data Guy
Download
51
How to Use Great Expectations for Data Quality Checks with Airflow
The Data Guy
Download
52
How to Use MongoDB with Airflow!
The Data Guy
Download
53
End to End ELT Pipeline Project for Financial Data Analytics!
The Data Guy
Download
54
How to Create an ELT Pipeline Using Airflow, Snowflake, and dbt!
The Data Guy
Download
55
How to Run Airflow Locally without Docker Desktop using Podman!
The Data Guy
Download
56
The Definitive Guide to Airflow XCom's!
The Data Guy
Download
57
Intro to Airflow! Beginner's Guide to Getting Started with Airflow
The Data Guy
Download
58
How to Build an ETL Pipeline from Spark to a Cassandra Database using Airflow!
The Data Guy
Download
59
The Airflow AI: Ask Astro
The Data Guy
Download
60
Introduction to Machine Learning with Airflow and Snowflake/Snowpark!
The Data Guy
Download
61
How to Migrate Data from MSSQL Server to Snowflake using Airflow!
The Data Guy
Download
62
How to Pass Pandas Dataframes as XCom's in Airflow using a Custom XCom Backend!
The Data Guy
Download
63
Introduction to Machine Learning with MLFlow and Airflow!
The Data Guy
Download
64
How to Use Redshift with Airflow!
The Data Guy
Download
65
How to Migrate Data from an Oracle Database to a Postgres Database Using Airflow!
The Data Guy
Download
66
How to Migrate Data from MongoDB to a MySql Database with Airflow!
The Data Guy
Download
67
How to Submit a PySpark Script to a Spark Cluster Using Airflow!
The Data Guy
Download
68
How to Build an ELT Pipeline with CrateDB and Airflow!
The Data Guy
Download
69
Submitting PySpark Jobs to Databricks: ETL Workflows with Airflow Explained!
The Data Guy
Download
70
How to Create a Cloud Hosted Airflow Environment in 5 Minutes!
The Data Guy
Download
71
End to End Data Processing Pipeline for Customer Analytics and Machine Learning!
The Data Guy
Download
72
Beginner Machine Learning: How to Use a Weaviate for Sentiment Analysis Using Airflow!
The Data Guy
Download
73
How to Create a Data Streaming Pipeline with Databricks Spark, Apache Kafka, and Airflow!
The Data Guy
Download
74
How to Perform Data Quality Checks on a Postgres Database
The Data Guy
Download
75
How to Build an ETL Pipeline with Postgres, Azure Data Factory, and Airflow!
The Data Guy
Download
76
How to Create an ML Prediction Pipeline using Pinecone & Airflow!
The Data Guy
Download
77
How to Create an Unstructured Data Processing Pipeline Using Airflow and OpenAI!
The Data Guy
Download
78
How to Use Kubernetes Locally with Airflow!
The Data Guy
Download
79
How to Run AWS Glue Jobs with Airflow! Use Airflow to Manage Amazon Glue Jobs
The Data Guy
Download
80
How to Pull Information from the Google Rest API and Store it in AWS S3 with Airflow!
The Data Guy
Download
81
How to Run AWS Lambda Functions Using Airflow! How to Get More out of Lambda Functions with Airflow!
The Data Guy
Download
82
How to Create a Data Pipeline with BigQuery and Airflow! Beginner's Guide to BigQuery + Airflow
The Data Guy
Download
83
End-to-End Data Pipeline Project with Change Data Capture Using Postgres, Airflow, and Snowflake!
The Data Guy
Download
84
How to Add Microsoft Teams Alerts to Your Airflow Pipeline!
The Data Guy
Download
85
End to End AI Recommendation Pipeline with Cohere and Airflow!
The Data Guy
Download
86
End to End Parallel Data Quality Check Pipeline Project with Airflow and Snowflake!
The Data Guy
Download
87
How to Sync Data Between S3 Buckets Using AWS DataSync and Airflow!
The Data Guy
Download
88
How to Back-Up/Migrate Data from AWS DynamoDB to an S3 Bucket with Airflow!
The Data Guy
Download
89
How to Migrate Data From an Oracle Database to a Snowflake Database Using Airflow!
The Data Guy
Download
90
How to Manage Google DataFlow Workflows with Apache Airflow!
The Data Guy
Download
91
How to Use Airflow's New Object Store Object! Airflow Object Store Guide!
The Data Guy
Download
92
End-to-End MLOp's Pipeline Example with Airflow, Weaviate, and OpenAI!
The Data Guy
Download
93
How to Build a Real Time Predictive Analytics Pipeline with Airflow, S3, OpenAI, and Weaviate!
The Data Guy
Download
94
Apache Airflow Vs. Mage.AI! Apache Airflow and Mage.AI Compared for Data Orchestration!
The Data Guy
Download
95
How to Do Machine Learning Operations in Production with Airflow! MLOp's Production Pipeline Example
The Data Guy
Download
96
How to Migrate Airflow DAG Metadata from One Airflow Environment to Another Using Airflow!
The Data Guy
Download
97
How to Migrate Data from a Redshift Database to Any SQL Database Using Airflow!
The Data Guy
Download
98
How to Migrate Data from a Postgres SQL Database to a Snowflake Database Using Apache Airflow!
The Data Guy
Download
99
End-to-End Postgres to Data Lake Data Processing Pipeline with Apache Spark, Airflow, and Minio!
The Data Guy
Download
100
End-to-End Parallel ETL Pipeline with Airflow, Snowflake, and S3 Buckets!
The Data Guy
Download
101
How to Build a BigQuery Ingestion Pipeline from API's, SFTP Servers, and Pub/Sub with Airflow!
The Data Guy
Download