Photo by Daniel Korpai on Unsplash

Install Airflow on MacOS (Guide)

For those looking to get started on their journey becoming an Airflow Developer please take a look at this course from Alexandra Abbas https://www.udemy.com/course/apache-airflow-course/

Follow these steps to install Airflow on Mac.

  1. Open a Terminal window.
  2. Navigate to your Desktop in your Terminal. We are going to create a working directory here but you can create anywhere else in your file system if you like.

3. Create a working directory here called airflow-tutorial

mkdir airflow-tutorial

4. Change to airflow-tutorial directory in your Terminal.

cd airflow-tutorial

5. Create a virtual environment using coda (or any other tool). Install Python 3.7 in your virtual environment. I’m going to call the environment airflow-tutorial.

6. Activate the virtual environment

conda activate airflow-tutorial

7. Print the absolute path to your working directory by typing pwd.

I get but you might get something different. Copy this path

8. Set the path as the AIRFLOW_HOME environment variable. Note that you have to do this every time you open a new Terminal window and wish to use the Airflow CLI. Alternatively, you can set a permanent environment variable in your bash_profile.

By default, airflow uses ~/airflow as it’s AIRFLOW_HOME directory. We can overwrite this by setting a different path. Airflow will initialize the airflow.cfg file here along with the logs folder. We’ll store our dags and plugins in this directory.

9. Install Airflow 1.10.10 + extras using pip

pip install apache-airflow[gcp,statsd,sentry]==1.10.10

If you’re using zsh like me then you need to put in quotes as shown below.

Running Airflow on Mac os Locally

Before starting make sure you set your airflow home variable to the working directory. Validate this by typing .

We are going to initialize the airflow database using the following command. Make sure you are in the airflow-tutorial working directory.

airflow initdb

You can run the webserver by typing

Now create a new tab , Don’t forget to again put the Airflow Home variable while

export AIRFLOW_HOME =/Users/your_username/Desktop/airflow-tutorial 
cd Desktop/airflow-tutorial
conda activate airflow-tutorial
airflow scheduler

Now both our scheduler and webserver is running on

Analytics Engineer , Airflow Developer, Tableau Developer, Bigquery, GCP, DBT @ACV AUCTIONS