This repository contains the code shown in the LIVE with Astronomer on 2023-03-07, showing how to use dag.test().
- Clone the repository.
- Make sure you are in an environment that has Airflow 2.5+ installed by running
airflow versionfrom your command line. - Make sure you are in an environment that has the Airflow Amazon provider installed. If you use
virtualenvyou can run the following commands:
virtualenv venv
source venv/bin/activate
pip3 install -r requirements.txt- Provide your own Amazon connection by creating a
connections.yamlfile based uponconnections_template.yaml. - Create an S3 bucket
live-dag-test-bucketcontaining one JSON file namedorder_values.jsoncontaining the following JSON string:{'1001': 301.27, '1002': 433.21, '1003': 502.22}. - Run
python dag_test_example.py.
- Concept guide: Test Airflow DAGs
- Airflow Documentation: Testing DAGs with dag.test()
- Python Documentation: The Python Debugger
- How to install Airflow locally
- Concept guide: Custom Hooks and Operators
- Source code dag object
- Debugging in VSCode