No description

Find a file

Ingrid Werner ee3a056449 Add unit tests for transform layer		2026-02-05 16:00:00 +00:00
config	Initial data pipeline scaffold	2026-01-08 09:00:00 +00:00
src	Add Redshift writer	2026-01-29 10:00:00 +00:00
tests	Add unit tests for transform layer	2026-02-05 16:00:00 +00:00
README.md	Initial data pipeline scaffold	2026-01-08 09:00:00 +00:00
requirements.txt	Initial data pipeline scaffold	2026-01-08 09:00:00 +00:00

data-pipeline

ETL pipeline for Nexus analytics platform. Python + Apache Airflow.

Setup

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
airflow db init
airflow dags list

S3 (raw events) → ingest → transform → load → Redshift (analytics)

Dags run nightly at 01:00 UTC. See config/pipeline.yml for full schedule config.