|
|
0b31344c4c
|
Reduce batch size — 50k was OOMing the Redshift loader, 10k works fine, don't ask why
|
2026-04-21 08:00:00 +00:00 |
|
|
|
4c29bcc54c
|
Add vacuum helper — please read the docstring before using
|
2026-04-07 17:45:00 +00:00 |
|
|
|
d657c8375a
|
Test PII anonymisation
|
2026-03-24 10:15:00 +00:00 |
|
|
|
773e577dad
|
Add PII anonymisation step — legal requirement from data protection review
|
2026-03-24 09:30:00 +00:00 |
|
|
|
15a5606f45
|
Move pipeline to 01:00 UTC — 03:00 was conflicting with Redshift maintenance
|
2026-03-10 08:10:00 +00:00 |
|
|
|
8b0cec826a
|
Add centralised logging utility
|
2026-03-04 14:00:00 +00:00 |
|
|
|
452ee4ed3e
|
Add Airflow DAG skeleton for events ETL
|
2026-02-19 11:30:00 +00:00 |
|
|
|
cf5e45713c
|
Fix typo in log message
|
2026-02-06 09:05:00 +00:00 |
|
|
|
ee3a056449
|
Add unit tests for transform layer
|
2026-02-05 16:00:00 +00:00 |
|
|
|
f9d124ab6f
|
Add Redshift writer
|
2026-01-29 10:00:00 +00:00 |
|
|
|
21fb333812
|
Add transform helpers: null drop, dedup, type casting
|
2026-01-22 14:20:00 +00:00 |
|
|
|
37c3266d9c
|
Initial data pipeline scaffold
|
2026-01-08 09:00:00 +00:00 |
|
