Making a 411.7M-event pipeline restartable
My first aggregation ran out of temporary storage. I split the work by month and session bucket so I could reuse finished work after a retry.
- Python
- DuckDB
- Parquet
- FastAPI
INDEPENDENT ENGINEERING & MSc RESEARCH
Read how I partitioned REES46, tested forecasting models on OTTO, and worked through a release failure in FulfillAI.
3 projects shown
My first aggregation ran out of temporary storage. I split the work by month and session bucket so I could reuse finished work after a retry.
I built a pipeline on synthetic fulfillment data, then set up a separate GKE API lab. I released a deployment pointing to a nonexistent image, then verified recovery.
I compared eight forecasting approaches on four weeks of hourly cart and order counts. Repeating last week’s pattern gave the lowest mean WAPE on the development folds.
OPEN TO JUNIOR OPPORTUNITIES
I’m looking for junior Data and ML Engineering roles. If you have a role in mind or a question about the work, I’d like to hear from you.
Contact me