Do you need MLflow, or do you need reproducible pipeline caching?
They sound like the same problem. They're not — and mixing them up is why so many ML projects still can't reproduce last week's result.
Blog¶
Stop rerunning your whole pipeline: caching intermediary DataFrames in Python
How data scientists waste hours recomputing steps that never changed — and a lightweight way to fix it.
4 Reasons why your machine learning code is bad
Problems with your current workflow
Your current workflow probably chains several functions together like in the example below. While quick, it likely has many problems: