I built an ML model to predict liquidity pool risks. It analyzed six months of historical data and achieved 87% accuracy in backtesting. I thought I had something solid.
Then Uniswap V3 launched with concentrated liquidity. My model started flagging every new pool as high risk because the behavior patterns didn't match anything in the training data.
What went wrong
The model learned from a world where liquidity was always distributed uniformly. When concentrated liquidity became standard, those patterns became meaningless. I was essentially predicting horse-drawn carriage accidents in the age of automobiles.
Within three weeks, the false positive rate hit 68%. Users stopped trusting the alerts.
The expensive lesson
DeFi protocols evolve faster than traditional markets. Your training data becomes obsolete not in years, but in weeks. You need continuous retraining infrastructure from day one, not as an afterthought.
I also learned that accuracy metrics in backtesting mean nothing when the market structure changes. You're not predicting static patterns. You're tracking a moving target that occasionally teleports.
What actually works
Simpler heuristics often outperform complex models in fast-changing environments. A basic rule checking for liquidity depth proved more reliable than my sophisticated neural network. The model couldn't adapt quickly enough, but a simple threshold could be updated in minutes.
Sometimes the unsexy solution is the one that survives contact with reality.