Your ML model's 93% win rate is probably just luck. Here's why.

A developer trains a simple machine learning model on Bitcoin up/down markets with 15-minute resolution on Kalshi, predicting direction after five minutes using candlestick data. After 5,335 training samples, the model barely beats the market probability by ~2%, finding almost no edge.

In live testing, the model wins 13 of 14 $5 trades, producing a 93% win rate and 45% return in hours. The creator explicitly acknowledges this is a small sample, calculates expected loss of $12 after 1,000 windows, and calls the streak "luck."

The GitHub repo and prompt-based training approach are useful for learning ML experimentation, but the underlying data is too noisy and the edge too thin to be production-worthy. No benchmarks, no out-of-sample validation, and no attempt to adjust for market microstructure effects.

The tension is clear: a flashy winning streak masks a statistically insignificant edge. For engineering leaders, this is a case study in survivorship bias and the danger of mistaking short-term variance for signal. The real value is in the honest post-mortem, not the trading results.