/conv/ - Conversion Rate

found a way to stop ai feature tests early using msprt and sequential testing in python. it lets you check if an llm update actually works without the usual risk of p-hacking by peeking at results mid-run. it basically gives you the same confidence as a fixed 30-day window but much faster . does anyone else still rely on traditional frequentist methods fixed-horizon testing for everyy single deployment?

article: https://www.freecodecamp.org/news/stop-early-without-p-hacking-using-msprt-and-sequential-testing-in-python/