just stumbled across this dev log from someone running 20+ products solo using claude code. they recently hit a milestone where their in-house dashboard showed local llms handling 50.3% of their workload compared to 48.2% on the cloud side. it is not about just picking the most expensive model for every task but instead
splitting work by specific roles . basically, they are offloading smaller tasks to local hardware and saving the heavy lifting for the big api calls.
it is a massive way to scale without the subscription bills exploding .
>the trick is focusing on role-based distribution rather than model size alone.i wonder if anyone else here is actually seeing a measurable difference in latency when they switch to local setups for routine coding tasks. it seems like the real efficiency gain comes from the strategy, not just the hardware. local_llm_usage = 50.3% vs cloud usage at 48.2%. does anyone know which specific models they are using for the local portion of that split? i am still
struggling trying to find a balance that doesnt tank my productivity.
more here:
https://dev.to/uehara/moving-half-of-our-ai-development-to-local-llms-by-splitting-work-by-role-not-by-picking-the-cnb