[ 🏠 Home / πŸ“‹ About / πŸ“§ Contact / πŸ† WOTM ] [ b ] [ wd / ui / css / resp ] [ seo / serp / loc / tech ] [ sm / cont / conv / ana ] [ case / tool / q / job ]

/tool/ - Tools & Resources

Software reviews, plugins & productivity tools
Name
Email
Subject
Comment
File
Password (For file deletion.)

File: 1788708634336.jpg (102.44 KB, 1024x1024, img_1788708624482_bvulcbzv.jpg)ImgOps Exif Google Yandex

382db No.2156

everyone is moving away from expensive subscriptions toward running small language models locally. it feels much more `
private
` to keep ur data off external servers using ollama run llama3. it actually works better than i expected if u have enough vram.

382db No.2157

File: 1788710055724.jpg (150.07 KB, 1024x1024, img_1788710013975_vmg9m5r7.jpg)ImgOps Exif Google Yandex

the latency is a huge killer if youre not on a decent gpu. i started using LM Studio to test different quantizations, and the difference btwn 4-bit and 8-bit is pretty noticeable once you get into larger models.



[Return] [Go to top] Catalog [Post a Reply]
Delete Post [ ]
[ 🏠 Home / πŸ“‹ About / πŸ“§ Contact / πŸ† WOTM ] [ b ] [ wd / ui / css / resp ] [ seo / serp / loc / tech ] [ sm / cont / conv / ana ] [ case / tool / q / job ]
. "http://www.w3.org/TR/html4/strict.dtd">