[ 🏠 Home / 📋 About / 📧 Contact / 🏆 WOTM ] [ b ] [ wd / ui / css / resp ] [ seo / serp / loc / tech ] [ sm / cont / conv / ana ] [ case / tool / q / job ]

/job/ - Job Board

Freelance opportunities, career advice & skill development
Name
Email
Subject
Comment
File
Password (For file deletion.)

File: 1788061505479.jpg (121.71 KB, 1024x1024, img_1788061466948_ivbkc55g.jpg)ImgOps Exif Google Yandex

abd36 No.2138

just pulled the new update via ollama and it seems way more stable than the previous version. you can grab the 12b version with ollama pull Natuworkguy/flash-onyx-2.2:12b if you are running standard hardware. there is also a 31b flagship model available, but youll definitely need a proper gpu to run that one smoothly. it sounds like the dev spent a lot of time tweaking it because the old version was almost there but not quite. fixes the logic errors i was seeing in my cli agent has anyone tried running the 31b version on a consumer setup yet?

article: https://dev.to/natuworkguy/flash-onyx-22-is-out-and-it-finally-finishes-the-job-2cki

abd36 No.2139

File: 1788061658560.jpg (179.18 KB, 1024x1024, img_1788061642540_5b73par8.jpg)ImgOps Exif Google Yandex

tried running that 31b on a single 3090 using 4-bit quantization and the slowdown was brutal. if you wanna avoid the massive lag, try setting your context window to smth smaller like
8192
instead of the default.
>the vram pressure is real once you start hitting higher token counts.



[Return] [Go to top] Catalog [Post a Reply]
Delete Post [ ]
[ 🏠 Home / 📋 About / 📧 Contact / 🏆 WOTM ] [ b ] [ wd / ui / css / resp ] [ seo / serp / loc / tech ] [ sm / cont / conv / ana ] [ case / tool / q / job ]
. "http://www.w3.org/TR/html4/strict.dtd">