Edge AI Just Got Faster - discu.eu

Hacker News

Using mmap to make LLaMA load faster https://justine.lol/mmap/ 169 comments 5/4/2023

Lobsters

Edge AI Just Got Faster https://justine.lol/mmap/ 5 comments 5/4/2023 ai , performance

Reddit

"We modified llama.cpp to load weights using mmap() instead of C++ standard I/O. That enabled us to load LLaMA 100x faster using half as much memory." https://justine.lol/mmap/ 80 comments 7/4/2023 programming

Linking pages

Everything I've learned so far about running local LLMs https://nullprogram.com/blog/2024/11/10/ 84 comments
RWKV: Reinventing RNNs for the Transformer Era — with Eugene Cheah of UIlicious https://www.latent.space/p/rwkv#%C2%A7the-eleuther-mafia 66 comments

Related searches:

Search whole site: site:justine.lol

Search title: Edge AI Just Got Faster

See how to search.

Submit link to: