PowerInfer: Fast Large Language Model Serving with a Consumer-Grade GPU [pdf] - discu.eu

Hacker News

PowerInfer: Fast Large Language Model Serving with a Consumer-Grade GPU [pdf] https://ipads.se.sjtu.edu.cn/_media/publications/powerinfer-20231219.pdf 9 comments 19/12/2023

Linking pages

GitHub - SJTU-IPADS/PowerInfer: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs https://github.com/SJTU-IPADS/PowerInfer 84 comments

Related searches:

Search whole site: site:ipads.se.sjtu.edu.cn

Search title: PowerInfer: Fast Large Language Model Serving with a Consumer-Grade GPU [pdf]

See how to search.

Submit link to: