OnnxStream running TinyLlama and Mistral 7B, with CUDA support - discu.eu

Hacker News

OnnxStream running TinyLlama and Mistral 7B, with CUDA support https://github.com/vitoplantamura/OnnxStream/blob/master/assets/LLM.md 2 comments 14/1/2024

Linking pages

GitHub - vitoplantamura/OnnxStream: Lightweight inference library for ONNX files, written in C++. It can run SDXL on a RPI Zero 2 but also Mistral 7B on desktops and servers. https://github.com/vitoplantamura/OnnxStream 74 comments

Related searches:

Search whole site: site:github.com

Search title: OnnxStream running TinyLlama and Mistral 7B, with CUDA support

See how to search.

Submit link to: