- Introducing FlashTokenizer: The World's Fastest CPU Tokenizer! https://github.com/NLPOptimize/flash-tokenizer 2 comments learnpython
Linked pages
- Trie - Wikipedia https://en.wikipedia.org/wiki/Trie 126 comments
- GitHub - martinus/robin-hood-hashing: Fast & memory efficient hashtable based on robin hood hashing for C++11/14/17/20 https://github.com/martinus/robin-hood-hashing 107 comments
- PyTorch http://pytorch.org/ 100 comments
- GitHub - microsoft/BlingFire: A lightning fast Finite State machine and REgular expression manipulation library. https://github.com/microsoft/blingfire 92 comments
- GitHub Star History https://star-history.com/#microsoft/playwright&cypress-io/cypress&Date 78 comments
- GitHub - openai/tiktoken https://github.com/openai/tiktoken 74 comments
- GitHub - huggingface/tokenizers: 💥 Fast State-of-the-Art Tokenizers optimized for Research and Production https://github.com/huggingface/tokenizers 47 comments
- GitHub - google/highway: Performance-portable, length-agnostic SIMD with runtime dispatch https://github.com/google/highway 39 comments
- GitHub - rapidsai/cudf: cuDF - GPU DataFrame Library https://github.com/rapidsai/cudf 32 comments
- [1810.04805] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding https://arxiv.org/abs/1810.04805 25 comments
- Bloom filter - Wikipedia https://en.wikipedia.org/wiki/Bloom_filter 10 comments
- Aho–Corasick algorithm - Wikipedia https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm 6 comments
- Home - OpenMP https://www.openmp.org/ 0 comments
- GitHub - google/sentencepiece: Unsupervised text tokenizer for Neural Network-based text generation. https://github.com/google/sentencepiece 0 comments
- Making WAF ML models go brrr: saving decades of processing time https://blog.cloudflare.com/making-waf-ai-models-go-brr 0 comments
- GitHub - Dao-AILab/flash-attention: Fast and memory-efficient exact attention https://github.com/Dao-AILab/flash-attention 0 comments
Related searches:
Search whole site: site:github.com
Search title: GitHub - NLPOptimize/flash-tokenizer: EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING
See how to search.