LIMoE: Learning Multiple Modalities with One Sparse Mixture-of-Experts Model

Linking pages

Google AI Introduces 'LIMoE': One Of The First Large-Scale Architecture That Processes Both Images And Text Using A Sparse Mixture Of Experts - MarkTechPost https://www.marktechpost.com/2022/06/11/google-ai-introduces-limoe-one-of-the-first-large-scale-architecture-that-processes-both-images-and-text-using-a-sparse-mixture-of-experts/ 0 comments

Linked pages

[2006.16668] GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding https://arxiv.org/abs/2006.16668 35 comments
Introducing Pathways: A next-generation AI architecture https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/ 33 comments
CLIP: Connecting Text and Images https://openai.com/blog/clip/ 15 comments
ImageNet http://image-net.org/index 12 comments
https://arxiv.org/abs/2101.03961 4 comments
[2111.10050] Combined Scaling for Open-Vocabulary Image Classification https://arxiv.org/abs/2111.10050 2 comments
Transformers for Image Recognition at Scale – Google AI Blog https://ai.googleblog.com/2020/12/transformers-for-image-recognition-at.html 1 comment
Google AI Blog: Scaling Vision with Sparse Mixture of Experts https://ai.googleblog.com/2022/01/scaling-vision-with-sparse-mixture-of.html 1 comment
Good News About the Carbon Footprint of Machine Learning Training – Google AI Blog https://ai.googleblog.com/2022/02/good-news-about-carbon-footprint-of.html 0 comments
ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision – Google AI Blog https://ai.googleblog.com/2021/05/align-scaling-up-visual-and-vision.html 0 comments
More Efficient In-Context Learning with GLaM – Google AI Blog https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html 0 comments
Mixture of experts - Wikipedia https://en.wikipedia.org/wiki/Mixture_of_experts 0 comments
[2202.08906] ST-MoE: Designing Stable and Transferable Sparse Expert Models https://arxiv.org/abs/2202.08906 0 comments