General and Scalable Parallelization for Neural Networks

Linking pages

Linked pages

TensorFlow http://tensorflow.org/ 440 comments
LaMDA: our breakthrough conversation technology https://blog.google/technology/ai/lamda/ 149 comments
[1706.03762] Attention Is All You Need https://arxiv.org/abs/1706.03762 145 comments
[1701.06538] Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer https://arxiv.org/abs/1701.06538 125 comments
GitHub - google/jax: Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more https://github.com/google/jax 99 comments
[2006.16668] GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding https://arxiv.org/abs/2006.16668 35 comments
Introducing Pathways: A next-generation AI architecture https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/ 33 comments
Introducing GPipe, an Open Source Library for Efficiently Training Large-scale Neural Network Models – Google AI Blog http://ai.googleblog.com/2019/03/introducing-gpipe-open-source-library.html 12 comments
Computer vision - Wikipedia https://en.wikipedia.org/wiki/Computer_vision 10 comments
Transformer: A Novel Neural Network Architecture for Language Understanding – Google AI Blog https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html 3 comments
[2106.05974] Scaling Vision with Sparse Mixture of Experts https://arxiv.org/abs/2106.05974 2 comments
[2105.04663] GSPMD: General and Scalable Parallelization for ML Computation Graphs https://arxiv.org/abs/2105.04663 1 comment
Google Research: Looking Back at 2020, and Forward to 2021 – Google AI Blog https://ai.googleblog.com/2021/01/google-research-looking-back-at-2020.html 0 comments
Google wins MLPerf benchmarks with TPU v4 | Google Cloud Blog https://cloud.google.com/blog/products/ai-machine-learning/google-wins-mlperf-benchmarks-with-tpu-v4 0 comments
MetNet-2: Deep Learning for 12-Hour Precipitation Forecasting – Google AI Blog https://ai.googleblog.com/2021/11/metnet-2-deep-learning-for-12-hour.html 0 comments
Ultra-High Resolution Image Analysis with Mesh-TensorFlow – Google AI Blog https://ai.googleblog.com/2020/02/ultra-high-resolution-image-analysis.html 0 comments
Cloud TPU v4 Pods, large model training, MLPerf v1.1 | Google Cloud Blog https://cloud.google.com/blog/topics/tpus/google-showcases-cloud-tpu-v4-pods-for-large-model-training 0 comments
MLCommons https://mlcommons.org/ 0 comments
XLA: Optimizing Compiler for Machine Learning | TensorFlow https://www.tensorflow.org/xla 0 comments