Posts by Collection

AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning

Siddharth Singh and Abhinav Bhatele
2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2022

Exploiting sparsity in pruned neural networks to optimize large model training

Siddharth Singh and Abhinav Bhatele
2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2023

A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training

Siddharth Singh, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He, and Abhinav Bhatele
ICS '23: Proceedings of the 37th International Conference on Supercomputing, 2023

Democratizing AI: Open-Source Scalable LLM Training on GPU-Based Supercomputers

Siddharth Singh, Prajwal Singhania, Aditya Ranjan, John Kirchenbauer, Jonas Geiping, Yuxin Wen, Neel Jain, Abhimanyu Hans, Manli Shu, Aditya Tomar, Tom Goldstein, and Abhinav Bhatele
SC'24: Proceedings of the ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, 2024

Be like a Goldfish, Don’t Memorize! Mitigating Memorization in Generative LLMs

Abhimanyu Hans, John Kirchenbauer, Yuxin Wen, Neel Jain, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, and Tom Goldstein
Advances in Neural Information Processing Systems 37 (NeurIPS), 2024

Loki: Low-Rank Keys for Efficient Sparse Attention

Prajwal Singhania, Siddharth Singh, Shwai He, Soheil Feizi, and Abhinav Bhatele
Advances in Neural Information Processing Systems 37 (NeurIPS), 2024

HPC-Coder-V2: Studying Code LLMs Across Low-Resource Parallel Languages

Aman Chaturvedi, Daniel Nichols, Siddharth Singh, Abhinav Bhatele
ISC High Performance Conference 2025, 2025

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.