NeurIPS 2026 · LCFM & DiffuLM Workshops · Under review at ICLR 2027
Machine learning & systems
Pranshu
Chaturvedi
Hi there! I recently graduated from Stanford University with a fully funded master’s in Computer Science (AI concentration) and Distinction in Research. Advised by Percy Liang, I worked on the Marin project, an open-development lab training frontier open-source language models. Check out our 535B MoE hero run here! (Percy’s announcement) In this hero run, I worked on implementing quantile balancing, a gradient-free expert load balancing algorithm that formulates a convex optimization objective for solving the optimal router bias updates for MoE experts.
I’ve also worked on diffusion language models (block diffusion hybrids and efficient distributed context parallelism for block diffusion) and data for relational foundation models at Stanford, and on federated learning and AI in biomedicine at Argonne National Laboratory under Eliu Huerta, including secure biomedical collaboration and privacy-preserving federated learning.
At Stanford, I’ve coauthored publications with professors Azalia Mirhoseini and Jure Leskovec and was fortunate to receive mentorship from Kaiyue Wen. At UIUC, I was fortunate to be advised by professors Eliu Huerta, Gagandeep Singh, Volodymyr Kindratenko, and Huan Zhang, where I worked on problems in distributed inference (gravitational wave detection on 192 V100 GPUs) and formal verification at the intersection of programming languages techniques and neural networks. This gravitational wave work was featured in NVIDIA’s special address at ISC ’22.
Some awards I’ve received include the $10,000 Modal for Academics Compute Grant (×2) in 2026 and HPCwire’s Readers’ Choice Award in the Best Use of High Performance Data Analytics & Artificial Intelligence category, presented at Supercomputing ’22 for our work published in Nature Scientific Data.
These days, I continue to work closely with Tarun Suresh and Hangoo Kang at Stanford on ML systems and architecture.
Publications
COLM · Non-AR Workshop · Under review at ICLR 2027
Training Hybrid Block Diffusion Language Models with Partial Bidirectionality
Block-local bidirectionality enables exact prefix caching in Mamba–attention diffusion hybrids.
ICLR · DATA-FM Workshop
RelBench v2: A Large-Scale Benchmark and Repository for Relational Data
Larger datasets and new predictive tasks for learning directly from relational databases.
Computational and Structural Biotechnology Journal
Enabling End-to-End Secure Federated Learning in Biomedical Research on Heterogeneous Computing Environments with APPFLx
ICLR
FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices Using a Computing Power-Aware Scheduler
Compute-aware scheduling for efficient federated learning across heterogeneous clients.
IEEE Computing in Science & Engineering
Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources: A Case Study on Federated Fine-Tuning of LLaMA 2
Frontiers in Artificial Intelligence
Inference-Optimized AI and High Performance Computing for Gravitational Wave Detection at Scale
Accelerated, distributed inference for gravitational-wave detection on supercomputers.