*: Equal contribution.

• Piotr Teterwak, Soren Nelson, Nikoli Dryden, Dina Bashkirova, Kate Saenko, and Bryan Plummer. “Learning to Compose SuperWeights for Neural Parameter Allocation Search.” WACV 2024.
• Julia Bazińska, Andrei Ivanov, Tal Ben-Nun, Nikoli Dryden, Maciej Besta, Siyuan Shen, and Torsten Hoefler. “Cached Operator Reordering: A Unified View for Fast GNN Training.” arXiv 2023.
• Andrei Ivanov, Nikoli Dryden, Tal Ben-Nun, Saleh Ashkboos, and Torsten Hoefler. “STen: Productive and Efficient Sparsity in PyTorch.” arXiv 2023.
• Maciej Besta, Patrick Iff, Florian Scheidl, Kazuki Osawa, Nikoli Dryden, Michal Podstawski, Tiancheng Chen, and Torsten Hoefler. “Neural Graph Databases.” Learning on Graphs Conference 2022.
• Nikoli Dryden and Torsten Hoefler. “Spatial Mixture-of-Experts.” NeurIPS 2022.
• Saleh Ashkboos, Langwen Huang, Nikoli Dryden, Tal Ben-Nun, Peter Dueben, Lukas Gianinazzi, Luca Kummer, and Torsten Hoefler. “ENS-10: A Dataset for Post-Processing Ensemble Weather Forecasts.” NeurIPS 2022.
• Maciej Besta, Raphael Grob, Cesare Miglioli, Nicola Bernold, Grzegorz Kwasniewski, Gabriel Gjini, Raghavendra Kanakagiri, Saleh Ashkboos, Lukas Gianinazzi, Nikoli Dryden, and Torsten Hoefler. “Motif Prediction with Graph Neural Networks.” SIGKDD 2022.
• Oliver Rausch, Tal Ben-Nun, Nikoli Dryden, Andrei Ivanov, Shigang Li, and Torsten Hoefler. “A Data-Centric Optimization Framework for Machine Learning.” ICS 2022.
• Bryan Plummer*, Nikoli Dryden*, Julius Frost, Torsten Hoefler, and Kate Saenko. “Neural Parameter Allocation Search.” ICLR 2022.
• Lukas Gianinazzi, Maximilian Fries, Nikoli Dryden, Tal Ben-Nun, and Torsten Hoefler. “Learning Combinatorial Node Labeling Algorithms.” arXiv 2021.
• Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. “Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks.” Journal of Machine Learning Research 2021.
• Nikoli Dryden, Roman Böhringer, Tal Ben-Nun, and Torsten Hoefler. “Clairvoyant Prefetching for Distributed Machine Learning I/O.” Supercomputing 2021.
• Andrei Ivanov*, Nikoli Dryden*, Tal Ben-Nun, Shigang Li, and Torsten Hoefler. “Data Movement Is All You Need: A Case Study on Optimizing Transformers.” MLSys 2021. (Outstanding Paper)
• Peter Grönquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Shigang Li, and Torsten Hoefler. “Deep Learning for Post-Processing Ensemble Weather Forecasts.” Philosophical Transactions of the Royal Society A, 2021.
• Yosuke Oyama, Naoya Maruyama, Nikoli Dryden, Erin McCarthy, Peter Harrington, Jan Balewski, Satoshi Matsuoka, Peter Nugent, and Brian Van Essen. “The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism.” IEEE Transactions on Parallel and Distributed Systems, 2020.
• Shigang Li, Tal Ben-Nun, Giorgi Nadiradze, Salvatore Di Girolamo, Nikoli Dryden, Dan Alistarh, and Torsten Hoefler. “Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging.” IEEE Transactions on Parallel and Distributed Systems, 2020.
• Peter Grönquist, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Luca Lavarini, Shigang Li, and Torsten Hoefler. “Predicting Weather Uncertainty with Deep Convnets.” ML4PS Workshop at NeurIPS 2019.
• Nikoli Dryden, Naoya Maruyama, Tim Moon, Tom Benson, Marc Snir, and Brian Van Essen. “Channel and Filter Parallelism for Large-Scale CNN Training.” Supercomputing 2019.
• Nikoli Dryden*, Naoya Maruyama*, Tom Benson, Tim Moon, Marc Snir, and Brian Van Essen. “Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism.” IPDPS 2019.
• Nikoli Dryden, Naoya Maruyama, Tim Moon, Tom Benson, Andy Yoo, Marc Snir, and Brian Van Essen. “Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems.” MLHPC 2018.
• Chen Wang, Nikoli Dryden, Franck Cappello, and Marc Snir. “Neural network based silent error detector.” Cluster 2018. (Best Paper)
• Roshan Dathathri, Gurbinder Gill, Loc Hoang, Hoang-Vu Dang, Alex Brooks, Nikoli Dryden, Andrew Lenharth, Marc Snir, Keshav Pingali. “Gluon: A Communication Optimizing Framework for Distributed Heterogeneous Graph Analytics.” PLDI 2018.
• Hoang-Vu Dang, Roshan Dathathri, Gurbinder Gill, Alex Brooks, Nikoli Dryden, Andrew Lenharth, Loc Hoang, Keshav Pingali, and Marc Snir. “A Lightweight Message Passing Runtime for Distributed Graph Analytics.” IPDPS 2018.
• Sam Adé Jacobs, Nikoli Dryden, Roger Pearce, and Brian Van Essen. “Towards Scalable Parallel Training of Deep Neural Networks.” MLHPC 2017.
• Nikoli Dryden, Tim Moon, Sam Ade Jacobs, and Brian Van Essen. “Communication Quantization for Data-parallel Training of Deep Neural Networks.” MLHPC 2016.
• Alex Brooks, Hoang-Vu Dang, Nikoli Dryden, and Marc Snir. “PPL: An abstract runtime system for hybrid parallel programming.” ESPM2 2015.
• Nikoli Dryden. “PGDB: A Debugger for MPI Applications.” XSEDE 2014.