Student opportunities

If you are interested in working in machine learning, systems and architecture feel free to reach out to me over email. If you contribute sufficiently, you can get a chance to be primary or secondary author in the paper. Currently, I am looking for students who can assist me in following projects:

Automatic Model Parallelism for PyTorch

Most frameworks such as TensorFlow, PyTorch and MxNet natively supports for automatic data parallelism across multiple GPU devices. However, for large models commonly seen in NLP or speech synthesis/recognition application, there is need to build efficient automatic model parallelism framework. Current automatic model parallelism frameworks such as GPipe or Mesh-Tensorflow requires rewrite of the models in their respective format. In this project, we would like to build an automatic model parallelism framework on top of pytorch that can efficiently partition the compute graph across multiple devices depending on a heuristic.

Programming skills: expertise in python, C and data structures and pytorch experience.

SparseDNN Challenge

Graph Challenge contest has been evolving since three years. Since 2019, graph challenge contest now encompasses large SparseDNN Challenge. This project essentially deals with writing codes on various optimizations on top of our current implementation.

Programming skills: Python, CUDA, C and data structures.

GPU Mircobenchmarking

We are building a set of measured statistics (memory bandwidth, power consumption, and others) on various generation of GPUs to understand how it has evolved over time. This project essentially involves writing microbenchmarks and creating plots for microarchitectural details using our internal libraries. The overall vision is to generate these stats to build an estimated GPU model for the future generation.

Programming skills: Python, CUDA, C and data structures.

Profiling Tools

Emerging deep learning models exhibit sparsity. However, there has been no study on the structure of sparsity in the past. We would like to build tools that can enable such a study.

Second part of the project involves efficient data visualization tool development. Here, we want to build a portfolio different representation of data using a python library.

Programming skills: Python, C and expertise in pytorch/tensorflow.

Moonshot ideas

1. Automatic model parallelism for multiple accelerators such as TPUs, FPGAs, or others.
2. Applications for the some of the research work I am doing. Check out my research overview page.