Curriculum Vitae

Anuj Jitendra Diwan

First Year Computer Science PhD at UT Austin

Q anuj.diwan@utexas.edu • Œ ajd12342.github.io • 7 anuj_diwan

‡ ajd12342

Education
University of Texas at Austin Austin, USA
PhD in Computer Science Aug 2021 - Present
Co-advised by Prof. David Harwath and Prof. Eunsol Choi
Indian Institute of Technology Bombay Mumbai, India
B.Tech in Computer Science and Engineering with Honors Jul 2017 - Aug 2021
Minor in Applied Statistics and Informatics
B.Tech (Major) CPI: 9.75/10 , Minor CPI: 10.00/10

Publications
1. Title: Reduce and Reconstruct: ASR for Low-Resource Phonetic Languages
Authors: Anuj Diwan, Preethi Jyothi
3 Shortlisted for the ISCA Best Student Paper Award 2021
INTERSPEECH 2021 [paper][slides]
2. Title: Multilingual and code-switching ASR challenges for low resource Indian languages
Authors: Anuj Diwan, Rakesh Vaideeswaran, Sanket Shah, Ankita Singh, Srinivasa Raghavan et.al.
INTERSPEECH 2021 [paper][code]
3. Title: Low Resource ASR: The surprising effectiveness of High Resource Transliteration
Authors: Shreya Khare*, Ashish Mittal*, Anuj Diwan*, Sunita Sarawagi, Preethi Jyothi, Samarth Bharadwaj
INTERSPEECH 2021 [paper][slides]

Internships
Big Data Experience Lab, Adobe Research Bangalore, India
Research Intern, Advisors: Sunav Choudhary, Subrata Mitra Apr - Jul 2020
Learning to Learn with Interruptions

INMA, ICTEAM, UCLouvain Louvain-la-Neuve, Belgium

Research Intern, Advisor: Prof. Pierre-Antoine Absil May - Jul 2019
Graph-regularized Matrix Completion using Riemannian manifolds [report][code]

Research Experience
Multilingual and code-switching ASR challenges for low resource Indian languages [paper][code]
B.Tech Thesis Project, Advisors: Prof. Preethi Jyothi Jan - Apr 2021
○␣ Served on the Shared Task Committee for the Interspeech Special Session on Multilingual and Code-Switching ASR.
○␣ Released ≈ 600 hours of labelled speech data in 7 Indian languages for multilingual and code-switching ASR tasks.
○␣ Wrote a technical paper describing the datasets and baselines released. Paper accepted at INTERSPEECH 2021.

Using Transliteration-based Transfer to improve Low-Resource ASR [paper][slides]

B.Tech Thesis Project, Advisors: Prof. Preethi Jyothi, Prof. Sunita Sarawagi Jul 2020 - Apr 2021
○␣ Worked on improving multilingual low-resource speech recognition in collaboration with IBM Research.
○␣ Proposed a new transfer learning strategy that pretrains using speech in a high-resource language with its text
transliterated to the low-resource language. Evaluated the approach for 6 Indian languages using two Transformer-
based ASR systems, attaining relative WER reductions of up to 8.2%. Paper accepted at INTERSPEECH 2021.

Reduce and Reconstruct: ASR for Low-Resource Phonetic Languages [paper][slides]

RnD Project, Advisor: Prof. Preethi Jyothi, IIT Bombay Jan 2020 - Apr 2021
○␣ Developed a novel algorithm for improving low-resource speech recognition that first makes a linguistically inspired
reduction followed by a reconstruction into the original vocabulary exploiting a larger textual context.
○␣ Showed improvements of 7% relative test WER on 2 Indian languages. Paper accepted at INTERSPEECH 2021.

1/3
Curriculum Vitae. Anuj Jitendra Diwan (anuj.diwan@utexas.edu)

Bridging Scripts by Grounding in Speech

B.Tech Thesis Project, Advisors: Prof. Preethi Jyothi, Prof. Sunita Sarawagi Jan - Jul 2021
○␣ Developed a novel round-trip transliteration system from a high-resource language A to a low-resource language B.
○␣ It requires no bilingual parallel data, unlike prior techniques; only requires monolingual speech data in both languages.

Stem2Morph: Low-Resource Morphological Inflection [slides][code]

Course Project (Natural Language Processing), Advisor: Prof. Pushpak Bhattacharya Jan - May 2021
○␣ Reimplemented the ‘Pushing the Limits of Low-Resource Morphological Inflection’ paper’s original Dynet implementation
in Pytorch. The new implementation supports GPU training and all Pytorch-supported model architectures.
○␣ Showed that transfer learning using related languages helps. Built a user-friendly demo to interact with the model.

Negative Interference in Multilingual Models and Code-Switching [slides]

Course Project (Advanced Machine Learning), Advisor: Prof. Sunita Sarawagi Jan - May 2021
○␣ Showed that pretraining a bilingual model on Hindi and English results in worse downstream Hindi-English code-switching
task performance as compared to monolingual pretraining on either language, indicating negative interference.
○␣ Enhancements from the ‘Negative Interference in Multilingual Models’ paper helped one of our two downstream tasks.

Anuvaadya: Instrumental Music Translation [report][code]

Course Project (Automatic Speech Recognition), Advisor: Prof. Preethi Jyothi Aug - Nov 2019
○␣ Implemented the ‘A Universal Music Translation Network’ paper for music translation from one instrument to another
using a CNN-based WaveNet Autoencoder and extended it for Indian instruments.
○␣ Extended the paper’s idea to arbitrary length sequence data by implementing LSTM RNN autoencoders in
Pytorch inspired by the novel elements of the paper (attention and a domain confusion network).

Learning to Learn with Interruptions

Research Intern, Advisors: Sunav Choudhary, Subrata Mitra, Adobe Research Apr - Jul 2020
○␣ Designed a novel reinforcement learning-based scheduler that can run ML jobs on interruptible cloud-based VMs.
○␣ The Python implementation consists of a job utility estimator and other predictive modules interfacing with AWS.

Improving Video Visual Relationship Grounding using Spatio-Temporal Graphs

RnD Project, Advisor: Prof. Ganesh Ramakrishnan, IIT Bombay Jul - Dec 2020
○␣ Working on improving the newly-introduced weakly-supervised Visual Relationship Grounding task for videos by
using a graph-based attention pooling technique over the video represented as a spatio-temporal graph.

Graph-regularized Matrix Completion using Riemannian manifolds [report][code]

Research Intern, Advisor: Prof. Pierre-Antoine Absil, UCLouvain May - Jul 2019
○␣ Implemented highly efficient Riemannian optimization algorithms using C++, MATLAB and LAPACK to solve a
matrix completion i.e. recommender system problem using similarity graph-based regularization techniques.
○␣ Added new functionality to ROPTLIB, an open-source Riemannian optimization library.

Major Academic Achievements

○␣ Awarded the Excellence in Teaching Assistantship Award 2021 by the IIT Bombay Computer Science
Department for my TAship duties for Automatic Speech Recognition taught by Prof. Preethi Jyothi.
○␣ Awarded the Excellence in Research Award 2021 by the IIT Bombay Computer Science Department

for my B.Tech Thesis Project titled ‘Multilingual, Code-switching and Low-Resource NLP and ASR’
○␣ Awarded the IIT Bombay Undergraduate Research Award (URA 01), Spring 2020 for conducting

exceptional research.
○␣ Awarded 7 AP grades for outstanding performance (top 1%) in 7 courses: Theoretical ML, Speech Recog-

nition, Data Analysis, Software Systems Lab, Linear Algebra, Electromagnetism and Chemistry.
○␣ Awarded the Institute Academic Prize for Academic Excellence by IIT Bombay for 2017-18.

○␣ Among top 35 in India selected for the International Mathematical Olympiad Training Camp in 2016.

○␣ Among top 50 students selected to attend the NLU track of the Google Research AI Summer School 2020.

○␣ Achieved All India Rank 118 in JEE Advanced 2017 and All India Rank 197 in JEE Mains 2017.

○␣ Awarded the National Talent Search (NTSE) 2015 and KVPY 2016 Scholarships by the Govt. of India.

Programming Skills
Programming Languages & Tools: C/C++, Python, Java, Bash, MATLAB, Javascript, Git, LATEX, Beamer.
AI/ML Libraries: Pytorch, Tensorflow, Numpy, Fairseq, ESPNet, Kaldi, OpenFST, Huggingface Transformers.

2/3
Curriculum Vitae. Anuj Jitendra Diwan (anuj.diwan@utexas.edu)

Positions of Responsibility
Member of the Shared Task Committee for an Interspeech 2021 Special Session on
Multilingual and code-switching ASR challenges for low resource Indian languages Jan - Aug 2021
Responsible for coming up with interesting tasks for the challenge and preparing the challenge datasets and baselines.
Teaching Assistant for Automatic Speech Recognition (CS 753)
Instructor: Prof. Preethi Jyothi Jan - May 2021
Responsible for preparing interesting programming assignments, solving doubts, helping with problem sets and grading
examinations for students taking the graduate-level Automatic Speech Recognition course
Teaching Assistant for Logic for Computer Science (CS 228, CS 228(M))
Instructors: Prof. Ashutosh Gupta and Prof. Krishna S. Jul 2019 - Jun 2020
Responsible for teaching, helping with problem sets, preparing programming labs and grading examinations for students
taking the CS 228 and CS228(M) courses. These courses teach propositional logic and automata theory.

Teaching Assistant for Computer Programming and Utilization (CS 101)

Instructors: Prof. Kameswari Chebrolu and Prof. Varsha Apte Jul - Dec 2018
Created introductory computer science problems for Bodhitree, an online educational repository by IIT Bombay
accessible to all students in the country, with the aim of making quality education accessible to all.

Mentor for Summer of Science

For Maths and Physics Club, IIT Bombay Apr - Jun 2020
Mentored three first-year students interested in AI and Logic by providing helpful resources and resolving doubts.

Other Selected Projects

Natural Language Processing for Chatbots [report][code-1][code-2]
Reading Project, Advisor: Prof. Kameswari Chebrolu, IIT Bombay Dec 2018
○␣ Studied seq2seq classification models for NLP and feature engineering using Bag-of-Words, Word2Vec, etc.
○␣ Wrote simple chatbots in Python and NodeJS and deployed them on Facebook Messenger via API endpoints.

SMALLBAG: Connecting Local Groceries [link]

Project, Indian Scientists’ Response to COVID-19 (ISRC), Prof. Bhaskaran Raman Apr - May 2020
○␣ Developed part of the SMALLBAG Android app, an app that enables customers to order online from local grocery
stores using an SMS-based e-token system that replaces physical queues with virtual queues.

Battleship Web Game [report][code]

Course Project (Software Systems Lab), Advisor: Prof. Soumen Chakrabarti Aug - Nov 2018
○␣ Implemented the arcade game Battleship, supporting score leaderboards, using Django, ReactJS and Redis.
○␣ Supported multiplayer timed matches with live chat, gameplay restoration on connection losses, etc.

Enigma Machine with Prefix-matching Cryptanalysis [report][code]

Course Project (Programming Abstractions & Paradigms), Advisor: Prof. Amitabha Sanyal Jan - Apr 2019
○␣ Faithfully emulated the German Enigma machine in Racket, a language based on the Lisp dialect Scheme.

Relevant Courses
Artificial Intelligence: Advanced ML, Deep Learning for NLP, Foundations of Intelligent and Learning Agents,
Theoretical ML, Automatic Speech Recognition, AI and ML, Medical Image Computing, Data Analysis
Statistics & Mathematics: Optimization, Regression Analysis, Statistical Inference, Stochastic Processes,
Probability Theory, Numerical Analysis, Linear Algebra, Differential Equations, Calculus
Computer Science: Implementation of Programming Languages, Automata Theory, Database and Information
Systems, Computer Architecture, Operating Systems, Abstractions and Paradigms for Programming, Design and
Analysis: Data Structures and Algorithms, Computer Networks, Digital Logic Design, Computer Programming

Extracurricular Activities
○␣ Successfully completed one year of learning Tabla under NSO, IIT Bombay. (2018)
○␣ Wrote an article titled ‘Summer Bucket List’ published in Insight, the Institute newsletter. (2018)
○␣ Placed 3rd in Wordgames and won the Overall Championship at the 3rd Inter-IIT Cultural Meet. (2018)
○␣ Won 1
st Prize in the Freshman Category in Matholympics, an intra-IIT Math competition. (2017)

3/3