Projects, Internships & Experience(s)

Image Encoding Schemes for Vision Transformers

CSCI-GA 3033 Special Topics: Large Language \& Vision Models (NYU)

• Guided by the instructor, Prof. Saining Xie.
• Reviewed literature on image tokenization methods in a Vision Transformer. Extensively studied the techniques 'Convolutions before Tokenization', 'Vector Quantization' & 'Mixed-Resolution Tokenization'.
• Working with the CIFAR10 \& CIFAR100 datasets, observed the following: 'Convolutions before Tokenization' improves performances, but adds a substantial training cost; 'Vector Quantization' does not improve performance; 'Mixed-Resolution Tokenization' improves performance at a marginal training cost.

Adaptive SphereFormer: Dynamic Radial Windows for Better Sparse Learning

CSCI-GA 2271: Computer Vision (NYU)

• Guided by the instructor, Prof. Rob Fergus.
• Reviewed literature on state of the art LiDAR segmentation techniques, with particular emphasis on the SphereFormer (Lai et al. 2023). Reproduced results of the baseline SphereFormer and observed the memory limitations encountered.
• Proposed a modified solution which we call, 'Adaptive SphereFormer' that improves the state of the art LiDAR 3D Segmentation accuracy, by a Mean Intersection over Union (mIoU) of 1%.

Graduate Employee Adjunct

Courant Institute of Mathematical Sciences, NYU

• Lead recitations for 4 sections of MATH-UA.121: Calculus 1.
• Hold office hours, proctor midterms & finals.
• Help students understand and practice the material taught during lecture.
• Communicate and coordinate with instructor to prepare and grade materials such as quizzes and worksheets.

Systems and Applications Engineer

Cirrus360 Corp.

• Directly assisted the team of Co-Founders in the development, optimization & release of an experimental Domain Specific Language with applications in the setup of private wireless networks.
• Used Python's multiprocessing tool to enable parallel execution of a Z3-based Constraint Solver resulting in a logarithmic speedup as the number of CPU cores is increased.
• Developed a Flask-hosted Automatic Speech Recognition (ASR) tool using OpenAI's Whisper API & TorchAudio to transcribe audio files, as a sample web-app deployed on this experimental network to demonstrate its feasibility.

Optimizing Diffusion Models for Image De-Noising

CSCI-GA 2565: Machine Learning (NYU)

• Guided by the instructor, Prof. Rajesh Ranganath.
• Reviewed literature on generative models (VAEs, GANs etc), with special emphasis on diffusion models.
• Reproduced benchmarks of Denoising Diffusion Probabilistic Model (DDPM) to set a baseline.
• Modified & trained diffusion models to accept noisy images as input, and reported effect of input noise level, diffusion input step and diffusion cycles on the de-noising output of DDPM. Also implemented & trained a class-conditioned diffusion model.
• Trained a diffusion model to re-generate images with missing pixels, essentially behaving as a Masked Auto-Encoder.

Multi-Agent RL with Unity (SoccerTwos)

DS-GA 3001: Special Topics: Reinforcement Learning (NYU)

• Guided by the instructor, Prof. Jeremy Curuksu.
• Used Unity's ML-Agents PettingZoo Wrapper to train and compare policies on the SoccerTwos environment (2v2 soccer).
• Used Self-Play to train agent against a former copy of this agent. Experimented with PPO (Proximal Policy Optimization), SAC (Soft Actor Critic) and POCA (Posthumous Credit Assignment) policies.
• Observed that POCA trained agent learned collaborative 'attack' and 'defence' strategies unaided by human feedback.

Forecasting of Extreme-Causing Weather Patterns Using Deep Learning

Centre for Excellence in Artificial Intelligence (CAI), IIT-KGP

• Supervised by Dr. Adway Mitra, performed literature review, with specific focus on Capsule Neural Networks and Analog Weather Forecasting.
• Used surface temperature (T2m) and geopotential height at 500 mbar (Z500) (from the NCAR CESM-LENS datatset) to make analogous predictions of the onset of heat/cold waves over North America, 1-5 days ahead.
• Observed that CapsNets outperformed CNNs and logistic regression; confirmed trends observed in the paper 'Analog Forecasting of Extreme-Causing Weather Patterns Using Deep Learning' (2020, Chattopadhyay et al.)

Human Pose Estimation Intern

Kabuni Sports

• Applied pre-trained Deep Learning models for the Human Pose Estimation (HPE) task. Used extracted Pose Information to suggest improvements in a Cricket (Sport) Learner's technique, preventing inefficiency and injuries.
• Fine-tuned Torchvision's KeypointRCNN class for best accuracy in the HPE task.
• Outputs from KeypointRCNN used to generate sequential pose data from video clips.
• Trained sequence models to detect potential errors in pose and technique from generated data.

Transformational AI Intern

Ministry of Electronics and Information Technology, Government of India

Interpretable Convolutional Neural Nets

CS60021: Scalable Data Mining (IIT-KGP)

• Guided by the instructor, Prof. Sourangshu Bhattacharya.
• For the class CS60021, Scalable Data Mining, taken in Fall 2021 at IIT Kharagpur.
• Implement "Convolutional Dynamic Alignment Networks for Interpretable Classifications" (2021, Böhle et al.) using PyTorch as Deep Learning framework of choice.

NumpyNet

Neural Net using nothing but NumPy

• Layers implemented: Dense, Sigmoid, ReLU, Tanh
• Error functions implemented: Binary Cross Entropy, Mean Absolute Error, Mean Squared Error
• Optimizers implemented: Stochastic Gradient Descent

SAT-based techniques in Low Power State Assignment

AI61005: Artificial Intelligence Foundations and Applications (IIT-KGP)

• This project is done under the guidance of Professor Partha Pratim Chakrabarti and Professor Arijit Mondal of IIT Kharagpur.
• Usage of AI techniques in your departmental domain (Electrical Engineering).
• Reference research paper: http://dx.doi.org/10.1142/S0218126611007980

Boolean Algebra Solver

EC31003: Digital Electronic Circuits (IIT-KGP)

• Guided by Prof.Goutam Saha.
• For the class EC31003, Digital Electronic Circuits, taken in Fall 2020, at IIT Kharagpur.
• Design and presented the software "Boolean Algebra Solver" that solves many modern day Boolean Algebra problems like K-Map, Quine-McCluskey Algorithm, Hazard Detection and Removal, Hamming Code Generation, etc.
• C# based implementation.

Autonomous Robotics & AI Intern

Ottonomy.IO

• Assisted with the automation of last mile delivery through self-driving rovers that operate on sidewalks.
• Set up cloud-based training and deployment pipelines for Deep Learning frameworks (TensorFlow, PyTorch) utilizing AWS services: S3, EC2, SageMaker.

Auto Encoders

Practical Applications of Auto Encoders

Image processing in Python using Keras with TensorFlow 2 backend.

parkMyCar

Microsoft presents Innovation Hackathon, an initiative of IncubateIND

• Developed in 24 hours at Microsoft presents Innovation Hackathon, an initiative of IncubateIND, powered by Microsoft Azure.
• Provides real-time car park availability, also enables you to list personal land for parking.
• Developed using MapMyIndia APIs and hosted on Microsoft Azure servers.
• Tools used: HTML, CSS, JavaScript, Python (Flask).