Projects

Articubot extension

• Worked on extending a zero-shot sim-to-real policy for articulated object manipulation (Articubot) by online history conditioning.
• Solved a critical real-world kinematic limitations by implementing a whole-body control maneuver, synchronizing the mobile base and XArm7’s movements, to overcome the robot’s IK limits.
• Improved real-world control by integrating compliant impedance control for safe interaction and optimizing the action execution pipeline (via parallel inference, selective action dropping, and other optimizations) to greatly speedup the real-world execution.

Sound Source Localization on videos

• Developed a system that combines semantic and temporal information to localize sound sources in videos
• Audio-visual grounding by aligning a large Vison-Language Model’s embeddings with audio embeddings (similar to ”Can CLIP Help Sound Source Localization?”)
• Addresses video challenges, including unidentified frames and instance disambiguation through our custom temporal module that utilizes a audio-motion transformer

ASAC- Assisted Spatial Audio Construction

• Automatically generating spatial audio (like Dolby Atmos) for movie scenes using video understanding.
• Developed a system for complex multi-speaker environments with identity switching, and occlusions, using lightweight tracking, facial recognition and Active Speaker Detection techniques.
• Currently working on improving spatial audio generation in scenes lacking proper visual cues through advanced scene understanding.

Contract-NLI

• Proposed a two-step BERT-based architecture, improving NLI and evidence inference for legal contracts.
• Improved handling of long-term dependencies in evidence-spans, and context size limits , due to our modified architecture and ensemble approach, achieving higher accuracy than the baseline ContractNLI-BERT.

Autonomous Omni-drive robot

• Assembled a mecanum wheel robot with four Planetary Gear DC Motor driven by two Cytron Mdds30.
• Integrated and calibrated, a Realsense T265 tracking camera and a Realsense D455 depth camera, along with voxfield for obtaining ESDFs utilized for motion planning.
• Implemented several planning algorithms, including CEM, MPPI, and MPC, on an onboard Intel NUC, utilizing ESDFs for collision costs; designed and implemented a PID controller for path-following.

8 legged Klann walker robot

• Designed a walker robot with 8 klann linkage mechanims for legs, performing comprehensive kinematics and dynamics analysis with consideration for different gaits for different purposes, on Fusion360.
• Built the mechanism using laser-cut acrylic for links and chassis, with 3D-printed joints and couplers.
• Wrote a simple controller to control the two dynamixel XL330 motors using U2D2 for communication.

Object centric reconstruction

• Designed a complete pipeline for 3D reconstruction of an object using a monocular fixed camera.
• Optimized the initial 3D reconstruction with ICP, pose graph optimization, and Bundle Adjustment for efficient and accurate results; Proposed future enhancements by utilizing NeRFs to learn appearance geometry.

Single image dehazing methods

• Implementation of various haze removal methods including Bayesian defogging, Single image haze removal using Dark Channel Prior, Single image haze removal using Colour Attenuation Prior.
• Improved upon the DCP prior paper, by adding an additional preprocessing step which increases the restored image quality by utilizing CLAHE; Demonstrated their use in stream tasks like object detection.
• Extensively benchmarked these algorithms along with other classical methods without a degradation model, and analyised their performance.

Y86 ISA Processor

• Designing and implementing sequential and pipelined Y86 ISA Processor in iverilog.
• proper handling of data, structural, and control hazards.
• A basic compiler using python for converting Y86-64 assembly code to machine code for that processor.

Teaching