The project will be related to the Development of state-of-the-art object fusion logic using camera and radar-based object lists for ADAS perception. My main objective is to optimize C++ fusion logic with 3D Mapping, Kalman Filtering, and Multi-Object Tracking in ROS for rapid testing. The solutions should be production ready, fast and efficient as it will need to run constantly on an embbedded controller.

Wheel Pick and Place : In RoboTire's operations, high-priced $12,000 Zivid cameras were relied upon for their exceptional point cloud capabilities. However, my internship marked a turning point for the perception system, achieving sub-millimeter precision in wheel placement onto hubs using the more cost-effective $300 OakD-SR camera. We leveraged stereo vision to furhter apply machine learning and pointcloud processing to focus on the interested region and command the Kawasaki Robotic arm through Socket programming and AS language.

Wheel Offset : The main challenge with Wheel swapping with summer and winter tires involved understanding the wheel offset of each Tire. As the RoboTire system has the camera on the Robotic arm end-effector, we were tasked with developing an external perpception system that can determine the wheel offset of each tire. This was possible using stereo vision and python libraries like numpy to undestand the pointcloud to accurately get the depth data at a short range.

YOLOv5 segmentation : During the Internship, we worked on YOLO segmentation to get lugnut Hex shape segmentation rather than just a bounding box around the lugnuts. This segmentation will open new applications related to lugnut detection.

MASTER'S THESIS

Thermal SLAM : Evaluating Thermal Deblurring Deep Learning Models and applying it as input to various SLAM pipelines

Thermal Camera Feature Evaluation : Developed a Novel Quantitative metric for evaluating Thermal Image restoration algorithms [ Report ]

Hardware T-rig system : Backpack System for Quantitative metric analysis of Stereo Thermal and Stereo Visible cameras

SLAM Evaluations : Generate Pseudo Ground Truth in 3D space to benchmark Thermal Odometry against Visual Odometry [ Report ]

Kinodynamic Fast Planner [ code ] : This project integrates mapping, planning, and control to facilitate autonomous drone navigation. Octomap forms the foundation, constructing detailed 3D maps for obstacle avoidance. We employ Fast-Planner, customized for seamless integration with Octomap, to generate optimal paths based on user-defined Cartesian goals. Our control system, relying on the reliable PX4-Autopilot firmware, ensures precise execution of trajectories. Mavros bridges the communication gap, enabling the seamless transmission of waypoints to the Pixhawk flight controller. This comprehensive system guarantees safe, accurate, and efficient drone operations in complex environments.

Visual-Inertial Navigation [ code ] : We have used VINS-Mono a real-time SLAM framework using Intel Realsense T265, This uses an optimization-based sliding window formulation for providing high-accuracy visual-inertial Odometry. This project uses the fisheye lenses for feature detection and the integrated IMU of T265 and this is essential for GPS-denied localization where features are present. It has loop detection and global pose graph optimization which is useful for autonomous drones. An accurate outdoor localization and tracking is shown in the video on the right.

Ewok Planner [ code ] : This is a reproduction of a real-time approch to trajectory replanning for microaerial vehicles Since most trajectory generation methods assume that environment is static and have prior knowledge of the map. This study maintains informationas an occupancy grid stored in a 3D circular bufferand represent the trajectories using uniform B-splines, This ensures thattrajectory is smooth and allows time for efficient optimization.

Real-time gesture control UAV with a low resource framework [ IEEE Paper ] : This study introduces a user-friendly framework for drone interaction without technical expertise. It leverages 2D computer vision and deep learning on an affordable micro drone with an RGB camera. The system enables real-time control of the drone through gesture recognition of human poses, ensuring safety with a linear PD controller and image processing for distance management. Implementation on the Robot Operating System (ROS) was evaluated indoors and outdoors. This versatile framework can extend to control robotic arms or mobile robots.

Aerial Manipulator [ code ] : In response to the demanding and high-risk process of harvesting areca nuts from towering trees, this project proposes the development of a UAV equipped with a 2DOF robotic arm featuring a prismatic arm designed for precise branch cutting. To ensure the UAV's stability during this task, advanced control techniques, including PID and LQR, are employed. The project encompasses modeling the drone using SolidWorks and simulating it in MATLAB SimMechanics and Simulink. Moreover, it involves the creation of a mathematical model for quadrotor dynamics, which is subsequently linearized to enable the implementation of the PID and LQR controllers, ultimately revolutionizing the areca nut harvesting process.