 |
Research Scientist, Waymo |
Bio
I am currently a research scientist at Waymo. Previously, I received my Ph.D. from Georgia Tech, where I was advised by James Hays and Frank Dellaert. Prior to joining Georgia Tech, I completed my Bachelor’s and Master’s degrees in Computer Science at Stanford University, specializing in artificial intelligence.
Research
My interests resolve around machine learning for robotics and autonomy. Past and present research areas have included image understanding, 3D perception, SLAM, and simulation. I’ve been involved in research for self-driving vehicle development since 2017. Machine learning and computer vision for robot autonomy currently present (and will continue to present) enormous benefits for people all over the world, with implications for safer transportation and safer workplaces.
News
- July 2022: Our paper SALVe: Semantic Alignment Verification for Floorplan Reconstruction from Sparse Panoramas has been accepted to ECCV 2022. [Project Page] [Paper]
- March 2022: I have joined Waymo Research as a research scientist.
- March 2022: I defended my PhD Thesis. Many thanks to Simon Lucey, Zsolt Kira, and Cedric Pradalier who joined my co-advisors, James Hays and Frank Dellaert, on my committee. The title of my thesis is “Deep Learning for Building and Validating Geometric and Semantic Maps” – coming soon to Arxiv.
- March 2022: The Trust, but Verify (TbV) Dataset is released. This is the first public dataset for HD map change detection, an important problem for self-driving vehicle perception. Available for download here.
- March 2022: The Argoverse 2.0 Datasets are released – featuring a Sensor Dataset (1000 scenarios w/ imagery, LiDAR, and paired HD maps), a Motion Forecasting Dataset (250K tracked scenarios w/ HD maps), and a LiDAR Dataset (20K thirty sec. scenarios, with LiDAR data and paired HD maps). Available for download here. [Devkit]
- January 2022: An updated journal-length version of our MSeg work has been accepted to TPAMI and is now available on ArXiv here.
- December 2021: I am on the industry job market for Spring 2022.
- October 2021: Two papers are accepted to NeurIPS 2021 – Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection and also Argoverse 2.0: Next Generation Datasets for Self-Driving Perception and Forecasting.
- September 2021: Code is released for our Trust but Verify HD map change detection work – public dataset release coming soon.
- June 2021: Winners have been announced for the CVPR 2021 Argoverse competitions. The talk is available here.
- May 2021: I am interning with Sing Bing Kang at Zillow Research, working on indoor scene understanding.
- April 2021: We are pleased to announce two new Argoverse competitions – Stereo and Motion Forecasting – at the CVPR 2021 Workshop on Autonomous Driving. Challenges are open through June 13th, 2021, and feature a total of $8,000 in prizes ($2000 for each first place winner, and $1000 for honorable mentions). We’ve put together a Jupyter notebook here to get started with the Stereo data with SGM.
- November 2020: I gave an invited talk at the ROS World 2020 virtual conference, discussing our MSeg work from CVPR. A recording is available here. Our lightweight 480p MSeg model can run at 25 fps in Pytorch.
- August 2020: We are the runner-up in the 2020 Robust Vision Challenge (semantic segmentation track), without training on four of the seven RVC test datasets (zero-shot, cross-dataset generalization). See our talk at the ECCV 2020 RVC workshop here.
- June 2020: Watch an excellent presentation from MachinesCanSee on our recent MSeg work, presented by Dr. Vladlen Koltun.
- June 2020: The CVPR 2020 WAD Argoverse competitions have concluded. Congratulations to the very impressive submissions from the winners. You can watch the results presentation here, or the summary presented at ICML 2020.
- April 2020: We are pleased to announce two Argoverse Competitions at the CVPR 2020 Workshop on Autonomous Driving. Argo AI is offering $5,000 in prizes for Motion Forecasting and 3D tracking methods. I’ve open-sourced my 3d tracking code that is currently 1st place on the leaderboard. Please consider participating! The competitions will remain open until June 10, 2020.
- April 2020: Our MSeg paper has been accepted to CVPR 2020 and took first place on WildDash. Pretrained models available here, data available here, and a Colab to try our demo on your own images and videos.
Teaching
Aside from research, another passion of mine is teaching. I enjoy creating teaching materials for topics related to computer vision, a field which relies heavily upon numerical optimization and statistical machine learning tools. A number of teaching modules I’ve written can be found below:
Module 1: Linear Algebra
Linear Algebra Without the Agonizing Pain
Necessary Linear Algebra Overview
Fast Nearest Neighbors
Vectorizing nearest neighbors (with no for-loops!)
Module 2: Numerical Linear Algebra
Direct Methods for Solving Systems of Linear Equations
backsubstitution and the LU, Cholesky, QR factorizations
Conjugate Gradients
large systems of equations, Krylov subspaces, Cayley-Hamilton Theorem
Least-Squares
QR decomposition for least-squares, modified Gram-Schmidt, GMRES
Module 3: SVMs and Optimization
The Kernel Trick
poorly taught but beautiful piece of insight that makes SVMs work
Gauss-Newton Optimization in 10 Minutes
Derivation, Trust-Region Variant (Levenberg-Marquardt), Numpy Implementation
Convex Optimization Without the Agonizing Pain
Constrained Optimization, Lagrangians, Duality, and Interior Point Methods
Subgradient Methods in 10 Minutes
Convex Optimization Part II
Module 4: State Estimation
What is State Estimation? and the Bayes Filter
linear dynamical systems, bayes rule, bayesian estimation, and filtering
Lie Groups and Rigid Body Kinematics
SO(2), SO(3), SE(2), SE(3), Lie algebras
Module 5: Geometry and Camera Calibration
Stereo and Disparity
disparity maps, cost volume, MC-CNN
Epipolar Geometry and the Fundamental Matrix
simple ideas that are normally poorly explained
Visual Odometry
The Essential matrix, Nister's 5-Pt Algorithm, and epipolar constraint derivation
Iterative Closest Point
registration, Sim(3) optimization, simple derivations and code examples
Module 6: Reinforcement Learning
Policy Gradients
intuition and simple derivations of REINFORCE, TRPO
Module 7: Convolutional Neural Networks
Backprop through a Conv Layer
Deriving Backprop through convolution to either the kernel weights or inputs
Generative Adversarial Networks (GANs)
Deriving minimax and non-saturating losses, DCGAN implementation
PyTorch Tutorial
PyTorch tensor operations, initializing CONV layers, groups, custom modules
JAX Tutorial
Intro to JAX, optax, flax, linen, and training loops for JAX
Module 8: Geometric Data Analysis
Module 9: Message Passing Interface (MPI)