Camera Calibration through Camera Projection Loss

Done with my MS-Thesis

I have just given my final thesis presentation. My thesis work got accepted at ICASSP 2022 last week. I am very very happy as I have got everything I wanted out of my MS. The following is about how it all started and the journey before the destination.

So, about 3.5 years back, I decided to start my MS in Pakistan instead of going to the UK as obviously it was way too expensive, and to be honest, I wasn’t sure about investing such a heavy amount on myself at that point in time just for a 1 year MS.

After my first semester, I decided to start working towards my Thesis and started looking for a supervisor, for which I sent emails to different professors and got a reply from Dr. Murtaza Taj.

I still remember it was 10th February 2019, when I met him for the first time. He gave an introduction to the lab and current projects, out of which I selected to work on Cross-View Image Retrieval as Numan Bhai, now Dr. Numan Khurshid was working on it and I thought it would be better to work with a Ph.D. student and I will have a better chance of a publication. I got lucky as we were able to publish a paper within 6 months but it wouldn't have been possible without Mohbat Bhai.

At this point in time, Ustaad Ji (Faisal Bhai) entered with his ideas about writing a paper as a first author and after some time I had decided that I will work on my own idea to go through the complete research cycle including finding a problem, proposing a solution, implementing it, and ideally publish it as the first author. By the grace of God, I am lucky that I have got what I wanted.

So, coming back to the story, I went to Dr. Murtaza and told him that I want to work on my own idea and I am really grateful that he agreed. I started working on Cyclist Detection but wasn’t able to propose a solution to detect small objects. I was working with Tsinghua-Daimler Cyclist Detection Dataset and it had a script having 2 functions to convert a 2D point to 3D. At the same time, Oneeb was working on performing basic mathematical operations using Neural Networks and I thought, how about embedding equations in a CNN for 2D to 3D point conversion and learning Camera Calibration parameters along the way. In this way, I had a new idea. As I had previously done, I went to Dr. Murtaza and told him that we should work on Camera Calibration instead of Cyclist Detection. He agreed.

I started off with a literature review and found out that no work on predicting calibration parameters by incorporating mathematical equations in a CNN existed.

Now the problem was to train a CNN for the task at hand for which we required a dataset but found out that a dataset with the required parameters and diversity in camera configurations didn’t exist so we decided to collect our own.

We used CARLA Simulator to collect the dataset and used the Tsinghua-Daimler Dataset to test our proposed approach on real data without training or transfer learning to test the generalizability of our method. To my surprise, it worked well and we sent our paper to ICASSP 2022 and got acceptance.

My paper on Camera Calibration can be accessed here while the code and dataset are available here.

This is a short story of my last 3.5 years. It was tough but necessary and totally worth it. Some of the people that were helpful along the way included but were not limited to Afaq Bhai, Fakhir Bhai, Waseem Bhai, Harooon Bhai, Wahab, Fezan, Amjad Bhai, Usman Bhai, Wadood, Sohaib, Ateeq Bhai, Zeeshan, and many many more.

That’s it for now. See you soon.