3D Object Localisation from Multi-view Image Detections (TPAMI 2017)

You can download here the code for the PAMI 2017 journal paper “3D Object Localisation from Multi-view Image Detections”. The file (code.zip) provides a demo sequence and the MATLAB functions to estimate the localisation of the objects given matched bounding box detections and camera parameters as input data.

Abstract

In this work we present a novel approach to recover objects 3D position and occupancy in a generic scene using only 2D object detections from multiple view images. The method reformulates the problem as the estimation of a quadric (ellipsoid) in 3D given a set of 2D ellipses fitted to the object detection bounding boxes in multiple views. We show that a closed-form solution exists in the dual-space using a minimum of three views while a solution with two views is possible through the use of non-linear optimisation and object constraints on the size of the object shape. In order to make the solution robust toward inaccurate bounding boxes, a likely occurrence in object detection methods, we introduce a data preconditioning technique and a non-linear refinement of the closed form solution based on implicit subspace constraints. Results on synthetic tests and on different real datasets, involving challenging scenarios, demonstrate the applicability and potential of our method in several realistic scenarios. DOI: 10.1109/TPAMI.2017.2701373

Please remember to cite our paper if you use code and dataset:

@inproceedings{rubino2017pami,

  title={3D Object Localization from Multi-view Image Detections}, 
  author={Rubino, Cosimo and Crocco, Marco and {Del Bue}, Alessio}, 
  booktitle={Pattern Analysis and Machine Intelligence (TPAMI), 2017 IEEE Transactions on}, 

  year={2017}
}