Based on repository activity, growth velocity and community engagement.
30
Growth3/30
Activity7/25
Popularity8/25
Trust13/20
89
Stars
High
Sentiment
Votes
89
README.md
DeT and DOT
Code and datasets for
"DepthTrack: Unveiling the Power of RGBD Tracking" (ICCV2021)
"Depth-only Object Tracking" (BMVC2021)
@InProceedings{yan2021det,
author = {Yan, Song and Yang, Jinyu and Kapyla, Jani and Zheng, Feng and Leonardis, Ales and Kamarainen, Joni-Kristian},
title = {DepthTrack: Unveiling the Power of RGBD Tracking},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {10725-10733}
}
@InProceedings{yan2021dot,
title = {Depth-only Object Tracking},
author = {Yan, Song and Yang, Jinyu and Leonardis, Ales and Kamarainen, Joni-Kristian},
booktitle = {Proceedings of the British Machine Vision Conference (BMVC)},
year = {2021},
organization= {British Machine Vision Association}
}
We manually remove bad sequences, and here are totally 646 sequences (some zip files may be broken, will be updated soon) used the DenseDepth method.
Original DenseDepth outputs are in range [0, 1.0], we multiply 2^16.
Please check LaSOT for RGB images and groundtruth.
We highly recommend to generate high quality depth data from the existing RGB tracking benchmarks,
such as LaSOT,
Got10K,
TrackingNet, and
COCO.
We show the examples of generated depth here.
The first row is the results from HighResDepth for LaSOT RGB images,
the second and the third are from DenseDepth for Got10K and COCO RGB images,
the forth row is for the failure cases in which the targets are too close to the background or floor.
The last row is from DenseDepth for CDTB RGB images.
In our paper, we used the DenseDepth monocular depth estimation method.
We calculate the Ordinal Error (ORD) on the generated depth for CDTB and our DepthTrack test set, and the mean ORD is about 0.386, which is sufficient for training D or RGBD trackers and we have tested it in our works.
And we also tried the recently HighResDepth from CVPR2021, which also performs very well.
@article{alhashim2018high,
title={High quality monocular depth estimation via transfer learning},
author={Alhashim, Ibraheem and Wonka, Peter},
journal={arXiv preprint arXiv:1812.11941},
year={2018}
}
@inproceedings{miangoleh2021boosting,
title={Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging},
author={Miangoleh, S Mahdi H and Dille, Sebastian and Mai, Long and Paris, Sylvain and Aksoy, Yagiz},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={9685--9694},
year={2021}
}
The generated depth maps by using HighResDepth will be uploaded soon.
If you find some excellent methods to generate high quality depth images, please share it.
Architecture
The settings are same as that of Pytracking, please read the document of Pytracking for details.
Actually the network architecture is very simple, just adding one ResNet50 feature extractor for Depth input and then merging the RGB and Depth feature maps.
Below figures are
the feature maps for RGB, D inputs and the merged RGBD ones,
the network for RGBD DiMP50, and
RGBD ATOM.
Download
Download the training dataset and edit the path in local.py
Download the checkpoints for DeT trackers (in install.sh)
