-
Notifications
You must be signed in to change notification settings - Fork 5.8k
LaSOT-based benchmark for trackers #2516
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 4.x
Are you sure you want to change the base?
Conversation
… problem is not solved
…roblem with averaging is not solved
@l-bat, please join to review |
…raging problem not solved
I noticed one thing when comparing the results of the benchmark on Ubuntu and Windows 10: |
@ieliz, first you need to check what exactly differs - before getting a final metric there are a lot of intermediate steps - dataset loading, tracker execution, metric calculation. |
# As a default argument used name of the original dataset folder | ||
parser.add_argument("--dataset", type=str, | ||
default="LaSOTTesting", help="Full path to LaSOT") | ||
parser.add_argument("--v", dest="visualization", action='store_true', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just -v
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
May I change some parts of the benchmark for adding DaSiamRPN in the list of trackers, due to changes in the DaSiamRPN tracker sample? from dasiamrpn_tracker import DaSiamRPNTracker |
Sure! That's what we exactly wanted to do (unify both benchmarks: for DaSiam and for the rest of trackers into one) |
…in the same directory (as for GOTURN)
Please unite all the numbers into a single table (seems to me that we need to run DaSiam again as a part of new script) |
I ran the DaSiamRPN tracker as a part of the new benchmark and compare results with the results of the integrated benchmark (in the description of the PR). Results for 'coin-3' video from the Lasot dataset are equal. |
import argparse | ||
import os | ||
|
||
from opencv.samples.dnn.dasiamrpn_tracker import DaSiamRPNTracker |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to suggest to use this construction:
import sys
path = cv.samples.findFile('samples/dnn/dasiamrpn_tracker.py')
sys.path.append(os.path.dirname(path))
from dasiamrpn_tracker import DaSiamRPNTracker
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
jenkins cn please retry a build |
Added Python3 script with a benchmark for trackers.
LaSOT paper: https://arxiv.org/abs/1809.07845
TrackingNet paper: https://arxiv.org/abs/1803.10794
TrackingNet repo: https://github.com/SilvioGiancola/TrackingNet-devkit/blob/master/metrics.py
For every tracker was used a particular rate of re-initialization (measured in frames).
UPD 07.07.2020:
Current values for the LaSOT dataset (testing part) on Ubuntu 18.04:
GOTURN still has some memory issues. The issue is reported.
DaSiamRPN results for the LaSOT dataset (testing part) on Ubuntu 18.04:
Version of the benchmark for DaSiamRPN:
UPD 29.07.2020:
Results for GOTURN:
Links to fixes for GOTURN will be provided here soon.
UPD 14.08.2020:
Link to PR with fixes for GOTURN tracker.
Table with all results:
UPD 07.09.2020:
Pull request for GOTURN fixes is merged with the test.