We ran the submission Docker using the following command:

docker load < teamname.tar.gz

docker run --gpus "device=0" --name teamname --rm -v \
  /home/xxx/tdsc/Test/DATA:/input:ro -v $(pwd)/predict:/predict --shm-size 8g teamname:latest

And obtained all the results using the code available on GitHub:
https://github.com/PerceptionComputingLab/TDSC-ABUS2023/tree/main/Final_Evaluation

All results are shown in the following table. The '-' symbol indicates that there are no results under the particular folder.

The segmentation task involves two metrics: the DICE coefficient and the Hausdorff distance (HD).
Initially, we eliminated teams that did not provide valid results.
Subsequently, we normalized the remaining teams' scores using min-max normalization,
calculated as (x - min(x)) / (max(x) - min(x)). Since lower HD scores are better,
we computed the final result as (1 + Norm_DICE - Norm_HD) / 2.

Some teams received an 'inf' result in the HD metric, presenting a significant challenge. Addressing these 'inf' results has been a topic of much debate. A common solution is to penalize them with a fixed value. However, selecting an appropriate value can be subjective and might compromise fairness for teams that produce robust results. Conversely, simply excluding teams with 'inf' results doesn't accurately represent the performance of all teams. After careful consideration, we've decided to rank two boards separately. In this ranking, we've replaced the 'inf' result with the worst HD score from all valid results for each case, multiplied by 105%. The normalization process and the overall calculation formula remain unchanged. It's evident that different penalization approaches can influence final rankings significantly. Hence, we've designated the aforementioned leaderboard (which employs elimination of the 'inf' scores) as the primary board. We'll provide certificates based on this primary leaderboard, with no associated cash rewards.

The evaluation of the detection performance utilizes the Free-Response Receiver Operating Characteristic (FROC) metric.
The FROC performance is presented in terms of sensitivities at different false positive (FP) levels. Specifically,
the average sensitivity at FP rates of 0.125, 0.25, 0.5, 1, 2, 4, and 8
is employed as the primary evaluation metric for assessing the detection performance.
These values are subsequently subjected to min-max normalization.

The overall performance is determined by considering only the teams that have submitted valid results for all metrics.
We apply similar normalization methods as before, but exclusively for teams with valid results.
The final overall result is obtained by:
(1 + Norm_DICE - Norm_HD) / 2 + (Norm_ACC + Norm_AUC) / 2 + Norm_FROC

Similar to the Segmentation task, we've substituted the 'inf' result with the worst HD score from all valid results for each case, multiplied by 105%. This allows us to rank teams that have an 'inf' result. Please note, this board will only offer certificate rewards too.