Danish Fungi 2020 - Not Just Another Image Recognition Dataset
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Danish Fungi 2020 - Not Just Another Image Recognition Dataset. / Picek, Lukas; Sulc, Milan; Matas, Jiri; Jeppesen, Thomas S.; Heilmann-Clausen, Jacob; Læssøe, Thomas; Frøslev, Tobias.
Proceedings - 2022 IEEE Winter Conference on Applications of Computer Vision, WACV 2022. ed. / Lisa O'Conner. IEEE, 2022. p. 3281-3291.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Danish Fungi 2020 - Not Just Another Image Recognition Dataset
AU - Picek, Lukas
AU - Sulc, Milan
AU - Matas, Jiri
AU - Jeppesen, Thomas S.
AU - Heilmann-Clausen, Jacob
AU - Læssøe, Thomas
AU - Frøslev, Tobias
N1 - Publisher Copyright: © 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - We introduce a novel fine-grained dataset and bench-mark, the Danish Fungi 2020 (DF20). The dataset, constructed from observations submitted to the Atlas of Danish Fungi, is unique in its taxonomy-accurate class labels, small number of errors, highly unbalanced long-tailed class distribution, rich observation metadata, and well-defined class hierarchy. DF20 has zero overlap with ImageNet, al-lowing unbiased comparison of models fine-tuned from publicly available ImageNet checkpoints. The proposed evaluation protocol enables testing the ability to improve classification using metadata - e.g. precise geographic location, habitat, and substrate, facilitates classifier calibration testing, and finally allows to study the impact of the device settings on the classification performance. Experiments using Convolutional Neural Networks (CNN) and the recent Vision Transformers (ViT) show that DF20 presents a challenging task. Interestingly, ViT achieves results su-perior to CNN baselines with 80.45% accuracy and 0.743 macro F1 score, reducing the CNN error by 9% and 12% respectively. A simple procedure for including metadata into the decision process improves the classification accuracy by more than 2.95 percentage points, reducing the error rate by 15%. The source code for all methods and experiments is available at https://sites.google.com/view/danish-fungi-dataset.
AB - We introduce a novel fine-grained dataset and bench-mark, the Danish Fungi 2020 (DF20). The dataset, constructed from observations submitted to the Atlas of Danish Fungi, is unique in its taxonomy-accurate class labels, small number of errors, highly unbalanced long-tailed class distribution, rich observation metadata, and well-defined class hierarchy. DF20 has zero overlap with ImageNet, al-lowing unbiased comparison of models fine-tuned from publicly available ImageNet checkpoints. The proposed evaluation protocol enables testing the ability to improve classification using metadata - e.g. precise geographic location, habitat, and substrate, facilitates classifier calibration testing, and finally allows to study the impact of the device settings on the classification performance. Experiments using Convolutional Neural Networks (CNN) and the recent Vision Transformers (ViT) show that DF20 presents a challenging task. Interestingly, ViT achieves results su-perior to CNN baselines with 80.45% accuracy and 0.743 macro F1 score, reducing the CNN error by 9% and 12% respectively. A simple procedure for including metadata into the decision process improves the classification accuracy by more than 2.95 percentage points, reducing the error rate by 15%. The source code for all methods and experiments is available at https://sites.google.com/view/danish-fungi-dataset.
KW - Datasets
KW - Evaluation and Comparison of Vision Algorithms Object Detection/Recognition/Categorization
U2 - 10.1109/WACV51458.2022.00334
DO - 10.1109/WACV51458.2022.00334
M3 - Article in proceedings
AN - SCOPUS:85122859027
SP - 3281
EP - 3291
BT - Proceedings - 2022 IEEE Winter Conference on Applications of Computer Vision, WACV 2022
A2 - O'Conner, Lisa
PB - IEEE
T2 - 22nd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022
Y2 - 4 January 2022 through 8 January 2022
ER -
ID: 326348571