Journal article
Lessons from a breast cell annotation competition series for school pupils
- Abstract:
- Due to COVID-19 outbreaks, most school pupils have had to be home-schooled for long periods of time. Two editions of a web-based competition “Beat the Pathologists” for school age participants in the UK ran to fill up pupils’ spare time after home-schooling and evaluate their ability on contributing to AI annotation. The two editions asked the participants to annotate different types of cells on Ki67 stained breast cancer images. The Main competition was at four levels with different level of complexity. We obtained annotations of four kinds of cells entered by school pupils and ground truth from expert pathologists. In this paper, we analyse school pupils’ performance on differentiating different kinds of cells and compare their performance with two neural networks (AlexNet and VGG16). It was observed that children tend to get very good performance in tumour cell annotation with the best F1 measure 0.81 which is a metrics taking both false positives and false negatives into account. Low accuracy was achieved with F1 score 0.75 on positive non-tumour cells and 0.59 on negative non-tumour cells. Superior performance on non-tumour cell detection was achieved by neural networks. VGG16 with training from scratch achieved an F1 score over 0.70 in all cell categories and 0.92 in tumour cell detection. We conclude that non-experts like school pupils have the potential to contribute to large-scale labelling for AI algorithm development if sufficient training activities are organised. We hope that competitions like this can promote public interest in pathology and encourage participation by more non-experts for annotation.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Version of record, 1.8MB, Terms of use)
-
- Publisher copy:
- 10.1038/s41598-022-11782-9
Authors
- Publisher:
- Springer Nature
- Journal:
- Scientific Reports More from this journal
- Volume:
- 12
- Article number:
- 7792
- Publication date:
- 2022-05-12
- Acceptance date:
- 2022-03-21
- DOI:
- EISSN:
-
2045-2322
- Language:
-
English
- Keywords:
- Pubs id:
-
1249522
- Local pid:
-
pubs:1249522
- Deposit date:
-
2022-04-05
Terms of use
- Copyright holder:
- Lu et al.
- Copyright date:
- 2022
- Rights statement:
- ©2022 The Author(s). Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
- Licence:
- CC Attribution (CC BY)
If you are the owner of this record, you can report an update to it here: Report update to this record