- Consensus, or multi-reader, single output: Have multiple labelers annotate a single task and record their inter-annotator agreement scores, i.e., measure the overlap between their annotations. The output of consensus is a single set of ground truth.
- Task duplication, or multi-reader, multiple output: Have multiple annotators annotate a single image and generate N unique ground truth records for a single image.

Task duplication vs. consensus