Algorithmic and human prediction of success in human collaboration from visual features

Journal article

This is the final version. Available on open access from Nature Research via the DOI in this recordData availability: The full dataset including aggregated features of each group of the >43K groups used in our analyses is available at the following link: https://doi.org/10.7910/DVN/HDT2RN. All photos used in this work are publicly available, posted on public Facebook pages. However, we do not release the raw images, or the individual-level raw features extracted using the Face++ API. More details are provided at the link above.The publisher correction to this article is available in ORE at http://hdl.handle.net/10871/124927As groups are increasingly taking over individual experts in many tasks, it is ever more important to understand the determinants of group success. In this paper, we study the patterns of group success in Escape The Room, a physical adventure game in which a group is tasked with escaping a maze by collectively solving a series of puzzles. We investigate (1) the characteristics of successful groups, and (2) how accurately humans and machines can spot them from a group photo. The relationship between these two questions is based on the hypothesis that the characteristics of successful groups are encoded by features that can be spotted in their photo. We analyze >43K group photos (one photo per group) taken after groups have completed the game—from which all explicit performance-signaling information has been removed. First, we find that groups that are larger, older and more gender but less age diverse are significantly more likely to escape. Second, we compare humans and off-the-shelf machine learning algorithms at predicting whether a group escaped or not based on the completion photo. We find that individual guesses by humans achieve 58.3% accuracy, better than random, but worse than machines which display 71.6% accuracy. When humans are trained to guess by observing only four labeled photos, their accuracy increases to 64%. However, training humans on more labeled examples (eight or twelve) leads to a slight, but statistically insignificant improvement in accuracy (67.4%). Humans in the best training condition perform on par with two, but worse than three out of the five machine learning algorithms we evaluated. Our work illustrates the potentials and the limitations of machine learning systems in evaluating group performance and identifying success factors based on sparse visual cues

Authors: Saveski, M; Awad, E; Rahwan, I; Cebrian, M

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Nature Research
• Journal: Scientific Reports
• Volume: 11
• Issue: 1
• Pages: 2756-2756
• Publication date: 2021-02-02
• DOI: 10.1038/s41598-021-81145-3
• EISSN: 2045-2322
• ISSN: 2045-2322
• Language: English
• Keywords: Random forest; Computer science; Group (periodic table); Artificial intelligence; Adventure; Machine learning