Citations

Every study behind our face rating tools in full, with design, sample, findings, limits and where we use it

Updated
On this page

Each study below is checked against the paper itself or its official abstract. Where we couldn't read the full paper, the entry says so. The short list is in the research base

Rhodes, 2006

Gillian Rhodes (2006). The evolutionary psychology of facial beauty. Annual Review of Psychology, 57(1), 199-226. doi:10.1146/annurev.psych.57.102904.190208

  • Design: Narrative review plus new meta-analyses of the link between attractiveness and three candidate biological cues: averageness, symmetry and sexual dimorphism
  • Sample: Meta-analyses: averageness 20 studies / 45 face samples; symmetry 23 studies / 63 face samples; femininity 10 studies / 18 face samples; masculinity 15 studies / 22 face samples

Findings

  • Averageness and attractiveness: mean R = 0.52 (95% CI 0.42-0.61); for real (unmanipulated) faces R = 0.40, for manipulated images R = 0.67
  • Symmetry and attractiveness: mean R = 0.25 (95% CI 0.16-0.33); for normal faces R = 0.23 (medium effect)
  • Sexual dimorphism: femininity is strongly attractive in female faces (R = 0.64); masculinity in male faces R = 0.35 for normal faces
  • Cross-cultural agreement on who is attractive and early emergence of preferences are cited as evidence that preferences are not purely cultural

Limits

  • Effect sizes are larger for composites and morphed images than for real faces (0.67 vs 0.40 for averageness), so part of the effect may come from non-average features of composites (paper's own discussion)
  • Evidence that attractive or symmetric faces signal health is weak; health was generally measured poorly, often by self-report (paper's own discussion)
  • A review plus effect-size aggregation of studies that mostly used Western student raters; cross-population coverage is limited (reviewer note, not a quote from the paper)

Langlois & Roggman, 1990

Judith H. Langlois, Lori A. Roggman (1990). Attractive faces are only average. Psychological Science, 1(2), 115-121. doi:10.1111/j.1467-9280.1990.tb00079.x

  • Design: Digitized male and female faces were averaged into composites of increasing numbers of faces; adults rated the individual faces and the composites for attractiveness

Findings

  • Composites of both sexes were rated more attractive than almost all of the individual faces that went into them (abstract)
  • Attractiveness of composites rose steadily as more faces were averaged into them (abstract)
  • Authors link the result to evolutionary pressure toward population-mean traits and to cognitive preference for prototypical category members (abstract)

Limits

  • Composites also lose blemishes and texture, so smoothing may contribute to their appeal; Rhodes 2006 reports a larger averageness effect for manipulated images (0.67) than for real faces (0.40)
  • Alley & Cunningham (1991) argued that very attractive faces are not average; the authors replied in Langlois, Roggman & Musselman (1994), doi 10.1111/j.1467-9280.1994.tb00503.x. Both exchanges were seen only as citations and titles

Langlois et al., 2000

Judith H. Langlois, Lisa Kalakanis, Adam J. Rubenstein, Andrea Larson, Monica Hallam, Monica Smoot (2000). Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological Bulletin, 126(3), 390-423. doi:10.1037/0033-2909.126.3.390

  • Design: Eleven meta-analyses of research on agreement about attractiveness (within and across cultures) and on the effects of attractiveness on judgments and treatment of children and adults
  • Sample: Adult within-culture reliability: 88 samples from 67 studies; child within-culture: 28 samples / 20 studies; adult cross-cultural: 17 samples / 9 studies (n = 12,146); adult cross-ethnic: 9 samples / 6 studies (n = 659)

Findings

  • Within-culture agreement of adult raters: effective reliability r = .90 for ratings of adults (mean pairwise r = .47), r = .85 for ratings of children
  • Cross-ethnic agreement: effective reliability r = .88 (mean pairwise r = .54); cross-cultural agreement: effective reliability r = .94 (mean pairwise r = .71)
  • Attractive adults were judged more positively than unattractive adults (overall d+ = .50; occupational competence d+ = .90)

Limits

  • 'Effective reliability' is a panel-level (Spearman-Brown type) coefficient that grows with the number of raters; mean pairwise agreement is much lower (.47 within-culture adults). Hönekopp (2006) argues the high coefficients were over-read as agreement between two raters
  • Many included reliabilities came from studies with only two raters and from Western samples; the authors note mean reliabilities were lower than effective reliabilities for this reason
  • Meta-analytic effects on judgments are correlational; the authors call for longitudinal work to test causal claims

Hönekopp, 2006

Johannes Hönekopp (2006). Once more: Is beauty in the eye of the beholder? Relative contributions of private and shared taste to judgments of facial attractiveness. Journal of Experimental Psychology: Human Perception and Performance, 32(2), 199-209. doi:10.1037/0096-1523.32.2.199

  • Design: Three experiments where judges rated the same faces twice, one week apart, so that variance could be split into shared taste (faces), private taste (judge x face interaction) and error
  • Sample: Exp 1: 77 Caucasian faces (56 women, 21 men), 31 judges. Exp 2: 60 faces (20 Asian, 20 Black, 20 Caucasian), 31 judges (10 Asian, 11 Black, 10 Caucasian). Exp 3: 54 Caucasian faces (27 women, 27 men), 100 judges (50 women, 50 men)

Findings

  • Exp 1: Cronbach's alpha of face scores = .95, but average agreement between two judges was only r = .40 (first session) and .39 (second session); retest reliability .74
  • Exp 1 variance: faces (shared taste) 34.0% vs judge x face interaction (private taste) 32.7%; beholder index .49-.57. Exp 2: faces 26.3% vs interaction 33.3%
  • Exp 3: alpha = .99, mean interjudge r = .43 (first session), beholder index .44-.57; private taste is about as strong as shared taste

Limits

  • Standardized black-and-white photographs and young judges (mostly students, ages 19-46), recruited in Germany
  • In Exp 2 the Asian and Black judges were not born in Germany and had lived there about 2 years on average, so cross-cultural conclusions are limited
  • Private taste here means stable, replicable disagreement with the group; it cannot separate individual preference from stable rater-specific response style

Grammer & Thornhill, 1994

Karl Grammer, Randy Thornhill (1994). Human (Homo sapiens) facial attractiveness and sexual selection: The role of symmetry and averageness. Journal of Comparative Psychology, 108(3), 233-242. doi:10.1037/0735-7036.108.3.233

  • Design: Men and women rated opposite-sex faces (and composites) on attractive, dominant, sexy and healthy; facial symmetry and averageness were related to the ratings

Findings

  • Facial symmetry affected attractiveness ratings
  • Hypotheses were supported except the predicted effect of averageness in male faces on attractiveness
  • Rhodes (2006) lists the study among those showing that natural variation in facial symmetry covaries with attractiveness

Limits

  • Composite faces were used for the averageness test, so the averageness result rests on manipulated images (design)

Penton-Voak et al., 2001

I. S. Penton-Voak, B. C. Jones, A. C. Little, S. Baker, B. Tiddeman, D. M. Burt, D. I. Perrett (2001). Symmetry, sexual dimorphism in facial proportions and male facial attractiveness. Proceedings of the Royal Society B: Biological Sciences, 268(1476), 1617-1623. doi:10.1098/rspb.2001.1703

  • Design: Real and computer-graphic male faces were used to test whether facial symmetry relates to rated attractiveness and masculinity (abstract)

Findings

  • Symmetric male faces were rated more attractive (abstract)
  • Symmetric male faces were not reliably rated more masculine (abstract)
  • Symmetric faces carry attractive characteristics that are independent of symmetry and not yet identified (abstract)

Limits

  • Authors state the symmetry-independent attractive characteristics are undefined, so symmetry alone cannot be credited with the effect (abstract)

Pallett et al., 2010

Pamela M. Pallett, Stephen Link, Kang Lee (2010). New 'golden' ratios for facial beauty. Vision Research, 50(2), 149-154. doi:10.1016/j.visres.2009.11.003

  • Design: Four experiments with paired attractiveness comparisons of faces that had identical features but different vertical (eye-mouth) and horizontal (inter-eye) distances

Findings

  • Attractiveness peaked when the vertical distance between eyes and mouth was about 36% of face length (confirmed in the abstract)
  • Attractiveness peaked when the horizontal distance between the eyes was about 46% of face width (confirmed in the abstract)
  • These optimal ratios match the ratios of an average face (abstract); a press-level snippet of the paper reports the average length ratio as .36 (SD .017) over 40 Caucasian female faces

Limits

  • The optimum coincides with the average face, so the study cannot separate a 'golden' preference from an averageness preference (follows from the abstract)
  • Only Caucasian female faces were studied

Holland, 2008

E. Holland (2008). Marquardt's Phi mask: Pitfalls of relying on fashion models and the golden ratio to describe a beautiful face. Aesthetic Plastic Surgery, 32(2), 200-208. doi:10.1007/s00266-007-9080-z

  • Design: Tested how well Marquardt's golden-ratio mask fits sample faces (Generalized Procrustes fit and thin-plate spline comparisons, per a search-engine summary of the abstract) and what the mask resembles

Findings

  • The method used to assess fit of the mask is faulty and the mask fits non-European populations poorly (abstract)
  • The mask resembles the face shape of masculinized white women, such as fashion models (abstract)
  • The mask does not describe an ideal face shape, particularly with respect to femininity; white women seeking facial surgery would be ill-advised to aim for a closer fit to it (abstract)

Limits

  • Critique paper: its argument rests on the mask's fit to chosen sample faces and on preference data cited from other studies

Carré & McCormick, 2008

Justin M. Carré, Cheryl M. McCormick (2008). In your face: facial metrics predict aggressive behaviour in the laboratory and in varsity and professional hockey players. Proceedings of the Royal Society B: Biological Sciences, 275(1651), 2651-2656. doi:10.1098/rspb.2008.0873

  • Design: Three studies relating facial width-to-height ratio (fWHR) to aggression: a lab task of reactive aggression, and penalty minutes per game in varsity and professional ice hockey players
  • Sample: Study 1: 88 undergraduates (37 men, 51 women). Study 2: 21 male varsity hockey players (goalkeepers excluded). Study 3: 112 NHL players from six Canadian teams (126 found; 2 excluded for head tilt, 12 goalkeepers excluded)

Findings

  • fWHR definition: bizygomatic width (distance between the left and right zygion) divided by upper facial height (distance between the upper lip and the brow), measured from photographs in ImageJ
  • Study 1: fWHR predicted reactive aggression in men (r = .38, p = .02, n = 37) but not in women (R2 = .03, p = .52)
  • Study 2 (varsity players): r = .54 with penalty minutes per game (p = .01). Study 3 (NHL players): r = .30 (p = .005); team-level correlations all positive, .17 to .51

Limits

  • Photographs were used instead of direct facial measurements; head posture could distort the vertical measure (authors)
  • Facial expression was not controlled in the hockey players; some were smiling (authors)
  • The link may reflect a third variable such as organizational effects of testosterone at puberty; unknown whether people actually use fWHR as a cue (authors). The varsity sample is small (n = 21)

Kosinski, 2017

Michal Kosinski (2017). Facial width-to-height ratio does not predict self-reported behavioral tendencies. Psychological Science, 28(11), 1675-1682. doi:10.1177/0956797617716929

  • Design: Two studies of Facebook-app users: fWHR measured manually (Carré and McCormick method) and with automatic landmark detection (Face++), then correlated with 55 self-report psychometric scales
  • Sample: Study 1: 1,703 images of 1,692 participants (58% female, median age 28). Study 2: 170,241 images of 137,163 participants (70% female, median age 27) from the myPersonality Facebook app

Findings

  • fWHR was not substantially linked with any of the 55 self-reported measures (intelligence, Big Five domains and facets, impulsiveness, sense of fairness, life satisfaction, etc.)
  • Mean fWHR (automatic, Study 2): 1.81 (SD 0.16) for females, 1.87 (SD 0.16) for males; Study 1 manual: 1.73 (SD 0.16) F, 1.83 (SD 0.17) M; Study 1 automatic: 1.65 (SD 0.15) F, 1.77 (SD 0.14) M
  • Manual and automatic fWHR estimates agree at r = .86; two images of the same person agree only at r = .60 (r = .76 for the highest-resolution subset)

Limits

  • Outcomes are self-reports, not observed behavior (design of the study; the title says so)
  • Automatic fWHR from profile photos is noisy and depends on resolution (test-retest r = .60 vs .76 at high resolution); manual and automatic means differ by about 0.06
  • Sample is Facebook users who installed one app, 70% female; photos are self-selected profile pictures

Geniole et al., 2015

Shawn N. Geniole, Thomas F. Denson, Barnaby J. Dixson, Justin M. Carré, Cheryl M. McCormick (2015). Evidence from meta-analyses of the facial width-to-height ratio as an evolved cue of threat. PLoS ONE, 10(7), e0132726. doi:10.1371/journal.pone.0132726

  • Design: Meta-analyses of the fWHR relation to sex, threat and dominance behavior, perceived threat/dominance/masculinity, attractiveness and BMI
  • Sample: Sex difference: 19 manuscripts, 32 samples (6,113 men, 4,740 women). Threat behavior: 19 manuscripts, 32 samples (4,573 men, 634 women). Perceived threat: 18 manuscripts, 38 samples

Findings

  • Sex difference in fWHR is small: d = .11 (95% CI 0.03 to 0.20)
  • fWHR and threat behavior: r = .12 (.07 to .17, k = 32); men r = .16 (.11 to .21), women r = .04 (-.04 to .12); aggressive behavior in men r = .13 (.07 to .18, k = 12)
  • Perceived threat from fWHR is much larger: r = .48 (.41 to .55, k = 38); perceived masculinity r = .30; attractiveness r = -.26; BMI r = .31

Limits

  • Few stimuli of female and older adult faces; the female-face masculinity estimate rests on 2 samples (authors)
  • Funnel-plot asymmetry for threat behavior suggests smaller samples gave larger effects, so the estimate may be biased (authors)
  • Few experimental studies (one manipulated fWHR for dominance, one for attractiveness); most dominance measures were questionnaires (authors)

Jenkins et al., 2011

Rob Jenkins, David White, Xandra Van Montfort, A. Mike Burton (2011). Variability in photos of the same face. Cognition, 121(3), 313-323. doi:10.1016/j.cognition.2011.08.001

  • Design: Four experiments using ambient photographs from the internet: card sorting by identity, familiarity, likeness ratings and attractiveness ratings across photos of the same person
  • Sample: Exp 1: 20 images of each of 2 Dutch celebrities (40 total), 20 UK undergraduates. Exp 2: same cards, 20 Dutch participants. Exp 3: 40 UK celebrities x 12 images (480), 20 UK undergraduates. Exp 4: 20 Dutch celebrities x 20 images (400), 40 UK undergraduates (20 men, 20 women)

Findings

  • Exp 1: unfamiliar viewers perceived a median of 7.5 identities (mode 9, range 3-16) in 40 photos of 2 people; none sorted correctly; t(19) = 7.82, d = 1.8
  • Exp 2: familiar (Dutch) viewers almost all sorted perfectly into 2 identities (median 2, range 2-5)
  • Exp 4: within-person variability in attractiveness exceeded between-person variability; for any pair of faces, photos could be chosen that reverse the preference

Limits

  • Only two celebrities in the sorting task (Exp 1-2); Exp 1 photos were converted to greyscale, roughly frontal, at least 150 pixels high
  • The authors tested only attractiveness and likeness; other social judgements are assumed to behave similarly but were not tested here
  • Celebrity images from Google Image search are not a random sample of everyday photos

Todorov & Porter, 2014

Alexander Todorov, Jenny M. Porter (2014). Misleading first impressions: Different for different facial images of the same person. Psychological Science, 25(7), 1404-1417. doi:10.1177/0956797614532474

  • Design: Four experiments: raters judged 5 different photos of each of 20 people on social traits, chose photos for different contexts, made decisions from photos, and judged photos shown for 40 ms
  • Sample: Exp 1: 800 Amazon Mechanical Turk participants rated 100 images (20 people x 5 FERET photos); Exp 2: 110 Princeton students; Exp 3a: 16 students, 3b: 22 New Jersey residents; Exp 4: 24 students

Findings

  • Exp 1: mean correlation between rank orders of the same people from different image sets was low (e.g., .20 for extraversion; range of between-set correlations .03 to .87)
  • For all judgments except attractiveness (and intelligence of male faces), within-person variance across images was comparable to or larger than between-person variance (M = 0.57 vs 0.35, t(13) = 2.64, p < .02)
  • Image preferences depend on context (Exp 2) and biased image choice changes decisions such as hiring or voting (Exp 3); the biases are present at 40 ms (Exp 4)

Limits

  • Only 20 individuals and 5 images each, from the FERET database (greyscale, spontaneous expressions); the people were much older than the raters
  • Attractiveness was the one trait where between-person variance clearly exceeded within-person variance, so the effect varies by trait
  • Raters were online workers and students; photos were not rated for expression, so expression and image differences are confounded (by design: authors treat expression as natural variation)

Ward et al., 2018

Brittany Ward, Max Ward, Ohad Fried, Boris Paskhover (2018). Nasal distortion in short-distance photographs: The selfie effect. JAMA Facial Plastic Surgery, 20(4), 333-335. doi:10.1001/jamafacial.2018.0009

  • Design: Mathematical (projection) model of how camera distance distorts facial features, used to estimate how much the nose looks larger in photos taken at short distances; no patient contact (IRB-exempt)

Findings

  • Taking a photo at arm's length distorts the face by projection, most visibly making the nose look larger (abstract)
  • At 12 in (30 cm), with cheekbone width held constant, nasal breadth looks 30% wider in men and 29% in women than in an orthographic projection; the distance between the nostrils looks 7% wider
  • At 5 ft (1.5 m) there is essentially no difference in perceived size

Limits

  • Model-based, not measured on a sample of people (mathematical model, no patient contact)
  • Only camera distance is modelled; vertical and horizontal camera angles need further models (paper)
  • A published letter (Cooper, 2018) argues the effect is a viewing-distance effect on perception rather than a distortion within the photographs

Kartynnik et al., 2019

Yury Kartynnik, Artsiom Ablavatski, Ivan Grishchenko, Matthias Grundmann (2019). Real-time facial surface geometry from monocular video on mobile GPUs. CVPR Workshop on Computer Vision for Augmented and Virtual Reality (arXiv:1907.06724). Link

  • Design: Neural network that predicts a 468-vertex 3D face mesh from a single camera frame in real time on mobile GPUs; ground truth built by an iterative bootstrapping procedure from 3DMM renderings and 2D contour annotations
  • Sample: Training: about 30K in-the-wild mobile camera photos. Evaluation: geographically diverse set of 1.7K images. Human annotation baseline: 11 annotators x 58 images

Findings

  • Mesh has 468 vertices arranged in fixed quads
  • Error normalized by interocular distance (mean absolute distance, 2D): full model 3.96%, light 5.15%, lightest 5.29%
  • Variance between trained human annotators on the same images: 2.56% IOD-normalized (abstract: quality 'comparable to the variance in manual annotations')
  • Inference time of the full model: 2.5 ms on iPhone XS and 7.4 ms on Pixel 3 (lightest: 0.7 ms / 2.6 ms)

Limits

  • z-coordinates come only from synthetic (3DMM) supervision, so the reported errors are 2D-only; depth accuracy is not measured (paper)
  • Ground truth for the 468 points is not manually annotated point by point; it is produced by an iterative, partly model-assisted process (paper)
  • Only an aggregate interocular-distance-normalized error is reported (Table 1); no breakdown by face region, pose or skin tone appears in the 4-page paper

Liang et al., 2018

Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, Mengru Li (2018). SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction. 2018 24th International Conference on Pattern Recognition (ICPR), 1598-1603. doi:10.1109/ICPR.2018.8546038

  • Design: Benchmark dataset of frontal face images with 1-5 beauty scores and 86 facial landmarks per image, plus baseline models for beauty regression, classification and score-distribution prediction
  • Sample: 5,500 frontal, unoccluded faces aged 15-60: 2,000 Asian female, 2,000 Asian male, 750 Caucasian female, 750 Caucasian male. 60 volunteer raters aged 18-27 (mean 21.6); the four subsets were labeled separately, about 10% of faces repeated for quality control

Findings

  • Images: 5,500. Scale: beauty score in [1, 5]
  • Raters per image: 60 (derived: 45,000 scores for 750 Caucasian-female faces and 120,000 scores for 2,000 Asian-female faces = 60 per face, from the paper's Table II)
  • Rater quality control: if the correlation between two ratings of the same repeated face was below 0.7, the volunteer re-rated the face
  • Licence: the dataset repository README says it can be used only for non-commercial research purposes; commercial use is NOT allowed

Limits

  • Only Asian and Caucasian subsets, unbalanced (4,000 vs 1,500 images); most images were collected from the internet, with some Asian faces from DataTang and some Caucasian faces from the 10k US Adult Faces database (paper)
  • All 60 raters are 18-27 years old; rater ethnicity is not reported in the paper text read here
  • Scores are averages of young volunteers' ratings, so they reflect that group's taste (see Honekopp 2006 on the share of private taste)

Ma et al., 2015

Debbie S. Ma, Joshua Correll, Bernd Wittenbrink (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47(4), 1122-1135. doi:10.3758/s13428-014-0532-5

  • Design: Standardized neutral-expression photographs of volunteer models with subjective norming ratings (14 dimensions, 1-7 scale) and objective physical measurements; Study 2 surveyed researchers about suitability of the faces
  • Sample: 158 models (Black and White, male and female, ages 18-40). Norming: 1,087 raters (552 female, 308 male, 227 not reported; mean age 26.75, SD 10.54; 516 White, 117 Asian, 74 Black, 72 multiracial, 57 Latino)

Findings

  • Each rater judged 15 randomly selected targets (first 168 raters) and then 10 targets; derived about 74 raters per face (reviewer arithmetic: (168x15 + 919x10)/158)
  • Attractiveness reliability (Cronbach's alpha across raters, n = 1,087) = .998
  • Terms of use on chicagofaces.org: non-commercial scientific research only; no publication without written consent; no redistribution, including to file hosting or cloud-based AI image manipulation services. Commercial use is NOT allowed
  • The current site lists 597 unique individuals in the extended set (original CFD plus expansions), 88 in CFD-MR and 142 in CFD-INDIA

Limits

  • Original 2015 set contains only Black and White targets aged 18-40 (abstract)
  • Norming raters were a convenience sample, mostly White and U.S.-based; only neutral expressions were normed
  • Attractiveness and the other 13 trait ratings were made relative to other people of the same race and gender (rater instruction in the paper), so scores are not comparable across groups

Lebedeva et al., 2022

Irina Lebedeva, Yi Guo, Fangli Ying (2022). MEBeauty: a multi-ethnic facial beauty dataset in-the-wild. Neural Computing and Applications, 34(17), 14169-14183. doi:10.1007/s00521-021-06535-0

  • Design: Facial beauty dataset of unconstrained (in-the-wild) face images of six ethnic groups, rated by volunteers of various ethnicity, age and gender
  • Sample: 2,550 images of Black, Asian, Caucasian, Hispanic, Indian and Middle Eastern women and men, rated by about 300 people of various cultural and social backgrounds; the Hugging Face mirror by the first author lists 2,462 images

Findings

  • Images: 2,550 in the original release; the Hugging Face mirror lists 2,462 images in fixed splits (1,962 train / 250 validation / 250 test)
  • Ratings per image: 8 to 92 on the mirror card
  • Licence: the Hugging Face mirror (by the first author) is tagged research-only: 'NON-COMMERCIAL ACADEMIC RESEARCH ONLY', no identity recognition, no re-upload; gated access. Commercial use is NOT allowed there. The original repository is also non-commercial research only

Limits

  • Mirror card: rater demographics are known for only 10 of the 593 attractiveness raters, so the claim of diverse raters cannot be checked
  • Mirror labels were recomputed (rater leniency removed), so results on the mirror are not directly comparable to the paper (card statement)
  • A third-party preprint (Hamel and Busch, arXiv:2509.24138) reports ethnic-group bias in models trained on SCUT-FBP5500 and MEBeauty

Koo & Li, 2016

Terry K. Koo, Mae Y. Li (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155-163. doi:10.1016/j.jcm.2016.02.012

  • Design: Practical guideline describing the 10 forms of ICC (model, type, definition) with advice on choosing a form and reporting it

Findings

  • Interpretation is based on the 95% confidence interval of the ICC estimate, not the point estimate alone
  • Thresholds: less than 0.5 = poor; 0.5 to 0.75 = moderate; 0.75 to 0.9 = good; greater than 0.90 = excellent reliability
  • There are 10 forms of ICC; the report should state model, type and definition

Limits

  • The cut-offs are worded as 'indicative of' reliability in the abstract, i.e. guidance, not a derived standard
  • An erratum exists (Journal of Chiropractic Medicine 2017;16(4):346, doi 10.1016/j.jcm.2017.10.001)

Buolamwini & Gebru, 2018

Joy Buolamwini, Timnit Gebru (2018). Gender Shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research (PMLR), 1st Conference on Fairness, Accountability and Transparency, 81, 77-91. Link

  • Design: Audit of three commercial gender classifiers (Microsoft, Face++, IBM) on a new benchmark balanced by gender and Fitzpatrick skin type
  • Sample: Pilot Parliaments Benchmark (PPB): 1,270 individuals from parliaments in Rwanda, Senegal, South Africa, Iceland, Finland, Sweden. Composition: darker female 21.3%, darker male 25.0%, lighter female 23.3%, lighter male 30.3% (darker = Fitzpatrick IV-VI, lighter = I-III)

Findings

  • Error rate for darker-skinned females: Microsoft 20.8%, Face++ 34.5%, IBM 34.7%
  • Error rate for lighter-skinned males: Microsoft 0.0%, Face++ 0.8%, IBM 0.3% (abstract: maximum 0.8%)
  • Darker females were 21.3% of the benchmark but 61.0% to 72.4% of the classification errors

Limits

  • Gender is binary (female/male labels for subjects perceived as women or men); the authors call this a reductionist view of gender
  • Images are constrained official portraits with cooperative subjects and fixed pose; the authors expect higher error rates on unconstrained images. European photos tend to be higher resolution, so South Africa was re-analysed separately
  • Labels were assigned by 3 annotators including the authors using the Fitzpatrick scale, which the authors call coarse and skewed toward lighter skin

Chardon et al., 1991

A. Chardon, I. Cretois, C. Hourseau (1991). Skin colour typology and suntanning pathways. International Journal of Cosmetic Science, 13(4), 191-208. doi:10.1111/j.1467-2494.1991.tb00561.x

  • Design: Introduced the Individual Typology Angle (ITA) computed from CIE Lab* skin colour measurements to classify skin colour and follow tanning

Findings

  • ITA is an angle in the L*-b* colour plane, computed from the point L* = 50, b* = 0; higher ITA means lighter skin: ITA° = arctan((L* − 50) / b*) × 180 / π
  • Original categories were four: very light, light, intermediate, tan; later work added brown and dark
  • The six categories in use today come from Del Bino et al. (2006): very light > 55° > light > 41° > intermediate > 28° > tan > 10° > brown > −30° > dark

Limits

  • A 2021 report found the six ITA categories correlate poorly with Fitzpatrick types, so the two should not be used as equivalents (PubMed 34796498)
  • Thresholds differ between sources: a non-peer-reviewed Episkin poster gives only three bands (light 0 < ITA <= 60, tanned -40 < ITA <= 0, brown ITA <= -40)

Zaidel & Cohen, 2005

D. W. Zaidel, J. A. Cohen (2005). The face, beauty, and symmetry: Perceiving asymmetry in beautiful faces. International Journal of Neuroscience, 115(8), 1165-1173. doi:10.1080/00207450590914464

  • Design: Head-on photographs of faces from professional modelling-agency collections were rated for beauty; left-left and right-right symmetrical composites were built from each face; a new group chose the more attractive image of each pair or answered 'same'

Findings

  • Participants did not prefer left-half composites over right-half composites
  • Participants distinguished the 'same' option from the two symmetrical composites, meaning they detected the asymmetry of the original beautiful faces
  • Authors conclude that very beautiful faces may be functionally asymmetrical, so perfect symmetry is not required for beauty

Limits

  • Faces came from modelling-agency collections, so they are a selected, highly attractive sample (design)

Maurer et al., 2002

Daphne Maurer, Richard Le Grand, Catherine J. Mondloch (2002). The many faces of configural processing. Trends in Cognitive Sciences, 6(6), 255-260. doi:10.1016/S1364-6613(02)01903-4

  • Design: Review of behavioural and developmental studies on how people process faces, separating featural processing from three types of configural processing

Findings

  • Three types of configural processing: first-order relations (two eyes above a nose above a mouth), holistic processing (features glued into one gestalt) and second-order relations (the spacing between features)
  • Turning a face upside down disrupts configural processing more than processing of individual features
  • Sensitivity to the spacing between features develops slowly and keeps improving into adolescence

Limits

  • A narrative review, with no new data of its own (design)

All methodology topics