Task:
The client needed a medically accurate dataset of male pattern baldness to support research and model development. Each participant was required to submit a complete photo set of five images, covering all key head angles:
- frontal view
- left profile
- right profile
- top view
- back view
Each image had to be carefully annotated according to the Norwood Scale, which classifies the severity of male hair loss. Given the medical context, annotation errors were not acceptable.
Key challenges included:
- Ensuring consistent photo quality and correct angles across all submissions
- Achieving precise differentiation between visually similar Norwood stages
- Training annotators to apply medical criteria consistently at scale
- Validating data accuracy before final delivery
Solution
-
- 01
-
Preparation and medical guidelines
- Conducted an in-depth review of the Norwood Scale, including clinical descriptions and visual criteria
- Worked with medical experts to develop a detailed annotation guide, describing each stage with clear visual references
- Included diagrams and measurement-based explanations to reduce subjective interpretation
- Standardized photo requirements to ensure all five angles were captured correctly
-
- 02
-
Assessor training and calibration
- Required all assessors to pass an entry qualification test with 50 labeled examples before starting work
- Ran practical annotation sessions covering a wide range of baldness patterns and edge cases
- Reviewed and discussed ambiguous and borderline cases to align interpretation across the team
- Performed assessor calibration rounds to ensure consistent labeling decisions
- Trained assessors to work confidently with annotation tools, spreadsheets, and data platforms
- Set up a dedicated helpdesk channel where assessors could quickly consult experts when needed
-
- 03
-
Data collection
- Used a large, reliable crowdsourcing platform as the primary source of participants
- Designed a clear and engaging task with step-by-step instructions and visual examples
- Offered fair compensation to motivate participants to submit complete and high-quality photo sets
- Continuously monitored incoming data to catch issues early
-
- 04
-
Validation
- Validation was handled in multiple stages: Automatic checks to confirm that all five images were present and met technical requirements
- Expert annotation of each photo set using the Norwood Scale
- Cross-validation, where assessors reviewed each other’s work to eliminate individual bias
Results:
Collected and annotated 350 complete male alopecia photo sets
Achieved annotation error rates below 1% through rigorous training and validation
Delivered a clean, medically reliable dataset accepted in full by the client
Enabled the client to proceed confidently with research and model development
