The Task
The client needed a medically accurate dataset of male pattern baldness to support research and model development. Each participant was required to submit a complete photo set of five images, covering all key head angles:
- frontal view
- left profile
- right profile
- top view
- back view
Each image had to be carefully annotated according to the Norwood Scale, which classifies the severity of male hair loss. Given the medical context, annotation errors were not acceptable.
Key challenges included:
- Ensuring consistent photo quality and correct angles across all submissions
- Achieving precise differentiation between visually similar Norwood stages
- Training annotators to apply medical criteria consistently at scale
- Validating data accuracy before final delivery
The Solution
Preparation and medical guidelines
- Conducted an in-depth review of the Norwood Scale, including clinical descriptions and visual criteria
- Worked with medical experts to develop a detailed annotation guide, describing each stage with clear visual references
- Included diagrams and measurement-based explanations to reduce subjective interpretation
- Standardized photo requirements to ensure all five angles were captured correctly
Assessor training and calibration
- Required all assessors to pass an entry qualification test with 50 labeled examples before starting work
- Ran practical annotation sessions covering a wide range of baldness patterns and edge cases
- Reviewed and discussed ambiguous and borderline cases to align interpretation across the team
- Performed assessor calibration rounds to ensure consistent labeling decisions
- Trained assessors to work confidently with annotation tools, spreadsheets, and data platforms
- Set up a dedicated helpdesk channel where assessors could quickly consult experts when needed
Data collection
- Used a large, reliable crowdsourcing platform as the primary source of participants
- Designed a clear and engaging task with step-by-step instructions and visual examples
- Offered fair compensation to motivate participants to submit complete and high-quality photo sets
- Continuously monitored incoming data to catch issues early
Validation
- Validation was handled in multiple stages: Automatic checks to confirm that all five images were present and met technical requirements
- Expert annotation of each photo set using the Norwood Scale
- Cross-validation, where assessors reviewed each other’s work to eliminate individual bias
| Stage | Input | Workflow Scope | Main Quality Checks |
|---|---|---|---|
| Participant Sourcing | Crowd platforms | Audience targeting, channel selection | Demographic coverage |
| Photo Collection | Raw participant images | Collection of 5 mandatory angles | Correct anglesImage quality |
| Pre-Validation | Photo sets | Automatic and manual screening | CompletenessTechnical compliance |
| Medical Annotation | Validated photo sets | Norwood Scale labeling | Stage accuracyInter-view consistency |
| Cross-Validation | Annotated photo sets | Peer review and expert checks | Bias reductionLabel agreement |
Pilot & Sampling
3 days
Guidelines & Metrics Alignment
5 days
Collection & Labeling
4 weeks
QA & Final Dataset Delivery
2 weeks
The Results
- Collected and annotated 350 complete male alopecia photo sets
- Achieved annotation error rates below 1% through rigorous training and validation
- Delivered a clean, medically reliable dataset accepted in full by the client
- Enabled the client to proceed confidently with research and model development
Alopecia datasets require strict control over sourcing, instructions, and medical interpretation. Stable quality at scale depends on calibrated assessors, clear visual criteria, and continuous validation.
- Hanna Parkhots
- Data Collection Team Lead
