The Task
A design-tech company approached us with an ambitious project. Their model needed pixel-perfect semantic segmentation of interior photos—for every object in view.
We had previously collaborated on a simpler task involving just floor and wall segmentation. This new project, however, scaled in complexity. Now, each photo had to be fully segmented by class, covering over 30 categories—furniture, fixtures, decor, appliances, and more.
The accuracy bar was set high. Precise labeling and consistent separation of foreground/background layers were critical. There was no room for ambiguity or missed elements.
The Solution
Preparation and Annotator Training
- Custom Entry Test: We developed a screening test to ensure annotators could differentiate between visually similar classes like armchairs vs. sofas, or shelves vs. cabinets.
- Structured Training Program: Annotators went through video-based training with examples of correct and incorrect segmentation.
- Error Feedback Loop: We set up a fast feedback system with video reviews of common errors, helping annotators improve rapidly.
Annotation Workflow
- Per-Object Segmentation: Every object on each photo was segmented individually and labeled according to the class hierarchy.
- Foreground–Background Separation: We created specific guidelines for annotating layered objects (e.g., a vase on a table in front of a wall).
- Consistency Protocols: The team followed a shared labeling structure to maintain consistency across the dataset.
Quality Assurance and Validation
- Layered Review: Each annotated image underwent a two-step validation—initial QA check, followed by expert review.
- Systemic Feedback: Repeated errors were flagged and addressed through daily syncs between annotators and validators.
- Hierarchy Tweaks: During the mid-project review, the client requested minor adjustments to layer priorities (foreground/background). We implemented changes without delaying delivery.
| Stage | Input | Workflow Scope | Main Quality Checks |
|---|---|---|---|
| Preparation & Screening | Sample images, class definitions | Annotator entry tests, setup guidelines, class hierarchy | Annotator capability / Understanding of complex classes |
| Training & Pilot | Guidelines, reference segmentations | Video-based training, error feedback, pilot annotations | Annotation accuracy / Class differentiation |
| Full Annotation | Interior photos | Per-object segmentation, foreground-background separation, class labeling | Consistency / Correct class assignment |
| QA & Validation | Annotated images | Two-step review, expert checks, error flagging | Precision of boundaries / Hierarchy compliance |
| Final Delivery | Validated dataset | Consolidation, final QA, submission to client | Dataset completeness / Accuracy across all classes |
Preparation & Annotator Screening
4 days
Structured Training & Pilot
3 days
Full Annotation Cycle
3 weeks
QA, Validation & Final Delivery
7 days
The Results
- 100% of data delivered on time with all requested objects correctly segmented and classified.
- Consistent precision across the dataset—especially in challenging areas like overlapping objects and shadow boundaries.
- Positive client feedback highlighted the clarity of object boundaries and the low correction rate.
Every object counts: consistent boundaries and class accuracy make the difference between a usable dataset and wasted effort
- Roman Lukoshin
- Speech and Generative Data Manager
