Objective
The strawberry producer approached Unidata to automate their berry-picking process and improve farm efficiency. Cameras installed on the farm captured close-up images of strawberries, and the task was to classify the strawberries based on ripeness to optimize the picking process.
Solution
Requirements and Pilot Project:
The client provided strawberry images and classification instructions to categorize berries into three groups: ripe, unripe, and partially ripe. We reviewed these requirements and suggested refinements to the classification criteria to make the annotation process as accurate and user-friendly as possible.
Data Annotation and Quality Review:
The Unidata team began annotating the images, sorting the berries into the specified ripeness categories. We considered client-provided criteria such as color, texture, and shape to enhance classification accuracy. Each annotator received additional training to standardize the approach and minimize errors.
Validation:
To ensure high accuracy, all annotated images underwent a multi-level review process. Our validation methodology involved selecting a specific data volume for re-evaluation to ensure adherence to the ripeness standard defined by the client. The validation process included several stages:
Primary Analysis:
The validation team conducted an initial review of the completed annotations to identify critical errors and detect systematic discrepancies.
Cross-Checking:
To eliminate subjective differences, several validation groups independently checked the same image sample, then compared results and refined the criteria to achieve a unified approach.
Performance Statistics and Analysis:
During validation, we closely monitored statistical data on annotator quality. This allowed us to quickly identify annotators with higher error rates and provide additional training as needed, while also recognizing top performers.
Our specialists also offered regular feedback and recommendations to annotators throughout the project. In addition to internal checks, Unidata organized clarification sessions with the client, which helped improve the evaluation criteria and achieve a higher level of consistency.
| Stage | Input | Workflow Scope | Main Quality Checks |
|---|---|---|---|
| Requirements Alignment | Client guidelines, strawberry images | Definition of ripeness categories, criteria refinement | Clarity, consistency, edge cases |
| Annotator Training | Finalized instructions | Training sessions, calibration tasks | Understanding of criteria, error rate |
| Image Annotation | Raw strawberry images | Classification into ripe, unripe, partially ripe | Label accuracy, consistency |
| Validation | Annotated dataset | Multi-level review, cross-checking | Agreement between reviewers, bias control |
| Performance Monitoring | Annotation statistics | Error tracking, retraining | Individual accuracy, stability |
| Final QA | Validated dataset | Dataset consolidation and delivery | Consistency, client acceptance |
The Results
- Based on the annotated and validated data, the client successfully implemented a system that optimizes the strawberry-picking process.
- The farm can now plan picking activities more accurately by directing workers to the areas with berries at the desired ripeness level.
Accurate ripeness classification depends on clearly defined visual criteria, consistent annotation standards, and rigorous multi-level validation. High-quality datasets require not only labeling but continuous alignment between annotators and client expectations.
- Roman Lukoshin
- Speech and Generative Data Manager
