How Do We Ensure the Highest Quality in Data Annotation?
Trusted by the world’s leading tech brands
- 6+
- years experience with various projects
Quality Control Department (QCD)
To ensure the highest quality annotation, Unidata employs a unique approach to quality control. Each project's validation is handled by a dedicated department, the QCD. The Quality Control Department includes validators who check the annotations and provide reports to the team leaders.QCD Work Stages
-
Conducting the pilot
01 -
Selecting validators
03 -
Regular monitoring of QCD performance
05 -
Defining metrics
02 -
Validation
04
Why is the QCD necessary?
The Quality Control Department is essential for timely quality management on the project. We validate the data from the previous or current day every day, allowing us to take prompt action, such as addressing problematic cases with performers, providing additional training, or discussing the technical assignment with clients. This helps us consistently guarantee high annotation quality.
Validators also work with team leaders and act as analysts. By processing large volumes of data, they can notice patterns, anomalies, or deviations in the project and send the appropriate reports to the team leaders, which helps reduce decision-making time.
Therefore, the QCD is an integral part of every Unidata project.
What does QCD expertise consist of?
It consists of five main points: testing, speed, consistency, re-monitoring, and error rate of appeals. Based on the analysis of these criteria, we can manage the department's work: hire or remove validators from the project and clarify necessary points during meetings.-
01.
Testing
For all projects lasting more than two weeks, we conduct testing. It includes 10 to 20 questions about the project's technical assignment. -
02.
Speed
The QCD affects the project's completion speed because all data requires validation. The sooner all necessary data is validated, the faster the project can be completed. Therefore, the Quality Control Department has speed metrics to ensure projects are delivered on time and without delays. -
03.
Consistency
It is important for us to maintain a level of uniformity in resolutions for the same case, so we evaluate consistency. For this, we give the same case to different validators on the project for independent resolution.
Team leaders, experts in the technical assignment, also participate in this metric by annotating the same case. -
04.
Re-monitoring
To control the QCD's quality, re-monitoring is performed. In each project, we review responses for each validator, checking their actions, whether cases are correctly confirmed, and if they are aware of the latest updates.
We also check each validator for the largest group of errors on the project and previous errors. It's important for us to ensure that planned adjustments are successfully made and that the validator correctly interprets such cases. -
05.
Error Rate of Appeals
It's crucial for us to check if there were any appeals by the team leader regarding errors found by the QCD, and if so, how many were successful in confirming QCD's errors. This metric is really important as it affects the error rate on the project and takes time of all participants in the appeal process. -
06.
Individual Approach
At Unidata, we offer the client an optimal approach to quality assessment on their project, considering the budget, required accuracy, and task specifics. The proposed approach is just an option, and the client always has the opportunity to accept it or make a counterproposal.
