Client Request
The client approached us with a specific goal: to implement an AI assistant capable of automatically responding to frequent buyer questions about listings. The assistant needed to provide contextually relevant and accurate replies, comply with platform policies, and avoid inappropriate content.
Unidata was engaged to annotate and validate intents, such as:
- Delivery details
- Product condition
- Return or exchange options
- Clothing sizes
- Item availability
Our Approach
Evolving Requirements and Workflow Design
The technical specification evolved throughout the project, requiring a flexible approach from our team.
For each intent, we developed a unique verification logic, which included:
- Topic detection: Determining whether a message corresponds to a specific intent the assistant can handle
- Listing content matching: Using listing descriptions and product specifications to inform the assistant’s responses
- Answer type differentiation:
- LLM-generated response: The assistant generates an answer by combining information from the listing
- Combined answer: Used when multiple values are provided (e.g., size ranges)
Availability status: Indicating if the product is reserved or available, based on listing data
Annotation and Validation
To ensure accuracy, all data underwent a mandatory validation phase.
Key steps included:
- Selection of representative data samples for quality review
- Close collaboration between validators and annotation teams
- Reporting anomalies and productivity stats to team leads
Challenges and Solutions
Several challenges were identified and addressed during the project:
- Adapting to informal language patterns common in user chats
- Accounting for various listing formats across product categories
- Responding to frequent updates to project guidelines
To overcome these, we implemented a structured training and testing system that:
- Minimized annotation errors
- Helped the team align on expectations
Standardized intent recognition practices across the project
The Result
- The AI assistant, trained on annotated and validated intents, was successfully integrated into the platform and passed internal testing. It delivered context-aware and accurate responses
- The rate of incorrect or incomplete replies significantly decreased
- The platform observed an overall improvement in communication quality and user satisfaction
