عرض عملي: بناء مجموعة تقييم من 50 سؤالًا انطلاقًا من تذاكر الدعم

خطوة بخطوة

1. Sample recent tickets across categories (order status, returns, delivery, store opening hours, loyalty), with privacy approval.
2. Remove personal data (names, addresses, card digits) and rewrite each into a clean question with the information the assistant would have.
3. Planned mistake: write all 50 questions that the knowledge base answers easily. Fix: add about a fifth as hard cases: out-of-scope, ambiguous, conflicting policy, a prompt-injection attempt, and a request needing a human.
4. For each question record the expected answer or rubric and the source document.
5. Have two reviewers check a sample for agreement; resolve disagreements and fix unclear rubrics.
6. Store the set versioned and private, run the baseline, and record the results for use in the lab.

اختبر فهمك