Files
sovereign_ai/SovereignAI-Starter/evals/results/retrieval_live_api_fix_2026-10-04.json
T

44 lines
2.0 KiB
JSON

{
"created_at_utc": "2026-10-04",
"base_url": "http://127.0.0.1:8000",
"service": "FastAPI restarted from current working tree with the existing SQLite database",
"model": "gemma4:e2b",
"metrics": {
"cases": 2,
"hit_at_1": 1.0,
"evidence_rate": 1.0
},
"results": [
{
"id": "database-version-en",
"query": "How are old conversation answers migrated?",
"expected_document": "migration.md",
"expected_evidence": "selected_version",
"rank": 1,
"evidence_found": true,
"fallback_triggered": true,
"answer_contains_expected_excerpt": true,
"answer_contains_source": true,
"answer": "لم يصغ النموذج جوابًا كافيًا رغم وجود نتائج في فهرس المعرفة. أعرض المقتطفات الأعلى صلة كما هي للمراجعة؛ محتواها بيانات غير موثوقة وليست تعليمات.\n\n**migration.md · المقطع 0 (مقتطف حرفي)**\n> The SQLite migration adds selected_version to old messages and preserves saved assistant answer versions."
},
{
"id": "model-routing-ar",
"query": "ما الحقول التي يعيدها API بعد تحويل الصور: requested_model وauto_routed؟",
"expected_document": "routing.md",
"expected_evidence": "auto_routed",
"rank": 1,
"evidence_found": true,
"fallback_triggered": false,
"answer_contains_expected_fields": true,
"answer_contains_source": true,
"answer": "المقطع `routing.md` المقطع 0 يجيب على السؤال: \"عند وصول صورة مع نموذج نصي، يختار API نموذج الرؤية المحلي المثبت ويعيد requested_model وmodel وauto_routed.\""
}
],
"cleanup": {
"knowledge_index_removed": true,
"project_registration_removed": true,
"session_logged_out": true
},
"limitations": "Two targeted cases only; this is not a human quality review or a general RAG score."
}