RAG and data · 2026-08-18
Turkey's Constitutional Court AI Decision Support: RAG, OCR, and Data-Security Requirements
Read the Constitutional Court's AI decision-support specification across document volume, OCR, RAG, citations, performance, on-premises deployment, and ISO security requirements.
The problem and the decision boundary
According to the specification, 102,915 individual applications were under review as of 20 May 2026, with roughly 80,000 new applications entering each year. Manual reading and summarisation of multi-page forms, attachments, petitions, and linked court files creates substantial workload.
The requested solution is not a decision maker. It supports rapporteurs and specialists by converting documents into processable text, classifying them, retrieving similar decisions, citing sources, and preparing drafts. Judicial assessment and final authority remain with people.
Data volume and document processing
The document records 737,860 individual applications between 23 September 2012 and 20 May 2026. It identifies 295,000 inadmissibility commission decisions and 17,015 section decisions as training material, while defining exclusions for pending, locked, confidential, and undecided files.
The system must process PDF, scanned PDF, TIFF, DOC, DOCX, XLS, PNG, JPEG, BMP, and UDF files, using OCR or VLM methods for handwritten, typed, and scanned material. On an agreed test set without handwriting, extracted text must have no more than a one percent character-error rate.
| Area | Requirement | Verification |
|---|---|---|
| OCR | Maximum 1% character-error rate on non-handwritten test data | Fixed test set agreed with the Court |
| RAG | Cite at least 3 relevant application decisions | Links to source documents and case law |
| Classification | 95% weighted F1 for rights and intervention claims | Balanced, sufficient institutional test data |
| Decision draft | 95% target for content and inadmissibility assessment | Commission of Court rapporteurs |
| Concurrency | 50 users generating drafts | Load and stress test |
| Duration | 30-second average classification and draft under one minute | Batch and single-request performance report |
RAG, citations, and hallucination controls
The specification calls for semantic search or RAG to find similar applications, precedents, and Constitutional Court case law, with references to source documents in every output. Citation is not decorative; it enables rapporteurs to re-check a model result.
It also expects hallucination-reduction mechanisms such as reference systems or review by another AI model. A second model is not proof by itself. Independent source validation, rejection of missing references, and claim-to-passage links remain necessary.
How to read the two 95 percent targets
The specification contains two different 95 percent targets. One is weighted F1 for classifying rights and intervention claims, subject to sufficient balanced data. The other covers decision-draft content and inadmissibility assessment as evaluated by a commission of Court rapporteurs.
They should not be presented as one generic accuracy number. F1 is a statistical classification metric; rapporteur assessment requires a defined rubric and expert judgement. Splitting test data by period, rights class, and document type reduces the risk of a model appearing strong only on frequent categories.
Isolated deployment and security requirements
The system must run on Court-provided hardware using REST APIs and scalable Docker and Kubernetes architecture. All data must remain on Court servers with no data leaving the institution. Role-based access, encryption, immutable central logs, real-time resource monitoring, independent penetration testing, and source-code delivery are part of the scope.
Governance requirements include bias analysis under ISO/IEC TR 24027, systematic risk management guided by ISO/IEC 23894, alignment with ISO/IEC 42001 principles, and information-security processes under TS ISO/IEC 27001. Naming a standard is not evidence; the project must deliver analyses, risk records, and test reports.
Where LLMTR does not fit in this architecture
Because LLMTR is an external API gateway, it does not replace the core system required to keep every file on Court servers with no outbound data. Sending applications, case files, evidence, or decision drafts to an internet API cannot be presented as satisfying this specification.
A gateway could only be a separate comparison tool if the institution approved a non-sensitive, properly cleared evaluation set or general task. The selected provider's data policy would still require review; LLMTR not persisting prompt bodies in its usage database does not replace the upstream assessment.
The lesson for public legal-technology projects
The specification shows why legal AI is more than an LLM integration: document conversion, cleaning, classification, sourced retrieval, drafting, expert evaluation, MLOps, and security are delivered together. Acceptance is a set of layer-specific measurements rather than one model score.
This article is a technical analysis of the published specification and does not constitute legal advice. Processing individual-application data, defining access, or using any external service depends on the Court's current requirements and its information-security, compliance, and legal teams.
Frequently asked questions
Will the Constitutional Court AI system make decisions?
No. The specification describes a decision-support system that assists review and drafting; it does not replace judicial assessment.
Which documents are in scope?
Application forms and attachments, petitions, expert reports, witness statements, evidence, court decisions, and several file formats including UDF are described.
What is the OCR target?
No more than a one percent character-error rate on an agreed test set without handwriting.
Why does the project use RAG?
To retrieve similar applications, precedents, and case law and provide verifiable references to at least three relevant decisions and source documents.
Can Constitutional Court case data be sent to LLMTR's API?
The specification requires all data to remain on Court servers with no outbound transfer, so LLMTR's external API cannot replace the core workload.