How to build an in-house AI assistant: architecture decisions for public institutions

What is an in-house AI assistant, and what is a public institution actually deciding?

An in-house AI assistant is a question-and-answer interface an institution opens only to its own staff, grounded in the institution's own documents and data. For public institutions the real decision is not which model to pick; it is where the data will be processed, which work will be delegated to the assistant, and who signs off on the output. These three decisions drive the cost and the legal exposure of the deployment far more than the model choice does.

Turkey has a verifiable state-side precedent for this approach. The Presidency's Directorate of Communications (İletişim Başkanlığı) announced CibGPT, a closed-loop AI platform for its own personnel, in a statement dated 13 June 2026. According to the Directorate's own statement, it built "an autonomous artificial intelligence architecture using our own GPU servers and infrastructure" and deployed "air-gapped domestic LLM and VLM models with institution-specific fine-tuning, enabling the secure processing of sensitive data." The same statement covers İlgen, a coding agent that accelerates software development, and ULAK, which allows natural-language interaction with legacy systems.

The Directorate also stated that it "extended this transformation across the public sector by providing integration guides to public institutions." That does not mean every institution should build the same architecture. The legal frame for the architecture decision is Presidential Circular No. 2019/12 on Information and Communication Security Measures, published in the Official Gazette (Resmî Gazete) No. 30823 on 6 July 2019. Item 3 of the circular states that data belonging to public institutions and organisations shall not be stored in cloud storage services, other than the institutions' own dedicated systems or domestic service providers under institutional control (translated from the Turkish original).

The circular separates critical data from other data. Item 2 requires that critical data held by public institutions be kept on a secure network that is closed to the internet and physically secured, with controlled device access and tamper-protected log retention (translated from the Turkish original). That distinction is the foundation of the architecture decision: a workload touching population or health records and a workload drafting internal correspondence do not require the same infrastructure.

• Decision 1 — Where the data is processed: the institution's own network, a Turkey-hosted service, or an overseas provider.
• Decision 2 — What the assistant will and will not do: the scope must be written down and bounded.
• Decision 3 — Who approves the output: outputs usable directly must be separated from outputs requiring sign-off.
• Decision 4 — Who may ask what: the existing authorisation matrix must survive on top of the assistant.
• Hardware or service procurement carried out before these decisions typically ends in paying for capacity nobody uses.

Three architecture options: own GPU infrastructure, Turkey-hosted API, global API

Public institutions building an in-house AI assistant choose, in practice, between three architectures. Not every institution needs to buy its own GPUs, and for most institutions that option is not economically defensible. The right choice depends on how critical the data is, whether usage is sustained or sporadic, and whether the institution has permanent technical staff.

The economics of owning GPUs depend entirely on volume. Hardware cost is fixed and continues while the system sits idle; API cost rises only with use. At sustained high volume, owned infrastructure becomes cheaper per request; at low or spiky volume, the cost of idle capacity exceeds the total cost of the API model. Publishing a general break-even figure would be misleading; each institution has to compute it from its own estimated daily request count and average request length.

Owned infrastructure has three genuine advantages that an API cannot substitute. First, air-gapped operation: it is the only option that can literally satisfy the circular's requirement of a secure network closed to the internet for critical data. Second, institution-specific fine-tuning: modifying model weights with institutional data is only possible when you run the model yourself. Third, provider independence: a third party's pricing, quota, or model-retirement decision cannot interrupt the institution's service. For an institution that handles critical data and keeps permanent technical staff, these three can outweigh the cost advantage of the API model.

For most institutions, the sensible path is not to commit to a single architecture up front. Piloting a non-critical workload over an API, measuring the real scope and volume, and then deciding on infrastructure investment based on that measurement carries far less risk than buying hardware first and looking for work to justify it. With an OpenAI-compatible interface, that migration changes configuration rather than application code.

Scoping: what the assistant will do, and what it will never do

The most frequently skipped step in in-house AI assistant projects is writing down what the assistant will not do. An assistant with an undefined scope gets pointed at everything, and the first bad output costs the whole project its credibility. Scope is an administrative decision rather than a technical one; the relevant unit heads and the legal counsel's office should approve it together.

The Turkish Data Protection Authority (KVKK) guide "İş Yerlerinde Üretken Yapay Zekâ Araçlarının Kullanımı" (Use of Generative AI Tools in Workplaces), published on 5 March 2026, draws a clear line on trusting output. The guide states that generative AI output should not be treated as the basis for final decisions, and should instead be handled as a supporting element under human oversight and evaluation (translated from the Turkish original). The guide also names the habit of accepting automated output without scrutiny as "automation bias".

The scope document should consist of two lists. The first names the work the assistant supports; the second names the work it must never touch. The second list matters more, because it is the only reference staff will reach for at the moment of doubt.

• Work the assistant can support: searching legislation and internal directives, drafting internal correspondence, summarising meeting notes and institutional documents, drafting reports, terminology and translation support, answering common HR questions.
• Work the assistant will not do: producing the final text of an administrative act, opining on disciplinary or personnel evaluations, recommending a tender evaluation decision, producing text that substitutes for legal opinion, sending unreviewed replies to citizens.
• Information never supplied as input under the non-air-gapped architectures: health data, biometric data, investigation files, classified documents, and tender information not yet made public.
• The scope document carries a version number and an effective date, and returns for approval whenever the scope widens.
• The assistant's response to an out-of-scope question is drafted in advance and enforced in the system prompt.

Grounding in institutional knowledge (RAG) and per-unit access separation

The value of an in-house AI assistant comes from the quality of the institutional documents it is connected to. A model's general knowledge does not include your directive, your fee schedule, or your internal circular; the retrieval-augmented generation layer is what carries that knowledge to the assistant. The real work in building this layer is not on the model side but in curating an approved source set and assigning ownership of it. In practice, most projects fail not because of model quality but because nobody owns keeping the source set current.

Every document in the approved source set needs three attributes: its version, its effective date, and the unit that owns it. As long as a repealed directive stays in the source set, the assistant will cite it as if it were in force, and the error belongs to data governance rather than to the model. Showing the name and effective date of the document behind each answer is the minimum condition for staff to be able to verify anything.

Access separation is a control distinct from the source set, and more critical than it. Personnel files, investigation records and tender pre-qualification documents are not open to everyone inside the institution either; an assistant that ignores this effectively dissolves the existing authorisation matrix. In practice each unit gets its own index and its own API key, queries are routed to the index of the unit the user belongs to, and no answer is generated from an unauthorised scope.

The legal basis for these controls is Article 12 of Law No. 6698 (KVKK), which requires the data controller to take all necessary technical and organisational measures to provide an appropriate level of security in order to prevent unlawful processing of personal data, prevent unlawful access to personal data, and ensure the retention of personal data (translated from the Turkish original). The assistant should be designed as an interface that enforces the institution's existing authorisation matrix, not as a shortcut around it.

• A single unit owns the source set; the person responsible for updates and the update cadence are written down.
• Each document is indexed with its version, effective date and owning unit; repealed documents are removed from the set.
• The assistant shows the document and effective date behind each answer, and says it cannot find an answer when the source set has none.
• Each unit gets its own index and its own API key; queries are routed by the user's unit.
• Spend caps and rate limits are set per key; per-unit usage reports are reviewed on a regular cycle.
• Query and access logs are retained as an audit trail, so who accessed which scope can be demonstrated afterwards.

Human review, usage policy, staff training and measurement

Once an in-house AI assistant goes live, the first document the institution needs to write is a usage policy. The KVKK guide of 5 March 2026 describes the contents of such a policy in detail and explicitly does not recommend a prohibition-based approach. The guide assesses that prohibitive approaches may encourage uncontrolled use, by pushing employees to use generative AI tools outside institutional control and visibility (translated from the Turkish original).

The guide calls this situation "Gölge YZ" (Shadow AI) and defines it as the use of generative AI tools in work processes by employees without the knowledge, approval or institutional control of the organisation (translated from the Turkish original). This is one of the strongest arguments for building an in-house assistant at all: if staff already use generative AI and the institution offers no controlled alternative, that use simply continues outside institutional visibility. The guide also recommends limiting employees to tools the organisation has designated with defined terms of use, and treating access on a role and need basis.

The approval flow splits in two. Output that stays inside the institution and carries no administrative consequence can be used directly under the member of staff's own judgement. Output that leaves the institution, requires a signature, or constitutes an administrative act cannot be used without the sign-off of the authorised manager. Writing this split explicitly into the usage policy spares staff from deciding case by case.

Without measurement there is no basis for widening the scope. An institution that cannot see where the assistant is actually used and where it goes wrong can neither justify additional budget nor narrow the scope responsibly. This is also why the guide recommends establishing feedback mechanisms where employees can share their experience and the problems they encounter. Institutions may also consult the Authority's "Üretken Yapay Zekâ ve Kişisel Verilerin Korunması Rehberi (15 Soruda)" (Guide on Generative AI and Personal Data Protection, in 15 Questions), which the 5 March 2026 guide explicitly references.

• The usage policy lists permitted tools, permitted purposes, the types of information that may be supplied as input, and the rules for using output.
• The policy is announced internally, kept somewhere staff can reach easily, and updates are communicated promptly.
• Staff training covers how to verify output and what information must never be entered, rather than how to operate the assistant.
• Minimum metrics to track: weekly active users, queries per unit, the share of not-found-in-sources answers, the number of reported bad answers, and average response time.
• The error reporting channel lives inside the assistant's own interface; every report is routed to the owner of the source set.
• Decisions to widen the scope are not taken before three months of usage and error data have been reviewed.

Where LLMTR fits into this deployment

LLMTR supplies only the model access layer of an in-house AI assistant; scoping, the approved source set and the approval flow remain the institution's own decision and responsibility. The platform exposes a single OpenAI-compatible /v1 surface through which both Turkey-hosted models and global providers are called from the same catalogue. That structure turns migration between the three architectures above from an application code change into a configuration change.

The per-unit access separation described in the fourth section is the one item with a direct counterpart on the platform side: each unit can hold its own API key, its own rate limit, its own spend cap and its own usage report. User prompts and model response bodies are not written to the usage and billing database. Customer API keys are not stored in plain text; they are stored as SHA-256 hashes. Credit top-ups carry an 8% platform margin; no margin is added to model prices.

LLMTR does not replace owned GPU infrastructure and does not claim to. Where critical data has to be processed on a network closed to the internet, the institution needs a deployment running on its own hardware; in that scenario LLMTR remains a complementary layer for the workloads that are not classified as critical. Which of an institution's data counts as critical, and which provider suits which workload, is an assessment for the institution's own information security and legal units.

The example below shows a model call from an institutional assistant using an OpenAI-compatible client. The architecture decision collapses into two lines: the base URL and the model identifier. If the institution later moves to its own infrastructure, the base URL points at its own endpoint and the rest of the application code stays the same. On the procurement side this work is usually specified as AI software leasing or a software service purchase; this is its technical counterpart.

• Turkey-hosted model identifiers can be called directly from the catalogue: llmtr/gemma-4, llmtr/qwen3-6-35b, llmtr/ornith-1-35b, among others.
• A separate API key, rate limit, spend cap and usage report can be defined for each unit.
• User prompts and model response bodies are not written to the usage and billing database.
• Customer API keys are stored as SHA-256 hashes, never in plain text.
• Credit top-ups carry an 8% platform margin; no margin is added to model prices.
• Because the /v1 surface is OpenAI-compatible, moving to owned infrastructure is limited to changing the base URL.

A model call from an institutional assistant: changing architecture changes only the base URL and the model identifier

import os
from openai import OpenAI

# The architecture decision collapses into two lines: base URL and model id.
client = OpenAI(
    api_key=os.environ["INSTITUTION_LLM_KEY"],  # never hard-code the key
    base_url="https://llmtr.com/v1"             # points at your own endpoint if you self-host
)

SYSTEM = (
    "You are the internal assistant of a public institution. "
    "Answer only from the approved institutional sources provided to you. "
    "If the sources do not cover it, say 'not found in institutional sources'. "
    "End every answer with the document name and its effective date."
)


def ask_institution(question: str, approved_sources: str,
                    model: str = "llmtr/gemma-4") -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content":
                f"Approved sources:\n{approved_sources}\n\nQuestion: {question}"}
        ],
        temperature=0.2
    )
    return response.choices[0].message.content


# Switching models is a one-line change; the rest of the application stays put.
# ask_institution(question, sources, model="llmtr/qwen3-6-35b")
# ask_institution(question, sources, model="llmtr/ornith-1-35b")

Sources

The regulatory references in this article were verified against the primary sources listed below. Last checked: 13 August 2026.

This content is informational and does not constitute legal advice. The final assessment rests with the institution's compliance and legal units.

• Law No. 6698 on the Protection of Personal Data (KVKK), Article 12 — mevzuat.gov.tr
• Guide on the Use of Generative AI Tools in Workplaces (İş Yerlerinde Üretken Yapay Zekâ Araçlarının Kullanımı), 5 March 2026 — kvkk.gov.tr
• Presidential Circular No. 2019/12 on Information and Communication Security Measures, Official Gazette (Resmî Gazete) No. 30823, 6 July 2019 — resmigazete.gov.tr
• Head of Communications Burhanettin Duran's statement on the Directorate's artificial intelligence initiatives, 13 June 2026 — iletisim.gov.tr
• Guide on Generative AI and Personal Data Protection in 15 Questions (Üretken Yapay Zekâ ve Kişisel Verilerin Korunması Rehberi) — kvkk.gov.tr

Steps to build an in-house AI assistant in a public institution

Steps for a public institution building an AI assistant for its own staff, from scoping through to measurement.

1. Write down the scope and the excluded work. Produce two separate lists: the work the assistant will support, and the work it must never do. Have the relevant unit heads and the legal counsel's office approve both. Give the document a version number and an effective date, and send it back through approval whenever the scope widens.
2. Classify the data and choose the architecture. Split the data the assistant will touch into critical and non-critical. Circular No. 2019/12 requires critical data to be kept on a secure network closed to the internet. Consider owned infrastructure for workloads involving critical data and a Turkey-hosted model over API for the rest. Do not buy hardware before you have an estimated daily request count.
3. Curate the approved institutional source set and assign ownership. Gather directives, circulars, fee schedules and FAQ documents into a single set. Attach a version, effective date and owning unit to each document. Remove repealed documents. Name in writing the unit responsible for updating the source set and the update cadence.
4. Deploy in a pilot unit and define access separation. Start with a single unit. Define a separate index and a separate API key per unit, with a rate limit and a spend cap on each key. In the system prompt, require the assistant to rely only on approved sources, to say it cannot find an answer when the sources do not cover the question, and to cite the document it used.
5. Publish the usage policy and train staff. State in the policy which tools may be used, for which purposes, what types of information may be supplied as input, and which outputs require sign-off. Announce the policy internally and keep it accessible. In training, cover how to verify output and what must never be entered rather than how to operate the assistant.
6. Measure, open a feedback channel, and widen the scope gradually. Track weekly active users, queries per unit, the share of not-found-in-sources answers, and the number of reported bad answers. Put the error reporting channel in the assistant's interface and route reports to the source-set owner. Widen the scope only after reviewing three months of usage and error data.

Frequently asked questions

Does every public institution need to buy its own GPU servers for an in-house AI assistant?

No. Owned GPU infrastructure makes sense for institutions that handle critical data, have sustained high-volume usage, and keep permanent technical staff. Circular No. 2019/12 requires critical data to be kept on a secure network closed to the internet, but not every workload in an institution involves critical data. Drafting internal correspondence, searching legislation or summarising meeting notes can run against a Turkey-hosted model over an API. Because hardware cost continues while the system sits idle, owned infrastructure loses on total cost against the API model at low or spiky volume.

What is CibGPT, and can other public institutions use it?

CibGPT is the closed-loop AI platform the Presidency's Directorate of Communications (İletişim Başkanlığı) built for its own personnel, announced in a statement dated 13 June 2026. According to the Directorate's statement, the system runs on the institution's own GPU servers with air-gapped domestic LLM and VLM models and institution-specific fine-tuning. The same statement says the transformation was extended across the public sector by providing integration guides to public institutions. There is no information indicating that CibGPT itself is offered to other institutions as a service; what is available is the architecture as a precedent for institutions building their own.

How should an in-house AI assistant be specified in a public procurement?

In Turkish public procurement practice this work is usually specified as AI software leasing or a software service purchase, while the technical specification covers model access, token- or request-based usage, the hosting country, and access authorisation. The specification should state the scope and the excluded work in writing, name explicitly the country where data will be processed, require per-unit authorisation and usage reporting, and define responsibility for error reporting and source-set updates. Procurement carried out without a defined scope tends to end in paying for capacity nobody uses.

Will institutional data be used to train the model behind the assistant?

That depends on the chosen architecture and the provider's contract, and it is a point each institution must verify separately. In a deployment running on owned infrastructure, data never leaves the institution's network, so the question does not arise. In API-based architectures, the provider's data handling and training behaviour must be stated explicitly in the contract. The KVKK guide of 5 March 2026 notes that sharing commercially sensitive information with third-party tools carries the risk that such information may be used in model development or improvement processes. On the LLMTR side, user prompts and model response bodies are not written to the usage and billing database.

How do you manage the risk of the assistant giving wrong or fabricated answers?

The risk is managed in three layers. First, the source set: the assistant should answer only from approved institutional documents and say it cannot find an answer when the sources do not cover the question. Second, transparency: every answer should show the document it relies on and that document's effective date, so staff can verify it. Third, human review: the KVKK guide of 5 March 2026 recommends that generative AI output not be treated as the basis for final decisions and instead be handled as a supporting element under human oversight. An error reporting channel should sit in the assistant's interface, with reports routed to the owner of the source set.

How does an in-house assistant differ from the general AI tools staff already use on their own?

An in-house assistant is bound to sources the institution has approved, enforces the institution's authorisation matrix, and its use is visible to the institution. General tools that staff adopt individually have none of those three properties. The KVKK guide of 5 March 2026 calls that second situation Shadow AI (Gölge YZ) and defines it as use occurring without the organisation's knowledge, approval or institutional control. The guide assesses that outright prohibition does not produce realistic results and can push staff to use these tools outside institutional visibility. Offering a controlled alternative is one of the core reasons to build an in-house assistant.

Related posts

• Are Turkey's domestic LLMs enough? BILGE, KUMRU, MAIN and multi-model strategy
• BİGR-compliant AI architecture: matching model choice to data classification in Turkish public institutions
• Municipal AI call centers in Turkey: tender analysis and solution-line design

• Model catalog
• Security documentation
• API keys
• Usage tracking