Access Requirements

Access is available to KIT employees, KIT students, and individuals with guest or partner accounts. Use of the KI-Toolbox is permitted exclusively for work-related or academic purposes!

Access to the KI-Toolbox requires successful completion of the self-paced course “Understanding and Applying Generative AI” on KOALA (for employees) or the ILIAS learning platform (for students). After completing the training and logging back into the KI-Toolbox, access will be automatically activated.

As of July 1, 2026, all users have a monthly standard budget of $5 available for using external models. You can check your current budget usage and other usage information yourself in the MyLiteLLM application (at https://my.litellm.scc.kit.edu). For more information and to learn how you, as a KIT employee, can set an individual budget if needed, see the “Budget” section below.

Please be sure to review the Terms of Use and Privacy Policy of the KI-Toolbox before using the service.

Quick Start

1. Please first complete the self-paced course “Understanding and Applying Generative AI” on KOALA (for employees) or on the ILIAS learning platform (for students). Successful completion of the course is a mandatory requirement for admission to the KI-Toolbox.
2. Go to https://ki-toolbox.scc.kit.edu in your browser, click “Continue with KIT Account,” and log in.
3. If necessary, accept the privacy policy and terms of use (only upon first login).
4. If necessary, create a new chat thread by clicking “New Chat” in the left sidebar.
5. If necessary, change the desired AI model by opening the model selection dropdown at the top of the page.
6. Enter your request in the middle section of the page (“How can I help you today?”) and press Enter to submit it.

AI Models

Currently, various AI models are available for selection in the KI-Toolbox. The local models are hosted directly at the SCC, and data processing takes place exclusively at KIT. External models (GPT, Gemini, Claude) are sourced from the cloud via European data centers. The Azure Cloud (Microsoft) is used for OpenAI models, while we use the Google Cloud Platform for models from Google and Anthropic.

Standard Models

For users with little experience working with AI models, we provide two models with optimized system prompts:

• We recommend trying the “Standard-Local” model first. This model uses the local “Mistral Small 4” model and can therefore also be used to process personal data.
• For more complex tasks that do not involve personal data, the “Standard-External” model can be used; this model utilizes the OpenAI “GPT-5 mini” model from the Azure Cloud.

List of available models

Costs are listed in U.S. dollars per 1 million tokens and are converted to euros at the current daily exchange rate (currently $1 ≈ 0.85 €). The calculations are based on official pricing information from Azure and Google. When calculating the request, a distinction is made between input tokens (in = your input + context, e.g., files) and output tokens (out = AI output). Tokens are words or parts of words; as a guideline, 100 tokens ≈ 75 words.

Notes:
(1) For context lengths exceeding 272,000 tokens, higher prices apply for GPT-5.4 and 5.5: GPT-5.4: $5.50 in, $24.75 out; GPT-5.5: $11 in, $49.50 out
(2) Claude Fable 5 is currently unavailable due to U.S. regulations.

Caching

Caching model inputs not only enables faster processing but can also lead to significant cost savings. Short messages (up to a size of between 1,024 and 4,096 tokens, depending on the model) are generally never cached.

For the GPT and Gemini models, caching is handled by Azure and Google, respectively, at no additional cost and is therefore enabled by default. The cache retention time varies between 5 minutes and one hour. The price reduction for cache hits is typically 10% of the input cost (e.g., GPT-5.5: $0.55 instead of $5.50).

For requests to the Claude models, LiteLLM injects corresponding “Cache-Control” directives into the messages, causing the inputs to be cached for 5 minutes. This is currently not configurable. Anthropic charges an additional 25% surcharge on the input tokens for cache writes. For a successful cache retrieval (cache hit), only 10% of the cost is incurred.

Web Search

To ensure the model can access up-to-date information, web searches can be performed, and the results of these searches can be processed by the models. You can also explicitly instruct the model to perform a web search in the prompt. Please note that the web search must be enabled as a tool in the prompt (enabled by default). If the AI model wishes to perform a web search, it uses SearXNG, which is hosted at KIT. SearXNG forwards the search query to various search engines on the Internet and consolidates the results into a single output that can then be processed by the model. This process does not allow for any identification of the person who initiated the search.

Speech Recognition

To convert speech to text (Speech-to-Text, STT), the KI-Toolbox uses OpenAI’s “Whisper-Large-v3” model, which is hosted at KIT. For voice input, the KI-Toolbox offers the “Dictate” or “Voice Mode” functions, which can be found under “Prompt Input” on the right-hand side.

Speech Generation

To generate speech (Text-to-Speech, TTS), the local “Voxtral-4b” model from Mistral AI is used. The model runs exclusively at KIT. Speech generation is used in the KI-Toolbox when you have an AI model’s response read aloud via the speaker icon.

Image Generation

The local model “Flux.2 Dev” from Black Forest Labs is used for image generation. Flux 2 Dev runs entirely locally at KIT. Please note the license terms when reusing the generated images (specifically, for non-commercial use only). For image generation, you can use any local model and instruct the prompt to create an image. The selected model generates a detailed image description, which is passed on to the image generation model. Please note that image generation must be enabled in the chat for this to work (enabled by default for local models).

Budget

Using the external models (GPT, Gemini, Claude) incurs costs of varying amounts. The incurred costs are calculated per user and can be viewed via MyLiteLLM. You can find the costs per model in the model table above. Please note that a maximum budget of 5 USD per user per month is currently available for using external models. These costs are not currently passed on to users but are covered centrally. If the budget is exceeded, a corresponding error message will be displayed when using an external model (“ExceededBudget: End-User=xy1234 ∂does-not-exist.kit edus over budget. Spend=xx.xxxxxxx, Budget=5.0”). The budget is reset at midnight (UTC) at the beginning of each month.

If higher usage is required, KIT employees can set an individual budget in MyLiteLLM. The budget may only be adjusted with prior approval from the budget manager. Unauthorized changes to the budget without consultation with the institute or DE head are prohibited! When a budget change is made, an email is sent to the user and to the responsible head as additional confirmation. For individually adjusted budgets, the SCC will charge the user’s cost center the full amount of the actual costs incurred at the end of the month. To do this, we convert the dollar amount to euros at the current exchange rate (currently 1 U.S. dollar is approximately 0.80–0.90 euros).

The budget adjustment is permanent, meaning it affects not only the current month but also subsequent months. Currently, adjustments of up to $100 are possible. If you have a higher need, please contact us.

The use of local models does not count toward the budget and remains available even if the budget is exceeded.

API Key

You can also use the AI models via an API (OpenAI-compatible interface). You can generate your own API key in the KI-Toolbox by going to “Settings” (bottom left of the menu) -> “Accounts” -> “API Key” (click “Show” if necessary). Please treat your API key like a personal password and protect it from unauthorized access. If you suspect that your API key may have been compromised, regenerate it immediately via the KI-Toolbox.

For external tools, the base URL for the API is usually “https://ki-toolbox.scc.kit.edu/api”. You can find the model ID in the table above or by hovering your mouse pointer over the model selection in the KI-Toolbox to display the tooltip. An overview of the available API endpoints can be found at https://ki-toolbox.scc.kit.edu/docs.

For more information on using the API, see the ZML documentation.

Service Accounts

To use the AI Toolbox in chatbots or other third-party applications that require permanent API access, we recommend using a non-personal API key via a KIT service account. You can request a service account at the SCC ServiceDesk or through the ticket system. Please include “Use of the AI Toolbox API” or similar in the service description so that the account can be activated for the AI Toolbox. Existing service accounts can also be activated via the ServiceDesk or the ticket system. Once activated, you can use the service account to generate an API key through the AI Toolbox.

Groups

Groups allow you to share various artifacts (models, knowledge, notes, prompts, etc.) with multiple people to enable collaborative editing or use. You cannot create groups directly in the KI-Toolbox; however, group synchronization via the SCC Group Management system is possible upon login. Specifically, groups from the SCC Group Management system with names in the format "<OE>-ki-toolbox-<name>" are automatically created in the KI-Toolbox when a person who is a member of that group logs in, and that person’s group membership is updated (i.e., new groups are created, new group memberships are added, and group memberships that no longer exist are removed).

Hardware and Software

The KI-Toolbox software stack consists exclusively of open-source software and runs in a Kubernetes environment on 4 nodes (DELL XE7745) at the SCC. Each node is equipped with 8 NVIDIA RTX6000 PRO Blackwell GPU cards. Open WebUI is used as the front end, while LiteLLM is used for load balancing, cost management, and logging. The local AI models are run using vLLM and LMCache and are loaded from the local S3 at the SCC.