🚀 Using Qwen on the Mahidol Cluster

Author: Snit Sanghlao, AI Assistant Gemini

High-Performance Coding & Chat via the AI Center Qwen endpoint

This guide provides the configuration and setup for the Qwen large language model served by the Mahidol AI Center. It is aimed at coding assistants (Continue.dev, agent frameworks) and general chat via an OpenAI-compatible API.

---

📌 Stable Endpoint (read this first)

All examples target a version-independent alias:

Setting	Value
API Endpoint	https://aicenter.mahidol.ac.th/qwen/v1
Model ID	qwen
API style	OpenAI-compatible (/chat/completions, /completions, /models)

The /qwen/v1 route is a stable alias maintained by the AI Center. It always points to the current production Qwen deployment. When the model is upgraded, the alias is repointed on the server side and your configuration does not change — the endpoint URL and the model ID qwen stay the same.

[!NOTE] For the AI Center administrator: the alias requires an NGINX route /qwen/v1/ → <internal vLLM service> and the vLLM server started with --served-model-name qwen. Serving-side details (hardware, parallelism, memory settings, node placement) are intentionally omitted from this guide.

Check what is currently deployed

curl -sk https://aicenter.mahidol.ac.th/qwen/v1/models

The response reports the upstream checkpoint (root) and the maximum context length (max_model_len) of whatever version is live. At the time of writing this resolves to Qwen3.8-27B with a 256K (262 144-token) context window.

---

💻 IDE Integration (VS Code)

Continue.dev Configuration

To use Qwen as your coding assistant, update your ~/.continue/config.yaml:

name: Local Config
version: 1.0.0
schema: v1
models:
  - name: Qwen
    provider: openai
    model: qwen
    apiBase: https://aicenter.mahidol.ac.th/qwen/v1
    systemMessage: "You are a helpful assistant."
    apiKey: "sk-xxxx"
    contextLength: 262144   # 256K context; lower it if your client is slow
    maxTokens: 4096         # Leave room for the model to respond
    requestOptions:
      extraBodyProperties:
        chat_template_kwargs:
          enable_thinking: false
context:
  - provider: web
    params:
      engine: "searxng"
      query: ""
      searxngBaseUrl: https://aicenter.mahidol.ac.th/metasearch/
      n: 5
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

---

🌐 Open WebUI Deployment

The easiest way to interact with the model is via Open WebUI. Run the following Docker command to connect to the cluster:

docker run -d -p 3000:8080 \
  --name open-webui \
  --restart always \
  -e WEBUI_AUTH=False \
  -e OPENAI_API_BASE_URL=https://aicenter.mahidol.ac.th/qwen/v1 \
  -e OPENAI_API_KEY=sk-xxxx \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:main

[!TIP] Use the -d (detached) flag instead of -it to keep the UI running in the background after you close your terminal.

---

🧪 Verification & Testing

Connectivity Test (cURL)

Run this in your terminal to verify the endpoint is reachable and the model is loaded:

curl https://aicenter.mahidol.ac.th/qwen/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-xxxx" \
  -d '{
    "model": "qwen",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "temperature": 0.7
  }'

---

🛠 Troubleshooting & Maintenance

Symptom	Action
404 / model not found	Confirm model is set to qwen; run curl .../qwen/v1/models to see the served IDs.
CUDA Out of Memory (self-hosted)	Lower gpu-memory-utilization or reduce max-model-len on your own server.
504 Gateway Timeout	The model is large; increase your client-side timeout (e.g., NGINX proxy-read-timeout).
401 Unauthorized	Verify your sk-xxxx API key is passed in the Authorization header.

Container Management

# View real-time logs
docker logs -f open-webui

# Stop and Clean up
docker stop open-webui && docker rm open-webui

---

📝 Usage Notes

• Version independence: Always target https://aicenter.mahidol.ac.th/qwen/v1 with model qwen. Do not hard-code a version number — when the deployment is upgraded the alias moves and your config keeps working.

• Privacy: All data remains within the Mahidol University infrastructure.

• Security: Never commit your actual apiKey to a public GitHub repository.

---

Last Updated: 2026-09-07