AxonOS Assistant

The AxonOS Assistant is a native GTK application that brings AI-powered help directly into the desktop. It connects to a local Ollama instance, so inference runs on your own machine — no data leaves the environment unless you ask it to search the web.

🧠Dual-model inference

Uses a text model (e.g. gemma4:31b) for conversation and a vision model (granite3.2-vision) for analyzing what's on screen.

👁️Vision & screenshots

Captures and analyzes the desktop with intelligent resizing (1920×1080 → 1344×1344) to understand plots, viewers, and UI state.

🔎Web search

Searches and summarizes the web (via Brave) for research queries when you need fresh information.

🧰Tool discovery

Scans system directories and reports which scientific applications are installed and available.

Capabilities

Answer questions about AxonOS and its components.
Help with scientific computing workflows.
Search the web for research information and summarize results.
List available tools and installed software.
Guide you through using scientific applications.
Assist with data analysis and visualization tasks.
Analyze desktop screenshots and explain visualizations, plots, and interface elements.
Render Markdown in responses, with conversation memory across interactions.

How it works

On the hosted platform the assistant is already running in your desktop — just open it and start chatting. It talks to a local Ollama instance, so inference happens on the session's hardware.

When you ask a visual question, the assistant captures the screen, resizes the image for the vision model, and includes it as context. Text questions go to the conversational model, optionally augmented with web-search results.

Self-hosting: choose your models

If you run your own instance, you pick which models to install at build time via the Launcher or a build config:

json

{
  "ollama_models": ["gemma4:31b", "granite3.2-vision"]
}

Local-first by design

Inference is local via Ollama. The assistant only reaches the network when a web search is explicitly requested. This keeps research data on the machine you control.

Model Context Protocol (MCP)

The assistant codebase includes MCP support for connecting the AI to external tools and data sources in a standardized way. See the in-repo axonos_assistant/MCP_README.md for the current integration details.

GPU recommended

Larger models (like the default text model) run far faster with an NVIDIA GPU. On CPU-only hosts, consider configuring a smaller Ollama model in your build config.