# chat_js Plugin Framework Developer API

Date: 2026-07-03
Repository: `C:\Users\Tee\projects\llmloader2`
Related reference: `docs/chat_js_plugin_framework_hook_inventory.md`
Playground guide: `docs/playground_plugin_guide.md`

## Purpose

This document turns the plugin hook inventory into a developer-facing API guide.

It explains:

- how frontend plugins are discovered and registered
- which hook surfaces `chat_js` exposes
- what each hook is for
- what data a plugin receives
- how backend helper plugins are mounted
- how frontend plugins should talk to backend helpers
- how model loader and AI route plugin families fit into the overall design

The intended use is practical plugin development inside `llmloader2` without adding plugin-specific logic to `chat_js.js` or `app.py`.

## Design Principle

The framework is built around extension points, not one-off framework edits.

In practice this means:

- frontend behavior belongs in `gui_js/plugins/<plugin>/plugin.js`
- frontend-facing server endpoints belong in `plugins/gui_helpers/<plugin>/routes.py`
- model/runtime integration belongs in `plugins/model_loader/*`
- AI routing behavior belongs in `plugins/ai_routes/*`

If a plugin can be built with an existing slot or helper API, it should use that instead of modifying the framework.

## 1. Frontend Plugin Module Contract

## What a frontend plugin is

A frontend plugin is a browser-side module under:

- `gui_js/plugins/<plugin>/plugin.js`

It is discovered from:

- `gui_js/plugins/manifest.json`

At runtime, `chat_js.js` loads the module and calls `register(host)`.

## Expected export shape

The minimal plugin shape is:

```js
const meta = {
  plugin_id: "example_plugin",
  name: "Example Plugin",
  kind: "ui",
  description: "Example plugin for chat_js.",
};

const plugin = {
  id: meta.plugin_id,
  name: meta.name,
  kind: meta.kind,
  description: meta.description,
  meta,
  register(host) {
    // hook registration here
  },
};

export default plugin;
```

## Notes

- `register(host)` is the main entrypoint.
- Plugins usually do not instantiate themselves elsewhere.
- Registration should be idempotent in spirit. Avoid spawning duplicate intervals, duplicate DOM, or duplicate global listeners on repeated enable/disable cycles.

## Example: minimal topbar plugin

```js
const meta = {
  plugin_id: "hello_topbar",
  name: "Hello Topbar",
  kind: "ui",
  description: "Adds a small topbar label.",
};

function buildNode() {
  const el = document.createElement("div");
  el.textContent = "Hello";
  el.style.fontSize = "12px";
  return el;
}

export default {
  id: meta.plugin_id,
  name: meta.name,
  kind: meta.kind,
  description: meta.description,
  meta,
  register(host) {
    host.addTranscriptTopbar(() => buildNode(), "right");
  },
};
```

## 2. Frontend Plugin Discovery API

## Manifest entry format

Each frontend plugin is listed in `gui_js/plugins/manifest.json`.

Typical entry:

```json
{
  "id": "theme_demo",
  "name": "Theme Demo",
  "kind": "ui",
  "description": "Switch between system/light/dark themes with accent colors.",
  "path": "./plugins/theme_demo/plugin.js"
}
```

## Required fields

- `id`: stable plugin identifier
- `name`: display name
- `kind`: plugin category shown in GUI/plugin management
- `description`: human-readable description
- `path`: browser module path

## Guidance

- Keep `id` stable. It is used in client enablement, permission checks, and helper route gating.
- The frontend plugin id should match the backend helper `GUI_PLUGIN_ID` when the plugin has a corresponding helper router.

## 3. Frontend Host API

The `host` object passed into `register(host)` is the main plugin API surface.

This section groups it by purpose.

## 3.1 UI placement hooks

These hooks add plugin-owned UI into framework-managed positions.

### `host.addToolbarAction(action)`

Adds a button/action into the toolbar action area.

Use it when:

- the plugin needs a compact global action
- a panel is not necessary for the initial entrypoint

Typical action shape:

```js
host.addToolbarAction({
  id: "example-open",
  label: "Example",
  title: "Open example",
  onClick(ctx) {
    ctx.openPluginPanel?.("example_plugin");
  },
});
```

### `host.addTopRightIconRow(nodeOrFactory)`

Adds a widget to the top-right icon/status row.

Use it for:

- compact status chips
- alert indicators
- launch buttons

Example:

```js
host.addTopRightIconRow((ctx) => {
  const btn = document.createElement("button");
  btn.textContent = "Jobs";
  btn.addEventListener("click", () => ctx.openPluginPanel?.("ai_jobs"));
  return btn;
});
```

### `host.addTranscriptTopbar(nodeOrFactory, side)`

Adds a widget to the transcript topbar.

Parameters:

- `nodeOrFactory`: DOM node or function returning a DOM node
- `side`: usually `"left"` or `"right"`

Used by:

- `theme_demo`
- `language`
- auth-related plugins

Example:

```js
host.addTranscriptTopbar((ctx) => {
  const wrap = document.createElement("div");
  wrap.textContent = "Context Ready";
  return wrap;
}, "left");
```

### `host.addTranscriptBottombar(nodeOrFactory, side)`

Adds a widget to the lower transcript bar near the composer.

This is one of the most common hooks for lightweight plugin controls.

Used by:

- `media_menu_demo`
- `media_upload`
- `voice_stt`
- `agent_flow`

Example:

```js
host.addTranscriptBottombar((ctx) => {
  const btn = document.createElement("button");
  btn.textContent = "Attach";
  btn.addEventListener("click", () => {
    ctx.log?.("Attach clicked", "info");
  });
  return btn;
}, "left");
```

### `host.addComposerLeft(nodeOrFactory)`

Adds plugin UI to the composer-left slot.

Use it when the plugin needs to live directly beside the text entry area rather than in a transcript bar.

Example:

```js
host.addComposerLeft(() => {
  const badge = document.createElement("div");
  badge.textContent = "Mode A";
  return badge;
});
```

### `host.addPanelTab(tab)`

Registers a plugin panel in the GUI plugin tools area.

Typical tab shape:

```js
host.addPanelTab({
  id: "example_plugin",
  title: "Example",
  render(container, ctx) {
    container.innerHTML = "";
    const box = document.createElement("div");
    box.textContent = "Example panel";
    container.appendChild(box);
  },
});
```

Optional properties commonly used:

- `id`
- `title`
- `render(container, ctx)`
- `renderFull(container, ctx)`
- `windowType`
- `pluginId`

`windowType: "full"` is used when the plugin wants a larger dedicated full-view experience.

Example:

```js
host.addPanelTab({
  id: "pins",
  title: "Pins",
  windowType: "full",
  render(container, ctx) {
    container.textContent = "Pinned messages";
  },
  renderFull(container, ctx) {
    container.textContent = "Full pin board";
  },
});
```

## 3.2 Message and transcript rendering hooks

These hooks customize how messages render after they are loaded from session state.

### `host.addMessagePreRenderer(renderer)`

Runs before normal message rendering.

Use it to:

- mutate a message before render
- suppress a message
- convert a message into a special local render form

Expected behavior:

- return the original or modified message
- return `false` to skip
- return `{ skip: true }`
- return `{ msg: nextMsg }`

Example:

```js
host.addMessagePreRenderer((msg, ctx) => {
  if (msg?.meta?.hidden_by_plugin) return false;
  if (msg?.meta?.badge) {
    return {
      msg: {
        ...msg,
        author: `${msg.author || "User"} [${msg.meta.badge}]`,
      },
    };
  }
  return msg;
});
```

### `host.addMessageRenderer(renderer)`

Lets a plugin claim full rendering of a message node.

Use it when:

- a message should render as a custom bubble
- the message contains multimodal or domain-specific layout

If the renderer returns a node, that node replaces the default message bubble rendering.

Example:

```js
host.addMessageRenderer((msg, ctx) => {
  if (!msg?.meta?.invoice_preview) return null;
  const wrap = document.createElement("div");
  wrap.className = "message assistant";
  const bubble = document.createElement("div");
  bubble.className = "bubble";
  bubble.textContent = `Invoice total: ${msg.meta.invoice_preview.total}`;
  wrap.appendChild(bubble);
  return wrap;
});
```

### `host.addBlockTransformer(transformer)`

Transforms parsed message blocks before rendering.

Use it when:

- text patterns should become structured blocks
- markdown output should be normalized into a plugin-specific block type

Transformers are sorted by `priority`.

Example:

```js
function transformBlocks(blocks) {
  return (blocks || []).map((block) => {
    if (block?.type === "text" && String(block.text || "").startsWith("NOTICE:")) {
      return { type: "notice", text: String(block.text || "").slice(7).trim() };
    }
    return block;
  });
}

transformBlocks.priority = 50;
host.addBlockTransformer(transformBlocks);
```

### `host.addBlockRenderer(renderer)`

Renders a plugin-owned block type.

Use it together with a block transformer or when a block type is already present.

Example:

```js
host.addBlockRenderer((block) => {
  if (block?.type !== "notice") return null;
  const box = document.createElement("div");
  box.className = "notice-block";
  box.textContent = block.text || "";
  return box;
});
```

### `host.addMessageFooterItem(item)`

Adds action UI below a message.

Typical shape:

```js
host.addMessageFooterItem({
  roles: ["assistant"],
  align: "right",
  render(msg, ctx) {
    const btn = document.createElement("button");
    btn.textContent = "Save";
    btn.addEventListener("click", () => {
      ctx.log?.(`Saved ${msg.msg_id}`, "info");
    });
    return btn;
  },
});
```

Use it for:

- message actions
- metrics
- pin/share/copy extensions

## 3.3 Send and completion hooks

These hooks are central to plugin workflow behavior.

### `host.addSendHook(handler, options)`

Runs after the user message is inserted locally, but before the completion request is sent.

This is useful because the user sees instant transcript feedback, while the plugin still gets a chance to adjust behavior before the backend call.

The handler receives:

- `payload`
- plugin runtime context

Payload typically includes:

- `pid`
- `sid`
- `text`
- `client_msg_id`
- `handled`

Possible handler outcomes:

- return nothing to leave payload unchanged
- return `{ text: "new text" }` to mutate outgoing text
- return `{ handled: true }` to stop default completion flow
- return `{ cancel: true }` to cancel and restore the draft

Example: append a command tag

```js
host.addSendHook((payload) => {
  if (!payload.text.startsWith("/search ")) return payload;
  return {
    ...payload,
    text: `<search>${payload.text.slice(8).trim()}</search>`,
  };
});
```

Example: handle the request entirely inside the plugin

```js
host.addSendHook(async (payload, ctx) => {
  if (!payload.text.startsWith("/echo ")) return payload;
  const body = payload.text.slice(6);
  await ctx.appendMessage({
    msg_id: ctx.randomId("assistant"),
    role: "assistant",
    content: `Echo: ${body}`,
  }, payload.sid);
  return { handled: true };
}, { timeoutMs: 5000 });
```

### `host.addCompletionPayloadHook(handler)`

Runs when the final completion payload is assembled from session history and preferences.

Use it to:

- inject system prompts
- attach plugin metadata
- transform the last user message into multimodal parts
- add plugin-specific ext fields

Example:

```js
host.addCompletionPayloadHook((payload, ctx) => {
  payload.ext = payload.ext || {};
  payload.ext.example_plugin = {
    mode: "fast",
    generated_at: Date.now(),
  };
  return payload;
});
```

Example: inject a plugin-owned system prompt

```js
host.addCompletionPayloadHook((payload) => {
  payload.ext = payload.ext || {};
  payload.ext.system_prompts_mode = "system";
  payload.ext.system_prompts = [
    ...(payload.ext.system_prompts || []),
    {
      id: "example_rules",
      content: "Prefer concise JSON answers when the user asks for structured output.",
    },
  ];
  return payload;
});
```

### `host.addSendContextMenuItem(buildItem)`

Adds items to the send-button context menu.

Use it when:

- a plugin wants alternate send actions
- a plugin wants to trigger assistant generation without typing

Example:

```js
host.addSendContextMenuItem(({ pid, sid }, ctx) => {
  return {
    label: "Send Assistant Response",
    onClick: async () => {
      await ctx.sendAssistantResponse?.();
    },
  };
});
```

## 3.4 Event and session hooks

### `host.addEventHandler(handler)`

Receives streaming and session event traffic.

This hook can observe:

- local `assistant_done`
- remote SSE events like `message`, `token`, and `done`

Use it for:

- metrics
- background synchronization
- stream-aware UI updates

Example:

```js
host.addEventHandler((event, data, ctx) => {
  if (event === "assistant_done") {
    ctx.log?.("Assistant stream finished", "info");
  }
  if (event === "message" && data?.msg?.meta?.important) {
    ctx.refreshTranscript?.();
  }
});
```

### `host.addRosterAction(handler)`

Registers a callback invoked from the roster UI.

Example:

```js
host.addRosterAction((ctx) => {
  ctx.openPluginPanel?.("team_presence");
});
```

### `host.setProjectCreateHandler(handler)`

Lets a plugin override the normal project creation flow.

The most recent handler wins for that plugin id.

Example:

```js
host.setProjectCreateHandler(async (ctx) => {
  ctx.openPluginPanel?.("auth_projects");
});
```

### `host.setSessionCreateHandler(handler)`

Lets a plugin override the normal session creation flow.

Example:

```js
host.setSessionCreateHandler(async (ctx, opts = {}) => {
  ctx.log?.(`Create session for pid=${opts.pid || ""}`, "info");
  ctx.openPluginPanel?.("auth_projects");
});
```

## 3.5 i18n hooks

These hooks are how plugins integrate with the language framework.

### `host.registerI18nBundle(bundle)`

Registers a translation bundle descriptor.

Typical use:

- plugin provides `lang/en.json`, `lang/es.json`, etc.
- language plugin or plugin code loads dictionaries on demand

Example:

```js
host.registerI18nBundle({
  id: "example_plugin",
  pluginId: "example_plugin",
  locales: ["en", "es", "ja"],
  pathTemplate: "./plugins/example_plugin/lang/{locale}.json",
});
```

### `host.installI18nDictionary(locale, dict, options)`

Installs a locale dictionary into runtime state.

Example:

```js
host.installI18nDictionary("en", {
  "plugin.example.title": "Example",
  "plugin.example.run": "Run",
});
```

### `host.t(key, fallback)`

Translate a key with fallback text.

Example:

```js
const title = host.t?.("plugin.example.title", "Example") || "Example";
```

### `host.translateContainer(root, pluginId)`

Translates a DOM container using registered keys.

Example:

```js
const box = document.createElement("div");
box.innerHTML = `<button data-i18n-key="plugin.example.run">Run</button>`;
host.translateContainer?.(box, "example_plugin");
```

### `host.onLanguageChange(callback)`

Use this to re-render plugin UI when language changes.

Example:

```js
host.onLanguageChange?.(() => {
  rerenderPanel();
});
```

## 3.6 Shared object and capability hooks

### `host.shareObject(obj)`

Publishes a reusable capability for other plugins.

Use it when:

- one plugin owns a reusable API
- another plugin should discover it dynamically
- you want to avoid direct frontend plugin imports

Example:

```js
host.shareObject({
  id: "example_actions_api",
  type: "api",
  service: "example_plugin",
  runExample(msg) {
    console.log("run", msg);
  },
});
```

### `host.getSharedObjects(filter)`

Retrieves shared objects by type or plugin id.

Example:

```js
const apis = host.getSharedObjects?.({ type: "api" }) || [];
const exampleApi = apis.find((item) => item.id === "example_actions_api");
exampleApi?.runExample?.("hello");
```

### `host.requestLoadPriority(options)`

Requests earlier plugin loading.

Use it for:

- plugins that need to attach hooks before others
- plugins that should reliably influence send/render flow

Example:

```js
host.requestLoadPriority?.({ position: "first" });
```

### `host.requestEmbedPreload(kind)`

Requests a preload hint for embedded mode.

Used by `page_json_retriever` for earlier capability setup.

Example:

```js
host.requestEmbedPreload?.("json_sniffer");
```

## 3.7 Framework service helpers

These are not registration hooks, but they are important for plugin behavior.

### Panel/window helpers

- `host.openTools(panelId)`
- `host.openPluginPanel(pluginId, options)`
- `host.openPluginFullView(pluginId, options)`

Example:

```js
host.openPluginPanel?.("setup_wizard", { openModal: true });
```

### Auth/account helpers

- `host.enableAccountMenu()`
- `host.disableAccountMenu()`
- `host.login(username, password)`
- `host.logout(announce)`
- `host.setAccountActions(actions)`

Example:

```js
host.setAccountActions?.([
  {
    id: "example-profile",
    label: "Profile",
    onClick(ctx) {
      ctx.openPluginPanel?.("profile_plugin");
    },
  },
]);
```

### Scope and data refresh helpers

- `host.setActiveScope(pid, sid)`
- `host.refreshProjects()`
- `host.refreshSessions()`
- `host.refreshMessages()`

Example:

```js
await host.refreshProjects?.();
await host.refreshSessions?.();
await host.setActiveScope?.("project1", "session1");
```

### Chat helpers

- `host.sendMessage()`
- `host.sendAssistantResponse()`
- `host.setChatsOverride(override)`
- `host.clearChatsOverride()`

Use these when the plugin needs to steer the chat/session browser experience.

### Logging and state

- `host.log(...)`
- `host.getState()`

Example:

```js
const state = host.getState?.() || {};
host.log?.(`activeSid=${state.ui?.activeSid || ""}`, "info");
```

## 4. Plugin Runtime Context API

When a hook executes, it usually receives `ctx`.

This is different from the registration-time `host`. `ctx` is the live runtime context for state and service calls.

## Core context fields

- `state`
- `apiJson`
- `streamSSE`
- `renderMarkdown`
- `refreshTranscript`
- `refreshMessages`
- `saveState`

## Composer helpers

- `getComposerText()`
- `setComposerText(text)`
- `clearComposerText()`
- `deleteLastWord()`
- `deleteLastLine()`

Example:

```js
const current = ctx.getComposerText?.() || "";
ctx.setComposerText?.(`${current}\n# Added by plugin`);
```

## Message/session helpers

- `appendMessage(msg, sidOverride)`
- `updateMessage(sid, msgId, content, force)`
- `appendToken(sid, msgId, text)`
- `markMessageDone(sid, msgId)`

Example:

```js
ctx.appendMessage?.({
  msg_id: ctx.randomId("assistant"),
  role: "assistant",
  content: "Plugin-generated answer",
});
```

## Streaming helpers

- `startCompletionStream(pid, sid, prompt, clientMsgId)`
- `startModelStream(pid, sid, prompt, clientMsgId)`
- `buildCompletionPayload(sid)`

Example:

```js
const msgId = ctx.randomId("msg");
await ctx.startCompletionStream?.("project1", "session1", "Continue", msgId);
```

## UI helpers

- `getEmbedMount()`
- `getOverlayMount()`
- `openPluginPanel(...)`
- `openPluginFullView(...)`

## Permissions helpers

- `hasPermission(key, fallback)`
- `canAccessPlugin(pluginId, action)`
- `refreshPermissions(options)`

Example:

```js
if (!ctx.hasPermission?.("theme.manage", false)) {
  ctx.log?.("Theme management not allowed", "warn");
  return;
}
```

## Theme helpers

- `getSavedUiTheme()`
- `getSharedUiThemeDefault()`
- `getUiThemeDefaults(target)`
- `applyUiTheme(snapshot, options)`
- `saveUiTheme(snapshot)`
- `saveSharedUiThemeDefault(payload)`
- `clearUiTheme(options)`

These are the reusable theme APIs surfaced by the framework and used by `theme_demo`.

## Example: call a backend helper route

```js
async function loadStatus(ctx) {
  const data = await ctx.apiJson("/v1/voice/stt/status");
  return data;
}
```

## 5. Backend GUI Helper Plugin API

Backend GUI helper plugins live under:

- `plugins/gui_helpers/<plugin>/`

The usual file structure is:

- `__init__.py`
- `routes.py`

Minimal `__init__.py`:

```python
from .routes import install
```

Minimal `routes.py`:

```python
from fastapi import APIRouter, Request
from plugins.gui_helpers._framework.utils import require_gui_plugin_enabled

GUI_PLUGIN_ID = "example_plugin"

def install(app) -> None:
    r = APIRouter()

    @r.get("/v1/example/status")
    def example_status(request: Request):
        require_gui_plugin_enabled(request, gui_plugin_id=GUI_PLUGIN_ID)
        return {"ok": True}

    app.include_router(r)
```

## What `install(app)` should do

- create an `APIRouter`
- declare plugin-owned endpoints
- gate routes when appropriate
- mount the router with `app.include_router(r)`

## Route gating

Most GUI helper routes should call:

```python
require_gui_plugin_enabled(request, gui_plugin_id=GUI_PLUGIN_ID)
```

This links the route to:

- enabled GUI plugin state
- permissions manager checks when available

## Example: persisted plugin settings

```python
import json
import os
from fastapi import APIRouter, Request, HTTPException
from plugins.gui_helpers._framework.utils import require_gui_plugin_enabled

GUI_PLUGIN_ID = "example_plugin"

def _settings_path(app):
    root = getattr(app.state, "data_dir", None) or os.path.abspath("./data")
    path = os.path.join(root, "gui_helpers", "example_plugin")
    os.makedirs(path, exist_ok=True)
    return os.path.join(path, "settings.json")

def install(app):
    r = APIRouter()

    @r.get("/v1/example/settings")
    def get_settings(request: Request):
      require_gui_plugin_enabled(request, gui_plugin_id=GUI_PLUGIN_ID)
      path = _settings_path(app)
      if not os.path.isfile(path):
          return {"ok": True, "settings": {}}
      with open(path, "r", encoding="utf-8") as fh:
          return {"ok": True, "settings": json.load(fh)}

    @r.post("/v1/example/settings")
    def save_settings(payload: dict, request: Request):
      require_gui_plugin_enabled(request, gui_plugin_id=GUI_PLUGIN_ID)
      path = _settings_path(app)
      with open(path, "w", encoding="utf-8") as fh:
          json.dump(payload or {}, fh, indent=2)
      return {"ok": True}

    app.include_router(r)
```

## 6. GUI Helper Framework Utilities API

The main utility surface used by helper routers is:

- `require_gui_plugin_enabled(request, gui_plugin_id=...)`
- `require_state(app, *names)`
- `get_user_rag(app)`
- `get_jobs(app)`

## `require_state(app, *names)`

Use it when a helper depends on app state objects that may be registered under one of several names.

Example:

```python
from plugins.gui_helpers._framework.utils import require_state

def _repo_api(app):
    return require_state(app, "repo_api", "repo_ingest", "repo_panel_api")
```

This keeps helper code decoupled from a single hardcoded app state key.

## 7. Backend Model Loader Plugin API

Model loader plugins live under:

- `plugins/model_loader/*`

They are installed by:

- `plugins.model_loader._framework.loader.install_model_loader_plugins(app)`

## Purpose

Use this family when the plugin is fundamentally about:

- model loading
- runtime adapters
- local model capabilities
- registry-managed loader behavior

Do not use `gui_helpers` for model loader internals unless the code is specifically a GUI-facing helper route.

## Registration pattern

A model loader package can expose one of two registration styles:

- `register_model_loader_plugin(app, reg)`
- `build_model_loader_plugin(app)`

## Example: registry-based registration

```python
def register_model_loader_plugin(app, reg):
    plugin = MyLoaderPlugin(app)
    reg.register(plugin)
```

## Example: builder style

```python
def build_model_loader_plugin(app):
    return MyLoaderPlugin(app)
```

## Practical examples in repo

- `plugins/model_loader/gguf`
- `plugins/model_loader/model_deck`
- `plugins/model_loader/model_deck/local_loaders/*`

## When frontend should call into model loader routes

Usually through:

- a GUI helper plugin
- or a model-loader-facing panel plugin in `gui_js/plugins`

This keeps raw model/runtime concerns out of generic chat framework code.

## 8. Backend AI Route Plugin API

AI route plugins live under:

- `plugins/ai_routes/*`

They are loaded by:

- `plugins.ai_routes.load_routes(core)`

## Purpose

Use this family when the plugin defines AI backend behavior such as:

- a router-selectable capability
- an agent route
- workflow/autoflow logic
- inference-side tool behavior

## Package contract

An `ai_routes` package typically provides:

- metadata constants
- `build_routes(core)`

## Example shape

```python
from plugins.ai_routes.base import BaseRoute, RouterCore

PLUGIN_ID = "example_route"
PLUGIN_TITLE = "Example Route"
PLUGIN_DESCRIPTION = "Example router plugin."

class ExampleRoute(BaseRoute):
    route_id = "example_route_run"
    title = "Example Route"
    short_description = "Handles example requests."

    def match(self, text: str, meta=None) -> float:
        return 1.0 if "example" in (text or "").lower() else 0.0

    def run(self, text: str, meta=None):
        return {"ok": True, "text": f"Example handled: {text}"}

def build_routes(core: RouterCore):
    return [ExampleRoute(core)]
```

## How frontend plugins usually interact with `ai_routes`

Frontend plugins normally do not import `ai_routes` directly.

Instead they:

- register completion payload hooks
- attach router plugin settings into `payload.ext`
- enable route ids in router config
- call helper routes that themselves invoke model loader or AI route logic

This is the separation to preserve.

## 8.1 How AI route plugins call model deck defaults

An `ai_routes` plugin does not need to manually open the deck JSON and bind a loader itself.

The framework already provides route-level helpers in:

- [base.py](C:\Users\Tee\projects\llmloader2\plugins\ai_routes\base.py)
- [model_deck_utils.py](C:\Users\Tee\projects\llmloader2\plugins\ai_routes\model_deck_utils.py)

The two main helpers are:

- `BaseRoute.resolve_model_deck_default(...)`
- `BaseRoute.prepare_model_deck_runner(...)`

## What `resolve_model_deck_default(...)` does

This helper resolves the current default model for a declared model deck type such as:

- `text_llm`
- `vlm`
- `image_gen`
- `video_gen`

Internally it:

1. Uses `settings["__model_loader_registry"]`
2. Finds the server app
3. Loads the model deck through `plugins.gui_helpers.model_deck.routes`
4. Resolves the configured default model for the requested type
5. Returns loader metadata such as:
   - `model_id`
   - `loader_id`
   - `settings`
   - `lazy`
   - `persist`

That means an AI route plugin can stay generic and refer to a model type instead of hardcoding a specific model file or loader configuration.

## What `prepare_model_deck_runner(...)` does

This helper creates a `ModelDeckRunner` instance for a deck type.

Example:

```python
runner = self.prepare_model_deck_runner(
    settings=settings,
    model_type="vlm",
    slot=self.route_id,
    prefer_worker=True,
    worker_mode="per_request",
    worker_timeout=120,
)
```

The runner handles:

- model deck default resolution
- lazy versus persistent loading
- worker-process execution when appropriate
- loader binding through the model loader registry
- cleanup and unload when the selected model is not persistent

This is the normal API that AI route plugins should use when they need a model deck type.

## 8.2 Lazy loading behavior for model deck routes

Lazy loading is already encoded in the selected model deck entry.

The resolved deck entry exposes:

- `lazy`
- `persist`
- `loader_id`
- `settings`

`ModelDeckRunner` interprets those fields in `model_deck_utils.py`.

At a high level:

1. The route asks for a deck type such as `vlm` or `image_gen`.
2. The framework resolves the current default model for that type.
3. If the deck entry is lazy and non-persistent, the runner loads it on demand.
4. If worker execution is preferred and supported, the runner can use the worker manager instead of binding directly in the main process.
5. If the deck entry is persistent, the runner keeps the model available instead of unloading it after the request.

This allows AI route plugins to request a capability by model type while letting the framework decide how the runtime should be loaded.

## Direct-binding path versus worker path

The runner can choose between two execution patterns:

- worker path:
  - preferred when the selected model is lazy, non-persistent, and worker execution is supported
- direct-binding path:
  - binds through the registered model loader and uses the loaded model directly

For example, `ModelDeckRunner` can:

- spawn a VLM worker for lazy GGUF/VLM work
- bind the model directly from `model_loader.gguf`
- unload the slot afterward if `persist` is false

The route plugin should not reimplement this decision logic.

## 8.3 Example: AI route using a model deck type

A typical pattern is:

```python
from plugins.ai_routes.base import BaseRoute, RouterCore

class ExampleVisionRoute(BaseRoute):
    route_id = "example_vision"
    MODEL_TYPE = "vlm"
    short_description = "Example route that uses the default VLM from model_deck."

    def handle(self, req):
        settings = dict(self.core.settings or {})
        runner = self.prepare_model_deck_runner(
            settings=settings,
            model_type=self.MODEL_TYPE,
            slot=self.route_id,
            prefer_worker=True,
            worker_mode="per_request",
            worker_timeout=120,
            require_mmproj=True,
        )
        if getattr(runner, "error", None):
            return {"route_id": self.route_id, "ok": False, "error": runner.error}

        messages = [
            {"role": "user", "content": "Describe this image precisely."}
        ]
        result = runner.plan(messages, {"max_new_tokens": 512})
        runner.close()
        return {"route_id": self.route_id, **result}
```

This is the same general pattern used by existing plugins such as:

- `image_reader`
- `vlm_code2`
- `os_auto3`
- `os_auto_cmd_browse`

For image and video generation routes, the framework provides specialized runners:

- `prepare_image_gen_runner(...)`
- `VideoGenRunner`

Those still resolve through model deck defaults rather than bypassing the model deck.

## 8.4 Agent Flow node-level lazy loading

There is a second lazy-loading path in:

- [agent_flow/__init__.py](C:\Users\Tee\projects\llmloader2\plugins\ai_routes\agent_flow\__init__.py)

This path is for flow nodes, not for ordinary standalone route plugins.

If a flow node sets:

- `lazy_load: true`

then Agent Flow will:

1. Read `model_loader_id` from the node, defaulting to `model_loader.gguf`
2. Read `model_settings` from the node
3. Resolve the loader from `settings["__model_loader_registry"]`
4. Call `loader_plugin.load_for(sid, loaded_slot, settings=model_settings)`
5. Call `loader_plugin.get_model_for(sid, loaded_slot)`
6. Temporarily bind that model to `self.core.chat_llm` for that node execution

Important distinction:

- ordinary AI route plugins should usually use `prepare_model_deck_runner(...)`
- Agent Flow node execution can lazy-load a specific model loader directly per node

The Agent Flow path is useful when the flow definition itself must select the exact loader and settings for a node.

## Example Agent Flow node

```json
{
  "plugin_id": "vlm_code",
  "lazy_load": true,
  "model_loader_id": "model_loader.gguf",
  "model_settings": {
    "model_id": "my-vlm-model",
    "mmproj_path": "my-mmproj.gguf"
  },
  "unload_policy": "on_step_end"
}
```

This is more explicit and lower-level than the model deck runner path.

## Which path to use

Use `prepare_model_deck_runner(...)` or the specialized image/video runners when:

- the route wants the current default model for a deck type
- the plugin should follow model deck configuration
- lazy/persist behavior should come from the deck entry

Use Agent Flow node `lazy_load` when:

- the flow node itself must name a specific loader
- the node needs explicit per-node model settings
- the behavior is part of a flow graph rather than a reusable route capability

In other words:

- model deck path = type-driven, framework-managed lazy loading
- Agent Flow node path = explicit loader-driven lazy loading

## 9. End-to-End Patterns

This section shows the common patterns you should follow.

## Pattern A: simple frontend-only UI plugin

Use when:

- all behavior can stay in browser state
- no server persistence or privileged action is needed

Example:

- transcript topbar widget
- local-only formatting controls

Skeleton:

```js
export default {
  id: "example_ui",
  register(host) {
    host.addTranscriptTopbar(() => {
      const el = document.createElement("button");
      el.textContent = "Example";
      return el;
    }, "right");
  },
};
```

## Pattern B: frontend plugin plus helper router

Use when:

- plugin needs persistence
- plugin needs host process access
- plugin needs filesystem, runtime, or privileged actions

Frontend:

```js
host.addPanelTab({
  id: "example_plugin",
  title: "Example",
  async render(container, ctx) {
    const data = await ctx.apiJson("/v1/example/status");
    container.textContent = JSON.stringify(data, null, 2);
  },
});
```

Backend:

```python
@r.get("/v1/example/status")
def example_status(request: Request):
    require_gui_plugin_enabled(request, gui_plugin_id=GUI_PLUGIN_ID)
    return {"ok": True, "status": "ready"}
```

## Pattern C: send/payload middleware plugin

Use when:

- plugin needs to alter what the model sees
- plugin needs to add metadata or hidden context
- plugin should remain generic rather than patching chat send flow

Example:

```js
host.addSendHook((payload) => {
  if (!payload.text.includes("#urgent")) return payload;
  return { ...payload, text: payload.text.replace("#urgent", "").trim() };
});

host.addCompletionPayloadHook((payload) => {
  payload.ext = payload.ext || {};
  payload.ext.priority = "urgent";
  return payload;
});
```

## Pattern D: custom render plugin

Use when:

- output contains structured content that the default markdown renderer should not own

Example:

```js
function transformBlocks(blocks) {
  return blocks.map((b) => {
    if (b.type === "text" && String(b.text || "").startsWith("SUMMARY:")) {
      return { type: "summary_card", text: String(b.text || "").slice(8).trim() };
    }
    return b;
  });
}

host.addBlockTransformer(transformBlocks);
host.addBlockRenderer((block) => {
  if (block.type !== "summary_card") return null;
  const el = document.createElement("div");
  el.className = "summary-card";
  el.textContent = block.text;
  return el;
});
```

## Pattern E: plugin-to-plugin capability sharing

Use when:

- plugin A owns a reusable capability
- plugin B should discover and use it dynamically

Plugin A:

```js
host.shareObject({
  id: "example_lookup_api",
  type: "api",
  lookup(value) {
    return value.toUpperCase();
  },
});
```

Plugin B:

```js
const apis = host.getSharedObjects?.({ type: "api" }) || [];
const lookupApi = apis.find((item) => item.id === "example_lookup_api");
const result = lookupApi?.lookup?.("hello");
```

## 10. Practical Recommendations

## Prefer hooks before framework edits

Before editing `chat_js.js`, check whether the problem can be solved with:

- `addTranscriptTopbar`
- `addTranscriptBottombar`
- `addPanelTab`
- `addSendHook`
- `addCompletionPayloadHook`
- `addMessagePreRenderer`
- `addMessageRenderer`
- `addBlockTransformer`
- `addBlockRenderer`
- `addMessageFooterItem`
- `shareObject`

In most cases it can.

## Prefer helper routes before `app.py` edits

Before editing `app.py`, check whether the server behavior can live in:

- `plugins/gui_helpers/<plugin>/routes.py`
- `plugins/model_loader/*`
- `plugins/ai_routes/*`

`app.py` should stay generic and only install plugin families or generic capabilities.

## Match frontend id and backend `GUI_PLUGIN_ID`

When a frontend plugin has a backend helper router, keep ids aligned.

This simplifies:

- route gating
- permissions
- plugin enablement
- maintainability

## Use the language plugin pattern for translatable UI

If the plugin has user-visible strings:

- register an i18n bundle
- install dictionaries
- react to language changes

This keeps plugin UI consistent with the rest of the framework.

## Keep plugin boundaries clean

Frontend plugin responsibilities:

- UI
- render behavior
- payload shaping
- local state

Backend GUI helper responsibilities:

- plugin-owned routes
- persistence
- privileged actions

Model loader responsibilities:

- runtime/model logic

AI route responsibilities:

- backend AI route behavior

## 11. Recommended Starter Templates

## Template: simple panel plugin

```js
const meta = {
  plugin_id: "starter_panel",
  name: "Starter Panel",
  kind: "panel",
  description: "Simple starter panel plugin.",
};

function renderPanel(container, ctx) {
  container.innerHTML = "";
  const box = document.createElement("div");
  box.textContent = "Starter panel";
  container.appendChild(box);
}

export default {
  id: meta.plugin_id,
  name: meta.name,
  kind: meta.kind,
  description: meta.description,
  meta,
  register(host) {
    host.addPanelTab({
      id: meta.plugin_id,
      title: meta.name,
      render: renderPanel,
    });
  },
};
```

## Template: panel plus backend helper

Frontend:

```js
const meta = {
  plugin_id: "starter_helper",
  name: "Starter Helper",
  kind: "plugin",
  description: "Starter helper-backed plugin.",
};

async function renderPanel(container, ctx) {
  container.innerHTML = "";
  const data = await ctx.apiJson("/v1/starter_helper/status");
  const pre = document.createElement("pre");
  pre.textContent = JSON.stringify(data, null, 2);
  container.appendChild(pre);
}

export default {
  id: meta.plugin_id,
  name: meta.name,
  kind: meta.kind,
  description: meta.description,
  meta,
  register(host) {
    host.addPanelTab({
      id: meta.plugin_id,
      title: meta.name,
      render: renderPanel,
    });
  },
};
```

Backend:

```python
from fastapi import APIRouter, Request
from plugins.gui_helpers._framework.utils import require_gui_plugin_enabled

GUI_PLUGIN_ID = "starter_helper"

def install(app) -> None:
    r = APIRouter()

    @r.get("/v1/starter_helper/status")
    def status(request: Request):
        require_gui_plugin_enabled(request, gui_plugin_id=GUI_PLUGIN_ID)
        return {"ok": True, "message": "starter helper ready"}

    app.include_router(r)
```

## Conclusion

The `chat_js` plugin framework already provides a large developer API surface for:

- UI insertion
- transcript rendering
- send-time middleware
- completion payload mutation
- session/event observation
- i18n
- shared plugin capabilities
- backend helper routing
- model loader extension
- AI route extension

For new plugin work, the correct default is to build on these APIs first. Only extend `chat_js.js` or `app.py` when the missing capability is genuinely generic and will benefit multiple plugins.