Liteskill

Liteskill is an event-sourced chat application built with Phoenix 1.8 and Elixir. It provides multi-model LLM integration, MCP tool support, RAG (Retrieval-Augmented Generation), an agent studio, collaborative reports, a built-in wiki, and scheduled agent runs.

Key Capabilities

• Multi-model chat — Connect any LLM provider (AWS Bedrock, OpenRouter, OpenAI-compatible endpoints) and switch models per conversation.
• MCP tool calling — Register MCP servers and let the LLM call external tools during conversation streaming, with optional user approval.
• RAG — Ingest documents, generate embeddings via Cohere/OpenAI, and augment conversations with semantic search results.
• Agent Studio — Define reusable AI agents with system prompts, model assignments, and scoped tool/data-source access via ACLs.
• Teams & Runs — Compose agents into teams with topologies (sequential, parallel, supervisor) and execute them as runs.
• Reports — Structured documents with nested sections, markdown rendering, and a comment/review workflow.
• Wiki — Built-in collaborative wiki with hierarchical pages, ACL-based sharing, and automatic RAG indexing.
• Schedules — Cron-based recurring execution of agent runs.
• Usage tracking — Per-user, per-model, per-conversation token and cost accounting.

Tech Stack

Current version: 0.2.37

So What’s the Difference?

If you’re evaluating self-hosted AI tools, you’ve probably noticed the space is crowded. Chat UIs, API gateways, workflow engines, wiki platforms, agent frameworks - there’s no shortage of options, and each one solves a piece of the puzzle. Liteskill was built to be the whole puzzle.

This page explains what makes Liteskill different from the categories of tools you’ll encounter, and why those differences matter.

vs. Chat Interfaces

Most open-source LLM chat UIs give you a clean frontend for talking to models. They handle streaming, multi-model switching, and sometimes RAG. But they’re built on standard CRUD persistence - conversations are mutable rows in a database.

What Liteskill does differently:

• Event-sourced conversations. Every state change is captured as an immutable event. You get a full audit trail, state replay, and temporal queries. For regulated industries, this is table stakes.
• Per-resource access control. Most chat UIs offer role-based permissions (admin vs. user). Liteskill provides granular ACLs on individual conversations, reports, agents, wiki spaces, and more - with owner, manager, editor, and viewer roles, plus group-based grants.
• Native MCP tool integration. Rather than a proprietary plugin system, Liteskill speaks the open Model Context Protocol over JSON-RPC 2.0. Any MCP-compliant tool server works without custom integration code.
• Agent Studio & Teams. Beyond personas and system prompts, Liteskill provides composable agents with strategies (ReAct, chain-of-thought, tree-of-thoughts), team topologies (sequential, parallel, supervisor), and full execution tracking with cost limits.
• Conversation forking. Branch off at any point in a conversation to explore alternate paths without losing the original thread.

vs. LLM Proxy Gateways

API gateways sit between your application and LLM providers, offering a unified interface, load balancing, rate limiting, and cost tracking. They’re useful infrastructure - but they’re not applications.

What Liteskill does differently:

• It’s a complete platform. A proxy gateway has no UI, no conversation management, no persistence, and no user-facing experience. Liteskill is something your team can actually use.
• Built-in provider management. Liteskill handles multi-provider routing, circuit breaking, concurrency gating, and token-bucket rate limiting through its own LLM Gateway layer. A separate proxy in front is optional, not required.
• No enterprise paywall for auth. Proxy gateways commonly gate SSO, audit logs, and advanced RBAC behind paid tiers. Liteskill ships all authentication and authorization features in the open-source release.
• Compatible, not competing. If you already run a proxy gateway, Liteskill can use it as a provider - they complement each other.

vs. Workflow Automation Platforms

Visual workflow builders let you connect triggers and actions into automated pipelines. Some have added AI nodes for LLM interactions. But AI is a bolt-on capability, not the core architecture.

What Liteskill does differently:

• Purpose-built for AI conversations. Streaming token-by-token responses, tool calling during generation, conversation forking, and message editing are all first-class features - not nodes wired into a generic workflow DAG.
• Event-sourced state. Workflow engines store execution history as flat records. Liteskill captures every conversation state change as an immutable event, enabling replay, auditing, and stream recovery.
• Open tool protocol. Workflow platforms typically connect to services through proprietary node/connector systems. Liteskill uses the open MCP protocol - any compliant tool server works without writing platform-specific integration code.
• Apache 2.0 licensing. Some workflow platforms use restrictive licenses that prohibit commercial embedding or redistribution. Liteskill is Apache 2.0 - use it however you want.
• SSO and RBAC included. No paid enterprise plan required for OIDC SSO, RBAC, or entity-level access controls.

vs. Wiki & Knowledge Platforms

Wiki platforms excel at structured documentation with hierarchical content, collaborative editing, and content-level permissions. Some have mature SSO support. But they’re static knowledge bases.

What Liteskill does differently:

• AI-native knowledge. Liteskill’s built-in wiki isn’t just a documentation tool - it’s automatically synced to the RAG pipeline, so your documentation becomes searchable context for AI conversations. Knowledge isn’t static; it’s active.
• Wiki is one feature, not the product. Conversations, agents, reports, MCP tools, RAG, and the wiki all live in the same platform under the same unified access control system.
• Event-sourced audit trail. Traditional wikis use mutable ORM persistence. Liteskill’s append-only event store provides an immutable record of every state change across the entire platform.

vs. Agent Frameworks

Agent frameworks provide libraries and abstractions for building multi-agent AI systems in code. Agents have roles, goals, and tools. Some support MCP. But they’re developer tools, not deployed applications.

What Liteskill does differently:

• It’s a platform, not a library. Agent frameworks require you to build an application around them - writing your own UI, user management, persistence layer, and deployment infrastructure. Liteskill is a deployed application with a full web interface, real-time streaming, user accounts, and persistent conversation history.
• Integrated chat + agents. Agents and teams are defined through the Agent Studio UI, then invoked within the same platform where conversations happen. No separate application to build and maintain.
• Enterprise auth built in. Most agent frameworks have no auth, no RBAC, and no SSO in their open-source releases. Enterprise tiers that add these features often come with execution quotas and significant cost. Liteskill includes OIDC SSO, RBAC, entity ACLs, and group-based authorization in the open-source release with no quotas.
• Event-sourced execution tracking. Agent runs are tracked with structured logs, usage metrics, cost limits, and full event histories - not just basic logging.
• Elixir’s concurrency model. Liteskill runs on the BEAM VM with lightweight processes, preemptive scheduling, and fault-tolerant supervision trees. Parallel agent topologies, concurrent tool calls, and streaming responses all benefit from these concurrency primitives without thread-safety complexity.

What Liteskill Brings Together

The Bottom Line

Most tools in this space solve one slice of the problem. You get a chat interface, or a provider gateway, or a workflow engine, or a wiki, or an agent framework - then stitch them together yourself.

Liteskill combines all of these into a single, event-sourced platform with enterprise access controls. And it ships everything under Apache 2.0 with no feature paywalls, no branding requirements, and no execution quotas.

Hello World Tutorial

Welcome! This tutorial gets you up and going with LiteSkill.

Docker Compose

The fastest way to get up and running is with Docker Compose. Ensure it’s installed on your system and then run the following:

git clone https://github.com/liteskill-ai/liteskill-oss.git && cd liteskill-oss

Before starting the app, generate the required secrets:

export SECRET_KEY_BASE=$(openssl rand -base64 64)
export ENCRYPTION_KEY=$(openssl rand -base64 32)

Then start everything:

docker compose up

This starts a PostgreSQL 16 database with pgvector and the Liteskill application. The app container uses host networking so it can reach the database on localhost.

Admin Setup

Open a browser and navigate to http://localhost:4000/ and follow the setup instructions.

For testing purposes, we recommend using OpenRouter as your model provider. For model selection, we recommend Claude Sonnet for inference and OpenAI: Text Embedding 3 Small for embeddings.

Your First Conversation

1. Navigate back to the main page by clicking Chat in the sidebar.
2. Click the “+” icon to the right of “Conversations” in the top left corner of the screen.
3. Type a message and hit Enter!

You should see the LLM stream its response in real-time. Congratulations — you’ve got Liteskill running!

Adding your first MCP Server

MCP (Model Context Protocol) servers give your LLM access to external tools. When an MCP server is connected, the LLM can request tool calls during conversation streaming, and Liteskill will execute them automatically or with user approval.

Prerequisites

• Liteskill running (see Initial Setup)
• An MCP server accessible via HTTP(S)

Register an MCP Server

1. Navigate to Tools in the sidebar (or go to /mcp).
2. Click the “Add Server” button.
3. Fill in the server details:
   • Name: A display name for the server (e.g. “My Tools”)
   • URL: The HTTP(S) endpoint of your MCP server
   • API Key (optional): If your server requires authentication
   • Custom Headers (optional): Any additional headers your server needs
4. Click Save.

Liteskill will connect to the server and discover its available tools via the tools/list JSON-RPC call.

Select the Server for Conversations

After registering the server, you need to select it for use in your conversations:

1. Start or continue a conversation.
2. Click the Tool icon to the left of “Type a message…” in the chat window.
3. Select the tools you’d like to use.
4. By default, tool calls execute automatically (auto_confirm: true).
5. When the LLM determines it needs to use a tool, it will emit a tool call request.

Tool calls appear in the conversation as collapsible sections showing the tool name, arguments, and result.

Docker Networking

If you’re running Liteskill via Docker Compose (which uses network_mode: host), MCP servers on your host machine are reachable at localhost:

http://localhost:4005

This requires Allow private MCP URLs to be enabled in server settings (Admin > Setup), since localhost resolves to a private address.

SSRF Protection

By default, only HTTPS URLs are accepted for MCP servers, and private/reserved addresses are blocked. To allow private URLs for self-hosted MCP servers, enable Allow private MCP URLs in the admin settings.

Configuring a RAG Data Source

RAG (Retrieval-Augmented Generation) lets you feed your own documents into conversations. When RAG is enabled, Liteskill searches your document collections for relevant context and includes it with your messages to the LLM.

Prerequisites

• Liteskill running (see Initial Setup)
• An LLM provider configured with an embedding model
• Documents you want to make searchable

Create a Wiki Space

1. Navigate to Wiki in the sidebar (or go to /wiki).
2. Click “New Space”.
3. Give it a name (e.g. “Engineering Docs”).

Add Documents

You can add nested documents in this page, called “children”.

External Connectors

For bulk document sync, configure an external data source:

1. Navigate to Sources and click “Add Source”.
2. Select a source type:
   • Google Drive — Service account JSON + folder ID
   • SharePoint — Tenant ID, site URL, client credentials
   • Confluence — Base URL, username, API token, space key
   • Jira — Base URL, username, API token, project key
   • GitHub — Personal access token, repository
   • GitLab — Personal access token, project path
3. Fill in the credentials and click Save.
4. Click “Sync” to pull documents from the source.

Using RAG in Conversations

Once your documents are embedded, RAG context is automatically injected into conversations:

1. Start a new conversation.
2. Ask a question related to your documents.
3. Liteskill embeds your query, searches across your accessible collections, and includes the most relevant chunks as context for the LLM.
4. RAG sources are displayed alongside the assistant’s response for transparency.

Wiki Integration

Wiki pages are automatically indexed for RAG. When you create or edit a wiki page, a background job updates the corresponding RAG document and re-embeds its chunks. See the Wiki documentation for details.

Prerequisites

Required Tools

Liteskill uses mise for tool version management. The mise.toml at the project root pins:

Install mise and run mise install to get the correct versions.

System Dependencies

A C compiler and make are required to build native extensions (e.g. argon2_elixir). Install your platform’s build tools before running mix deps.compile:

• Debian/Ubuntu: sudo apt install build-essential
• Fedora/RHEL: sudo dnf groupinstall "Development Tools"
• macOS: xcode-select --install
• Arch Linux: sudo pacman -S base-devel

PostgreSQL

PostgreSQL 16 with the pgvector extension is required.

Option A: Local install

Install PostgreSQL and pgvector via your system package manager. The default dev config expects:

• Host: localhost
• Port: 5432
• User/password: postgres/postgres

Option B: Docker

Use the included test script which starts a disposable Postgres container:

./scripts/test-with-docker.sh test

Or use the production docker-compose.yml which provides a pgvector/pgvector:pg16 service.

Quick Start (Docker Compose)

If you just want to run Liteskill without setting up a development environment:

export SECRET_KEY_BASE=$(openssl rand -base64 64)
export ENCRYPTION_KEY=$(openssl rand -base64 32)
docker compose up

This starts PostgreSQL with pgvector and the Liteskill application on port 4000.

Local Development

Quick Start

# Install tool versions
mise install

# Install deps, create DB, run migrations, build assets
mix setup

# Start the dev server
mix phx.server

The app will be available at http://localhost:4000.

Without Local Postgres

If you don’t have PostgreSQL installed locally, use the Docker-based scripts:

# Run tests with a temporary Docker Postgres
./scripts/test-with-docker.sh test

# Full precommit with Docker Postgres
./scripts/test-with-docker.sh precommit

Single-User Mode

For desktop or self-hosted single-user setups:

SINGLE_USER_MODE=true mix phx.server

Or use the mise task:

mise run singleuser

This skips the login screen and auto-provisions an admin user.

Desktop Mode

Liteskill can run as a desktop application via Tauri (ex_tauri):

LITESKILL_DESKTOP=true mix phx.server

In desktop mode:

• A bundled PostgreSQL instance is started automatically
• An encryption key and secret key base are auto-generated and stored in desktop_config.json
• Single-user mode is enabled automatically
• Data is stored in platform-specific directories:
  • macOS: ~/Library/Application Support/Liteskill
  • Linux: $XDG_DATA_HOME/liteskill (default: ~/.local/share/liteskill)
  • Windows: %APPDATA%/Liteskill

Docker Compose

For production-like local development:

export SECRET_KEY_BASE=$(openssl rand -base64 64)
export ENCRYPTION_KEY=$(openssl rand -base64 32)
docker compose up

The docker-compose.yml includes:

• db — pgvector/pgvector:pg16 with persistent volume and health checks
• app — The Liteskill release, using network_mode: host for direct database access
• migrate — One-shot migration runner (activated with docker compose run migrate)

First Run

Initial Setup

On first boot, Liteskill:

1. Validates the encryption key (ENCRYPTION_KEY env var or auto-generated in desktop mode)
2. Creates ETS tables for rate limiting and LLM token buckets
3. Ensures a root admin user exists ([email protected])
4. Bootstraps system RBAC roles
5. Ensures environment-configured providers are created (e.g. AWS Bedrock via AWS_BEARER_TOKEN_BEDROCK)
6. Loads application settings

Admin Setup

Navigate to http://localhost:4000/setup to set the admin password. This route is only available when the admin user hasn’t been configured yet.

After setup, log in at http://localhost:4000/login.

Configuring LLM Providers

Before you can chat, configure at least one LLM provider:

1. Go to Admin > Providers (/admin/providers)
2. Add a provider (e.g. Amazon Bedrock, OpenRouter, or any OpenAI-compatible endpoint)
3. Go to Admin > Models (/admin/models) and add models under that provider

Users can also add personal providers at Profile > Providers (/profile/providers).

Configuring MCP Servers

To enable tool calling:

1. Go to MCP Servers (/mcp)
2. Add an MCP server URL with optional API key and custom headers
3. The server’s tools will be available in conversations when selected

Configuring RAG

To enable document-augmented conversations:

1. Go to Admin > RAG (/admin/rag) and configure the default embedding model
2. Navigate to Sources (/sources) to create collections and add documents
3. Documents are automatically chunked and embedded for semantic search

Architecture Overview

Liteskill is an event-sourced Phoenix application organized around bounded contexts with enforced boundaries (via the boundary library).

High-Level Architecture

┌─────────────────────────────────────────────────────────┐
│                    LiteskillWeb                          │
│  LiveView UI │ REST API │ Auth Plugs │ Rate Limiting     │
└──────────┬──────────────┬────────────────────────────────┘
           │              │
┌──────────▼──────────────▼────────────────────────────────┐
│                   Context Layer                           │
│  Chat │ Accounts │ Authorization │ Groups │ LLM │ ...    │
└──────────┬───────────────────────────────────────────────┘
           │
┌──────────▼───────────────────────────────────────────────┐
│              Infrastructure Layer                         │
│  EventStore │ Aggregate │ Crypto │ Repo │ PubSub         │
└──────────────────────────────────────────────────────────┘

Bounded Contexts

Each context is a top-level Boundary module that declares its dependencies and exports:

Key Design Decisions

• Event sourcing for chat — Conversations are modeled as event streams for full auditability and state replay. All other domains use standard CRUD with Ecto.
• CQRS — Write operations go through the event sourcing pipeline; reads query projection tables directly.
• Boundary enforcement — The boundary library enforces context dependencies at compile time, preventing unauthorized cross-context calls.
• ReqLLM for all LLM transport — No direct HTTP calls to LLM providers. ReqLLM abstracts provider differences.
• Unified ACL table — A single entity_acls table handles access control for all entity types (conversations, reports, wiki spaces, agents, etc.).
• Oban for background jobs — Embedding, data source sync, and agent runs are processed as Oban jobs with configurable queues and concurrency.

Event Sourcing

Liteskill uses event sourcing for the chat domain. Conversations are modeled as event streams rather than mutable rows.

Flow

Command → Aggregate → Event → EventStore (append) → PubSub broadcast → Projector → Projection tables

1. A context function (e.g. Chat.send_message/4) constructs a command tuple
2. Aggregate.Loader.execute/3 loads the aggregate state by replaying events (or from snapshot), executes the command, and appends resulting events
3. EventStore.Postgres.append_events/3 inserts events in a transaction with optimistic concurrency (unique index on stream_id, stream_version)
4. After successful append, events are broadcast via Phoenix.PubSub on topic "event_store:<stream_id>"
5. Chat.Projector (a GenServer in the supervision tree) receives events and updates projection tables

EventStore

Liteskill.EventStore.Postgres implements the Liteskill.EventStore behaviour:

• append_events(stream_id, expected_version, events_data) — Append with optimistic concurrency
• read_stream_forward(stream_id) — Read all events for a stream
• stream_version(stream_id) — Get the current version
• subscribe(stream_id) — Subscribe to PubSub notifications
• save_snapshot/4 and get_latest_snapshot/1 — Snapshot support for performance

ConversationAggregate

State machine with states: :created → :active ↔ :streaming → :archived

Commands:

• :create_conversation — Creates a new conversation
• :add_user_message — Adds a user message
• :start_stream — Begins LLM streaming
• :add_chunk — Appends a streaming chunk
• :complete_stream — Marks stream as complete
• :fail_stream — Records a stream failure
• :start_tool_call / :complete_tool_call — Tool calling lifecycle
• :archive — Archives the conversation
• :update_title — Updates the title
• :truncate_conversation — Truncates at a message boundary

Events

The system defines 12 domain event types:

Event Serialization

Events are stored with string keys (via stringify_keys). Deserialization back to structs happens through Events.deserialize/1. The event type string (e.g. "ConversationCreated") is mapped to its module via Events.module_for/1.

Stream IDs

Stream IDs follow the format "conversation-<uuid>".

Projector

Liteskill.Chat.Projector is a GenServer in the main supervision tree that projects events into read-model tables:

• Synchronous — project_events/2 blocks until projection is complete (used for write operations that need immediate read consistency)
• Asynchronous — project_events_async/2 casts without waiting (used for streaming chunks where eventual consistency is acceptable)
• Rebuild — rebuild_projections/0 replays all events from scratch to reconstruct projection tables

Snapshots

The aggregate loader supports snapshots for performance. When loading an aggregate, it first checks for the latest snapshot and replays only events after the snapshot version, avoiding full event replay for long-lived streams.

Supervision Tree

Liteskill uses a rest_for_one supervision strategy. If an infrastructure child (like Repo or PubSub) crashes, all children started after it restart too, re-establishing subscriptions and connections.

Child Process Order

The supervision tree starts children in this order:

1. LiteskillWeb.Telemetry — Telemetry metrics and poller
2. Liteskill.Desktop.PostgresManager — (Desktop mode only) Starts bundled PostgreSQL
3. Liteskill.Repo — Ecto repository / database connection pool
4. DNSCluster — DNS-based node clustering
5. Phoenix.PubSub — PubSub server (Liteskill.PubSub)
6. Liteskill.Rag.EmbedQueue — Embedding request queue (skipped in test)
7. Oban — Background job processing (queues: default, rag_ingest, data_sync, agent_runs)
8. Boot tasks — (non-test) Ensures admin user, system roles, env providers, and settings; auto-provisions admin in single-user mode
9. Liteskill.OpenRouter.StateStore — OAuth PKCE state storage for OpenRouter
10. LiteskillWeb.Plugs.RateLimiter.Sweeper — Periodic ETS cleanup for rate limiter
11. Task.Supervisor (Liteskill.TaskSupervisor) — For LLM streaming and async work
12. Liteskill.Chat.StreamRegistry — Monitors active LLM stream tasks, triggers recovery on crash
13. Registry (Liteskill.LlmGateway.GateRegistry) — Per-provider gate registry
14. DynamicSupervisor (Liteskill.LlmGateway.GateSupervisor) — Dynamic circuit breaker gates
15. Liteskill.LlmGateway.TokenBucket.Sweeper — Periodic ETS cleanup for token buckets
16. Liteskill.Chat.Projector — Event → projection table updater
17. Liteskill.Chat.StreamRecovery — Periodic sweep for stuck streaming conversations
18. Liteskill.Schedules.ScheduleTick — (non-test) Periodic check for due schedules
19. LiteskillWeb.Endpoint — Phoenix HTTP endpoint (started last)

LLM Gateway

The LLM Gateway provides per-provider concurrency control and circuit breaking:

• GateRegistry — Unique registry keyed by provider ID
• GateSupervisor — Dynamically spawns Gate processes per provider
• TokenBucket — ETS-based rate limiting with periodic sweeper

Stream Registry

Liteskill.Chat.StreamRegistry monitors active LLM streaming tasks. If a streaming task crashes, the registry detects the :DOWN message and triggers stream recovery, preventing conversations from getting stuck in the :streaming state.

Stream Recovery

Liteskill.Chat.StreamRecovery is a periodic sweeper that detects conversations stuck in streaming state for more than 5 minutes and automatically recovers them by failing the stream.

Database

Liteskill uses PostgreSQL 16 with the pgvector extension for vector similarity search.

Conventions

• Binary UUIDs for all primary keys
• :utc_datetime for all timestamps
• All schemas use field :name, :string even for text columns
• Foreign keys (e.g. user_id) are set programmatically, never in cast
• Custom Postgrex types registered via Liteskill.Repo.PostgrexTypes

Key Tables

Event Store

• events — Append-only event log with (stream_id, stream_version) unique index
• snapshots — Aggregate snapshots for performance

Projections (Chat)

• conversations — Current conversation state (stream_id, title, status, model, message_count)
• messages — Projected messages (role, content, status, token counts, position)
• message_chunks — Streaming chunks (delta_text, delta_type, chunk_index)
• tool_calls — Tool call records (name, arguments, status, result)

Accounts & Auth

• users — User records (OIDC and password auth)
• invitations — Admin-created invite tokens
• entity_acls — Unified ACL table for all entity types
• rbac_roles / rbac_role_permissions / rbac_user_roles — Role-based access control

LLM

• llm_providers — Provider configurations (API keys encrypted, regions, types)
• llm_models — Model definitions linked to providers (costs, context window, type)
• usage_records — Token/cost tracking per API call

RAG

• rag_collections — Top-level grouping for embeddings (configurable dimension)
• rag_sources — Sources within collections
• rag_documents — Documents to be chunked and embedded
• rag_chunks — Chunked text with pgvector embeddings

Data Sources

• data_sources — External connectors (Google Drive, Confluence, Jira, GitHub, GitLab, SharePoint)
• data_source_documents — Documents synced from connectors

Agents & Teams

• agent_definitions — AI agent configurations (backstory, strategy, model)
• team_definitions — Agent team compositions
• team_members — Position-ordered team membership with roles

Runs

• runs — Agent/team execution records (status, prompt, topology)
• run_tasks — Individual task tracking within a run
• run_logs — Structured execution logs (level, step, message, metadata)

Features

• reports / report_sections / section_comments — Structured report documents
• schedules — Cron-based scheduled runs
• mcp_servers — MCP server registrations
• user_tool_selections — Per-user MCP server selections
• settings — Application-wide settings

Background Jobs

• oban_jobs / oban_peers — Oban background job queue tables

Conversations

Conversations are the core feature of Liteskill. Each conversation is an event-sourced stream that records every user message, LLM response, and tool call.

Creating Conversations

Users create conversations from the main UI. Each conversation has:

• A title (auto-generated or user-set)
• A model selection (from configured LLM models)
• An optional system prompt
• An optional LLM model override

Messaging

When a user sends a message:

1. A UserMessageAdded event is appended to the conversation stream
2. The LLM StreamHandler starts a streaming response
3. Chunks arrive as AssistantChunkReceived events
4. The stream completes with AssistantStreamCompleted

Tool Calling

During streaming, the LLM may request tool calls:

• Auto-confirm mode: Tool calls execute automatically via MCP servers
• Manual mode: The UI pauses for user approval before executing each tool call

Tool calls are recorded as ToolCallStarted and ToolCallCompleted events.

RAG Context

When RAG is configured, the user’s message is embedded and searched against accessible collections. Relevant document chunks are included as context for the LLM. RAG sources are tracked per message for transparency.

Forking

Conversations can be forked at any message position, creating a new independent conversation with the history up to that point. The fork creates a new event stream with copied events and remapped IDs.

Editing Messages

Users can edit a previously sent message. This truncates the conversation at that message position and re-sends with the new content, effectively creating a new branch point.

Sharing

Conversation access is controlled via ACLs:

• The creator is automatically the owner
• Owners can grant manager access to other users or groups
• Shared conversations appear in the recipient’s conversation list

Archiving

Conversations can be archived individually or in bulk. Archived conversations are hidden from the default list but retain their full event history.

Stream Recovery

A periodic sweeper (Liteskill.Chat.StreamRecovery) detects conversations stuck in streaming state for more than 5 minutes and automatically recovers them by failing the stream. Additionally, the StreamRegistry monitors active streaming tasks and triggers immediate recovery if a task crashes.

LLM Providers

Liteskill supports multiple LLM providers through a pluggable provider system powered by ReqLLM.

Provider Types

Providers are configured in the database via the admin UI. ReqLLM supports 56+ providers including:

• Amazon Bedrock — AWS-hosted models (Claude, Llama, etc.)
• OpenRouter — Multi-model gateway with OAuth PKCE support
• OpenAI-compatible — Any endpoint that speaks the OpenAI API format (vLLM, LiteLLM, etc.)
• Anthropic — Direct Anthropic API access
• Google — Gemini models via Google AI
• Azure OpenAI — Azure-hosted OpenAI models

Provider Configuration

Each provider record stores:

• Name — Display name
• Provider type — Determines the API protocol
• API key — Encrypted at rest via Liteskill.Crypto
• Base URL — Optional override for custom endpoints (e.g. LiteLLM proxies, local vLLM)
• Provider config — Type-specific settings (e.g. AWS region, Azure deployment ID)
• Instance-wide flag — If true, available to all users
• Status — Active or inactive

Access Control

• Instance-wide providers are available to all users
• User-owned providers are private to their creator (configured at /profile/providers)
• Admin-granted access — Admins can grant viewer role on a provider to specific users via ACLs

Models

Models are defined under providers. Each model specifies:

• Model ID — The provider’s model identifier (e.g. anthropic/claude-3-5-haiku)
• Display name — Shown in the UI
• Model type — inference, embedding, or rerank
• Cost rates — Input/output cost per million tokens
• Context window — Maximum token limit
• Instance-wide flag — Available to all users when set
• Active/inactive status

Users select models when creating conversations or configuring agents.

Environment-Based Providers

On boot, Liteskill checks for environment variables and auto-creates providers:

• AWS_BEARER_TOKEN_BEDROCK + AWS_REGION — Creates an instance-wide Bedrock provider

LLM Gateway

The LLM Gateway provides per-provider infrastructure:

• Circuit breaker — Tracks failures per provider and opens the circuit after a threshold, preventing cascading failures
• Concurrency gates — Limits concurrent requests per provider
• Token bucket — ETS-based rate limiting with periodic cleanup

ReqLLM Configuration

ReqLLM connection pools and timeouts are configured in config/runtime.exs:

• Stream receive timeout: 120 seconds (accommodates tool-calling rounds)
• HTTP/1.1 pool with 25 connections

MCP Tools

Liteskill integrates with Model Context Protocol (MCP) servers to give LLMs access to external tools.

How It Works

1. Register an MCP server with its HTTP URL
2. Liteskill discovers available tools via tools/list
3. During conversation streaming, the LLM can request tool calls
4. Liteskill executes tool calls via tools/call and feeds results back

MCP Client

Liteskill.McpServers.Client implements the MCP JSON-RPC 2.0 Streamable HTTP transport:

1. Initialize — Sends initialize request, receives session ID
2. Initialized — Sends notifications/initialized notification
3. Request — Sends tools/list or tools/call with the session ID

The client supports:

• Automatic retry with exponential backoff on 429/5xx errors (up to 2 retries)
• Session ID tracking via mcp-session-id header
• Custom headers per server
• API key authentication (sent as Authorization: Bearer <key>)
• SSE response parsing

Server Management

• Users can create, update, and delete their own MCP servers
• Global servers (set by admin) are available to all users
• Servers can be shared via ACLs

Tool Selections

Users select which MCP servers are active for their conversations:

• Selections are persisted per user via UserToolSelection
• Selected servers’ tools are included in LLM requests
• Stale selections (for deleted servers) are automatically pruned

Built-in Servers

Virtual servers prefixed with builtin: are defined in code (Liteskill.BuiltinTools) and merged into the server list. They cannot be modified or deleted. These provide tools for built-in features like reports and wiki.

SSRF Protection

Server URLs are validated to prevent Server-Side Request Forgery (SSRF). By default:

• Only HTTPS URLs are accepted
• Private and reserved addresses are blocked (localhost, 127.*, 10.*, 172.16-31.*, 192.168.*, 169.254.*, IPv6 loopback, and host.docker.internal)

To allow private URLs (e.g. for self-hosted MCP servers), enable Allow private MCP URLs in server settings.

Docker Networking

The Liteskill app container uses network_mode: host, which means it shares the host’s network stack directly. MCP servers running on the host machine are reachable at localhost:

http://localhost:4005

This requires Allow private MCP URLs enabled in server settings, since localhost resolves to a private address.

RAG (Retrieval-Augmented Generation)

Liteskill includes a full RAG pipeline: document ingestion, chunking, embedding generation, semantic search, and reranking.

Pipeline

1. Ingest — Documents are added to a collection via URL, manual upload, or data source sync
2. Chunk — Documents are split into chunks using a recursive text splitter (paragraph → line → sentence → word boundaries)
3. Embed — Chunks are embedded using a configured embedding model (Cohere or OpenAI-compatible)
4. Search — User queries are embedded and matched against chunks using pgvector cosine similarity
5. Rerank — Search results are optionally reranked using a Cohere rerank model

Data Model

• Collection — Top-level grouping (e.g. “Wiki”, “Engineering Docs”) with configurable embedding dimension (256–1536, default 1024)
• Source — A source within a collection (e.g. “wiki”, “manual”)
• Document — A single document with content, metadata, and content hash
• Chunk — A text chunk with position, token count, and its pgvector embedding

Embedding

Embeddings are generated via Liteskill.Rag.EmbedQueue, a GenServer that batches requests and manages throughput. Two client implementations:

• CohereClient — For Cohere’s embed and rerank APIs (including Cohere on AWS Bedrock)
• OpenAIEmbeddingClient — For OpenAI-compatible embedding endpoints

Context Augmentation

During conversations, RAG context is injected automatically:

1. The user’s message is embedded
2. All accessible collections are searched
3. Top results are optionally reranked
4. Relevant chunks are included as context for the LLM
5. RAG sources are tracked per message for citation

Wiki Integration

Wiki pages are automatically synced to RAG collections. When a wiki page is created or updated, a background job (WikiSyncWorker) updates the corresponding RAG document and re-embeds its chunks.

URL Ingestion

The IngestWorker (Oban job) handles URL-based document ingestion:

1. Fetches URL via Req
2. Validates content type (rejects binary content)
3. Auto-creates a source from the domain name
4. Chunks the response body
5. Enqueues embedding via EmbedQueue
6. Retries up to 3 times on transient failures

Re-embedding

Admins can trigger a full re-embedding of all documents (e.g. after changing the embedding model) via the ReembedWorker.

Agent Studio

The Agent Studio lets users define reusable AI agents with specific configurations, tool access, and data source access.

Agent Definitions

Each agent definition includes:

• Name and backstory — Identity and persona
• Strategy — Behavioral approach (react, chain_of_thought, tree_of_thoughts, direct)
• Opinions — Behavioral guidance and perspective
• LLM model — The model the agent uses for inference
• System prompt — Optional per-agent system prompt override
• Tool access — ACL-controlled access to specific MCP servers
• Data source access — ACL-controlled access to specific data sources for RAG

Tool Access

Agents are granted access to MCP servers via ACLs on the mcp_server entity type. During execution, the ToolResolver resolves which tools the agent can use based on its ACLs.

Data Source Access

Similarly, agents can be granted access to data sources. During runs, the agent’s RAG context is scoped to only the data sources it has access to.

Teams

Agents can be composed into teams with different topologies:

• Sequential (Pipeline) — Agents execute in order, passing results forward. Later agents see the accumulated context from prior stages.
• Parallel — Agents execute concurrently
• Supervisor — A supervisor agent coordinates other agents

Team members are position-ordered and assigned roles (lead, analyst, reviewer, editor).

Execution

Agents execute through the Runs system. A run tracks the full lifecycle of an agent or team execution, including:

• Prompt — Input for the pipeline
• Logs — Structured execution logs with level, step, message, and metadata
• Tasks — Individual task tracking within a run
• Usage — Token/cost accounting per run
• Report — Optional deliverable report generated from the run output

Jido Integration

Under the hood, agent execution uses the Jido framework (v2.0) for structured agent actions and workflows. The LlmGenerate action provides a synchronous agentic loop with tool calling, context pruning, and cost limits.

Access Control

Agent definitions use the standard ACL system:

• Creator is the owner
• Access can be granted to other users
• Only accessible agents appear in the agent list

Routes

• /agents — Agent Studio landing page
• /agents/list — List all agents
• /agents/new — Create a new agent
• /agents/:agent_id — View agent details
• /agents/:agent_id/edit — Edit an agent
• /teams — List teams
• /teams/new — Create a new team
• /teams/:team_id — View/edit a team
• /runs — List runs
• /runs/new — Start a new run
• /runs/:run_id — View run details and logs

Reports

Reports are structured documents with hierarchical sections, markdown rendering, and a comment/review workflow.

Structure

• Report — Top-level document with a title and owner
• Sections — Infinitely nested sections with titles, content, and position ordering
• Comments — Threaded comments on sections or the report itself

Section Management

Sections are addressed by path (e.g. "Executive Summary > Key Findings"). The path-based API supports:

• Upsert — Create or update a section at a path
• Delete — Remove a section and its children
• Move — Reposition a section among its siblings

Batch operations via modify_sections/3 allow multiple actions in a single transaction.

Comments

Comments support a review workflow:

• Comments have an author_type ("user" or "agent")
• Comments can be open or addressed
• Resolving a comment creates a reply and marks it as addressed
• Comments can be threaded (replies to replies)

An LLM agent can be instructed to address open comments, making edits and resolving each one.

Markdown Rendering

render_markdown/2 produces a complete markdown document from a report:

• Section titles become #, ##, ### headers based on nesting depth
• Comments are rendered as blockquotes with status labels
• Comments can be included or excluded

Access Control

Reports use the standard ACL system:

• Creator is the owner
• Access can be granted to other users at the manager level
• Shared reports appear in the recipient’s report list

Run Integration

Reports can be created as part of an agent run, linking the report to its source run for traceability. The ReportBuilder in the runs system generates structured report output from pipeline execution.

Routes

• /reports — List all reports
• /reports/:report_id — View/edit a report

Wiki

Liteskill includes a built-in collaborative wiki for documentation and knowledge management.

Structure

The wiki uses a hierarchical document model:

• Spaces — Top-level wiki documents (root pages with no parent)
• Pages — Child documents nested under spaces
• Pages can be nested to arbitrary depth

All wiki documents use the built-in "builtin:wiki" source reference.

Creating Content

• Create a new wiki space from the wiki page (/wiki)
• Add child pages under any space or page
• Edit pages with markdown content
• Pages have titles, slugs, and optional content

Sharing

Wiki spaces use ACL-based sharing:

• The space creator is the owner
• Owners can grant viewer, editor, or manager access to other users
• Editors can create and edit pages within the space
• Managers can also delete pages and manage access
• Shared spaces appear in the recipient’s wiki

RAG Integration

Wiki content is automatically indexed for RAG:

• When a page is created or updated, a WikiSyncWorker job is enqueued
• The worker creates/updates a RAG document and re-embeds chunks
• Wiki collections are shared with users who have access to the wiki space
• This means RAG search results include relevant wiki content from shared spaces

Export & Import

• Wiki spaces can be exported via the /wiki/:space_id/export route
• Reports can be exported to wiki pages via DataSources.export_report_to_wiki/3
• Wiki content can be imported from external sources

Routes

• /wiki — Wiki home (list all accessible spaces)
• /wiki/:document_id — View/edit a wiki page

Schedules

Schedules enable cron-based recurring execution of agent runs.

How It Works

1. Create a schedule with a cron expression, timezone, and run template
2. ScheduleTick (a periodic GenServer) checks for due schedules every minute
3. When a schedule is due, it enqueues a ScheduleWorker Oban job
4. The worker creates and starts a run based on the schedule’s template
5. next_run_at is recalculated for the next occurrence

Schedule Configuration

Each schedule defines:

• Name — Display name
• Cron expression — Standard 5-field format (minute, hour, day-of-month, month, day-of-week)
• Timezone — The cron expression is interpreted in this timezone
• Team definition — The agent team to execute
• Enabled flag — Schedules can be toggled on/off
• Status — active or other lifecycle states

Cron Parser

The built-in cron parser supports:

• Wildcards (*)
• Step values (*/5)
• Exact values (30)
• Comma-separated lists (1,15,30)

Access Control

Schedules use the standard ACL system. Only the owner can update or delete a schedule.

Routes

• /schedules — List all schedules
• /schedules/new — Create a new schedule
• /schedules/:schedule_id — View/edit a schedule

Chat Context

Liteskill.Chat is the primary context for conversation management. It provides write and read APIs backed by event sourcing.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Aggregate, Liteskill.Authorization, Liteskill.EventStore, Liteskill.Rbac, Liteskill.LlmModels],
  exports: [Conversation, ConversationAggregate, Events, Message, MessageBuilder, MessageChunk, Projector, StreamRecovery, StreamRegistry, ToolCall]

Write API

All write operations go through the event sourcing pipeline: command → aggregate → event store → projector.

Read API

Read operations query the projection tables directly.

ACL Management

Delegates to Liteskill.Authorization:

• grant_conversation_access/4 — Grant user access (normalizes “member” to “manager”)
• revoke_conversation_access/3 — Revoke user access
• leave_conversation/2 — User leaves a shared conversation
• grant_group_access/4 — Grant group access

Streaming

Streaming is initiated after a user message is sent. The StreamHandler (in Liteskill.LLM) manages the LLM streaming lifecycle, emitting events back to the conversation’s event stream. The Projector updates projection tables as events arrive, and LiveView receives real-time updates via PubSub.

Message Builder

Liteskill.Chat.MessageBuilder constructs the message history for LLM requests from the conversation’s projected messages, including tool call results and RAG context.

Accounts Context

Liteskill.Accounts manages user records, authentication, and invitations.

Boundary

use Boundary, top_level?: true, deps: [], exports: [User, Invitation]

Authentication

Liteskill supports dual authentication:

OIDC (via Ueberauth)

• find_or_create_from_oidc(attrs) — Finds or creates a user by OIDC subject + issuer
• Idempotent — safe to call on every login callback

Password (via Argon2)

• register_user(attrs) — Creates a user with email/password
• authenticate_by_email_password(email, password) — Validates credentials
• change_password(user, current_password, new_password) — Changes password (requires current)
• setup_admin_password(user, password) — First-time admin setup (no current password required)
• set_temporary_password(user, password) — Admin sets a temp password (forces change on login)

User Management

Root Admin

On boot, ensure_admin_user/0 creates [email protected] if missing and forces its role to "admin".

Single-User Mode

When SINGLE_USER_MODE=true, SingleUser.auto_provision_admin/0 runs on boot to ensure the admin user is ready without manual setup.

Invitations

• create_invitation(email, admin_user_id) — Creates an invite token
• get_invitation_by_token(token) — Looks up an invitation
• accept_invitation(token, attrs) — Accepts: validates token, creates user, marks as used (transactional)
• revoke_invitation(id) — Deletes a pending invitation
• list_invitations() — Lists all invitations

Authorization Context

Liteskill.Authorization provides centralized access control for all entity types via a single entity_acls table.

Boundary

use Boundary, top_level?: true, deps: [Liteskill.Groups], exports: [EntityAcl, Roles]

Role Hierarchy

viewer < editor < manager < owner

Access Checks

ACL Management

Query Helpers

• accessible_entity_ids(entity_type, user_id) — Subquery of entity IDs the user can access
• usage_accessible_entity_ids(entity_type, user_id) — Same but excludes “owner” role

Entity Types

The ACL system is used across: conversation, report, wiki_space, mcp_server, agent_definition, team_definition, run, schedule, source, llm_provider, llm_model.

Groups Context

Liteskill.Groups manages user groups and memberships, used for group-based ACL authorization.

Boundary

use Boundary, top_level?: true, deps: [], exports: [Group, GroupMembership]

Group Management

Membership Management

Admin Functions

Admin functions bypass creator/membership checks:

• list_all_groups/0 — Lists all groups with memberships and creator preloaded
• admin_get_group/1, admin_get_group_by_name/1
• admin_list_members/1, admin_add_member/3, admin_remove_member/2
• admin_delete_group/1

Usage with ACLs

Groups are used in the Authorization context for group-based access. When a group is granted access to an entity (e.g. a conversation), all members of that group inherit that access via a join with group_memberships.

LLM Context

Liteskill.LLM is the public API for LLM interactions, providing both streaming and non-streaming completions.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Chat, Liteskill.Aggregate, Liteskill.EventStore, Liteskill.Usage, Liteskill.LlmModels, Liteskill.LlmGateway, Liteskill.McpServers],
  exports: [StreamHandler, ToolUtils, RagContext]

Non-Streaming Completions

LLM.complete(messages, opts) sends a single-turn completion request via ReqLLM. Used for tasks like auto-generating conversation titles.

Options:

• :llm_model — A %LlmModel{} struct with full provider config
• :model_id — Model ID string (requires :provider_options)
• :max_tokens, :temperature, :system — Standard LLM parameters
• :generate_fn — Override for testing

Streaming

Streaming is handled by LLM.StreamHandler, which orchestrates:

1. Starting a streaming request via ReqLLM
2. Appending events to the conversation’s event stream
3. Retrying on 429/503/408 errors and transport failures (up to 3 retries with exponential backoff)
4. Executing tool calls (auto-confirm or pause for approval)
5. Recording usage after each round

Model Resolution

Models are configured in the database via LlmModels and LlmProviders:

• available_models(user_id) — Returns active inference models accessible to the user
• No hardcoded model IDs or env-var fallbacks for model selection

LLM Gateway

The gateway provides per-provider infrastructure:

• Circuit breaker — Tracks failures and opens the circuit after threshold
• Concurrency gates — Limits concurrent requests per provider
• Token bucket — ETS-based rate limiting with periodic cleanup

Usage Recording

After each streaming round, usage is recorded via Liteskill.Usage.record_from_response/2. This captures input tokens, output tokens, reasoning tokens, cached tokens, costs, and latency.

RagContext

Liteskill.LLM.RagContext builds RAG context for prompts:

• Searches accessible RAG collections for relevant documents
• Injects top-k chunks as system context
• Tracks RAG sources for citation and traceability

ToolUtils

Liteskill.LLM.ToolUtils provides shared utilities:

• Convert tool specs to ReqLLM format
• Dispatch tool execution to MCP servers
• Format tool outputs for LLM consumption
• Normalize tool call formats across providers

MCP Servers Context

Liteskill.McpServers manages MCP server registrations and user tool selections.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Authorization, Liteskill.Rbac, Liteskill.BuiltinTools, Liteskill.Settings],
  exports: [McpServer, Client, UserToolSelection]

Server CRUD

Built-in Servers

Virtual servers prefixed with builtin: are defined in code (Liteskill.BuiltinTools) and merged into the server list. They cannot be modified or deleted.

Tool Selections

Users select which servers are active for their conversations:

Client

Liteskill.McpServers.Client implements JSON-RPC 2.0 over HTTP:

• list_tools(server, opts) — Discovers available tools
• call_tool(server, tool_name, arguments, opts) — Executes a tool call

Features:

• MCP initialization handshake (initialize → initialized → request)
• Automatic retry with exponential backoff (up to 2 retries)
• Session ID tracking via mcp-session-id header
• Custom header support with blocked header filtering for security
• API key authentication via Authorization: Bearer <key> header

RAG Context

Liteskill.Rag manages the RAG pipeline: collections, sources, documents, chunks, embeddings, and search.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Authorization, Liteskill.DataSources, Liteskill.LlmModels, Liteskill.LlmProviders, Liteskill.Settings],
  exports: [Collection, Source, Document, Chunk, Chunker, CohereClient, DocumentSyncWorker, EmbedQueue, EmbeddingClient, EmbeddingRequest, IngestWorker, OpenAIEmbeddingClient, Pipeline, ReembedWorker, WikiSyncWorker]

Data Model

• Collection — Top-level grouping, per user, with configurable embedding dimension (256–1536, default 1024)
• Source — A source within a collection (e.g. “wiki”, “manual”)
• Document — Content with title, status, metadata, and content hash
• Chunk — Text chunk with position, token count, and pgvector embedding

Collection & Source CRUD

Standard CRUD with user ownership checks. Collections and sources are scoped to the creating user.

Embedding

embed_chunks(document_id, chunks, user_id, opts):

1. Validates ownership chain (document → source → collection)
2. Sends texts to EmbedQueue for embedding
3. Inserts chunk rows with pgvector embeddings in a transaction
4. Updates document status to "embedded"

Search

Wiki Integration

• find_or_create_wiki_collection(user_id) — Gets or creates the “Wiki” collection
• find_or_create_wiki_source(collection_id, user_id) — Gets or creates the “wiki” source
• find_rag_document_by_wiki_id(wiki_document_id, user_id) — Finds RAG doc by wiki doc ID

URL Ingestion

The IngestWorker (Oban job) handles URL-based ingestion:

1. Fetches URL content via Req
2. Validates text content (rejects binary types)
3. Auto-creates source from domain name
4. Chunks content and enqueues for embedding
5. Retries up to 3 times on transient failures

Background Workers

Reports Context

Liteskill.Reports manages structured reports with nested sections and a comment/review workflow.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Accounts, Liteskill.Authorization, Liteskill.Rbac],
  exports: [Report, ReportSection, SectionComment]

Report CRUD

Section Management

Sections are addressed by > delimited paths (e.g. "Summary > Findings").

Comments

Rendering

• render_markdown(report, opts) — Full markdown with headers and optional comments
• section_tree(report) — Nested tree structure for UI rendering

Agents Context

Liteskill.Agents manages agent definitions — the “character sheets” for AI agents.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Authorization, Liteskill.Rbac, Liteskill.McpServers, Liteskill.LLM, Liteskill.LlmGateway, Liteskill.Usage, Liteskill.LlmModels, Liteskill.Rag, Liteskill.DataSources],
  exports: [AgentDefinition, ToolResolver, JidoAgent, Actions.LlmGenerate]

Agent CRUD

Tool Access

Agents are granted access to MCP servers via entity ACLs:

Data Source Access

Agents can also be granted access to data sources for RAG:

Model Validation

When creating or updating an agent with an llm_model_id, the context validates that the user has access to that model via LlmModels.get_model/2.

Tool Resolution

Liteskill.Agents.ToolResolver resolves the set of tools available to an agent at execution time by querying its MCP server ACLs and fetching tool lists from each accessible server.

Execution

Agent execution is handled by Liteskill.Agents.Actions.LlmGenerate, which provides a synchronous agentic loop:

1. Build messages from context
2. Send to LLM with available tools
3. If LLM requests tool calls, execute them and loop
4. Apply context pruning if needed
5. Enforce cost limits per run

Usage Context

Liteskill.Usage records and queries LLM token usage and costs.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Groups, Liteskill.LlmModels, Liteskill.Rag],
  exports: [UsageRecord, CostCalculator]

Recording Usage

record_from_response/2 resolves costs via CostCalculator, assembles all fields, and inserts. No-ops when user_id is nil.

Each record tracks: input_tokens, output_tokens, total_tokens, reasoning_tokens, cached_tokens, cache_creation_tokens, input_cost, output_cost, reasoning_cost, total_cost, latency_ms, call_type, and tool_round.

Aggregation Queries

Cost Limits

• check_cost_limit(:conversation, id, limit) — Checks if conversation cost exceeds limit
• check_cost_limit(:run, id, limit) — Checks if run cost exceeds limit

Cost Calculator

Liteskill.Usage.CostCalculator computes costs from token counts and model pricing:

• Resolves per-model input/output rates from the LlmModel record
• Calculates separate costs for input, output, and reasoning tokens
• Handles cached token pricing

Embedding Usage

Separate tracking for embedding API calls:

• embedding_totals(opts) — Aggregate embedding usage

Data Sources Context

Liteskill.DataSources manages external data connectors and their documents.

Boundary

use Boundary,
  top_level?: true,
  deps: [Liteskill.Authorization, Liteskill.Rbac, Liteskill.BuiltinSources, Liteskill.Reports],
  exports: [Source, Document, SyncWorker, Connector, ConnectorRegistry, ContentExtractor, Connectors.GoogleDrive, Connectors.Wiki, WikiExport, WikiImport]

Supported Source Types

Source CRUD

Document Management

Sync Pipeline

• start_sync(source_id, user_id) — Enqueues a SyncWorker Oban job
• SyncWorker uses the ConnectorRegistry to dispatch to the correct connector (Google Drive, SharePoint, etc.)
• Each connector fetches updated documents, creates/updates local records, and triggers RAG re-embedding

Wiki Integration

• Connectors.Wiki handles built-in wiki documents
• WikiExport exports wiki spaces as structured documents
• WikiImport imports wiki content from external sources
• export_report_to_wiki/3 converts a report to wiki pages

Content Extraction

ContentExtractor handles converting various document formats to plain text for embedding.

Built-in Sources

Liteskill.BuiltinSources defines virtual sources that exist in code (e.g. the Wiki source). These appear in the source list but cannot be modified or deleted.

Router

LiteskillWeb.Router defines all routes for the application. Routes are organized into scopes with different pipelines.

Pipelines

Auth Routes (/auth)

Public LiveView Routes

Admin LiveView Routes

Require admin role via require_admin mount hook.

Authenticated LiveView Routes

Chat

Profile

Wiki

Sources & RAG

MCP Servers

Reports

Agent Studio

LiveView

Liteskill’s UI is built entirely with Phoenix LiveView. The interface is composed of several LiveView modules, each handling a different feature area.

Live Sessions

The router defines several live sessions with different auth requirements:

LiveView Modules

ChatLive

The primary chat interface. Handles:

• Message list with real-time streaming updates
• Tool call approval UI
• RAG source display
• Conversation forking and editing
• Model selection

AdminLive

Multi-tab admin dashboard with separate tab modules:

• UsageTab — Usage analytics and reporting
• ServerTab — MCP server registry
• UsersTab — User management and invitations
• GroupsTab — Group management
• ProvidersTab — LLM provider configuration
• ModelsTab — LLM model configuration
• RolesTab — RBAC role management
• RagTab — RAG settings and collection management

SetupLive

Setup wizard used for both first-time admin setup (/setup) and admin re-run (/admin/setup). Dual-mode: :initial skips existing config; :admin_rerun loads current DB state.

AgentStudioLive

Agent, team, run, and schedule management with live actions:

• Agent CRUD (list, new, show, edit)
• Team composition
• Run execution with real-time log streaming
• Schedule management

ProfileLive

User account settings: profile info, password change, personal providers and models.

ReportsLive

Report viewing and editing with section management and comment threads.

SourcesLive

RAG collection browser with document and chunk management.

PipelineLive

RAG ingestion pipeline UI for URL-based document ingestion.

WikiLive

Wiki editor and browser for creating and editing wiki spaces and pages.

McpLive

MCP server management: CRUD, tool listing, and per-user tool selection.

AuthLive

Login, registration, and invitation acceptance forms.

SetupLive

First-time admin setup wizard.

Auth Hooks

LiteskillWeb.Plugs.LiveAuth provides on_mount callbacks:

• :require_authenticated — Redirects to /login if not authenticated
• :redirect_if_authenticated — Redirects to / if already logged in
• :require_admin — Requires admin role
• :require_setup_needed — Only allows access during initial setup

Real-Time Updates

LiveView receives real-time updates via PubSub:

• Streaming chunks — LLM response chunks update the UI in real-time
• Tool call status — Tool call progress and results
• Run updates — Agent run status and log entries
• Conversation list — New messages update conversation metadata

API

Liteskill provides a JSON REST API at /api for programmatic access.

Authentication

API routes require session-based authentication. The :api pipeline:

1. Accepts JSON ("application/json")
2. Fetches session
3. Loads current user via LiteskillWeb.Plugs.Auth
4. Applies rate limiting (1000 requests per 60 seconds)

The :require_auth pipeline then verifies an authenticated user is present.

Endpoints

Conversations

Groups

Rate Limiting

The API uses an ETS-based rate limiter (LiteskillWeb.Plugs.RateLimiter) with:

• Limit: 1000 requests per window
• Window: 60 seconds
• Periodic sweeper cleans stale ETS entries

Authentication

Liteskill supports two authentication methods: OIDC (OpenID Connect) and password-based.

Session-Based Auth

All authentication is session-based. The session flow:

1. User authenticates via OIDC callback or password login
2. user_id is stored in the session
3. LiteskillWeb.Plugs.Auth loads the user from the session on each request
4. LiteskillWeb.Plugs.LiveAuth handles the same for LiveView mounts

OIDC Authentication

Configured via environment variables:

• OIDC_ISSUER — OpenID Connect issuer URL
• OIDC_CLIENT_ID — Client ID
• OIDC_CLIENT_SECRET — Client secret

Uses Ueberauth with the OIDCC strategy. On callback, Accounts.find_or_create_from_oidc/1 finds or creates the user by OIDC subject + issuer.

Password Authentication

Password auth uses Argon2 for hashing (configured with t_cost: 1, m_cost: 8 in test for speed).

Registration

• Available at /register (LiveView) or POST /auth/register (API)
• Can be restricted to invited users only

Login

• Available at /login (LiveView) or POST /auth/login (API)
• Returns session with user ID

Password Changes

• Users change passwords at /profile/password
• Requires current password verification
• Admins can set temporary passwords that force a change on next login

Single-User Mode

When SINGLE_USER_MODE=true:

• The login screen is skipped entirely
• SingleUser.auto_user/0 returns the admin user for every request
• An admin user is auto-provisioned on boot if not present

Invitations

Admins can create invitation tokens for specific email addresses:

1. Admin creates invitation at /admin/users
2. Invitation generates a unique token
3. User visits /invite/:token to register
4. Token is marked as used after acceptance

OpenRouter OAuth

Liteskill supports OpenRouter OAuth PKCE flow for connecting user-level OpenRouter accounts:

1. User initiates at GET /auth/openrouter
2. PKCE challenge is generated and state stored in Liteskill.OpenRouter.StateStore
3. User authorizes on OpenRouter
4. Callback at GET /auth/openrouter/callback exchanges the code for credentials

Helm Chart

Liteskill publishes a Helm chart for deploying to Kubernetes. The chart handles the application deployment, database migrations, ingress, and optional autoscaling.

Add the Helm Repository

helm repo add liteskill https://docs.liteskill.ai/helm
helm repo update

Prerequisites

Before installing, you need:

• A running Kubernetes cluster
• A PostgreSQL database accessible from the cluster
• The following secrets ready:
  • DATABASE_URL — Ecto connection string (ecto://USER:PASS@HOST/DATABASE)
  • SECRET_KEY_BASE — generate with mix phx.gen.secret or openssl rand -base64 64
  • ENCRYPTION_KEY — generate with openssl rand -base64 32
  • AWS_BEARER_TOKEN_BEDROCK — your AWS Bedrock bearer token (if using Bedrock)
  • AWS_REGION — e.g. us-east-1

Quick Install

helm install liteskill liteskill/liteskill \
  --namespace liteskill --create-namespace \
  --set config.phxHost=liteskill.example.com \
  --set secrets.databaseUrl="ecto://user:pass@db-host/liteskill" \
  --set secrets.secretKeyBase="$(openssl rand -base64 64)" \
  --set secrets.encryptionKey="$(openssl rand -base64 32)" \
  --set secrets.awsBearerToken="your-token" \
  --set secrets.awsRegion="us-east-1"

Using a Values File

For production deployments, use a values file instead of inline --set flags.

# values-production.yaml

config:
  phxHost: liteskill.example.com
  port: "4000"
  poolSize: "10"

ingress:
  enabled: true
  className: nginx
  hosts:
    - host: liteskill.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: liteskill-tls
      hosts:
        - liteskill.example.com

secrets:
  existingSecret: liteskill-secrets

resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    memory: 1Gi

helm install liteskill liteskill/liteskill \
  --namespace liteskill --create-namespace \
  -f values-production.yaml

Using an Existing Secret

Rather than passing secrets through Helm values (which end up in Helm release metadata), you can reference a pre-created Kubernetes Secret:

kubectl create secret generic liteskill-secrets \
  --namespace liteskill \
  --from-literal=DATABASE_URL="ecto://user:pass@db-host/liteskill" \
  --from-literal=SECRET_KEY_BASE="$(openssl rand -base64 64)" \
  --from-literal=ENCRYPTION_KEY="$(openssl rand -base64 32)" \
  --from-literal=AWS_BEARER_TOKEN_BEDROCK="your-token" \
  --from-literal=AWS_REGION="us-east-1"

Then reference it in your values:

secrets:
  existingSecret: liteskill-secrets

The chart maps secret keys using existingSecretKeys, which defaults to:

existingSecretKeys:
  databaseUrl: DATABASE_URL
  secretKeyBase: SECRET_KEY_BASE
  encryptionKey: ENCRYPTION_KEY
  awsBearerToken: AWS_BEARER_TOKEN_BEDROCK
  awsRegion: AWS_REGION
  oidcClientSecret: OIDC_CLIENT_SECRET

Database Migrations

The chart includes a migration Job that runs as a Helm pre-install and pre-upgrade hook. It executes bin/liteskill eval Liteskill.Release.migrate() before the application starts.

Migrations are enabled by default. To configure:

migration:
  enabled: true
  backoffLimit: 3              # retry attempts on failure
  activeDeadlineSeconds: 600   # timeout in seconds (default: 10 minutes)

Ingress

Enable ingress to expose the application externally:

ingress:
  enabled: true
  className: nginx    # or your ingress controller class
  annotations: {}
  hosts:
    - host: liteskill.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: liteskill-tls
      hosts:
        - liteskill.example.com

Autoscaling

Horizontal Pod Autoscaling is available but disabled by default:

replicaCount: 2

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 5
  targetCPUUtilizationPercentage: 80

When using multiple replicas, set dnsClusterQuery to enable Erlang node clustering:

config:
  dnsClusterQuery: "liteskill-headless.liteskill.svc.cluster.local"

Pod Disruption Budget

Enable a PodDisruptionBudget to ensure availability during voluntary disruptions (node drains, cluster upgrades):

podDisruptionBudget:
  enabled: true
  minAvailable: 1

OIDC Authentication

To enable OpenID Connect authentication:

oidc:
  enabled: true
  issuer: "https://your-idp.example.com"
  clientId: "liteskill"
  clientSecret: "your-client-secret"

Or with an existing secret that includes the OIDC_CLIENT_SECRET key:

oidc:
  enabled: true
  issuer: "https://your-idp.example.com"
  clientId: "liteskill"

secrets:
  existingSecret: liteskill-secrets

SAML Authentication

To enable SAML authentication, set saml.enabled: true and provide the IdP metadata file via a volume mount:

saml:
  enabled: true
  idpMetadataFile: "/saml/metadata.xml"
  spId: "liteskill"
  spEntityId: "urn:liteskill:sp"
  spCertfile: "/saml/sp.crt"
  spKeyfile: "/saml/sp.key"
  idpId: "saml"
  baseUrl: "https://liteskill.example.com/sso"

extraVolumes:
  - name: saml-config
    secret:
      secretName: liteskill-saml

extraVolumeMounts:
  - name: saml-config
    mountPath: /saml
    readOnly: true

Create the Kubernetes secret containing the SAML files:

kubectl create secret generic liteskill-saml \
  --namespace liteskill \
  --from-file=metadata.xml=./idp-metadata.xml \
  --from-file=sp.crt=./sp-certificate.pem \
  --from-file=sp.key=./sp-private-key.pem

Session Configuration

Configure session timeouts:

config:
  sessionMaxAgeSeconds: "86400"         # max session age (default: 14 days)
  sessionIdleTimeoutSeconds: "3600"     # idle timeout (default: none)

Single User Mode

Enable single user mode to skip authentication entirely (useful for personal/local deployments):

config:
  singleUserMode: true

Security Context

The chart sets pod and container security contexts by default for hardened deployments:

# Pod-level (default)
securityContext:
  runAsNonRoot: true
  runAsUser: 65534
  fsGroup: 65534

# Container-level (default)
containerSecurityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop: ["ALL"]

To disable or customize, override the values:

securityContext: {}
containerSecurityContext: {}

Startup Probe

The chart includes a startup probe to handle slow-starting containers. This allows the application up to 60 seconds (30 failures x 2s period) to start before the liveness probe kicks in:

startupProbe:
  httpGet:
    path: /login
    port: http
  failureThreshold: 30
  periodSeconds: 2

Extra Volumes

Mount additional volumes into the application and migration containers (e.g. for SAML certificates, CA bundles, or database TLS certificates):

extraVolumes:
  - name: db-certs
    secret:
      secretName: db-tls-certs

extraVolumeMounts:
  - name: db-certs
    mountPath: /certs
    readOnly: true

Topology Spread Constraints

Spread pods across failure domains for high availability:

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: liteskill

Upgrading

helm repo update
helm upgrade liteskill liteskill/liteskill \
  --namespace liteskill \
  -f values-production.yaml

The migration hook runs automatically before each upgrade.

ArgoCD

An example ArgoCD Application manifest is included in the repository at deploy/argocd/application.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: liteskill
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://docs.liteskill.ai/helm
    chart: liteskill
    targetRevision: 0.2.51
    helm:
      releaseName: liteskill
      valuesObject:
        config:
          phxHost: liteskill.example.com
        ingress:
          enabled: true
          className: nginx
          hosts:
            - host: liteskill.example.com
              paths:
                - path: /
                  pathType: Prefix
          tls:
            - secretName: liteskill-tls
              hosts:
                - liteskill.example.com
        secrets:
          existingSecret: liteskill-secrets
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
          limits:
            memory: 1Gi
  destination:
    server: https://kubernetes.default.svc
    namespace: liteskill
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true

All Values

For a full reference of available values, see the chart’s values.yaml or run:

helm show values liteskill/liteskill

Environment Variables

Required (Production)

Optional (Auto-generated if not set)

Encryption (Off by Default)

Server

Authentication

LLM

Mode

Desktop Mode

When LITESKILL_DESKTOP=true:

• A bundled PostgreSQL instance is managed automatically
• Secret key base is auto-generated and stored in desktop_config.json
• Single-user mode is enabled automatically
• Data is stored in platform-specific directories:
  • macOS: ~/Library/Application Support/Liteskill
  • Linux: $XDG_DATA_HOME/liteskill (default: ~/.local/share/liteskill)
  • Windows: %APPDATA%/Liteskill

Docker Compose Defaults

The docker-compose.yml uses these defaults (overridable via shell environment):

SECRET_KEY_BASE is auto-generated on first boot if not provided. To persist across container restarts, mount a volume and set LITESKILL_SECRETS_FILE.

Encryption

Liteskill can optionally encrypt sensitive data at rest using AES-256-GCM via Liteskill.Crypto. Encryption is off by default and must be explicitly enabled.

Enabling Encryption

Set both environment variables:

export LITESKILL_ENCRYPTION=true
export ENCRYPTION_KEY="$(openssl rand -base64 32)"

When LITESKILL_ENCRYPTION is not set to true, the Crypto module is a passthrough — values are stored and returned as-is with no encryption overhead.

How It Works

When enabled:

1. A 32-byte encryption key is derived from the ENCRYPTION_KEY config value via SHA-256
2. Each encrypt operation generates a random 12-byte IV
3. Ciphertext format: IV (12 bytes) || tag (16 bytes) || ciphertext
4. The result is base64-encoded for storage in string columns

Validation

On application boot, Crypto.validate_key!/0 checks the configuration. If encryption is enabled but no key is set, the app will crash on startup rather than failing on first encrypt/decrypt. If encryption is not enabled, this is a no-op.

What’s Encrypted (When Enabled)

The following fields use encryption-aware Ecto types:

• MCP server API keys — via Liteskill.Crypto.EncryptedField Ecto type
• LLM provider API keys — via Liteskill.Crypto.EncryptedField
• Data source credentials — via Liteskill.Crypto.EncryptedMap Ecto type

When encryption is off, these types store values as plaintext.

Ecto Custom Types

• Liteskill.Crypto.EncryptedField — Encrypts/decrypts a single string value
• Liteskill.Crypto.EncryptedMap — Encrypts/decrypts a JSON map (for structured credentials)

These types handle encryption transparently at the Ecto layer — data is encrypted before insert and decrypted after load.

Development Setup

Prerequisites

1. Install mise
2. Clone the repository
3. Run mise install to get Elixir 1.18, Erlang/OTP 28, Node.js 24, and mdbook

Setup

mix setup

This runs:

1. deps.get — Install Elixir dependencies
2. ecto.create — Create the database
3. ecto.migrate — Run migrations
4. run priv/repo/seeds.exs — Seed data
5. npm install --prefix assets — Install Node dependencies
6. tailwind.install --if-missing — Install Tailwind
7. esbuild.install --if-missing — Install esbuild
8. gen.jr_prompt — Generate JSON render prompt
9. Compile and build assets

Database Reset

mix ecto.reset

Drops, creates, migrates, and seeds the database.

Running the Server

mix phx.server

Visit http://localhost:4000.

Docker-Based Development

If you don’t have PostgreSQL installed locally:

# Run tests with Docker Postgres
./scripts/test-with-docker.sh test

# Full precommit with Docker Postgres
./scripts/test-with-docker.sh precommit

The script starts a temporary pgvector/pgvector:pg16 container, sets DATABASE_URL, runs the specified command, and cleans up after.

Building the Docker Image

docker build -t liteskill .

The multi-stage Dockerfile:

1. Stage 0 — Copies Node.js binaries from node:24-bookworm-slim
2. Stage 1 (Builder) — Compiles the Elixir release with mix release liteskill
3. Stage 2 (Runtime) — Minimal Debian image with the release binary

Running with Docker Compose

export SECRET_KEY_BASE=$(openssl rand -base64 64)
export ENCRYPTION_KEY=$(openssl rand -base64 32)
docker compose up

This starts:

• db — PostgreSQL 16 with pgvector
• app — Liteskill release with host networking

To run migrations separately:

docker compose run --rm migrate

Testing

Running Tests

# All tests
mix test

# Single file
mix test test/path_test.exs

# Re-run failures
mix test --failed

Coverage

100% test coverage is required, enforced by ExCoveralls. Files excluded from coverage are listed in coveralls.json.

Use # coveralls-ignore-start / # coveralls-ignore-stop for genuinely unreachable branches (e.g. desktop-only code paths, production-only error handling).

Test Configuration

DataCase

use Liteskill.DataCase, async: false

All database tests use a shared Ecto sandbox (async: false).

Unit Tests

Pure unit tests (aggregates, events, parsers) use:

use ExUnit.Case, async: true

Argon2

Test config uses t_cost: 1, m_cost: 8 for fast password hashing.

HTTP Mocking with Req.Test

Use plug: {Req.Test, ModuleName} to mock HTTP calls:

Req.Test.stub(Liteskill.McpServers.Client, fn conn ->
  Req.Test.json(conn, %{"result" => %{"tools" => []}})
end)

Important: Req.Test does NOT trigger into: callbacks. Use the plug: option when constructing Req requests in production code to enable test substitution.

Projector

The Chat.Projector runs in the main supervision tree — never start_supervised! it in tests. It’s already running.

Process Synchronization

Chat context write functions include Process.sleep(50) for projection consistency. In tests, prefer synchronization over additional sleeps:

# Prefer this
_ = :sys.get_state(Liteskill.Chat.Projector)

# Or use Projector.sync/0
:ok = Liteskill.Chat.Projector.sync()

Stateful Stubs

Use Agent for varying responses across retries:

{:ok, counter} = Agent.start_link(fn -> 0 end)

Req.Test.stub(ModuleName, fn conn ->
  call = Agent.get_and_update(counter, &{&1, &1 + 1})
  case call do
    0 -> Plug.Conn.send_resp(conn, 429, "rate limited")
    _ -> Req.Test.json(conn, %{"result" => "ok"})
  end
end)

Set backoff_ms: 1 for retry tests to avoid slow test runs.

MCP Client Testing

Req.Test.stub(Liteskill.McpServers.Client, fn conn ->
  Req.Test.json(conn, %{
    "jsonrpc" => "2.0",
    "id" => 1,
    "result" => %{"tools" => []}
  })
end)

Docker-Based Testing

# Run tests with a temporary Docker Postgres
./scripts/test-with-docker.sh test

# Full precommit with Docker Postgres
./scripts/test-with-docker.sh precommit

Conventions

IDs and Timestamps

• Binary UUIDs for all primary keys
• :utc_datetime for all timestamp fields

Schemas

• All schemas use field :name, :string even for text columns
• Foreign keys (e.g. user_id) are set programmatically, never included in cast

HTTP

• Req library only for all HTTP calls — never httpoison, tesla, or httpc
• HTTP mocking via Req.Test with plug: option

Boundaries

Every context uses the Boundary library to declare:

• top_level?: true — Marks it as a top-level bounded context
• deps: [...] — Allowed dependencies on other contexts
• exports: [...] — Modules visible to other contexts

The boundary compiler enforces these at compile time.

Event Sourcing

• Events stored with string keys via stringify_keys
• Deserialized back to structs via Events.deserialize/1
• Stream IDs: "conversation-<uuid>"
• Optimistic concurrency via unique index on (stream_id, stream_version)

Authorization

• All entity types share a single entity_acls table
• Owner ACL auto-created on resource creation
• Role hierarchy: viewer < editor < manager < owner
• RBAC permissions checked via Liteskill.Rbac.authorize/2

Tailwind CSS

• Tailwind v4 — No tailwind.config.js
• Uses @import "tailwindcss" syntax in assets/css/app.css

Oban Queues

LLM Integration

• All LLM transport via ReqLLM (req_llm ~> 1.5)
• ReqLLM.stream_text/3 for streaming, ReqLLM.generate_text/3 for single-turn
• ReqLLM.embed/3 for embeddings
• No hardcoded model IDs — all models configured in the database

Security

• Sensitive fields encrypted at rest via Liteskill.Crypto (AES-256-GCM)
• SSRF protection on MCP server URLs (configurable)
• ETS-based rate limiting on API routes
• CSRF protection via Phoenix plugs
• Per-provider circuit breakers prevent cascading LLM failures

Commands

Development

Testing

Code Quality

Precommit

mix precommit runs the full quality pipeline:

1. compile --warnings-as-errors
2. deps.unlock --unused
3. format
4. credo --strict
5. sobelow --config --exit low
6. dialyzer
7. ecto.create --quiet && ecto.migrate --quiet
8. coveralls (tests with 100% coverage check)
9. mdbook build docs/

Always run mix precommit after completing changes.

Docker-Based

Desktop