openviking-context-database

OpenViking Context Database

Skill by ara.so — Daily 2026 Skills collection.

OpenViking is an open-source context database for AI Agents that replaces fragmented vector stores with a unified filesystem paradigm. It manages agent memory, resources, and skills in a tiered L0/L1/L2 structure, enabling hierarchical context delivery, observable retrieval trajectories, and self-evolving session memory.

---

Installation

Python Package

pip install openviking --upgrade --force-reinstall

Optional Rust CLI

# Install via script
curl -fsSL https://raw.githubusercontent.com/volcengine/OpenViking/main/crates/ov_cli/install.sh | bash

# Or build from source (requires Rust toolchain)
cargo install --git https://github.com/volcengine/OpenViking ov_cli

Prerequisites

• Python 3.10+
• Go 1.22+ (for AGFS components)
• GCC 9+ or Clang 11+ (for core extensions)

---

Configuration

Create ~/.openviking/ov.conf:

{
  "storage": {
    "workspace": "/home/user/openviking_workspace"
  },
  "log": {
    "level": "INFO",
    "output": "stdout"
  },
  "embedding": {
    "dense": {
      "api_base": "https://api.openai.com/v1",
      "api_key": "$OPENAI_API_KEY",
      "provider": "openai",
      "dimension": 1536,
      "model": "text-embedding-3-large"
    },
    "max_concurrent": 10
  },
  "vlm": {
    "api_base": "https://api.openai.com/v1",
    "api_key": "$OPENAI_API_KEY",
    "provider": "openai",
    "model": "gpt-4o",
    "max_concurrent": 100
  }
}

Note: OpenViking reads api_key values as strings; use environment variable injection at startup rather than literal secrets.

Provider Options

Role	Provider Value	Example Model
VLM	openai	gpt-4o
VLM	volcengine	doubao-seed-2-0-pro-260215
VLM	litellm	claude-3-5-sonnet-20240620, ollama/llama3.1
Embedding	openai	text-embedding-3-large
Embedding	volcengine	doubao-embedding-vision-250615
Embedding	jina	jina-embeddings-v3

LiteLLM VLM Examples

{
  "vlm": {
    "provider": "litellm",
    "model": "claude-3-5-sonnet-20240620",
    "api_key": "$ANTHROPIC_API_KEY"
  }
}

{
  "vlm": {
    "provider": "litellm",
    "model": "ollama/llama3.1",
    "api_base": "http://localhost:11434"
  }
}

{
  "vlm": {
    "provider": "litellm",
    "model": "deepseek-chat",
    "api_key": "$DEEPSEEK_API_KEY"
  }
}

---

Core Concepts

Filesystem Paradigm

OpenViking organizes agent context like a filesystem:

workspace/
├── memories/          # Long-term agent memories (L0 always loaded)
│   ├── user_prefs/
│   └── task_history/
├── resources/         # External knowledge, documents (L1 on demand)
│   ├── codebase/
│   └── docs/
└── skills/            # Reusable agent capabilities (L2 retrieved)
    ├── coding/
    └── analysis/

Tiered Context Loading (L0/L1/L2)

• L0: Always loaded — core identity, persistent preferences
• L1: Loaded on demand — relevant resources fetched per task
• L2: Semantically retrieved — skills pulled by similarity search

This tiered approach minimizes token consumption while maximizing context relevance.

---

Python API Usage

Basic Setup

import os
from openviking import OpenViking

# Initialize with config file
ov = OpenViking(config_path="~/.openviking/ov.conf")

# Or initialize programmatically
ov = OpenViking(
    workspace="/home/user/openviking_workspace",
    vlm_provider="openai",
    vlm_model="gpt-4o",
    vlm_api_key=os.environ["OPENAI_API_KEY"],
    embedding_provider="openai",
    embedding_model="text-embedding-3-large",
    embedding_api_key=os.environ["OPENAI_API_KEY"],
    embedding_dimension=1536,
)

Managing a Context Namespace (Agent Brain)

# Create or open a namespace (like a filesystem root for one agent)
brain = ov.namespace("my_agent")

# Add a memory file
brain.write("memories/user_prefs.md", """
# User Preferences
- Language: Python
- Code style: PEP8
- Preferred framework: FastAPI
""")

# Add a resource document
brain.write("resources/api_docs/stripe.md", open("stripe_docs.md").read())

# Add a skill
brain.write("skills/coding/write_tests.md", """
# Skill: Write Unit Tests
When asked to write tests, use pytest with fixtures.
Always mock external API calls. Aim for 80%+ coverage.
""")

Querying Context

# Semantic search across the namespace
results = brain.search("how does the user prefer code to be formatted?")
for result in results:
    print(result.path, result.score, result.content[:200])

# Directory-scoped retrieval (recursive)
skill_results = brain.search(
    query="write unit tests for a FastAPI endpoint",
    directory="skills/",
    top_k=3,
)

# Direct path read (L0 always available)
prefs = brain.read("memories/user_prefs.md")
print(prefs.content)

Session Memory & Auto-Compression

# Start a session — OpenViking tracks turns and auto-compresses
session = brain.session("task_build_api")

# Add conversation turns
session.add_turn(role="user", content="Build me a REST API for todo items")
session.add_turn(role="assistant", content="I'll create a FastAPI app with CRUD operations...")

# After many turns, trigger compression to extract long-term memory
summary = session.compress()
# Compressed insights are automatically written to memories/

# End session — persists extracted memories
session.close()

Retrieval Trajectory (Observable RAG)

# Enable trajectory tracking to observe retrieval decisions
with brain.observe() as tracker:
    results = brain.search("authentication best practices")
    
trajectory = tracker.trajectory()
for step in trajectory.steps:
    print(f"[{step.level}] {step.path} → score={step.score:.3f}")
    # Output:
    # [L0] memories/user_prefs.md → score=0.82
    # [L1] resources/security/auth.md → score=0.91
    # [L2] skills/coding/jwt_auth.md → score=0.88

---

Common Patterns

Pattern 1: Agent with Persistent Memory

import os
from openviking import OpenViking

ov = OpenViking(config_path="~/.openviking/ov.conf")
brain = ov.namespace("coding_agent")

def agent_respond(user_message: str, conversation_history: list) -> str:
    # Retrieve relevant context
    context_results = brain.search(user_message, top_k=5)
    context_text = "\n\n".join(r.content for r in context_results)
    
    # Build prompt with retrieved context
    system_prompt = f"""You are a coding assistant.

## Relevant Context
{context_text}
"""
    # ... call your LLM here with system_prompt + conversation_history
    response = call_llm(system_prompt, conversation_history, user_message)
    
    # Store interaction for future memory
    brain.session("current").add_turn("user", user_message)
    brain.session("current").add_turn("assistant", response)
    
    return response

Pattern 2: Hierarchical Skill Loading

# Register skills from a directory structure
import pathlib

skills_dir = pathlib.Path("./agent_skills")
for skill_file in skills_dir.rglob("*.md"):
    relative = skill_file.relative_to(skills_dir)
    brain.write(f"skills/{relative}", skill_file.read_text())

# At runtime, retrieve only relevant skills
def get_relevant_skills(task: str) -> list[str]:
    results = brain.search(task, directory="skills/", top_k=3)
    return [r.content for r in results]

task = "Refactor this class to use dependency injection"
skills = get_relevant_skills(task)
# Returns only DI-related skills, not all registered skills

Pattern 3: RAG over Codebase

import subprocess
import pathlib

brain = ov.namespace("codebase_agent")

# Index a codebase
def index_codebase(repo_path: str):
    for f in pathlib.Path(repo_path).rglob("*.py"):
        content = f.read_text(errors="ignore")
        # Store with relative path as key
        rel = f.relative_to(repo_path)
        brain.write(f"resources/codebase/{rel}", content)

index_codebase("/home/user/myproject")

# Query with directory scoping
def find_relevant_code(query: str) -> list:
    return brain.search(
        query=query,
        directory="resources/codebase/",
        top_k=5,
    )

hits = find_relevant_code("database connection pooling")
for h in hits:
    print(h.path, "\n", h.content[:300])

Pattern 4: Multi-Agent Shared Context

# Agent 1 writes discoveries
agent1_brain = ov.namespace("researcher_agent")
agent1_brain.write("memories/findings/api_rate_limits.md", """
# API Rate Limits Discovered
- Stripe: 100 req/s in live mode
- SendGrid: 600 req/min
""")

# Agent 2 reads shared workspace findings
agent2_brain = ov.namespace("coder_agent")
# Cross-namespace read (if permitted)
shared = ov.namespace("shared_knowledge")
rate_limits = shared.read("memories/findings/api_rate_limits.md")

---

CLI Commands (ov_cli)

# Check version
ov --version

# List namespaces
ov namespace list

# Create a namespace
ov namespace create my_agent

# Write context file
ov write my_agent/memories/prefs.md --file ./prefs.md

# Read a file
ov read my_agent/memories/prefs.md

# Search context
ov search my_agent "how to handle authentication" --top-k 5

# Show retrieval trajectory for a query
ov search my_agent "database migrations" --trace

# Compress a session
ov session compress my_agent/task_build_api

# List files in namespace
ov ls my_agent/skills/

# Delete a context file
ov rm my_agent/resources/outdated_docs.md

# Export namespace to local directory
ov export my_agent ./exported_brain/

# Import from local directory
ov import ./exported_brain/ my_agent_restored

---

Troubleshooting

Config Not Found

# Verify config location
ls -la ~/.openviking/ov.conf

# OpenViking also checks OV_CONFIG env var
export OV_CONFIG=/path/to/custom/ov.conf

Embedding Dimension Mismatch

If you switch embedding models, the stored vector dimensions will conflict:

# Check current dimension setting vs stored index
# Solution: re-index after model change
brain.reindex(force=True)

Workspace Permission Errors

# Ensure workspace directory is writable
chmod -R 755 /home/user/openviking_workspace

# Check disk space (embedding indexes can be large)
df -h /home/user/openviking_workspace

LiteLLM Provider Not Detected

# Use explicit prefix for ambiguous models
{
  "vlm": {
    "provider": "litellm",
    "model": "openrouter/anthropic/claude-3-5-sonnet",  # full prefix required
    "api_key": "$OPENROUTER_API_KEY",
    "api_base": "https://openrouter.ai/api/v1"
  }
}

High Token Usage

Enable tiered loading to reduce L1/L2 fetches:

# Scope searches tightly to avoid over-fetching
results = brain.search(
    query=user_message,
    directory="skills/relevant_domain/",  # narrow scope
    top_k=2,                               # fewer results
    min_score=0.75,                        # quality threshold
)

Slow Indexing on Large Codebases

# Increase concurrency in config
{
  "embedding": {
    "max_concurrent": 20  # increase from default 10
  },
  "vlm": {
    "max_concurrent": 50
  }
}

# Or batch-write with async
import asyncio

async def index_async(files):
    tasks = [brain.awrite(f"resources/{p}", c) for p, c in files]
    await asyncio.gather(*tasks)

---

Environment Variables Reference

Variable	Purpose
OV_CONFIG	Path to ov.conf override
OPENAI_API_KEY	OpenAI API key for VLM/embedding
ANTHROPIC_API_KEY	Anthropic Claude via LiteLLM
DEEPSEEK_API_KEY	DeepSeek via LiteLLM
GEMINI_API_KEY	Google Gemini via LiteLLM
OV_LOG_LEVEL	Override log level (DEBUG, INFO, WARN)
OV_WORKSPACE	Override workspace path

---

Resources

• Website: https://openviking.ai
• Docs: https://www.openviking.ai/docs
• GitHub: https://github.com/volcengine/OpenViking
• Issues: https://github.com/volcengine/OpenViking/issues
• Discord: https://discord.com/invite/eHvx8E9XF3
• LiteLLM Providers: https://docs.litellm.ai/docs/providers