Documentation
RaggieCode Architecture
How RaggieCode parses codebases with tree-sitter, stores every symbol in SQLite, traverses dependency graphs, retrieves contextual symbols, orchestrates multi-step tasks, and preserves state across context window limits.
System Overview
RaggieCode is built around a semantic code index .; a persistent SQLite database that stores every function, class, method, import, and variable extracted from your codebase using tree-sitter AST parsing. The index is the single source of truth that powers every code exploration tool, every dependency graph traversal, and every context-aware code change the agent makes.
At a high level, data flows through the system in five stages: indexing (parsing with tree-sitter), storage (SQLite persistence), retrieval (RAG-style symbol lookup), reasoning (LLM-driven agent planning with tool calls), and execution (file write, shell, git commit). The following diagram shows the full pipeline.
parses args (role + project-dir), loads config, starts agent loop
Tables: symbols, files, dependencies, references, imports
call_graph, symbol_overview, code_chunks
- symbols . name, kind (function/class/variable), file, line, docstring
- dependencies . caller → callee, importer → imported
- call_graph . BFS-ready parent/child edges with depth tracking
agent.py → prompt loop:
- User message → LLM (with chat history + tool definitions)
- LLM responds with text and/or tool calls
- ToolRegistry (tools.py) dispatches each call to a handler
- Tool results → back to LLM for next iteration
- Repeat until LLM sends a text-only response
- ◆ Re-index after every tool call
- ◆ Commit all changes to built-in git when done
- ◆ Handover if context window is nearly full
Todo list system (todo_list.py):
Create plan → Approve → ExecuteNextTask → Subagent
Each stage is designed to be incrementally modifiable: the index is rebuilt only for changed files, the dependency graph is lazily traversed, and the agent loop re-indexes after every single tool call to ensure it always works with fresh context.
Project Directory Structure
RaggieCode's source is organised into six main directories under src/,
each with a single responsibility. The entry point is raggie.py.
raggie.py Entry point. parses args, runs agent loop. src/ ├── cli.py Argument parser (argparse). ├── chat.py Watermelon-themed status messages. ├── config/ Default configuration files (roles.json, tools.json, coder_system_prompt.md). ├── Agent/ │ ├── agent.py Core Agent class. prompt loop, tool execution, commit. │ ├── config.py Config loader. reads from ~/.config/raggie/. │ ├── tools.py ToolRegistry. maps tool names to handler functions. │ ├── chat_history_db.py SQLite DB. sessions, messages, skills, todo lists. │ └── git_manager.py Local git repo in .raggie/git/ for versioning. ├── Tools/ 31 tool handlers (read, write, shell, code analysis, etc.). ├── indexing/ │ ├── code_indexer.py Tree-sitter based code indexer (orchestrator). │ ├── code_index_sdk.py SDK for querying the code index. │ ├── parse_worker.py Multiprocessing parse worker. │ ├── extracts.py Symbol extraction per language. │ ├── models.py Data models (Symbol, Function, Class, etc.). │ ├── db_schema.py SQLite schema for the code index. │ ├── language_config.py Language parser configurations. │ ├── node_utils.py Tree-sitter node helpers. │ ├── queries.py Tree-sitter query patterns. │ ├── file_utils.py File walking utilities. │ └── export_to_json.py Export index to JSON. ├── RAG/ │ ├── find.py Find symbols in the index (exact + fuzzy). │ └── graph.py Dependency graph traversal (BFS, cycle detection). └── skills/ ├── __init__.py Exports SkillManager. ├── manager.py SkillManager. CRUD for named skills. └── tool.py SetSkill + GetSkill tool handlers.
Each directory contains a single __init__.py that exports the
public API. The Tools/ directory registers all 31 tool handlers with the
ToolRegistry, which the agent.py loop uses to dispatch LLM-generated
tool calls at runtime.
Config directory
~/.config/raggie/ stores user-specific config: keys.json (API keys), roles.json (role definitions), and tools.json (tool definitions). These are copied from src/config/ on first run and customisable at any time.
Indexing Pipeline
The indexing pipeline is the foundation of everything RaggieCode does. It converts raw source files into a queryable semantic index. The pipeline runs at startup, after every tool call (to pick up new files or changes), and uses multiprocessing for performance.
Step-by-step flow
┌───────────────────────────────────────────────────────────────────────────┐ │ Indexing Pipeline │ └───────────────────────────────────────────────────────────────────────────┘ 1. File Discovery file_utils.py walks the project directory recursively. Ignores files matched by .gitignore (or .aiignore if present). Skips binary files, node_modules, .venv, build/, dist/, etc. Returns a list of file paths with their detected language. 2. Multiprocessing Dispatch code_indexer.py splits file list across N worker processes (one per CPU core). Each worker (parse_worker.py) receives a subset of files. 3. AST Parsing (per worker, per file) language_config.py loads the correct tree-sitter grammar for each file. Supports 15 languages: Python, JS, TS, TSX, Go, Rust, C, C++, C#, PHP, Elixir, Zig, Dart, Java, Kotlin. queries.py contains language-specific tree-sitter query patterns. tree-sitter parses the file into a Concrete Syntax Tree (CST). 4. Symbol Extraction extracts.py + node_utils.py walk the CST to extract: • Function/method definitions (name, parameters, return type, docstring) • Class definitions (name, base classes, member variables, methods) • Top-level variable assignments • Import statements (what is imported, from which module) • Function calls (caller name, callee name, arguments) • Each symbol gets: file path, start/end line, source snippet 5. Dependency Resolution For every function call found in step 4, resolve which symbol it refers to: • Same-file references → direct link • Imported symbols → resolve through import chain • External/third-party → marked as unresolved (ext:name) The result is a dependency table: caller ──► callee edges. 6. SQLite Write db_schema.py writes all extracted data to .raggie/.code_index.raggie. Uses INSERT OR REPLACE for incremental updates. Tables: symbols, files, dependencies, references, imports, call_graph.
Why tree-sitter?
tree-sitter is the same parser engine used by Neovim, GitHub code search, and countless language servers. It produces a full Concrete Syntax Tree with error recovery .; meaning it can parse incomplete or syntactically incorrect files without crashing. Its language grammars are fast, incremental, and run in-process (no subprocess per file).
The index is incrementally updated. code_indexer.py tracks file
modification timestamps and only re-parses files that have changed since the last
index. This means subsequent indexing cycles (which happen after every tool call)
are near-instant for all but the largest projects.
SQLite Storage Schema
The code index is stored in a single SQLite database file at .raggie/.code_index.raggie.
The schema is designed for fast lookups by symbol name, file path, and dependency
edges. No separate database server is needed .; everything is local and embedded.
Core Tables
| Table | Purpose | Key Columns |
|---|---|---|
| symbols | Every function, class, method, and variable | name, kind, file_id, line_start, line_end, docstring |
| files | Indexed files with metadata | path, language, last_modified, checksum |
| dependencies | Caller-callee edges (who calls who) | caller_id, callee_id, call_type, line_number |
| references | Variable/method references across files | symbol_id, ref_file_id, ref_line, ref_column |
| imports | Import statements and their resolutions | file_id, imported_name, source_module, resolved_file_id |
| call_graph | BFS-ready graph edges with depth tracking | parent_id, child_id, depth, is_cycle |
| code_chunks | Full source text for each symbol | symbol_id, source_text, token_count |
| symbol_overview | Materialised view for fast listing | name, kind, file_path, line, summary |
The code_index_sdk.py module provides the query layer. It exposes
methods like find_symbol(name), get_callers(symbol_id),
get_callees(symbol_id), and fuzzy_search(partial_name). These are the
primitives used by the tool handlers in Tools/ and the RAG layer in
src/RAG/.
Two databases
RaggieCode maintains two separate SQLite databases in the project's .raggie/
directory. The code index (.code_index.raggie) stores parsed symbols
and dependency graphs. The chat database (.raggie.chat) stores sessions,
messages, skills, and todo lists. Both are gitignored by default.
Dependency Graph
The dependency graph is what gives RaggieCode its blast radius analysis capability. Unlike a simple grep-based tool that finds strings, the dependency graph knows exactly which functions call which, which classes inherit from which, and which files import from which .; across your entire codebase.
Graph Construction
The graph is built during indexing (step 5 of the pipeline). Every function call,
class instantiation, import statement, and variable reference is extracted as an edge
in the graph. Edges are directional: caller ──► callee.
WalkCallTree: BFS Traversal
The WalkCallTree tool (implemented in src/Tools/walk_call_tree.py) performs
a breadth-first traversal of the graph starting from any symbol. It is the primary
way the agent understands execution flow.
WalkCallTree(symbol_name="handle_request", max_depth=3) Depth 0 [handle_request] ─── src/api/handler.py:42 │ ├──► validate_auth ─── src/auth/validate.py:18 depth 1 ├──► parse_payload ─── src/api/parser.py:55 depth 1 └──► execute_query ─── src/db/queries.py:102 depth 1 │ ├──► connect_db ─── src/db/connection.py:7 depth 2 └──► sanitize_input ─── src/db/sanitizer.py:31 depth 2 │ └──► escape_string ─── src/utils/strings.py:12 depth 3 ───────────────────────────────────────────────────── 4 symbols found, max depth 3, 0 cycles detected.
Key Properties
Cycle Detection
The graph is a directed graph that may contain cycles (A calls B, B calls A). WalkCallTree detects cycles by tracking the ancestor chain. When a cycle is found, traversal stops at that branch and the node is marked with cycle: true.
Max Depth
Traversal defaults to depth 5 (configurable). This prevents runaway exploration in large codebases while still giving the agent enough context for meaningful blast radius analysis.
External Nodes
Third-party calls (e.g., json.dumps, requests.get) are tracked as unresolved ext:name nodes. They can be included or excluded from results via the include_external flag.
Two-Way Traversal
The graph supports both forward traversal (what does this function call?) and reverse traversal (what calls this function?). This enables both "trace execution flow" and "find all callers for refactoring."
The graph.py module in src/RAG/ contains the core graph traversal
logic. The WalkCallTree tool is a thin wrapper that formats the traversal
output as structured JSON for the LLM.
Retrieval Flow
When the LLM decides it needs to understand part of the codebase, it calls one of
the code exploration tools. Each tool queries the SQLite index through the
code_index_sdk.py layer. The following diagram shows how a tool call flows
through the system:
LLM decides: "I need to see the source of `validate_user`"
Tool call: GetSymbolSourceCode(symbol_name="validate_user")
SELECT * FROM symbols WHERE name = 'validate_user'
- • If exact match: return symbol_id, file_id, line, docstring
- • If no exact match: fall back to fuzzy search
- (subsequence matching, ranks by similarity)
SELECT source_text FROM code_chunks WHERE symbol_id = ?
- • Returns the full source body as a string
- • Includes docstring, type annotations, and decorators
SELECT * FROM dependencies WHERE caller_id = ?
SELECT * FROM imports WHERE file_id = ?
- • Attaches dependency info so the LLM sees the full picture
Return structured JSON to LLM:
{
"name": "validate_user",
"file": "src/auth/validate.py:18",
"source": "def validate_user(user: User) -> bool:...",
"dependencies": ["hash_password", "check_rate_limit"]
}
The three code exploration tools (GetSymbolSourceCode,
GetFileCodeSemantics, and WalkCallTree) are the preferred way
for the LLM to explore code. The UglyWholeFileContentDump tool exists
as a fallback but is throttled and not recommended for routine use .; the
semantic tools provide richer context with fewer tokens.
Re-indexing after every change
After every tool call that modifies files (WriteFile, ReplaceText, RemoveFile), the agent re-indexes the affected files. This ensures the next tool call always has fresh symbol data. The incremental indexer only re-parses changed files, so it is typically sub-second.
Agent Orchestration Loop
The agent.py module contains the core orchestration loop. It manages the
conversation with the LLM, dispatches tool calls, handles context window limits,
and coordinates subagents.
The Main Loop
- • Load role config from ~/.config/raggie/
- • Connect to LLM API (any OpenAI-compatible)
- • Index codebase (tree-sitter, multiproc.)
- • Initialise local git repo (.raggie/git/)
- • Load skill summaries from database
- • Check for incomplete todo lists
User sends message to agent
- • System prompt (role + skills + AGENTS.md)
- • Chat history (trimmed to window size)
- • Tool definitions (31 tools)
- • User's latest message
Text response + optional tool call requests
No Tool Calls (text only)
- • Dispatch to ToolRegistry
- • Execute handler
- • Re-index affected files
- • Return results to LLM
(loop back to step 3)
- • Git commit all changes to .raggie/git/
- • Show response to user
- • Print /undo hint
(back to step 2 for next user message)
Context Window Management
Before each LLM request, the agent checks how close it is to the configured context
window size (/windowSize command). When the total token count exceeds 80% of
the limit, the agent triggers an automatic handover. It generates a
comprehensive handover document (see Agent Handover), saves it to the
database, and spawns a fresh agent session that picks up exactly where it left off.
Todo List System
For complex multi-step tasks, the agent can create a todo list. The todo
list system (implemented in src/Tools/todo_list.py) provides a structured
workflow:
Todo List Workflow GetActiveTodoList ──► check for existing incomplete list CreateTodoList ──► create a new list AddTask (x N) ──► add tasks with goal, requirements, notes GetTodoList ──► review the plan ApproveTodoList ──► present to user for y/n approval ExecuteNextTask ──► dispatch subagent for each task, sequentially Subagents run in isolated sessions (max 3 deep) Auto-deletion when all tasks are done Crash recovery: interrupted lists are detected and offered for resumption on next startup
Each task is executed by a dedicated subagent (via DispatchSubagent).
Subagents are isolated: they receive only the task context (goal, requirements, notes,
summary of prior completed tasks) and cannot see the full parent conversation. This
keeps each subtask focused and prevents context bleed.
Testing Workflow
RaggieCode does not have a separate "testing phase" baked into the agent loop. Instead,
testing is integrated into the agent's skill system. The default
code role includes a testing skill that instructs the agent to always
write and run tests after making code changes.
┌───────────────────────────────────────────────────────────────────────┐ │ Testing Flow (as executed by the agent via skills + tools) │ └───────────────────────────────────────────────────────────────────────┘ 1. Agent makes code changes (WriteFile, ReplaceText, etc.) │ ▼ 2. Testing skill kicks in (loaded from skill database): "Run the existing test suite to verify nothing is broken." │ ▼ 3. Shell tool call: Shell(command="cd /project && python -m pytest tests/ -x --tb=short") │ ├── All tests pass ──► proceed to commit └── Tests fail ──► agent diagnoses failure, fixes code, re-runs │ ▼ 4. (Optional) Write new tests for the changed functionality WriteFile(file_path="tests/test_new_feature.py", ...) Shell(command="python -m pytest tests/test_new_feature.py -v") │ ▼ 5. All passing ──► commit with test results in message
The testing skill is a named instruction set stored in the database. It is advertised
at startup as a one-line summary (code/testing: Always write tests after implementing. Use pytest.).
The LLM calls GetSkill("code", "testing") to fetch the full instructions
when it decides it needs them. This on-demand loading saves tokens compared to stuffing
the full testing instructions into every system prompt.
Custom testing instructions
You can override the default testing skill or add project-specific instructions
via AGENTS.md. For example: "Use unittest instead of pytest" or
"Run linters before tests". The agent dynamically fetches these instructions
as part of its system prompt assembly.
Revert & History Mechanism
Every change the agent makes is versioned in a local git repository at
.raggie/git/. This is not your project's git repo .; it is a separate,
hidden repository that only RaggieCode writes to. It exists so the agent can undo
and redo changes without touching your project's commit history.
Commit Cycle
┌───────────────────────────────────────────────────────────────────────┐ │ Git Integration (git_manager.py) │ └───────────────────────────────────────────────────────────────────────┘ 1. Agent finishes responding (no more tool calls) │ ▼ 2. git_manager.commit() • Stages all project files (respects .gitignore/.aiignore) • Creates a commit with structured message: User prompt: "Add input validation to login" Tool calls: 4 (GetSymbolSourceCode, WriteFile, Shell) Summary: Created src/auth/validator.py, updated handlers • Stores proper nested trees (subdirs = git tree objects) │ ▼ 3. /undo • Reads the HEAD commit's parent tree • Restores all files to their state before the commit • Writes a marker file (.raggie/.undoing) for crash safety • Moves HEAD back by one commit • Pushes the undone commit onto the redo stack • Removes the marker file │ ▼ 4. /redo • Pops the top commit from the redo stack • Restores files to that commit's state • Moves HEAD forward │ ▼ 5. ViewChanges tool (can be called by the agent or user) • ViewChanges("status") ──► files added/modified/deleted • ViewChanges("diff") ──► line-by-line diffs with path filter • ViewChanges("log") ──► commit history with tool call counts
Crash Safety
The undo and redo operations write marker files (.raggie/.undoing and
.raggie/.redoing) before deleting or modifying files, and remove them
after successful restoration. If the process crashes mid-operation, the marker is
detected on next startup and a warning is displayed. The redo stack
(.raggie/.redo_stack) tracks undone commits; it is cleared when a new
commit is made (because the redo history is no longer valid after new changes).
Why a separate git repo?
Using your project's existing git repo would pollute your commit history with
hundreds of agent-generated commits. The separate .raggie/git/
repository stays out of your way while still giving the agent full version
control. It is also automatically gitignored via .raggie/ patterns.
Handover Lifecycle
The automatic context handover is what allows RaggieCode to work on tasks that exceed a single LLM context window. When the agent's token usage approaches the configured limit, it generates a handover document and continues in a fresh session .; with zero loss of progress.
When Handover Triggers
Handover is triggered before each LLM request when the total token count (system
prompt + chat history) exceeds 80% of the configured context window size.
The window size defaults to 128K tokens (matching most modern LLMs) and can be
changed at runtime with /windowSize <number>.
Handover Flow
┌───────────────────────────────────────────────────────────────────────┐ │ Handover Lifecycle │ └───────────────────────────────────────────────────────────────────────┘ 1. Token check before LLM call • System prompt: ~5K tokens (role + skills + instructions) • Chat history: variable (grows with each exchange) • If total > 80% of window ──► trigger handover │ ▼ 2. Agent generates handover document (via LLM call) The handover doc includes: • Original goal (from the first user message) • Current objective (what the agent was doing right now) • Code state (files changed, functions modified) • Important files & symbols discovered • Decisions made (architecture choices, rejected alternatives) • Changes applied (file paths, line ranges, summaries) • Test results (what passed, what failed, error messages) • Errors encountered (blockers, workarounds) • User constraints (requirements, preferences) • Next step (exact instruction for the new session) • Things not to repeat (avoid redundant exploration) │ ▼ 3. Save handover to database (chat_history_db.py) • Stored in the sessions table with status="handover" │ ▼ 4. Spawn new agent session • Fresh Python process, fresh LLM connection • System prompt: same as original (role + skills) • First user message: the handover document • Code index and git repo: already up to date │ ▼ 5. New session continues • The handover document acts as the first user message • The agent reads it, understands state, and continues • The old session is marked as "handed_off" • The new session is linked to the old one via parent_id
Handover vs. Truncation
Most AI coding tools simply truncate the chat history when the context window fills up. This means the agent forgets everything beyond the truncation point. RaggieCode's handover is fundamentally different:
| Aspect | Truncation | Handover |
|---|---|---|
| What is lost | Everything before the cut point | Nothing .; summarised in handover doc |
| Task awareness | Lost (agent may repeat work) | Preserved (next step is explicit) |
| State continuity | Broken | Seamless |
| Token cost | Zero (just drops history) | One extra LLM call to generate handover |
| Recovery on restart | Not possible | Fully supported via database |
Handover documents are persisted in the chat database. If the entire application crashes mid-handover, the document is already saved and can be resumed on the next startup. The session chain (parent session → handover → child session) is tracked so the full history of a multi-handover task can be reconstructed.
For a deeper dive into the handover mechanism, including the exact fields of the handover document and a worked example, see the Agent Handover documentation.
Next Steps
Now that you understand the architecture, explore the other documentation pages:
Getting Started
Install RaggieCode, run the setup wizard, and execute your first automated code change.
Agent Handover
Deep dive into the handover document format, trigger conditions, and recovery mechanics.
GitHub Repository
Browse the full source code, explore the indexing pipeline, and contribute to the project.
FAQ
Common questions about data privacy, local models, supported languages, and more.