Storage Systems

This guide covers accessing and debugging data in Eloquent's storage systems using CLI tools.

Storage Overview

PostgreSQL

Multi-Tenancy

Each organization has its own schema:

org_0197b2d3_982f_7b79_b2bf_710f6def104a
├── entity_definitions
├── entity_records
├── entity_views
└── ...

Common Queries

-- List all org schemas
SELECT schema_name FROM information_schema.schemata
WHERE schema_name LIKE 'org_%';

-- Query entities in an org
SET search_path TO org_0197b2d3_982f_7b79_b2bf_710f6def104a;

SELECT * FROM entity_definitions;
SELECT * FROM entity_records WHERE definition_name = 'customer';

Using psql

# Connect
psql $DATABASE_URL

# List schemas
\dn

# Switch to org schema
SET search_path TO org_xxx;

# List tables
\dt

# Describe table
\d entity_records

ClickHouse

Knowledge Graph Storage

Per-org databases with KG tables:

org_0197b2d3_982f_7b79_b2bf_710f6def104a
├── kg_nodes
├── kg_edges
└── kg_embeddings

Common Queries

-- List databases
SHOW DATABASES;

-- Use org database
USE org_0197b2d3_982f_7b79_b2bf_710f6def104a;

-- Query nodes
SELECT id, label, name, properties
FROM kg_nodes
WHERE graph_id = 'products'
LIMIT 10;

-- Vector search (approximate)
SELECT id, name, L2Distance(embedding, [0.1, 0.2, ...]) as distance
FROM kg_nodes
ORDER BY distance
LIMIT 10;

Using clickhouse-client

# Connect
clickhouse-client --host $CLICKHOUSE_HOST --user $CLICKHOUSE_USER

# Run query
clickhouse-client --query "SELECT count() FROM system.databases"

NATS JetStream

KV Stores

Chat history is stored in NATS KV:

chats_{orgId}
├── chat-{chatId}
├── chat-{chatId}/messages
└── ...

Common Commands

# List streams
nats stream ls

# View KV buckets
nats kv ls

# Get value from KV
nats kv get chats_org123 chat-abc123

# Watch KV changes
nats kv watch chats_org123 "chat-*"

Streams

# View stream info
nats stream info CHAT_ANALYTICS

# View messages
nats stream view CHAT_ANALYTICS --last 10

# Subscribe to subject
nats sub "chat.events.>"

Redis

Data Patterns

Common Commands

# Connect
redis-cli -h $REDIS_HOST

# Get key
GET session:abc123

# List keys
KEYS cache:entities:org123:*

# Check TTL
TTL cache:entities:org123:customers

# Delete key
DEL cache:entities:org123:customers

Debugging

# Monitor all commands
redis-cli MONITOR

# Get memory usage
redis-cli INFO memory

# Scan for keys (production-safe)
redis-cli SCAN 0 MATCH "session:*" COUNT 100

Docker Access

When running locally with Docker:

# PostgreSQL
docker exec -it postgres psql -U postgres

# ClickHouse
docker exec -it clickhouse clickhouse-client

# Redis
docker exec -it redis redis-cli

# NATS
nats stream ls  # Uses local NATS

Debugging Tips

Entity Not Found

# Check PostgreSQL
psql -c "SELECT id, name FROM entity_definitions WHERE name = 'customer'"

# Check if org schema exists
psql -c "SELECT schema_name FROM information_schema.schemata WHERE schema_name LIKE 'org_%'"

Chat History Missing

# Check NATS KV
nats kv ls
nats kv get chats_{orgId} chat-{chatId}

Cache Issues

# Clear entity cache
redis-cli KEYS "cache:entities:*" | xargs redis-cli DEL

Best Practices

1. Use org-specific schemas/databases - Never query across orgs
2. Index frequently queried fields - Performance optimization
3. Set appropriate TTLs - Prevent stale cache
4. Use SCAN, not KEYS - Production-safe Redis queries
5. Monitor storage usage - Set up alerts for capacity

Next Steps

• Entities Management - Working with entities
• Knowledge Graph - KG queries
• Dev Environment - Local development