AI Chat Assistant

A daily-use LLM conversation tool. Compared to the official web clients, it solves three pain points: multi-turn context management, a typewriter-style streaming experience, and fully local history — all conversation data stays in your own browser.

Architecture

The frontend is React (Vite); the backend is FastAPI. The two keep a persistent connection over SSE (Server-Sent Events):

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio

app = FastAPI()

@app.post("/chat")
async def chat(messages: list[dict]):
    async def stream():
        # Simulated LLM streaming output
        for token in ["Hello", ", ", "I", "am", "AI", "assistant", "."]:
            yield f"data: {token}\n\n"
            await asyncio.sleep(0.05)
    return StreamingResponse(stream(), media_type="text/event-stream")

Feature List

• Multi-turn context: automatically trims history beyond the window limit
• Streaming output: tokens pushed via SSE, rendered word by word
• Local-first: history stored in IndexedDB, exportable as JSON
• Custom system prompts: per-conversation settings

Technical Decisions

• Why SSE instead of WebSocket? For one-way push, SSE is simpler — built-in reconnection and event IDs, zero extra libraries.
• Why not deploy the backend on Vercel? Streaming endpoints need long-lived connections; edge function timeouts are a poor fit. FastAPI runs on a small cloud VPS instead.

Roadmap

Next up: a plugin system (web search, code sandbox), voice input, and personal knowledge-base Q&A backed by a local vector store. Ideas are welcome — open an issue on GitHub.