Academic retrieval workspace
Arcus indexes your PDFs for hybrid retrieval — dense vector search and PostgreSQL full-text search, fused by rank — then answers only from what it actually found.
Retrieval
Embeddings are good at paraphrase and bad at exact tokens — course codes, theorem names, notation. Full-text search is the reverse. Arcus runs both on every question and combines them with Reciprocal Rank Fusion, so neither weakness reaches your answer.
Cosine distance over 768-dimensional Gemini embeddings, served by an HNSW index. Finds passages that mean the same thing in different words.
embedding <=> query::vector
USING hnsw (vector_cosine_ops)
PostgreSQL full-text search over a generated tsvector column, served by a GIN index. Finds the exact term you typed, even when the embedding blurs it.
searchVector @@ websearch_to_tsquery(...)
USING gin ("searchVector")
Fusion
score(d) = Σarms weight ÷ (k + rankarm(d))
Cosine similarity and ts_rank_cd live on incomparable scales, so blending their scores needs constant recalibration. Rank position doesn't — which is why RRF stays honest as the corpus grows. A passage both arms rank moderately well beats one that a single arm ranks first.
Ingestion
Upload triggers a durable background job: fetch, parse, chunk, validate, embed, upsert. Every stage is measured, and every failure is either retried or explained.
Every chunk is keyed by the SHA-256 of its text under a unique constraint. A retried job re-inserts the same rows and the database absorbs them — a half-finished run can be replayed verbatim.
FAILED means the retry budget is exhausted, not that something went wrong once. Terminal input skips retries entirely; everything else is redriveable from the UI with its error trace intact.
Each attempt records latency, per-step timings, chunk yield, dedupe count, and estimated token spend — so p95 latency and cost per document are one SQL query away.
fetch-and-chunkParse the PDF, split it, normalise whitespace, hash each chunk, and validate against a zod schema. Malformed fragments are rejected individually and counted.
embed-and-storeEach batch is embedded and written inside a single step, so vectors never cross a step boundary. Writes are ON CONFLICT DO NOTHING.
finalizeCount what actually landed, then close the run with latency, yield, dedupe count, and token spend.
Workspace
Answers are constructed only from retrieved passages, with the source document and page attached to every claim.
Decks generated from a document's own content as structured output, not free-form prose.
Multiple-choice sets with explanations, scored in place and saved back to your workspace.
Tool calls pull dates and syllabus items out of documents and onto an academic calendar.
Stack