Demonstration ProjectIntermediate AIHR / internal tools

PolicyPal

Ask the policy. Get the page.

PolicyPal is an internal assistant for Northgate Logistics, a UK freight company. Employees ask questions about annual leave, expenses, remote work or sickness absence and get a short answer with numbered citations to the exact policy and page. When retrieval is weak, it says it could not find the answer and points to HR instead of guessing. HR admins manage the document library and review the questions the bot was unsure about.

  • Next.js
  • TypeScript
  • Node.js
  • PostgreSQL
  • pgvector
  • RAG
  • LLM APIs
  • REST API
  • Queue
Chat with citations. Interface designed and built by Vikrant

Overview

Business problem
HR policies live in PDFs and Word files across a shared drive. Employees ask the same questions in chat or by email, answers drift from the written policy, and old versions keep circulating. A general chatbot would answer confidently with text that is not Northgate's policy.
Product objective
Let an employee get a correct, sourced answer in seconds, make every answer checkable against the original page, and give HR a clear view of what the bot could not answer so the policies themselves can be fixed.
Target users
Employees across depots and the head office who need a quick answer about their entitlements, and a small HR team that owns the documents and wants fewer repeated questions.
Solution
A retrieval-augmented assistant restricted to approved policy documents. Answers are generated only from retrieved passages, each claim carries a citation back to a document, version and page, and weak retrieval produces an explicit fallback that is logged for HR review.

Project classification

Project type
Document Q&A bot
Industry
HR / internal tools
Frontend
Next.js with TypeScript: chat, source viewer and HR admin screens
Backend
Node.js API with a queue-driven ingestion worker and a retrieval service
Database
PostgreSQL with a pgvector index for chunk embeddings
Architecture
Ingestion pipeline, retrieval with rerank, then a cited answer generator

User roles

  • Employee
  • HR admin

My role

  • System architecture: the split between the Next.js app, the Node.js API, the ingestion worker and the PostgreSQL store.
  • Ingestion pipeline design: parsing PDF and DOCX, chunking, embedding, and re-indexing when a document version changes.
  • Retrieval design: vector search with pgvector, reranking, a confidence threshold and the weak-retrieval fallback.
  • Citation mapping: tying each answer sentence to chunk ids, then to document, version and page.
  • Access model: department-scoped documents enforced in the retrieval query, not in the prompt.
  • Evaluation: a labelled question set and a script that compares retrieval and answers between versions.
  • Frontend: chat, source viewer and admin screens in Next.js and TypeScript.
  • This is a self-initiated concept; there is no client, live deployment or user base.

Key features

  • Chat with citations. Interface designed and built by Vikrant

    Answers that show their sources

    An employee asks a plain question and gets a short answer with numbered citations. Each citation names the document, version and page, and the source rail beside the chat lists the same passages, so the answer can be checked in one click rather than trusted.

  • Source viewer. Interface designed and built by Vikrant

    Source viewer with the passage highlighted

    Opening a citation shows the original document page with the supporting passage highlighted, along with the document version, owner and effective date. It makes it obvious when an answer paraphrases correctly and when it does not.

  • Document library. Interface designed and built by Vikrant

    Document library with ingestion status

    HR uploads PDFs and DOCX files, assigns a department scope and a version, and watches each document move through parsing, chunking and embedding. Failed or superseded documents are flagged so they do not quietly degrade answers.

  • Low-confidence review queue. Interface designed and built by Vikrant

    Low-confidence review queue

    When retrieval is weak the bot says it could not find the answer, and the question lands in a queue. HR sees the question, the best passages that were found and their scores, then replies, links the right policy, or marks a gap in the documents.

  • Retrieval settings. Interface designed and built by Vikrant

    Retrieval settings HR can reason about

    Similarity and confidence thresholds, chunk size and overlap, the number of passages sent to the model, the rerank step and department access scopes are visible in one place, with the evaluation set results alongside so a change can be judged before it is published.

Also in the product

  • Chat with numbered citations that link to the source page
  • Source viewer with the supporting passage highlighted
  • Honest fallback when retrieval is weak
  • Document library with versions, departments and ingestion status
  • Low-confidence review queue for HR
  • Retrieval settings: thresholds, chunking, rerank and access scopes

User flow

  1. HR admin uploads a policy document and sets its department scope and version
  2. Ingestion parses, chunks, embeds and indexes the document in the background
  3. Superseded versions are archived and excluded from retrieval
  4. Employee asks a question in the chat
  5. The question is embedded and searched, filtered by the employee's department access
  6. Top passages are reranked; if the best score is under the threshold the bot gives the fallback answer
  7. Otherwise the model answers from the passages only, with numbered citations
  8. Employee opens a citation to read the highlighted source page
  9. Low-confidence questions appear in the HR review queue
  10. HR fixes the document or replies, then re-runs the evaluation set

Technical architecture

  1. Next.js app: employee chat, source viewer and HR admin screens
  2. REST API (Node.js) with session auth and department-aware authorisation
  3. Ingestion queue and worker: parse, chunk, embed, index
  4. Retrieval service: query embedding, pgvector search, rerank, confidence check
  5. Answer generator (LLM API) constrained to retrieved passages, returning citation ids
  6. PostgreSQL with pgvector: documents, versions, chunks, conversations, review items
  7. Evaluation runner: labelled question set against retrieval and answers

Database and backend

Ingestion pipeline

  • Upload: the API stores the file, creates a Document and DocumentVersion row, enqueues a job and returns immediately; the UI polls the ingestion status.
  • Parse: PDF and DOCX are converted to text with page numbers and heading paths preserved, since both are needed for citations.
  • Chunk: structure-aware splitting on headings and clauses with a token target and a small overlap; every chunk keeps its page range and heading path.
  • Embed: batched embedding calls, with a content hash per chunk so unchanged text in a new version is not re-embedded.
  • Index: vectors are written to a pgvector column with an approximate-nearest-neighbour index; the previous version is marked superseded in the same transaction.

Retrieval, rerank and fallback

  • Search: the question is embedded and matched against current-version chunks only, with the department filter applied inside the SQL query.
  • Hybrid match: a keyword match is merged in so exact terms such as policy names and form codes are not missed by vector search alone.
  • Rerank: the top candidates are re-scored against the question and trimmed to the few passages that fit the context budget.
  • Weak-retrieval fallback: if the best rerank score is under the configured threshold, no answer is generated; the bot replies that it could not find this in the policies and offers HR contact details.
  • Every fallback is stored with the question and the passages that were closest, which is what fills the review queue.

Citation mapping

  • The model receives passages labelled with short ids and must return its answer together with the ids it used.
  • The server validates that every cited id was in the retrieved set and drops or flags answers that cite something else.
  • Ids are resolved to document name, version and page range for display, and to a highlight range for the source viewer.
  • Answers with no valid citation are treated as a failed generation and fall back to the not-found reply.

Access scoping and data model

  • Document, DocumentVersion, Chunk (with embedding), Department, Conversation, Message, ReviewItem, EvalQuestion, EvalRun.
  • Each document carries a department scope (All staff, People and Culture, Finance, Operations); the retrieval query joins the user's departments, so out-of-scope text never reaches the model.
  • Conversations store the retrieved chunk ids and scores per answer, which makes any answer reproducible and auditable.
  • Evaluation questions are labelled with the expected document and page; a run records retrieval hit rate and answer correctness notes per version of the settings.

Challenges and solutions

  • Chunking policy documents without losing the clause

    Fixed-size splitting cut conditions away from the rule they modify. Chunks follow headings and numbered clauses, keep their heading path, and overlap slightly, so an entitlement and its exceptions tend to travel together.

  • Citations that point to the right page

    Page numbers are captured at parse time and stored on every chunk. The model returns passage ids rather than free-text references, and the server resolves and validates them, so a citation cannot name a document that was not retrieved.

  • Stale documents and versions

    Each upload is a new version. Indexing swaps the current version in a transaction and marks the old one superseded, so retrieval only searches current text, while the source viewer can still open an older version that a past answer cited.

  • Keeping department-only documents private

    Scope is enforced in the retrieval query, not in the prompt. Chunks the user may not read are never fetched, so there is nothing for the model to leak or for a prompt injection to extract.

  • Knowing when not to answer

    A confidence threshold on the reranked score decides between answering and the fallback. The threshold is set from the evaluation set, and every fallback is queued for HR, so a gap in the documents becomes a visible task.

  • Judging whether a change made answers better

    A labelled evaluation set runs after any change to chunking, thresholds or prompts. It reports whether the expected passage was retrieved and flags answers that changed, so settings are adjusted on evidence rather than on a few hand-picked questions.

Results

  • A complete case study of a retrieval-augmented assistant covering ingestion, retrieval, citation checking and refusal behaviour.
  • Shows how to make AI answers verifiable: every claim points to a document, version and page.
  • Treats a wrong answer as worse than no answer, with a fallback and a review loop designed in from the start.
  • Keeps access control in the data layer and keeps document versions explicit, which are the parts that usually fail in internal bots.

Qualitative outcomes only; no usage or revenue figures.

Visual identity

  • Primary #0F3D3E
  • Secondary #334155
  • Accent #F2A33A
  • Background #F6F8F7

IBM Plex Sans for the interface, IBM Plex Mono for references and settings. Folded-page mark with a quote tick in amber. Calm top-navigation workspace, a single conversation column with a source rail, quiet green and amber accents.

Related projects

  • Demonstration ProjectAI / automation

    AgentForge

    Build, ground and monitor AI agents with RAG, tools, a testing playground and usage tracking.

    • Next.js
    • React.js
    • TypeScript
    • Node.js
    • Laravel
    • PostgreSQL
    • pgvector
    • RAG
    • LLM APIs
    • REST API
    • Webhooks
  • Demonstration ProjectBasic AISmall business / e-commerce

    Pebble Chat

    An embeddable FAQ chat widget for a small shop that answers from the owner's own policies and hands anything uncertain over by email.

    • React.js
    • Node.js
    • Express.js
    • LLM API
    • REST API
    • SQLite