Knowledgebase RAG System - Intelligent Chatbot Integration

RAG-based Question-Answering with Document, FAQ and News Integration

2025-2026 Personal Project

Project Description

This project implements a comprehensive knowledgebase system using Retrieval-Augmented Generation (RAG) technology for intelligent question-answering in the chatbot. The system integrates documents, FAQs, and news articles to provide context-based answers using OpenAI GPT-4o-mini. It features vector search for document similarity, TOON format for efficient context formatting, automatic storage of unanswered questions, and seamless integration with the document management system.

Key Features

RAG-based Answer Generation

OpenAI GPT-4o-mini with structured JSON responses

Vector Search

Embedding-based document similarity search

Document Integration

Automatic indexing and vectorization of documents

FAQ Integration

Keyword-based FAQ search with scoring

News Integration

News article search as fallback information source

Question Management

Automatic storage and management of unanswered questions

Technology Stack

Backend Framework

Laravel 11
PHP 8.2+
MySQL

AI & ML

OpenAI GPT-4o-mini
Vector Embeddings
RAG Technology

Frontend

Livewire
Alpine.js
Bootstrap

Data Management

Vector Store
TOON Format
JSON Schema

Workflow

  1. User Query: User asks a question in the chatbot
  2. FAQ Search: System searches in FAQ files with keyword matching and scoring
  3. Knowledgebase Search: If FAQ score is low, system searches in vectorized documents
  4. Context Formatting: Search results are formatted as TOON (Token-Oriented Object Notation)
  5. Answer Generation: OpenAI GPT-4o-mini generates answer based on context with JSON schema
  6. Source Display: Relevant document sources are displayed with chunk information
  7. Question Storage: If no answer found, question is automatically stored for review
  8. News Fallback: If no document match, system searches in news articles