PLCT AI Context Builder

A tool for building AI context datasets from online courses developed with PLCT-CLI and PetljaDoc. It processes course content (lessons, quizzes, coding problems), generates summaries using OpenAI/Azure OpenAI, computes text embeddings, and produces a structured context dataset consumed by the AI Assistant in PLCT-Server.

Features

• Course summarization — generates per-activity and consolidated course summaries using GPT-4o-mini
• Text chunking & embedding — splits course content into token-sized chunks and computes embeddings via OpenAI or Azure OpenAI
• Multiple embedding dimensions — supports configurable embedding sizes (e.g. 256, 1536)
• RST and Markdown support — normalizes both reStructuredText (PetljaDoc) and MyST Markdown (PLCT-CLI) sources, resolving literalinclude and activecode directives
• Coding quiz processing — extracts and embeds coding problem statements alongside regular lesson content
• Incremental builds — skips already-generated summaries unless forced, with optional cleanup of inactive chunks

Prerequisites

• Python 3.10+
• The uv tool (recommended) or pip
• An OpenAI API key or an Azure OpenAI deployment
• Have cloned PLCT-Server repo in the sibling PLCT-Server directory (see pyproject.toml source path)

Installation

Clone from GitHub

Clone this repo:

git clone https://github.com/Petlja/PLCT-AI-Ctx.git

If not already done, clone PLCT-Server into the same parent directory:

git clone https://github.com/Petlja/PLCT-Server.git

Your directory layout should look like:

parent-dir/
├── PLCT-AI-Ctx/
└── PLCT-Server/

In-place installation

Navigate to the cloned repo:

cd PLCT-AI-Ctx

Install with uv (recommended):

uv sync

Or install with pip (requires an activated virtual environment):

pip install -e .

Installation inside another project

If your project consumes PLCT AI Context Builder as a dependency, both repos should share the same parent directory:

parent-dir/
├── PLCT-AI-Ctx/
├── PLCT-Server/
└── your-project/

This way you separate your work from PLCT AI Context Builder itself, and may clearly combine other tools into your project.

Option 1 — pip: From your project's activated virtual environment:

pip install -e ../PLCT-AI-Ctx

Option 2 — uv (recommended): Add the dependency to your project's pyproject.toml:

[project]
# ... other settings
dependencies = [
    "plct-ai-ctx",
    # ... other dependencies
]

[tool.uv.sources]
plct-ai-ctx = { path = "../PLCT-AI-Ctx", editable = true }

Then run:

uv sync

Configuration

1. Copy plct-ai-ctx-config-sample.yaml to plct-ai-ctx-config.yaml in your project's working directory and fill in the values:

course_paths:
  - "path/to/course1"
  - "path/to/course2"
embedding_sizes:
  - 256
  - 1536
embedding_model: "text-embedding-3-large"
chunk_size: 3072
chunk_overlap: 1524
base_dir: "ai-context"
coding_problems_dir: "path/to/problems"

# Optional — omit for direct OpenAI usage
azure_endpoint: "https://<resource>.openai.azure.com"
azure_api_version: "2023-03-15-preview"
azure_embedding_api_version: "2023-05-15"

Each course path shoud point to an online coure project built with ith PLCT-CLI and PetljaDoc. Many of them you can find at https://github.com/Petlja/.

2. Set the required environment variable(s):

# For OpenAI
export CHATAI_OPENAI_API_KEY="sk-..."

# For Azure OpenAI
export CHATAI_AZURE_API_KEY="..."

Usage

Run the CLI from the project root (where plct-ai-ctx-config.yaml lives):

plct-ai-ctx-build

CLI Options

Flag	Description
--force_activity_summary	Regenerate all per-activity summaries even if they already exist
--force_course_summary	Regenerate the short course summary even if it already exists
--delete_inactive_chunks	Remove chunks from the index that are no longer referenced

Examples

# Full rebuild of summaries and embeddings
plct-ai-ctx-build --force_activity_summary --force_course_summary

# Incremental build (only new/missing content)
plct-ai-ctx-build

# Clean up stale chunks after removing a course
plct-ai-ctx-build --delete_inactive_chunks

How It Works

The build pipeline processes each course in course_paths through these stages:

1. Activity summarization — each lesson/quiz is sent to GPT-4o-mini to extract key learning points, programming techniques, and outcomes. Results are saved under summaries/<courseId>/.
2. Consolidated summary — individual activity summaries are merged into a single document per course.
3. Short summary — the consolidated summary is compressed into a ~700-word overview.
4. Table of contents — a structured TOC is generated from the course metadata.
5. Chunking & embedding — source content is tokenized (using tiktoken), split into chunks of chunk_size tokens with chunk_overlap overlap, and embedded using the configured model and dimensions.
6. Index update — the final context dataset is written to base_dir (default: ai-context/).

Project Structure

plct_ai_ctx/
├── cli.py             # Entry point & pipeline orchestration
├── ai_context.py      # Embedding generation & context export
├── config_options.py  # Configuration loading & validation
├── course_metadata.py # Course structure parsing (YAML / Sphinx pickle)
└── util.py            # Utility for decompressing .zst files

Output

The build produces the following artifacts:

• summaries/<courseId>/ — per-activity summaries, consolidated summary, short summary, and TOC
• ai-context/chunks/ — content-addressed text chunks with their embeddings
• ai-context/<courseId>/ — course-level summary metadata

License

See the project license file for details.