Playground

The AI Playground (aio_playground) module provides an interactive web workspace and analytics engine for prompt engineering, real-time testing, conversation branching, and evaluation of AI agents in OpenEduCat.

Overview

The AI Playground is an interactive workspace designed for testing, evaluating, and refining AI agents in real time. It enables administrators, instructors, developers, and AI prompt engineers to test agent behaviors, evaluate tool calls, analyze model performance, branch conversation threads, and monitor AI spending.

Key Features

  • Interactive Chat Workspace: Test agents in a dedicated chat interface with parameter controls (Temperature, Max Output Tokens, System Prompt overrides).

  • Live Word & Tool Streaming: Stream agent responses word-by-word and view real-time function tool execution logs.

  • Conversation Branching: Create parallel conversation threads from any past message turn to compare prompt variations side-by-side.

  • Session Management & Tagging: Categorize, filter, and save testing sessions (e.g., Academic Advisor Test, Admissions Assistant, Prompt V2).

  • Multi-Format Export: Export complete test transcripts into PDF, CSV, JSON, or Markdown formats for compliance auditing and performance evaluation.

  • Feedback & Response Evaluation: Rate AI answers using Thumbs Up / Thumbs Down rating controls and edit past prompts to evaluate regenerated responses.

  • Analytics & Cost Optimization: Monitor token consumption, average latency, cost per session, and budget thresholds.

How to Use the AI Playground

1. Accessing the Playground Dashboard

  • Go to AIO ‣ Playground ‣ Playground to open the chat testing environment.

Welcome to AI Playground Dashboard

2. Starting a New Testing Session

  • Select AI Agent: Pick an active AI agent from the top agent dropdown selector or select a tool card from the dashboard.

  • New Session: Click New Session or select an existing session from the history sidebar.

  • Session Tags: Assign descriptive tags (e.g., Grading Test, Prompt Tuning) for easy search and categorization.

  • Parameter Overrides: Optionally adjust temperature, top_p, max tokens, or attached knowledge bases for experimental runs.

3. Testing Agents & Live Streaming

  • Type user prompts into the chat input bar and click Send.

  • Observe real-time token streaming output, reasoning thoughts, and function tool execution details.

  • Inspect tool call inputs and outputs (e.g., database record retriever, email notification trigger, RAG search results).

  • Rate agent outputs with Thumbs Up / Thumbs Down or click Edit Prompt to test alternative user inputs.

Active AI Playground Chat Session

4. Conversation Branching

  • Hover over any message turn in the conversation thread and click Branch Conversation.

  • The playground spawns a new parallel session thread starting from that exact point, allowing side-by-side comparison of different prompts without losing the original session history.

5. Exporting Transcripts & Data

  • Click Export Session in the top control bar.

  • Choose your desired format: * PDF: Formatted transcript with metadata headers, timestamps, and charts. * CSV / JSON: Raw structured JSON/CSV data for machine learning model evaluation and automated logging. * Markdown: Clean text format for sharing test results with team members.

6. Cost Optimization & Performance Analytics

  • Navigate to AIO ‣ Playground ‣ Analytics & Costs.

  • Review token consumption graphs (input vs. output tokens per model/agent).

  • Monitor average response latency, total cost per query, monthly spending, and cost reduction recommendations.