llama-tutor

TA Tutor - LLaMA-based AI Tutoring System

TA Tutor is an AI-powered tutoring system built using LLaMA 2, providing interactive learning experiences through Socratic dialogue. The system runs locally on your machine, utilizing GPU acceleration for optimal performance.

System Requirements

• Windows 11
• NVIDIA GPU with CUDA support
• Python 3.8 or higher
• At least 8GB of GPU VRAM (recommended)
• Minimum 16GB system RAM

Installation

1. Clone or download this repository:

   git clone [repository-url]
   cd llama-tutor
   

2. Create and activate a virtual environment:

   python -m venv venv
   .\venv\Scripts\activate
   

3. Install required Python packages:

   pip install gradio
   pip install ctransformers[cuda]
   pip install psutil
   pip install gputil
   

4. Download the LLaMA model:

5. Visit TheBloke's Hugging Face page

6. Download llama-2-7b-chat.gguf (approximately 4GB)

7. Place the downloaded file in the models directory:

   llama-tutor/
   ├── models/
   │   └── llama-2-7b-chat.gguf
   ├── main.py
   └── README.md
   

Directory Structure

llama-tutor/
├── models/              # Model storage directory
├── logs/               # Session logs (created automatically)
├── main.py             # Main application file
└── README.md           # This file

Usage

1. Ensure your virtual environment is activated:

   .\venv\Scripts\activate
   

2. Run the application:

   python main.py
   

3. Access the web interface:

4. Open your browser and navigate to http://localhost:7860

5. The interface will load with a chat interface and system monitoring

Features

• Interactive chat interface using Gradio
• GPU-accelerated inference
• System resource monitoring (CPU, Memory, GPU utilization)
• Session logging
• Example questions for various subjects
• Automatic conversation length management
• Real-time system statistics

Troubleshooting

1. CUDA Issues:

   • Ensure you have the latest NVIDIA drivers installed
   • Verify CUDA toolkit is properly installed
   • Check GPU compatibility with nvidia-smi command

2. Memory Issues:

   • If you encounter memory errors, try reducing gpu_layers in main.py
   • Adjust batch_size and threads parameters based on your system capabilities

3. Model Loading Issues:

   • Verify the model file path in main.py matches your actual model location
   • Ensure the model file is not corrupted by checking its size (should be ~4GB)

Performance Optimization

• Adjust these parameters in main.py based on your system:

  gpu_layers=32  # Reduce if experiencing GPU memory issues
  threads=6      # Adjust based on CPU cores
  batch_size=256 # Modify based on available memory
  

Logging

• Session logs are automatically saved in the logs directory
• Each session creates a new JSON file with timestamp
• Logs include: user messages, AI responses, and system statistics

License

This project uses the LLaMA 2 model which is subject to the Meta AI license. Ensure compliance with all relevant licenses and terms of use.

Acknowledgments

• Built with LLaMA 2 by Meta AI
• Uses CTransformers for GPU acceleration
• Interface built with Gradio