|
|
@@ -0,0 +1,120 @@
|
|
|
+# TA Tutor - LLaMA-based AI Tutoring System
|
|
|
+
|
|
|
+TA Tutor is an AI-powered tutoring system built using LLaMA 2, providing interactive learning experiences through Socratic dialogue. The system runs locally on your machine, utilizing GPU acceleration for optimal performance.
|
|
|
+
|
|
|
+## System Requirements
|
|
|
+
|
|
|
+- Windows 11
|
|
|
+- NVIDIA GPU with CUDA support
|
|
|
+- Python 3.8 or higher
|
|
|
+- At least 8GB of GPU VRAM (recommended)
|
|
|
+- Minimum 16GB system RAM
|
|
|
+
|
|
|
+## Installation
|
|
|
+
|
|
|
+1. Clone or download this repository:
|
|
|
+```bash
|
|
|
+git clone [repository-url]
|
|
|
+cd llama-tutor
|
|
|
+```
|
|
|
+
|
|
|
+2. Create and activate a virtual environment:
|
|
|
+```bash
|
|
|
+python -m venv venv
|
|
|
+.\venv\Scripts\activate
|
|
|
+```
|
|
|
+
|
|
|
+3. Install required Python packages:
|
|
|
+```bash
|
|
|
+pip install gradio
|
|
|
+pip install ctransformers[cuda]
|
|
|
+pip install psutil
|
|
|
+pip install gputil
|
|
|
+```
|
|
|
+
|
|
|
+4. Download the LLaMA model:
|
|
|
+- Visit [TheBloke's Hugging Face page](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/tree/main)
|
|
|
+- Download `llama-2-7b-chat.gguf` (approximately 4GB)
|
|
|
+- Place the downloaded file in the `models` directory:
|
|
|
+ ```
|
|
|
+ llama-tutor/
|
|
|
+ ├── models/
|
|
|
+ │ └── llama-2-7b-chat.gguf
|
|
|
+ ├── main.py
|
|
|
+ └── README.md
|
|
|
+ ```
|
|
|
+
|
|
|
+## Directory Structure
|
|
|
+```
|
|
|
+llama-tutor/
|
|
|
+├── models/ # Model storage directory
|
|
|
+├── logs/ # Session logs (created automatically)
|
|
|
+├── main.py # Main application file
|
|
|
+└── README.md # This file
|
|
|
+```
|
|
|
+
|
|
|
+## Usage
|
|
|
+
|
|
|
+1. Ensure your virtual environment is activated:
|
|
|
+```bash
|
|
|
+.\venv\Scripts\activate
|
|
|
+```
|
|
|
+
|
|
|
+2. Run the application:
|
|
|
+```bash
|
|
|
+python main.py
|
|
|
+```
|
|
|
+
|
|
|
+3. Access the web interface:
|
|
|
+- Open your browser and navigate to `http://localhost:7860`
|
|
|
+- The interface will load with a chat interface and system monitoring
|
|
|
+
|
|
|
+## Features
|
|
|
+
|
|
|
+- Interactive chat interface using Gradio
|
|
|
+- GPU-accelerated inference
|
|
|
+- System resource monitoring (CPU, Memory, GPU utilization)
|
|
|
+- Session logging
|
|
|
+- Example questions for various subjects
|
|
|
+- Automatic conversation length management
|
|
|
+- Real-time system statistics
|
|
|
+
|
|
|
+## Troubleshooting
|
|
|
+
|
|
|
+1. **CUDA Issues**:
|
|
|
+ - Ensure you have the latest NVIDIA drivers installed
|
|
|
+ - Verify CUDA toolkit is properly installed
|
|
|
+ - Check GPU compatibility with `nvidia-smi` command
|
|
|
+
|
|
|
+2. **Memory Issues**:
|
|
|
+ - If you encounter memory errors, try reducing `gpu_layers` in `main.py`
|
|
|
+ - Adjust `batch_size` and `threads` parameters based on your system capabilities
|
|
|
+
|
|
|
+3. **Model Loading Issues**:
|
|
|
+ - Verify the model file path in `main.py` matches your actual model location
|
|
|
+ - Ensure the model file is not corrupted by checking its size (should be ~4GB)
|
|
|
+
|
|
|
+## Performance Optimization
|
|
|
+
|
|
|
+- Adjust these parameters in `main.py` based on your system:
|
|
|
+ ```python
|
|
|
+ gpu_layers=32 # Reduce if experiencing GPU memory issues
|
|
|
+ threads=6 # Adjust based on CPU cores
|
|
|
+ batch_size=256 # Modify based on available memory
|
|
|
+ ```
|
|
|
+
|
|
|
+## Logging
|
|
|
+
|
|
|
+- Session logs are automatically saved in the `logs` directory
|
|
|
+- Each session creates a new JSON file with timestamp
|
|
|
+- Logs include: user messages, AI responses, and system statistics
|
|
|
+
|
|
|
+## License
|
|
|
+
|
|
|
+This project uses the LLaMA 2 model which is subject to the Meta AI license. Ensure compliance with all relevant licenses and terms of use.
|
|
|
+
|
|
|
+## Acknowledgments
|
|
|
+
|
|
|
+- Built with [LLaMA 2](https://ai.meta.com/llama/) by Meta AI
|
|
|
+- Uses [CTransformers](https://github.com/marella/ctransformers) for GPU acceleration
|
|
|
+- Interface built with [Gradio](https://gradio.app/)
|