diff --git a/README.md b/README.md deleted file mode 100644 index 2127233..0000000 --- a/README.md +++ /dev/null @@ -1,382 +0,0 @@ -# 🚀 LLM-Powered URL Health Checker Generator - -[![Python](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://python.org) -[![LLM APIs](https://img.shields.io/badge/LLM-Multi--Provider-green.svg)](https://github.com) -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -[![Jupyter](https://img.shields.io/badge/Jupyter-Notebook-orange.svg)](https://jupyter.org) - -🤖 **AI-Powered Code Generation** | 🌐 **URL Health Monitoring** | 📊 **DevOps Tooling** | ⚡ **Automated CLI Tools** - -> **Transform ideas into code with ANY LLM API!** This project showcases how to harness various Large Language Model APIs (OpenAI, Anthropic Claude, Google Gemini, Azure, local models, and more) to automatically generate production-ready Python CLI tools. Watch as AI creates a sophisticated URL health monitoring system that rivals hand-coded solutions. - -## 🎯 What This Project Does - -This project demonstrates cutting-edge AI-assisted development by: -- 🧠 **Intelligently generating** complete Python applications using any LLM API -- 🔍 **Creating a professional URL health checker** with advanced monitoring capabilities -- 📈 **Showcasing practical AI applications** in DevOps and system administration -- 🛠️ **Providing a universal framework** for AI-powered code generation across multiple providers - -## 🌟 Supported LLM Providers - -| Provider | Models | Configuration | -|----------|---------|--------------| -| **OpenAI** | GPT-4, GPT-3.5-Turbo | Standard OpenAI API | -| **Anthropic** | Claude 3 Opus, Sonnet, Haiku | Claude API | -| **Google** | Gemini Pro, Gemini Ultra | Google AI Studio | -| **Azure OpenAI** | GPT-4, GPT-3.5 | Azure OpenAI Service | -| **Cohere** | Command, Generate | Cohere API | -| **Hugging Face** | Various open models | Inference API | -| **Local Models** | LLaMA, Mistral, etc. | Ollama, LM Studio | -| **Custom APIs** | Any OpenAI-compatible | Custom endpoints | - -## 🚀 Features - -### Core Framework -- **Universal LLM Support**: Works with any OpenAI-compatible API endpoint -- **Provider Flexibility**: Easy switching between different LLM providers -- **Smart Configuration Management**: Secure environment variable handling with validation -- **AI-Powered Code Generation**: Uses your choice of LLM to generate complete, functional Python scripts -- **Modular Design**: Clean, reusable configuration class for API integration -- **Error Handling**: Robust validation and error management across providers - -### Generated URL Health Checker -- **Batch URL Testing**: Process multiple URLs from input files -- **Performance Monitoring**: Measure and report response times -- **Status Code Tracking**: Monitor HTTP response codes -- **Configurable Parameters**: Customizable timeouts and retry mechanisms -- **CSV Export**: Structured data output for analysis -- **Summary Statistics**: Real-time reporting of test results -- **CLI Interface**: Professional command-line tool with argument parsing - -## ⚡ Quick Start - -```bash -# 1. Clone and setup -git clone https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator.git -cd LLM-URL-Health-Checker-Generator - -# 2. Install dependencies -pip install openai anthropic google-generativeai python-dotenv requests - -# 3. Configure environment (see provider examples below) -cp .env.example .env -# Edit .env with your preferred provider - -# 4. Run the notebook to generate the tool -jupyter notebook Final.ipynb - -# 5. Use the generated URL checker -echo "https://github.com" > urls.txt -python url_checker.py --input urls.txt --output results.csv -``` - -## 📋 Prerequisites - -- Python 3.10+ -- Access to at least one LLM API (see supported providers) -- Required Python packages (see installation) - -## 🔧 Detailed Installation - -### 1. **Clone the Repository** - ```bash - git clone https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator.git - cd LLM-URL-Health-Checker-Generator - ``` - -### 2. **Set Up Python Environment** (Recommended) - ```bash - # Create virtual environment - python -m venv venv - - # Activate it (Windows) - venv\Scripts\activate - # Or on Linux/Mac - source venv/bin/activate - ``` - -### 3. **Install Dependencies** - ```bash - # Core dependencies - pip install python-dotenv requests jupyter - - # Install based on your provider: - pip install openai # For OpenAI - pip install anthropic # For Claude - pip install google-generativeai # For Gemini - # Or install all: - pip install -r requirements.txt - ``` - -### 4. **Configure Environment Variables** - -Create a `.env` file based on your chosen provider: - -#### OpenAI Configuration -```env -LLM_PROVIDER=openai -OPENAI_API_KEY=sk-your_api_key_here -BASE_URL=https://api.openai.com/v1 -MODEL_NAME=gpt-4 -``` - -#### Anthropic Claude Configuration -```env -LLM_PROVIDER=anthropic -ANTHROPIC_API_KEY=sk-ant-your_api_key_here -BASE_URL=https://api.anthropic.com -MODEL_NAME=claude-3-opus-20240229 -``` - -#### Google Gemini Configuration -```env -LLM_PROVIDER=google -GOOGLE_API_KEY=your_api_key_here -MODEL_NAME=gemini-pro -``` - -#### Azure OpenAI Configuration -```env -LLM_PROVIDER=azure -AZURE_API_KEY=your_api_key_here -BASE_URL=https://your-resource.openai.azure.com -AZURE_DEPLOYMENT_NAME=your-deployment -MODEL_NAME=gpt-4 -``` - -#### Local Models (Ollama) Configuration -```env -LLM_PROVIDER=ollama -BASE_URL=http://localhost:11434/v1 -MODEL_NAME=llama2:13b -# No API key needed for local models -``` - -#### Custom OpenAI-Compatible API -```env -LLM_PROVIDER=custom -API_KEY=your_api_key_here -BASE_URL=https://your-custom-endpoint.com/v1 -MODEL_NAME=your-model-name -``` - -## 🎯 Usage Guide - -### Step 1: Generate the URL Checker -1. **Open the Jupyter Notebook:** - ```bash - jupyter notebook Final.ipynb - ``` - -2. **Select your LLM provider** in the configuration cell - -3. **Execute all cells** to: - - Configure your chosen LLM API connection - - Generate the URL health checker script - - Save it as `url_checker.py` - -### Step 2: Prepare Your URL List -Create a `urls.txt` file with websites to monitor: -```text -https://www.google.com -https://github.com -https://stackoverflow.com -https://httpbin.org/status/200 -https://httpbin.org/delay/3 -# Comments are ignored -https://nonexistent-website-12345.com -``` - -### Step 3: Run the Health Checker -```bash -# Basic usage -python url_checker.py - -# Custom parameters -python url_checker.py --input my_sites.txt --output health_report.csv --timeout 10 --retries 5 -``` - -## 📊 Example Output - -### Console Output -``` -🔍 URL Health Checker Starting... -✅ https://www.google.com - 200 OK (156ms) -✅ https://github.com - 200 OK (245ms) -❌ https://nonexistent-site.com - Connection failed -📊 Summary: 2/3 successful (66.7%), Average: 200ms -``` - -### CSV Report (`results.csv`) -```csv -url,status_code,response_time_ms,success -https://www.google.com,200,156.3,True -https://github.com,200,245.7,True -https://nonexistent-site.com,0,0.0,False -``` - -## 🛠️ Command Line Options - -| Option | Default | Description | Example | -|--------|---------|-------------|---------| -| `--input` | `urls.txt` | Input file containing URLs | `--input websites.txt` | -| `--output` | `results.csv` | Output CSV file for results | `--output report.csv` | -| `--timeout` | `5` | Request timeout in seconds | `--timeout 10` | -| `--retries` | `3` | Number of retry attempts | `--retries 5` | - -## 🏗️ Project Architecture - -``` -LLM-URL-Health-Checker-Generator/ -├── 📓 Final.ipynb # Main LLM integration notebook -├── 📄 README.md # This documentation -├── 🐍 url_checker.py # Generated URL health checker -├── ⚙️ .env # Environment variables (create this) -├── ⚙️ .env.example # Example environment configuration -├── 📝 urls.txt # Input URLs list -├── 📊 results.csv # Output results -├── 📦 requirements.txt # Python dependencies -└── 📜 .gitignore # Git ignore file -``` - -## 🔐 Security & Best Practices - -### Environment Security -- ✅ **Never commit `.env` files** - Add to `.gitignore` -- ✅ **Use environment variables** for all API keys -- ✅ **Rotate API keys regularly** -- ✅ **Monitor API usage** and set billing alerts -- ✅ **Use provider-specific security features** (API key restrictions, rate limits) - -### Code Safety -- ✅ **Review generated code** before execution -- ✅ **Test with safe URLs** first -- ✅ **Use virtual environments** to isolate dependencies -- ✅ **Validate input files** before processing - - - -## 🤖 LLM Integration Details - -### Model Configuration Examples - -#### OpenAI GPT-4 -```python -model="gpt-4" -temperature=0.2 # Low for consistent code -max_tokens=8000 # Balanced for complete scripts -``` - -#### Claude 3 Opus -```python -model="claude-3-opus-20240229" -temperature=0.2 -max_tokens=4000 -``` - -#### Gemini Pro -```python -model="gemini-pro" -temperature=0.2 -max_tokens=8192 -``` - -#### Local Models (Ollama) -```python -model="codellama:13b" -temperature=0.1 # Very low for code consistency -max_tokens=4096 -``` - -### Prompt Engineering -The project uses carefully crafted prompts optimized for each provider to ensure: -- 🎯 **Focused output** - Only relevant code generation -- 🔧 **Practical functionality** - Real-world utility -- 📚 **Documentation** - Self-explaining code -- ⚡ **Efficiency** - Optimized performance -- 🔄 **Provider compatibility** - Works across different LLMs - -## 📈 Use Cases - -### DevOps & Monitoring -- **Website uptime monitoring** -- **API endpoint health checks** -- **Load balancer status verification** -- **CDN performance testing** -- **Multi-region availability checking** - -### Development & Testing -- **Integration test automation** -- **Environment validation** -- **Dependency verification** -- **Service discovery** -- **CI/CD pipeline validation** - - -## 🤝 Contributing - -We welcome contributions! Here's how to get started: - -### 1. Fork & Clone -```bash -# Fork the repository on GitHub first, then: -git clone https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator.git -``` - -### 2. Create Feature Branch -```bash -git checkout -b feature/amazing-new-feature -``` -### 3. Make Changes -- Add support for new LLM providers -- Improve prompt engineering -- Add new tool generation capabilities -- Update documentation -- Add tests if applicable - -### 4. Submit Pull Request -```bash -git add . -git commit -m "✨ Add amazing new feature" -git push origin feature/amazing-new-feature -``` - - -## 🙏 Acknowledgments - -- **LLM Providers** - OpenAI, Anthropic, Google, and the open-source community -- **Python Community** - For excellent libraries (`requests`, `python-dotenv`) -- **Jupyter Project** - For the interactive development environment -- **Open Source Models** - LLaMA, Mistral, and other contributors -- **Contributors** - Everyone who helps improve this project - -## 📞 Support & Community - -### Getting Help -- 🐛 **Bug Reports**: [GitHub Issues](https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator/issues) -- 💡 **Feature Requests**: [GitHub Discussions](https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator/discussions) -- 📖 **Documentation**: This README and notebook comments -- 💬 **Community**: Join our discussions for tips and tricks - -### Troubleshooting -| Problem | Solution | -|---------|----------| -| `ModuleNotFoundError` | Run `pip install -r requirements.txt` | -| `API Key Error` | Check your `.env` file and provider configuration | -| `Timeout Issues` | Increase `--timeout` parameter or switch providers | -| `Generation Fails` | Verify API credits, permissions, and model availability | -| `Provider Not Working` | Check API endpoint, model name, and authentication | - ---- - -
- -**🌟 Star this repository if you find it useful!** - -**Made with ❤️ and AI** | *Supporting ALL Major LLM Providers* - -[![GitHub stars](https://img.shields.io/github/stars/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator.svg?style=social)](https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator/stargazers) -[![GitHub forks](https://img.shields.io/github/forks/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator.svg?style=social)](https://github.com/Mahmoud-Zaafan/LLM-URL-Health-Checker-Generator/network) - -
\ No newline at end of file