🚀 -> Project on GitHub <-

Cloud LLM Integration Guide

CrawLlama now supports cloud-based LLM providers in addition to local Ollama models.

Supported Providers | Provider | Models | Speed | Cost | API Key Required |

|———-|——–|——-|——|——————| | Ollama (default) | Qwen, Llama, Mistral, etc. | Fast (local) | Free | No | | OpenAI | GPT-3.5, GPT-4, GPT-4-Turbo | Fast | Paid | Yes | | Anthropic | Claude 3 (Opus, Sonnet, Haiku) | Fast | Paid | Yes | | Groq | Mixtral, LLaMA 2, Gemma | Very Fast | Free tier available | Yes |

Quick Start

1. Install Cloud Provider Libraries

Uncomment the desired providers in requirements.txt:

# For OpenAI
pip install openai>=1.54.0

# For Anthropic
pip install anthropic>=0.40.0

# For Groq
pip install groq>=0.15.0

# Or install all at once
pip install openai anthropic groq

2. Add API Keys to .env

Copy .env.example to .env and add your API keys:

# OpenAI
OPENAI_API_KEY=sk-proj-your_key_here

# Anthropic
ANTHROPIC_API_KEY=sk-ant-your_key_here

# Groq
GROQ_API_KEY=gsk_your_key_here

3. Configure Provider in config.json

Edit the llm section in config.json:

{
 "llm": {
 "provider": "openai",
 "model": "gpt-4-turbo",
 "temperature": 0.7,
 "max_tokens": 4096
 }
}

Configuration Examples

OpenAI GPT Models

{
 "llm": {
 "provider": "openai",
 "model": "gpt-4-turbo",
 "temperature": 0.7,
 "max_tokens": 4096
 }
}

Available Models:

Pricing: See OpenAI Pricing

Anthropic Claude

{
 "llm": {
 "provider": "anthropic",
 "model": "claude-3-sonnet-20240229",
 "temperature": 0.7,
 "max_tokens": 4096
 }
}

Available Models:

Pricing: See Anthropic Pricing

Groq (Fast Inference)

{
 "llm": {
 "provider": "groq",
 "model": "mixtral-8x7b-32768",
 "temperature": 0.7,
 "max_tokens": 4096
 }
}

Available Models:

Pricing: Free tier available! See Groq Console

Local Ollama (Default)

{
 "llm": {
 "provider": "ollama",
 "base_url": "http://127.0.0.1:11434",
 "model": "qwen2.5:3b",
 "temperature": 0.7,
 "max_tokens": 4096
 }
}

No API key required - runs 100% locally.

Programmatic Usage

Using the Factory Function

from core.cloud_llm_client import get_llm_client

# Get OpenAI client
client = get_llm_client("openai", model="gpt-4")
response = client.generate("What is the capital of France?")
print(response)

# Get Anthropic client
client = get_llm_client("anthropic", model="claude-3-sonnet-20240229")
response = client.chat([
 {"role": "user", "content": "Hello, Claude!"}
])
print(response)

# Get Groq client
client = get_llm_client("groq", model="mixtral-8x7b-32768")
response = client.generate("Explain quantum computing")
print(response)

Direct Client Initialization

from core.cloud_llm_client import OpenAIClient, AnthropicClient, GroqClient

# OpenAI
openai_client = OpenAIClient(
 api_key="sk-proj-...", # Or from .env
 model="gpt-4",
 temperature=0.7
)

# Anthropic
anthropic_client = AnthropicClient(
 api_key="sk-ant-...",
 model="claude-3-opus-20240229",
 temperature=0.7
)

# Groq
groq_client = GroqClient(
 api_key="gsk_...",
 model="mixtral-8x7b-32768",
 temperature=0.7
)

Advanced Configuration

Temperature Settings

Controls randomness in responses:

{
 "llm": {
 "temperature": 0.0 // Deterministic (0.0) to Creative (2.0)
 }
}

Token Limits

{
 "llm": {
 "max_tokens": 4096 // Maximum response length
 }
}

Note: Different models have different context windows:

Testing

Run tests for cloud LLM clients:

# Run all cloud LLM tests
python -m pytest tests/unit/test_cloud_llm_client.py -v

# Run specific provider tests
python -m pytest tests/unit/test_cloud_llm_client.py::TestOpenAIClient -v
python -m pytest tests/unit/test_cloud_llm_client.py::TestAnthropicClient -v
python -m pytest tests/unit/test_cloud_llm_client.py::TestGroqClient -v

Security Best Practices

  1. Never commit .env to Git 
     # .gitignore already includes .env
 echo ".env" >> .gitignore
  2. Use environment variables in production 
     export OPENAI_API_KEY="sk-proj-..."
 export ANTHROPIC_API_KEY="sk-ant-..."
 export GROQ_API_KEY="gsk_..."
  3. Rotate API keys regularly
    • OpenAI: https://platform.openai.com/api-keys
    • Anthropic: https://console.anthropic.com/settings/keys
    • Groq: https://console.groq.com/keys
  4. Monitor API usage and costs
    • Set spending limits in provider dashboards
    • Enable usage alerts

Performance Comparison | Provider | Avg Response Time | Cost (1M tokens) | Context Window |

|———-|——————-|——————|—————-| | Ollama | 1-5s (local) | Free | 4K-128K (model dependent) | | OpenAI GPT-4 | 2-10s | $30 (input) / $60 (output) | 8K-128K | | Anthropic Claude 3 | 2-8s | $15 (Sonnet) | 200K | | Groq Mixtral | 0.5-2s | Free tier | 32K |

Recommendation:

Troubleshooting

“API key not found” error

# Check if .env file exists and contains the key
cat .env | grep API_KEY

# Ensure .env is loaded
python -c "from dotenv import load_dotenv; load_dotenv(); import os; print(os.getenv('OPENAI_API_KEY'))"

“Module not found” error

# Install missing provider library
pip install openai anthropic groq

“Invalid API key” error

  1. Verify key format:
    • OpenAI: sk-proj-...
    • Anthropic: sk-ant-...
    • Groq: gsk_...

  2. Check key validity in provider dashboard

  3. Ensure no extra spaces/quotes in .env: ```bash # BAD OPENAI_API_KEY=”sk-proj-…”

# GOOD OPENAI_API_KEY=sk-proj-… ```

“Rate limit exceeded” error

  1. OpenAI: Upgrade to paid tier or wait
  2. Anthropic: Check usage limits in console
  3. Groq: Free tier has limits, consider paid tier

Additional Resources

Contributing

Found a bug or want to add support for more providers? See CONTRIBUTING.md

Potential future providers: