Real-Time llmswap Demo Session
Archived illustrative transcript. Model IDs, prices, and generated output reflect an older release and are not current documentation. See the 5.7.0 docs for supported workflows and current defaults.
Developer Workflow: Multi-Provider AI for DevOps
$ llmswap providers
╭─────────────┬──────────┬──────────────────────┬─────────────────────╮
│ Provider │ Status │ Model │ Issue │
├─────────────┼──────────┼──────────────────────┼─────────────────────┤
│ ANTHROPIC │ ✓ Ready │ claude-3-5-sonnet │ │
│ OPENAI │ ✓ Ready │ gpt-4o │ │
│ GEMINI │ ✓ Ready │ gemini-1.5-pro │ │
│ GROQ │ ✓ Ready │ llama-3.3-70b │ │
│ OLLAMA │ ✓ Ready │ llama3.1 │ │
╰─────────────┴──────────┴──────────────────────┴─────────────────────╯
$ llmswap chat
🤖 Starting chat with claude-3-5-sonnet (Tokens: 0, Cost: $0.00)
You: I'm debugging a Kubernetes deployment that's failing. The pods keep restarting.
Claude: I can help you troubleshoot the pod restart issue. Let's start by gathering some information:
1. First, let's check the pod status and recent events:
```bash
kubectl get pods -o wide
kubectl describe pod <pod-name>
kubectl logs <pod-name> --previous
- Common causes for pod restarts include:
- Memory/CPU limits exceeded (OOMKilled)
- Failed health checks
- Application crashes
- Configuration issues
What’s the current status when you run kubectl get pods? Are you seeing any specific error messages?
You: Here’s what I see:
NAME READY STATUS RESTARTS AGE
web-app-7d4b8c6f9-x2k8r 0/1 CrashLoopBackOff 5 8m
The logs show: “Error: ENOENT: no such file or directory, open ‘/app/config.json’”
Claude: Perfect! This is a common issue. The error indicates your application is looking for /app/config.json but it doesn’t exist in the container. Here are a few solutions:
Option 1: ConfigMap approach
# Create a ConfigMap with your config
kubectl create configmap app-config --from-file=config.json=./local-config.json
# Then mount it in your deployment
Option 2: Check your Docker build Make sure your Dockerfile includes:
COPY config.json /app/config.json
Which approach fits your setup better? Do you have the config file locally, or should we create it as a ConfigMap?
You: /switch groq
🔄 Switched to groq (llama-3.3-70b-versatile) Tokens: 289 → 289 | Cost: $0.00 → $0.00
You: Actually, let me try the ConfigMap approach. How do I mount it properly?
Groq: Here’s how to mount the ConfigMap properly in your deployment:
1. Create the ConfigMap:
kubectl create configmap app-config --from-literal=config.json='{"port": 3000, "env": "production"}'
2. Update your deployment YAML:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
spec:
template:
spec:
containers:
- name: web-app
image: your-image:latest
volumeMounts:
- name: config-volume
mountPath: /app/config.json
subPath: config.json
volumes:
- name: config-volume
configMap:
name: app-config
3. Apply the changes:
kubectl apply -f deployment.yaml
kubectl rollout restart deployment/web-app
The key is using subPath: config.json to mount just the file, not the entire directory. This prevents overwriting your /app folder.
Want me to help you customize this for your specific config needs?
You: Perfect! Let me try this…
$ kubectl create configmap app-config --from-file=config.json
configmap/app-config created
$ kubectl patch deployment web-app -p '{"spec":{"template":{"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"app-config"}}],"containers":[{"name":"web-app","volumeMounts":[{"name":"config-volume","mountPath":"/app/config.json","subPath":"config.json"}]}]}}}}'
deployment.apps/web-app patched
$ kubectl get pods -w
NAME READY STATUS RESTARTS AGE
web-app-85f7d6c4b6-m9x2k 1/1 Running 0 30s
It’s working! The pod is now running successfully.
Groq: Excellent work! 🎉 Your deployment is now healthy. The ConfigMap approach solved the missing config file issue perfectly.
Quick validation steps:
# Verify the config is mounted correctly
kubectl exec web-app-85f7d6c4b6-m9x2k -- cat /app/config.json
# Check application logs to ensure it's starting properly
kubectl logs web-app-85f7d6c4b6-m9x2k
Pro tip: For future deployments, consider using tools like Helm or Kustomize to manage ConfigMaps and volume mounts more systematically. This pattern of “missing file → ConfigMap mount” is super common in Kubernetes.
Want to explore any other aspects of your deployment, or shall we look at monitoring/alerting to catch issues like this earlier?
You: /switch openai
🔄 Switched to openai (gpt-4o) Tokens: 289 → 634 | Cost: $0.00 → $0.02
You: Let’s generate a monitoring alert for this type of issue
OpenAI: Great idea! Here’s a comprehensive monitoring setup to catch similar issues early:
1. Prometheus Alert for CrashLoopBackOff:
# alerts.yaml
groups:
- name: kubernetes-pods
rules:
- alert: PodCrashLoopBackOff
expr: |
kube_pod_container_status_waiting_reason{reason="CrashLoopBackOff"} > 0
for: 5m
labels:
severity: warning
team: platform
annotations:
summary: "Pod / is in CrashLoopBackOff"
description: |
Pod / has been in CrashLoopBackOff for more than 5 minutes.
This usually indicates:
- Missing configuration files
- Application startup failures
- Resource constraints
Check: kubectl logs -n --previous
2. Log-based Alert for Missing Files:
- alert: MissingConfigurationFile
expr: |
increase(log_messages_total{level="error", message=~".*ENOENT.*config.*"}[5m]) > 0
labels:
severity: critical
team: development
annotations:
summary: "Application missing configuration file"
description: "Application is failing due to missing configuration files"
3. Quick kubectl alias for debugging:
# Add to ~/.bashrc or ~/.zshrc
alias k-debug='kubectl get pods --all-namespaces | grep -E "(Error|CrashLoop|ImagePull)" && echo "=== Recent Events ===" && kubectl get events --sort-by=.metadata.creationTimestamp --all-namespaces | tail -10'
Want me to help set up the Prometheus configuration, or would you prefer a different monitoring solution like Datadog/New Relic?
You: /stats
📊 Session Statistics: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Messages: 12 Providers used: claude-3-5-sonnet, llama-3.3-70b-versatile, gpt-4o Total tokens: 1,247 (634 input, 613 output) Total cost: $0.02 Session duration: 15m 32s ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Provider breakdown:
• claude-3-5-sonnet: 289 tokens, $0.00
• llama-3.3-70b-versatile: 345 tokens, $0.00
• gpt-4o: 613 tokens, $0.02
You: This is exactly what I needed! Can you help me generate a shell script that automates the ConfigMap creation for new deployments?
You: /switch anthropic
🔄 Switched to anthropic (claude-3-5-sonnet-20241220) Tokens: 1,247 → 1,247 | Cost: $0.02 → $0.02
Claude: Absolutely! Here’s a comprehensive shell script for automating ConfigMap deployments:
#!/bin/bash
# deploy-with-configmap.sh - Automated ConfigMap deployment script
set -euo pipefail
APP_NAME=${1:-}
CONFIG_FILE=${2:-}
NAMESPACE=${3:-default}
usage() {
echo "Usage: $0 <app-name> <config-file> [namespace]"
echo "Example: $0 web-app ./config.json production"
exit 1
}
[[ -z "$APP_NAME" || -z "$CONFIG_FILE" ]] && usage
echo "🚀 Deploying $APP_NAME with ConfigMap..."
# Validate config file exists
[[ ! -f "$CONFIG_FILE" ]] && echo "❌ Config file not found: $CONFIG_FILE" && exit 1
# Create ConfigMap
echo "📦 Creating ConfigMap: ${APP_NAME}-config"
kubectl create configmap "${APP_NAME}-config" \
--from-file="config.json=$CONFIG_FILE" \
--namespace="$NAMESPACE" \
--dry-run=client -o yaml | kubectl apply -f -
# Update deployment with ConfigMap mount
echo "🔧 Updating deployment with ConfigMap mount..."
kubectl patch deployment "$APP_NAME" \
--namespace="$NAMESPACE" \
--type='json' \
-p='[
{
"op": "add",
"path": "/spec/template/spec/volumes",
"value": [{
"name": "config-volume",
"configMap": {"name": "'${APP_NAME}'-config"}
}]
},
{
"op": "add",
"path": "/spec/template/spec/containers/0/volumeMounts",
"value": [{
"name": "config-volume",
"mountPath": "/app/config.json",
"subPath": "config.json"
}]
}
]'
# Wait for rollout
echo "⏳ Waiting for rollout to complete..."
kubectl rollout status deployment "$APP_NAME" --namespace="$NAMESPACE" --timeout=300s
# Verify deployment
echo "✅ Verifying deployment..."
kubectl get pods -l app="$APP_NAME" --namespace="$NAMESPACE"
echo "🎉 Deployment complete! ConfigMap mounted successfully."
Usage examples:
# Deploy web-app with config.json in default namespace
./deploy-with-configmap.sh web-app ./config.json
# Deploy to specific namespace
./deploy-with-configmap.sh api-server ./api-config.json production
# Make it executable
chmod +x deploy-with-configmap.sh
This script handles error checking, creates the ConfigMap, patches the deployment, and verifies success. Want me to add any other automation features?
You: Perfect! Now let me test the cost comparison feature
$ llmswap compare --input-tokens 2000 --output-tokens 1000
Provider Cost Comparison (2000 input, 1000 output tokens):
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Provider │ Cost │ Model │ Savings vs Most Expensive
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ollama │ $0.0000 │ llama3.1 │ 100.0%
Groq │ $0.0002 │ llama-3.3-70b │ 99.4%
Gemini │ $0.0038 │ gemini-1.5-pro │ 88.1%
Anthropic │ $0.0240 │ claude-3-5-sonnet │ 24.8%
OpenAI │ $0.0320 │ gpt-4o │ 0.0%
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Monthly projection (100 requests/day):
• Ollama: $0.00/month (Local processing)
• Groq: $0.60/month (Lightning fast)
• Gemini: $11.40/month (Google's latest)
• Anthropic: $72.00/month (Reasoning excellence)
• OpenAI: $96.00/month (Industry standard)
💡 Recommendation: Use Groq for speed-critical tasks, Claude for complex reasoning
Claude: Excellent demonstration of the cost comparison feature! This shows exactly why llmswap is so valuable - you can make informed decisions about provider selection based on both cost and performance needs.
A few advanced cost optimization strategies you could implement:
# 1. Set cost budgets per provider
llmswap config set budget.daily.openai 5.00
llmswap config set budget.monthly.anthropic 50.00
# 2. Auto-fallback to cheaper providers
llmswap config set fallback.strategy cost-optimized
llmswap config set fallback.max-cost 0.01
# 3. Smart routing based on query complexity
llmswap ask "What's 2+2?" --auto-route # Routes to Groq (fast + cheap)
llmswap ask "Explain quantum computing" --auto-route # Routes to Claude (complex reasoning)
The monthly projections are particularly useful for budget planning. Teams can now make data-driven decisions about which providers to use for different types of workloads.
You: /usage –days 30
📊 Usage Analytics (Last 30 days): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Provider │ Requests │ Tokens │ Cost │ Avg Response Time │ Success Rate ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ OpenAI │ 142 │ 245,231 │ $18.25 │ 2.3s │ 98.6% Anthropic │ 89 │ 198,452 │ $15.87 │ 3.1s │ 99.1% Gemini │ 203 │ 567,891 │ $12.45 │ 1.8s │ 97.5% Groq │ 445 │ 1,234,567 │ $1.23 │ 0.4s │ 99.8% Ollama │ 67 │ 89,234 │ $0.00 │ 8.2s │ 95.2% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Total │ 946 │ 2,335,375 │ $47.80 │ 2.1s avg │ 98.4%
🎯 Insights: • Groq handling 47% of requests (speed advantage) • Claude highest success rate at 99.1% • Ollama for offline/privacy-sensitive tasks • $47.80 total spend vs $156.80 single-provider equivalent (69% savings)
💰 Cost Optimization Opportunities: • Route simple queries to Groq (potential $8.50/month savings) • Use Gemini for content generation (25% faster than GPT-4) • Consider Ollama for development/testing workloads
You: /quit
👋 Session complete!
📊 Final Statistics: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Messages: 18 Providers used: 4 (claude, groq, openai, anthropic) Total tokens: 3,089 Total cost: $0.08 Session duration: 28m 15s Conversation memory: Maintained across all providers ✓ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
💡 This session demonstrated:
• Multi-provider debugging workflow
• Real-time provider switching
• Cost-aware decision making
• Comprehensive usage analytics
• Persistent conversation context
Thanks for using llmswap! 🚀 ```