Available Tools

All Converra MCP tools with examples and common use cases.

Prompts

list_prompts

List all your prompts.

You say: "Show me my prompts"

Response:

Found 3 prompts:
- Customer Support (gpt-4o) - Active
- Sales Assistant (gpt-4o) - Active
- Code Review (claude-3.5-sonnet) - Draft

create_prompt

Create a new prompt. Requires: name, content, llmModel.

You say: "Create a customer support prompt for gpt-4o"

Example with full content:

Create a prompt with:
- name: "Customer Support Agent"
- llmModel: "gpt-4o"
- content: "You are a helpful customer support agent. Be friendly,
  concise, and always try to resolve issues on first contact."
- description: "Main support chatbot"
- tags: ["support", "production"]

Supported models (examples): gpt-4o, gpt-4.1, gpt-4o-mini, gpt-o3, o4-mini, o1-mini, claude-3.5-sonnet, claude-sonnet-4, claude-opus-4, gemini-2.5-pro, gemini-2.5-flash

update_prompt

Update an existing prompt.

You say: "Update my support prompt to be more friendly"

Example:

Update prompt abc123:
- content: "You are an exceptionally friendly customer support agent..."

get_prompt_status

Get details about a specific prompt including performance metrics.

You say: "Show details for my support prompt"

Response:

Prompt: Customer Support Agent
Model: gpt-4o
Status: Active
Conversations: 1,247
Last optimized: 3 days ago
Performance: 87% task completion

---

Optimization

trigger_optimization

Start an optimization to improve your prompt.

You say: "Optimize my support prompt with 3 variants"

Full example:

Optimize prompt abc123:
- mode: "exploratory"  (or "validation" for statistical rigor)
- variantCount: 3
- intent:
  - targetImprovements: ["clarity", "task completion"]
  - hypothesis: "Adding examples will help users understand better"

What happens:

1. Converra generates variant prompts
2. Simulates conversations with AI personas
3. Evaluates which variant performs best
4. Reports results with improvement percentages

get_optimization_details

Check progress and results of an optimization.

You say: "How's my optimization going?"

Response (in progress):

Optimization abc123
Status: Running (Iteration 2/5)
Progress: Simulating conversations...
Variants: 3 being tested

Response (complete):

Optimization abc123
Status: Complete
Winner: Variant B
Improvement: +23% task completion, +15% clarity
Recommendation: Apply Variant B

list_optimizations

See recent optimization runs.

You say: "Show my recent optimizations"

Response:

Recent optimizations:
1. Customer Support - Completed 2h ago - Variant B won (+23%)
2. Sales Assistant - Running - Iteration 3/5
3. Code Review - Completed yesterday - No clear winner

get_variant_details

Compare variants from an optimization.

You say: "Show me the variants from my last optimization"

Response:

Variant A (Control):
- Task completion: 72%
- Clarity: 68%

Variant B (Winner):
- Task completion: 89% (+17%)
- Clarity: 83% (+15%)
- Key change: Added step-by-step instructions

Variant C:
- Task completion: 75% (+3%)
- Clarity: 71% (+3%)

apply_variant

Deploy a winning variant to your prompt.

You say: "Apply the winning variant"

Response:

Applied Variant B to "Customer Support Agent"
Previous version saved. You can revert anytime.

stop_optimization

Stop a running optimization.

You say: "Stop the current optimization"

---

Insights

get_insights

Get aggregated performance insights for a prompt based on logged conversations.

You say: "How is my support prompt performing?"

Response:

Insights for Customer Support (last 30 days):
- Task completion: 87%
- Avg sentiment: Positive
- Common topics: order status, refunds, shipping
- Improvement opportunity: Users often confused about return policy

get_conversation_insights

Get detailed insights for a specific conversation.

You say: "Show me the insights for conversation xyz789"

Response:

Conversation: Order Status Inquiry
Success Score: 85%
AI Relevancy: 88%
User Sentiment: 72%

Summary: Customer inquired about order status. AI provided tracking info.
Issues: None
AI Performance: Quick response, accurate information

regenerate_conversation_insights

Re-run insights analysis for a conversation when insights are missing or incorrect.

You say: "Regenerate insights for conversation xyz789"

Response:

Insights regeneration started for conversation xyz789
Existing insights will be refreshed asynchronously.

batch_regenerate_insights

Regenerate insights for all conversations of a prompt. Useful for bulk fixes.

You say: "Regenerate insights for all conversations of my support prompt"

Response:

Batch regeneration initiated:
- Total conversations: 85
- Queued for regeneration: 83
- Errors: 2
Insights will be generated asynchronously.

refresh_prompt_analysis

Re-analyze a prompt's structure (strengths, weaknesses, quality metrics).

You say: "Re-analyze my support prompt"

Response:

Analysis refreshed for Customer Support:
Strengths:
- Clear role definition
- Good tone instructions

Weaknesses:
- Missing product context
- No edge case handling

Opportunities:
- Add example interactions
- Include escalation guidelines

refresh_prompt_insights

Regenerate aggregated prompt insights from conversation data.

You say: "Refresh the insights for my support prompt"

Response:

Insights refreshed for Customer Support:
- Conversations analyzed: 142
- Success patterns: Clear explanations, quick responses
- Failure patterns: Difficulty with technical queries
- Summary: Strong performance with room to improve technical support

---

Conversations

list_conversations

List logged conversations for a prompt.

You say: "Show recent conversations for my support prompt"

get_conversation

Get details of a specific conversation including insights.

You say: "Show me conversation xyz789"

create_conversation

Log a conversation for analysis.

Example:

Log conversation:
- promptId: "abc123"
- content: "User: I need help with my order\nAI: Happy to help! What's your order number?"
- status: "completed"

---

Personas

list_personas

List simulation personas for testing.

You say: "What personas are available for testing?"

Response:

Available personas:
- Frustrated Customer (impatient, had bad experiences)
- Enterprise Buyer (technical, detail-oriented)
- First-time User (needs guidance, asks basic questions)
- Power User (efficient, knows what they want)

create_persona

Create a custom persona for simulations.

Example:

Create persona:
- name: "Confused Senior"
- description: "An elderly user unfamiliar with technology,
  needs patient explanations, may ask the same thing twice"
- tags: ["senior", "patience-test"]

---

Simulation

simulate_prompt

Test your prompt against personas without optimization.

You say: "Test my support prompt against 5 personas"

Response:

Simulation complete:
- 5 conversations generated
- Avg task completion: 78%
- Issues found: Struggled with technical users
- Recommendation: Add more technical details

analyze_prompt

Get structural analysis and improvement recommendations.

You say: "Analyze my support prompt for weaknesses"

Response:

Analysis of Customer Support:
Strengths:
- Clear role definition
- Good tone instructions

Weaknesses:
- No examples provided
- Missing edge case handling
- Could be more concise

Recommendations:
1. Add 2-3 example interactions
2. Add instructions for handling complaints
3. Remove redundant phrases

simulate_ab_test

Run A/B simulation test comparing two prompts. Executes multi-turn simulated conversations for both prompts against identical personas and scenarios, then compares performance to determine which is better.

Note: This tool was previously named run_head_to_head.

You say: "Compare my old and new support prompts with simulations"

Response:

A/B Simulation Results:
Baseline vs Variant across 9 conversations

Recommendation: variant
Lift: +12.3 successScore

Comparison:
- Variant wins: 6
- Baseline wins: 2
- Ties: 1

Evidence level: high

regression_test

Test a variant prompt against a prompt's golden test suite to ensure it doesn't break existing functionality. Uses pre-validated scenarios with known success criteria.

You say: "Run regression test on my new support prompt"

Response:

Regression Test: PASSED

Summary:
- Total scenarios: 8
- Passed: 8
- Regressed: 0
- Pass rate: 100%

All golden scenarios passed. Safe to deploy.

---

Account

get_account

Get account info and usage.

You say: "What's my Converra usage?"

get_settings

Get optimization settings.

update_settings

Update default settings.

---

Integrations

list_integrations

List all configured integrations and their status.

You say: "Show me my integrations"

Response:

✓ LANGSMITH
  Project: My Production Bot
  Last import: Jan 3, 2:30 PM (47 imported)
  Available projects:
    - My Production Bot (proj_abc) ← selected
    - Development (proj_def)
    - Testing (proj_ghi)

✓ LANGFUSE
  Region: US
  Last import: Jan 2, 8:00 AM (23 imported)

sync_conversations

Sync conversations from LangSmith or Langfuse. Optionally configure auto-sync.

You say: "Sync my LangSmith conversations"

Example with options:

Sync conversations from langsmith:
- lookbackDays: 30
- maxTraces: 500
- enableAutoSync: true
- syncIntervalMinutes: 1440  (daily)

Response:

✓ Import successful

Conversations imported: 47
Prompts created: 3
Prompts reused: 2
Skipped: 12 (8 single-turn)
Auto-sync enabled (daily).

Parameters:

Parameter	Options	Description
source	langsmith, langfuse	Required. Integration to sync from
lookbackDays	7, 30, 90	Days to look back (default: 30)
maxTraces	1-2000	Max traces to sync (default: 500)
enableAutoSync	true/false	Enable automatic sync
syncIntervalMinutes	60, 360, 720, 1440	Hourly, 6h, 12h, or daily

---

Webhooks

list_webhooks

List configured webhooks.

create_webhook

Create a webhook for events.

Example:

Create webhook:
- url: "https://myapp.com/converra-webhook"
- events: ["optimization.completed", "prompt.updated", "regression_test.completed"]

delete_webhook

Remove a webhook.

---

Common Workflows

Bring Your Prompt

1. "Here's my prompt that isn't working well: [paste prompt]"
2. "What's wrong with it?"
3. "Test it against difficult users"
4. "What changed in the winning version?"
5. "Apply it"

Analyze Conversation Logs

1. "Here are some conversations from my bot: [paste logs]"
2. "What patterns do you see?"
3. "Optimize the prompt based on these issues"
4. "Apply the improvement"

Test Before Going Live

1. "Here's a new prompt I'm considering: [paste prompt]"
2. "Simulate against frustrated and confused users"
3. "What issues were found?"
4. "Fix those and test again"