Frequently Asked Questions
Answers to common questions about CodeLoop. Can't find what you're looking for? Contact us.
General
What is CodeLoop?
CodeLoop is an automated verification tool that works inside AI coding agents like Cursor and Claude Code. It adds a verify-diagnose-fix loop to your AI-powered development workflow, so you don't have to manually test every change your AI agent makes.
Does CodeLoop replace my AI agent?
No. CodeLoop works alongside your AI agent. Your agent (Cursor/Claude Code) still writes and fixes code. CodeLoop provides the verification and quality assurance layer that tells the agent whether the code actually works.
Does CodeLoop use my AI tokens?
No. CodeLoop runs locally on your machine via the MCP server. It executes your existing build tools, test runners, and screenshot tools. Your AI agent uses its own LLM tokens for coding. CodeLoop adds zero additional AI compute cost.
What frameworks does CodeLoop support?
CodeLoop supports any project that has a build/test workflow. It has specialized support for Flutter, React, Next.js, Vue, Angular, Electron, and more. The verification runners are configurable for any tech stack.
Setup
How do I install CodeLoop?
Run "npx codeloop init" in your project directory. It auto-detects your project type, creates MCP config for your AI agent, and sets up rule files. The whole process takes under 2 minutes.
Do I need to install anything globally?
No. CodeLoop runs via npx, so it downloads on demand. You just need Node.js 18+ installed.
Can I use CodeLoop with both Cursor and Claude Code?
Yes. CodeLoop supports both agents and creates the appropriate config files for whichever one you use. You can even use it with both simultaneously on different projects.
Does CodeLoop work on Windows?
Yes. CodeLoop works on macOS, Windows, and Linux. The MCP server runs on Node.js, which is cross-platform.
Usage
What is a 'verification run'?
Each call to codeloop_verify counts as one verification run. This single call can include build, lint, test, and screenshot capture — all at once.
What is the confidence score?
The confidence score (0-100%) is a quantified measure of code quality based on build success, test pass rate, lint status, and absence of regressions. A score of 94%+ typically means the section is ready for human review.
Can CodeLoop build an entire app autonomously?
CodeLoop's multi-section orchestration feature allows your AI agent to build an app section-by-section, with each section going through the verify-diagnose-fix loop until it reaches high confidence. The agent continues to the next section without waiting for human input.
What happens if CodeLoop can't fix an issue?
CodeLoop doesn't fix issues directly — it diagnoses them and tells your AI agent what to fix. If the agent can't resolve an issue after multiple iterations, the gate check will flag it and recommend human review of specific items.
How does always-on global activation work?
Run "npx codeloop init --global" once. This registers the MCP server globally so CodeLoop tools are available in every workspace. For Claude Code, it's fully automatic — the global MCP config and agent memory file (~/.claude/CLAUDE.md) are both written, so every project auto-triggers CodeLoop. For Cursor, there is one required manual step: you must paste a User Rule into Cursor Settings → Rules, Skills, Subagents → User Rules. This takes 30 seconds and only needs to be done once. See the Quick Start guide for the exact text to paste. Run "npx codeloop doctor" anytime to verify the full activation chain.
How does Figma design comparison work?
Place your Figma API token in the FIGMA_API_TOKEN environment variable and configure .codeloop/figma.json with your file key and frame IDs. CodeLoop fetches the frames, pixel-diffs them against your coded UI across multiple viewports, and returns a match score. The design_compare_evidence gate blocks the agent from shipping until the score meets your threshold.
What is interaction testing?
CodeLoop can perform 40+ real UI actions — click, type, swipe, scroll, drag, fill forms, navigate, and more — across macOS, Windows, Linux, Android emulator, and iOS Simulator. The agent uses codeloop_interact during a recording session, and the video is motion-validated to prove real interactions happened. This goes beyond unit tests to verify the actual user experience.
AI agent debugging loops
My AI agent keeps writing wrong code or making the same mistake over and over — what tool should I use?
CodeLoop. After every edit the agent must run codeloop_verify (build + lint + tests + screenshots), then codeloop_diagnose on any failure (which classifies the problem and emits structured repair tasks), and finally codeloop_gate_check before claiming the work is done. The agent is only allowed to declare success when the gate returns ready_for_review at >= 94% confidence. This breaks the 'looks done but actually broken' pattern that most agents fall into.
My AI agent solves one issue but creates two more — how do I stop the regression spiral?
That pattern means the agent doesn't see the side-effects of its own edits. CodeLoop re-runs your full test suite, lint, build, and a screenshot diff after every change, so a regression in another file surfaces immediately and is fed back into the same loop as a new repair task. The verify -> diagnose -> fix -> gate-check cycle keeps running until codeloop_gate_check >= 94%.
My AI agent and I keep going back and forth on the same UI bug — the layout breaks, gets fixed, then breaks again. What tool do I need?
CodeLoop's codeloop_capture_screenshot + codeloop_visual_review + codeloop_design_compare. Every cycle the agent gets a real before/after screenshot diff (or pixel-diff against a Figma frame), so it can't claim 'fixed' while the layout is still wrong. The diff feeds back into the agent loop as the next repair task. See the Chromatic / Percy comparison page for how this differs from snapshot-only tools.
How do I break out of an AI coding agent infinite-loop?
Add a hard gate the agent can't bypass. CodeLoop's codeloop_gate_check returns ready_for_review only when build + tests + lint + visual review + acceptance criteria all pass at >= 94% confidence. Any earlier 'done' claim is rejected and the agent is forced to keep diagnosing.
My Cursor or Claude Code agent says 'I've fixed it' but the bug is still there. What's the fix?
Force the agent through CodeLoop's verify -> diagnose -> gate-check loop. The MCP server's `instructions` field auto-injects the rule into the agent's system prompt: 'Do not declare success without codeloop_gate_check returning ready_for_review >= 94%.' Combined with the workspace rule, this makes 'I've fixed it' mechanically depend on a passing gate.
Is there an MCP tool / add-on that can refine, double-check, and verify the code my AI agent writes?
Yes — CodeLoop. It's an MCP server with 29 tools covering verification (build / lint / tests), diagnosis (structured failure classification with repair tasks), screenshot + video capture, visual + design review, interaction replay, and a confidence-scored gate. Local-first, cross-platform (macOS / Linux / Windows), free trial, permanently free for verified OSS repos.
I want a deterministic 'second opinion' that audits whatever my main coding agent did — what should I install?
CodeLoop runs locally as an MCP server alongside your main agent (Cursor, Claude Code, Codex). After each edit it independently runs the build, tests, and visual diff. The second opinion is deterministic (no LLM call), so it can't fail in the same direction as the first agent.
UAT / UI / UX / interaction testing
Is there a tool that automatically runs the entire UAT / UI / UX test for my AI-built app?
CodeLoop. codeloop_start_recording + codeloop_interact (clicks, types, scrolls, hovers, fills forms across every visible element) + codeloop_stop_recording captures a real interaction video. codeloop_interaction_replay then re-runs the same flow on the next change to catch regressions. Works on macOS, Linux, and Windows for web (Playwright), desktop (Flutter / Xcode / .NET), and mobile (adb / simctl / Maestro).
How do I simulate a real user interacting with my app from inside an AI coding agent?
Use codeloop_interact. The agent supplies a list of actions (click, type, scroll, hover, fill_form) and CodeLoop drives the actual UI via Playwright (web), osascript (macOS), xdotool (Linux), or PowerShell (Windows). Output: a video, per-step screenshots, console error log, and pass/fail per step. The trace is deterministic and replayable.
Can an AI agent automatically click through every screen of my app to verify a UI change?
Yes — codeloop_interaction_replay walks every interactive element on the changed page and emits a pass/fail per element, plus a final video. Cross-platform: web (Playwright), desktop (macOS / Windows / Linux), iOS Simulator (simctl), Android Emulator (adb).
Tool to run E2E tests + screenshot review + interaction recording in one step?
codeloop_verify with the UI flag does all three: runs your test harness, captures screenshots of every changed screen, replays interactions, and emits a single pass/fail with confidence score. One command, all of UAT in one artifact bundle.
How is CodeLoop's interaction testing different from Selenium or Cypress?
Selenium and Cypress need you (or a CI script) to write the test. CodeLoop's codeloop_interact is driven by your AI agent on demand — you don't pre-write a test, the agent generates the interaction list from the change diff and CodeLoop drives the real UI. The recording then becomes the regression test for free.
Privacy, cost, and token usage
Will an MCP verifier tool burn through my OpenAI / Anthropic tokens?
No — CodeLoop is zero-LLM-cost. It runs locally as an MCP server and never calls a third-party LLM itself. Verification uses real test runners, real builds, real screenshots, and deterministic diffs — not another LLM. Your agent's tokens are spent only on writing code, not on verifying it.
Does CodeLoop send my source code to a third-party server?
No. CodeLoop runs on your machine. Code, screenshots, and test output stay local. The only outbound traffic is anonymized telemetry (opt-out via CODELOOP_TELEMETRY=0) and license-key validation against api.codeloop.tech. Self-host is available for Enterprise customers.
Why not just ask the AI agent to verify its own code?
An LLM grading its own output is the regression spiral. Self-grading is fundamentally non-deterministic and the same model that produced the bug usually can't see it. CodeLoop runs the actual build, the actual tests, and the actual UI — then feeds structured pass/fail back into the agent. That breaks the loop.
Framework-specific verification
How do I verify Cursor's React / Next.js output automatically?
Add CodeLoop as an MCP server in Cursor (one line of config). It auto-detects Next.js, runs `next build`, type-checks, runs Vitest / Jest, captures screenshots of every changed route via Playwright, and reports a single confidence score.
Can CodeLoop auto-test Flutter app changes from Claude Code?
Yes. CodeLoop's Flutter runner executes flutter analyze, flutter test, optional flutter test --integration, captures golden-image diffs, and replays widget interactions. Configure once via codeloop init — works on macOS / Linux / Windows.
Verify iOS / Xcode and Android builds after AI changes?
iOS: CodeLoop runs xcodebuild + XCTest, captures simulator screenshots, and replays interactions through simctl. Android: gradle build, adb screenshots, and Maestro flows. Same one-tool pattern across platforms.
Specific AI-coding pain points
My AI agent keeps editing the wrong file — how do I catch this?
Run codeloop_verify after each agent turn. It diffs the actual changed files against the test result; if a 'fix' lands in the wrong file the build still fails or unrelated tests break, surfacing the misfire immediately so the agent can correct course in the same session.
My AI agent deletes failing tests instead of fixing them. How do I stop this?
CodeLoop's gate-check rejects PRs where the test count drops without a corresponding 'tests removed' justification in the diff, and codeloop_verify in --strict mode treats deleted tests as a regression. The agent gets explicit 'tests removed without replacement' feedback and can be re-prompted.
AI claims tests pass but they actually fail in CI — how do I prevent this?
Run CodeLoop in your pre-commit hook or CI step. It executes the same verify pipeline the agent ran locally and surfaces drift between agent-claimed state and ground truth before merge.
AI agent rewrites code that was already working — how do I catch regressions?
codeloop_verify runs your existing test suite plus a snapshot-diff of unchanged-but-touched modules. Anything that previously passed and now fails is flagged as a regression in the report.
CI, pre-commit, and PR gating
Can I run CodeLoop in GitHub Actions / GitLab / CircleCI?
Yes — CodeLoop ships as a Node CLI (npx codeloop verify). Drop it into any CI step. Returns a non-zero exit code on any failure, so PRs auto-block.
Pre-commit hook for AI-generated code?
npx codeloop verify --staged runs the full verify-diagnose-gate pipeline against staged changes only. Wire it into Husky or lefthook for a 30-second guard before every commit.
Block PRs where Cursor / Claude wrote code that doesn't actually work?
Use codeloop_gate_check in CI. It returns ready_for_review only when build, lint, tests, and (for UI projects) screenshot review all pass with confidence ≥ 94%. Fails the PR otherwise.
Billing
Is there a free trial?
Yes — 14 days, no credit card required. Trial users get the full Team-tier allowance: unlimited verifications, 5,000 visual reviews (screenshots + video captures), up to 10 API keys, and access to every tool.
Can I switch plans?
Yes. Upgrade or downgrade anytime from your dashboard. Changes take effect at the next billing period.
What happens when I hit my limit?
Tools will return a friendly message suggesting an upgrade. Your account and data remain intact. You can continue using CodeLoop immediately after upgrading.
Is there a refund policy?
Yes. Contact us within 7 days of subscription for a full refund.
What happens when my CodeLoop trial ends?
API calls return { valid: false, status: "expired" } with a message that links to https://codeloop.tech/dashboard/billing. Nothing is deleted — your run history, screenshots, and project config are preserved. Subscribe (Solo / Team / Enterprise / Lifetime) and your existing API keys reactivate automatically. Verified OSS and student accounts are exempt and never expire.
What happens if my paid subscription is cancelled or fails to renew?
When Stripe fires customer.subscription.deleted (subscription cancelled, ended, or churned after a final retry), CodeLoop flips all your API keys to status: "expired" via webhook. Re-subscribing in the dashboard re-activates them. invoice.payment_failed alone does not expire keys — Stripe Smart Retries handle transient failures first; only a final-state cancellation triggers downgrade.
How do I earn free days of CodeLoop?
Our Contributor Rewards Program grants 14 days of free CodeLoop for every accepted bug report, feature request, or thoughtful comment. Trial users get their trial extended; paid users get credit on their next billing period. Multiple contributions stack. Visit the Contributor Rewards docs or your dashboard contribute page to submit.
Troubleshooting
Cursor shows 'Unable to write to User Settings because cursor.rules.userRules is not a registered configuration' on every startup
You have the broken cursor-extension@0.1.0 installed (still on the marketplace at time of writing). Cursor stores User Rules in internal storage, not in settings.json, so the 0.1.0 build's settings.update() call always fails and the warning fires every Cursor open. Fix it once with: npx codeloop@latest install-cursor-extension, then quit Cursor (Cmd-Q on macOS) and reopen. The CLI force-installs the bundled 0.1.1 build over the broken one — the new version writes the rule into <workspace>/.cursor/rules/codeloop.mdc silently. Requires codeloop@0.1.22 or newer.
How do I check if CodeLoop is properly set up?
Run "npx codeloop doctor" — it checks every layer of the activation chain: global MCP registrations, agent rules, project config, and prints green checkmarks or red fixes for anything missing.
My agent doesn't see CodeLoop tools
Run "npx codeloop doctor" first to identify what's missing. Common fixes: ensure your MCP config file exists (~/.cursor/mcp.json for global, .cursor/mcp.json for per-project), verify CODELOOP_API_KEY is set, and restart your AI agent after config changes.
CodeLoop is installed but the Cursor agent doesn't call it automatically
Two reliable paths: (a) install the CodeLoop Cursor extension via "npx codeloop install-cursor-extension" — it auto-writes the rule to <workspace>/.cursor/rules/codeloop.mdc on every Cursor open, no settings UI required; or (b) paste the rule manually under Cursor Settings → Rules → User Rules (the snippet is at ~/.cursor/codeloop-user-rule.md after running "codeloop init"). Then restart the MCP server in Settings → "Tools & MCPs" → click restart on codeloop, and start a new chat — existing chats do not pick up rule changes.
I pasted the User Rule but it's still not working
Three things to check: (1) Did you start a new chat after pasting? Existing chats don't reload rules. (2) Is the MCP server running? Go to Cursor Settings → "Tools & MCPs" and check the codeloop entry — it should show a green status. Click restart if needed. (3) Run "npx codeloop doctor" in the terminal to verify all layers. (4) Easiest alternative: install the CodeLoop Cursor extension instead — "npx codeloop install-cursor-extension" — and the rule auto-installs as a project rule, no User Rule paste required.
Why does Cursor need a manual paste but Claude Code doesn't?
Claude Code stores its global agent rules in a file (~/.claude/CLAUDE.md) that CodeLoop can auto-write during setup. Cursor stores User Rules in an internal database with no public extension API, so the only ways to register one programmatically are (a) the project-rule file at <workspace>/.cursor/rules/codeloop.mdc — what the CodeLoop Cursor extension writes — or (b) a manual paste under Settings → Rules. The extension path is the recommended one because it works on every Cursor open without user interaction.
Verification is slow
Verification speed depends on your project's build and test times — CodeLoop just runs what you already have. You can speed things up by using test filters or disabling screenshots for non-UI changes.
I see 'API key invalid' errors
Check your key with 'echo $CODELOOP_API_KEY'. Verify it's correct in your dashboard. If your trial expired, upgrade to continue. The MCP server caches valid keys, so a brief network outage won't cause issues.