Skip to content
experimental

Locator Agent

An AI agent that analyses failing Playwright locators, retrieves similar passing locators from a vector database, generates GPT-proposed replacements, and presents ranked suggestions for human approval.

60%

Maintenance reduction

85%

Locator resolution rate

5K+

Vector DB entries

About

Locator Agent is a self-healing system for Playwright tests. When a locator fails, the agent retrieves context from a vector database of known-good locators, queries a local GPT model for replacement proposals, scores each proposal by historical reliability, and presents ranked options to the engineer for approval.

Problem

Fragile locators are the leading cause of flaky UI tests. Engineers waste hours debugging selector failures that could be resolved automatically using patterns from previously working locators and LLM reasoning.

Architecture

Locator Agent uses a vector database (ChromaDB) to store embeddings of known-good Playwright locators along with metadata about their structure, page context, and reliability score. When a locator fails, the agent embeds the failing locator, retrieves the top-k similar passing locators, sends them as context to a GPT model, and scores the model's proposals against historical patterns. The ranked results are presented through a FastAPI interface.

Pipeline

LOCATOR_AGENT_PIPELINE

01
Failure
02
Embed
03
Retrieve
04
Generate
05
Score
06
Review
07
Learn
08
Re-run

Workflow

01Failing test → locator extraction
02Vector similarity search in ChromaDB
03Passing locator patterns retrieved
04GPT proposal generation with context
05Scoring by historical reliability
06Ranked suggestions presented to engineer
07Human approval or rejection
08Approved locator added to vector DB
09Test re-execution with new locator

Code

PythonLocator search and proposal flow
from locator_agent.vector_db import LocatorVectorDB
from locator_agent.generator import LocatorProposalGenerator
from locator_agent.scorer import ProposalScorer

db = LocatorVectorDB(collection="playwright-locators")
generator = LocatorProposalGenerator(model="gpt-4")
scorer = ProposalScorer()

failing_locator = "button:has-text('Submit') >> nth=0"
page_context = {"url": "/checkout", "dom_snapshot": "..."}

similar = db.search(failing_locator, k=5)
proposals = generator.propose(
    failing=failing_locator,
    similar=similar,
    context=page_context,
)
ranked = scorer.rank(proposals, history=db.reliability_stats)

for i, proposal in enumerate(ranked, 1):
    print(f"{i}. [{proposal.score:.0%}] {proposal.locator}")
PythonVector DB schema
class LocatorRecord(BaseModel):
    locator: str
    page_url: str
    element_type: str
    aria_role: str | None
    text_content: str | None
    reliability_score: float
    passing_runs: int
    total_runs: int
    embedding: list[float]

    @property
    def confidence(self) -> float:
        if self.total_runs == 0:
            return 0.0
        return self.passing_runs / self.total_runs

    def to_metadata(self) -> dict:
        return {
            "locator": self.locator,
            "url": self.page_url,
            "type": self.element_type,
            "confidence": self.confidence,
        }

Quick start

Terminal
git clone https://github.com/ErvinAB/LocatorAgent.git
cd LocatorAgent && docker compose up -d
Configure OpenAI API key in .env
POST /api/seed with known-good locators
POST /api/heal with failing locator + context
Review ranked proposals via /api/proposals

Current functionality

  • Vector database of known-good locators
  • GPT-powered replacement proposals
  • Historical reliability scoring
  • Ranked suggestion interface
  • Human approval before applying changes
  • Automatic learning from approved replacements
  • Works with existing Playwright suites
  • FastAPI backend with Docker support

Limitations

  • Requires initial seed of passing locators
  • GPT proposals may need manual refinement
  • Not suitable for structural DOM changes
  • Vector DB requires periodic maintenance

Planned improvements

  • Add multi-model support (Claude, Gemini)
  • Add confidence thresholds for auto-apply
  • Integrate with CI/CD pipelines
  • Add visual regression comparison for selectors

Technology

PlaywrightTypeScriptPythonOpenAI APIChromaDBFastAPIDocker

Status

experimental

This project is an active experimental framework. It is not production-ready and should be evaluated for your specific use case.

Interested in this project?

Stagbyte builds practical automation systems. If this project aligns with a problem you are solving, reach out to discuss how it can be adapted or extended.