AI News·16 Nov 2026·via Dev Community

    Give your AI agent real Google rankings with a self-hosted MCP server

    A new open-source tool, SERP Maven, enables AI agents like Claude Code and Gemini CLI to access Google's actual search results, complete with specific rankings, AI Overviews, and country-specific context. This capability addresses limitations in standard agent web searches, offering a self-hosted solution for SEO and competitive research tasks.

    Most AI agents that offer "web search" typically provide a curated list of sources, often lacking the granular detail of a full Google search results page. This includes the absence of explicit search positions, Google's AI Overview, "People also ask" sections, or the ability to specify a country for results. For professionals engaged in search engine optimization (SEO) work, rank checks, or competitor analysis, these missing elements are crucial for accurate insights.

    SERP Maven, an open-source Google scraper developed by NodeMaven, aims to bridge this gap. It operates by running a self-hosted Multi-Client Protocol (MCP) server on a user's machine. This server then integrates Google's actual results page directly into various AI agent interfaces, including Claude Code, Cursor, Gemini CLI, and Claude Desktop, functioning as a specialized tool.

    The system uses a residential proxy to mimic a genuine user browsing experience, which helps bypass common scraping deterrents. A key operational detail is the initial "prewarming" phase for each new proxy IP. During this period, which can take three to five minutes, SERP Maven opens a real Chrome browser instance and navigates several ordinary web pages before initiating any Google searches. This process helps Google accept cookies for the proxy IP, leading to faster subsequent searches that complete in seconds.

    Once configured, the `google_search` tool returns compact JSON data containing Google's own ordering of organic results. It can also include related searches, "People also ask" sections, Google's AI Overview, and map pack details, provided these elements are present on the actual Google search page. This direct access to comprehensive search data allows AI agents to perform more informed and targeted research.

    The primary operational cost is associated with the residential proxy traffic. The initial proxy IP warm-up consumes approximately 60 MB of data, while each subsequent search averages about 0.4 MB. Users are advised to limit the number of pages requested per search to manage data consumption effectively, as every additional page incurs more traffic on their proxy plan.

    Share this article

    Go deeper
    Original reporting by Dev CommunityWe don't republish, read the full story →

    Related reading

    6 stories