Google Search Grounding vs a Search API: 40-Question Test

Without search, two Gemini models answered none of 40 fresh questions. One API Pick search call lifted both to 35 of 40; grounding with Google Search reached 30 to 40 depending on the model and the prompt. Full results, prices checked on 2026-11-14, and the question set.

TL;DR

• •With no search, Gemini 3.8 Flash and Gemini 3.5 Flash-Lite answered 0 of 40 questions about October–November 2026 news. One API Pick web search per question lifted both to 35 of 40, with no wrong answers.
• •Grounding with Google Search was the most accurate and the fastest: 39 of 40 on Flash-Lite in 2.5 seconds, and 40 of 40 on 3.8 Flash once the prompt stopped inviting it to skip the search.
• •Grounding is billed per search query, not per question: the model ran 1.9 to 2.85 queries per question, so 1,000 questions cost $24 to $40 in search fees after 5,000 free queries a month. An API Pick call is a flat $15 per 1,000.
• •Pick grounding for Gemini-only apps under about 1,750 questions a month. Pick a search API when you use other models, need one predictable call per question, or want to see and filter the sources yourself.

How much does a web search lift an LLM's answers?

On 40 questions about news from October and November 2026, Gemini 3.8 Flash and Gemini 3.5 Flash-Lite answered none correctly without search. A single API Pick web search per question lifted both models to 35 of 40 (87.5%) with no wrong answers, and grounding with Google Search lifted them to between 30 and 40 of 40, depending on the model and the prompt. We ran every question on 2026-11-14 in four setups and graded the answers against sources we checked by hand.

We build API Pick, the search API in this test. Google's built-in grounding beat it on accuracy and speed in most runs, and this page says where and why.

Every other answer was "I don't know". Neither model made anything up when it had no search, which is the behavior you want. On the 10 general-knowledge control questions, all setups scored 10 of 10 except Flash-Lite with news search, which gave up on two questions it knew because the news results were off topic.

What does grounding with Google Search cost?

Grounding with Google Search on Gemini 3 and newer models costs $14 per 1,000 search queries after 5,000 free queries a month. The catch is the unit: you pay for each query the model decides to run, not for each question you ask. These are the terms on Google's pricing pages on 2026-11-14:

• Free allowance: 5,000 search queries a month, shared across all Gemini 3 and newer models.
• Paid rate: $14 per 1,000 queries, with a default limit of 1,500 grounding requests a day per project on the Gemini Developer API paid tier.
• Queries per question: Gemini ran 1.9 to 2.85 on average in our runs, and up to 16 on one question.
• Older models: Gemini 2.5 models are billed per grounded prompt, at $35 per 1,000 after 1,500 free a day. From December 1, 2026, requests to gemini-2.5-flash are routed to Gemini 3.5 Flash-Lite.

The text Google feeds the model from search is not billed as input tokens, which is why the grounded runs used only about 90 input tokens per question against about 1,400 for the runs where we pasted five API Pick results into the prompt. Model prices used here: Gemini 3.8 Flash at $0.75 per million input tokens and $3.75 per million output tokens through December 31, 2026, and Gemini 3.5 Flash-Lite at $0.30 and $2.50.

Why did Gemini 3.8 Flash skip the search?

Because with grounding the model decides whether to search, and our prompt gave it an easy way out. In the first run, Gemini 3.8 Flash ran no search at all on 9 of the 40 questions and answered "I don't know". We then removed the sentence that allowed "I don't know" and reran Gemini 3.8 Flash on the 40 news questions:

Without the escape hatch, grounding searched every time, ran more queries (2.85 per question instead of 1.9) and got all 40 right. The API Pick run barely changed, because the search had already happened before the model saw the question.

Where did the search API miss?

API Pick web search missed 5 of the 40 questions. In four, none of the five results it returned contained the fact; in the fifth, the fact was there only indirectly. Each time the model said it did not know rather than guessing:

• Uber's Q3 2026 revenue: the results were earnings coverage that discussed bookings and profit but not the revenue figure.
• Palantir's U.S. commercial revenue: the results had the growth rate (148%) but not the dollar amount.
• The top model on Firecrawl's search-lift leaderboard: the results were general LLM leaderboards.
• Composio's Standard plan price: the results included Composio's pricing page, but the snippet showed "$18 in credits per seat" rather than the plan price. With the "I don't know" option the model declined; without it, the model answered $18 per seat per month, which is right.
• Where Exa launched on November 13: the results were about Exa's Singapore office. In the second run, without the "I don't know" option, the model answered Singapore, which is wrong (it was the EU).

The news endpoint did worse than web search on product questions: it answered 5 of 13 questions about AI and developer tools, because pricing pages, docs and changelogs are not news. Pick the endpoint for the question: web search for pages and docs, news search for coverage of events, and specialist endpoints such as SEC filings search when the answer lives in a filing.

What does each setup cost per 1,000 questions?

After the free allowances, an API Pick web search costs $15 per 1,000 questions plus $0.46 to $2.29 in model tokens, while grounding with Google Search costs $24 to $40 per 1,000 questions in search fees plus $0.11 to $1.64 in tokens. API Pick charges 15 credits per call, $1 buys 1,000 credits, and only HTTP 200 responses are charged (pricing).

The free allowances point the other way. Google's 5,000 free queries a month cover roughly 1,750 to 2,600 questions at the query rates we measured. API Pick gives 100 credits once, which is six searches. A Gemini app under about 1,750 questions a month pays nothing for grounding; above that, the per-question price of grounding is about double.

Speed favors Google. Grounding answered in a median 2.5 seconds on Flash-Lite and 7.1 to 7.4 seconds on 3.8 Flash. An API Pick web search took a median 5.75 seconds by itself (7.8 seconds at the 90th percentile), so the full answer took 7.5 to 11 seconds. News search had a slow tail: 16 seconds at the 90th percentile.

When should you use grounding, and when a search API?

This is the pipeline we used for the API Pick runs, cut down to the essentials. It works the same with any model that takes a text prompt; for a full agent loop with tool calling, see how to add web search to an OpenAI or Claude agent.

import os, requests

APIPICK_KEY = os.environ["APIPICK_API_KEY"]
GEMINI_KEY = os.environ["GEMINI_API_KEY"]

def search(question: str) -> str:
    r = requests.post(
        "https://www.apipick.com/api/search/web",
        headers={"x-api-key": APIPICK_KEY},
        json={"query": question, "max_num_results": 5},
        timeout=60,
    )
    r.raise_for_status()
    return "\n\n".join(
        f"[{i + 1}] {x['title']} ({x['url']})\n{x['snippet']}"
        for i, x in enumerate(r.json()["results"])
    )

def answer(question: str, model: str = "gemini-3.8-flash") -> str:
    prompt = (
        "Today's date is November 14, 2026. Answer in one short sentence.\n\n"
        f"Search results:\n{search(question)}\n\nQuestion: {question}"
    )
    r = requests.post(
        f"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent",
        headers={"x-goog-api-key": GEMINI_KEY},
        json={"contents": [{"parts": [{"text": prompt}]}],
              "generationConfig": {"temperature": 0}},
        timeout=120,
    )
    r.raise_for_status()
    return r.json()["candidates"][0]["content"]["parts"][0]["text"]

print(answer("What revenue did AMD report for the third quarter of 2026?"))

Grounding with Google Search, by contrast, is one extra field in the Gemini request body:

{"contents": [{"parts": [{"text": "Who won the 2026 World Series?"}]}],
 "tools": [{"google_search": {}}]}

How does this compare with Firecrawl's search-lift leaderboard?

Firecrawl published a Web Search LLM Leaderboard with data as of November 10, 2026. It tests 10 models on 4,703 hard research questions from BrowseComp, DeepSearchQA, HLE and its own private set, lets each model search and read pages repeatedly within a tool budget, and reports an average lift of 40.6 points with its Alexandria data layer. Claude Opus 5.5 scored highest at 79.0.

Our test answers a narrower question: what does one search do for simple, recent facts that a model cannot know? Our questions need one lookup, not a chain of searches, so our lifts are larger (0 to 87.5% and more) and are not comparable with Firecrawl's numbers. Our 40 questions and answers are listed below; Firecrawl keeps its own question set private.

How did we run the test?

All runs took place on 2026-11-14:

• Questions: 40 about events from early October to November 13, 2026 in four groups (prizes and sport 7, central banks 5, company earnings 15, AI and developer tools 13), plus 10 general-knowledge controls. Every answer comes from a primary source we read ourselves: nobelprize.org, the Federal Reserve, ECB and Bank of England releases, SEC 8-K exhibits, and vendors' own blogs and pricing pages.
• Models: gemini-3.8-flash and gemini-3.5-flash-lite through Vertex AI, temperature 0, default thinking settings.
• Setups: no search; one API Pick web search (5 results, the question as the query); one API Pick news search; and Gemini's built-in Google Search tool. Search results were fetched once per question and given to both models.
• Prompt: today's date, "answer in one short sentence", and in the first run "if you are not sure, reply I don't know". The second run removed that last sentence.
• Grading: two models (Gemini 3.8 Flash and Gemini 3.1 Pro Preview) labeled each of the 520 answers correct, wrong or abstained against the reference. We checked every disagreement (6) and every wrong answer by hand against the source, and changed one label both graders agreed on: a claim that Exa had not announced Exa 3.0 counts as wrong, not as abstaining. Numbers had to match the reference's precision, so "AWS grew 39%" counted as wrong against 39.3%.

The full question set with reference answers and sources:

id   question (shortened)                         reference answer           source
N1   2026 Nobel Prize in Physics                  Francis Halzen             nobelprize.org
N2   2026 Nobel Prize in Chemistry                Henri B. Kagan, Kenso Soai nobelprize.org
N3   2026 Nobel Prize in Physiology or Medicine   Deisseroth, Hegemann, Nagel nobelprize.org
N4   2026 Nobel Prize in Literature               Anne Carson                nobelprize.org
N5   2026 Nobel Peace Prize                       Navanethem Pillay          nobelprize.org
N6   2026 Nobel Prize in Economic Sciences        Robert E. Hall, M. Woodford nobelprize.org
S1   2026 World Series winner                     Milwaukee Brewers          mlb.com
C1   FOMC target range, 2026-10-28                3.75% to 4.00%, unchanged  federalreserve.gov
C2   FOMC dissents, 2026-10-28                    2 (Cook, Hammack)          federalreserve.gov
C3   ECB deposit facility rate, 2026-10-29        2.50%, unchanged           ecb.europa.eu
C4   Bank of England Bank Rate, Nov 2026          4%, raised 0.25 pt         bankofengland.co.uk
C5   BoE MPC vote, meeting ending 2026-11-04      8-1                        bankofengland.co.uk
E1   Apple revenue, FQ4 2026                      $114.5 billion             SEC 8-K Ex. 99.1
E2   Apple revenue, fiscal 2026                   $479 billion ($478.834 bn) SEC 8-K Ex. 99.1
E3   Microsoft revenue, quarter to 2026-09-30     $92.3 billion              SEC 8-K Ex. 99.1
E4   Microsoft Cloud revenue, same quarter        $63.9 billion              SEC 8-K Ex. 99.1
E5   Alphabet revenue, Q3 2026                    $127.6 billion             SEC 8-K Ex. 99.1
E6   Amazon net sales, Q3 2026                    $201.9 billion             SEC 8-K Ex. 99.1
E7   AWS net sales growth, Q3 2026                39.3%                      SEC 8-K Ex. 99.1
E8   Meta revenue, Q3 2026                        $64.60 billion             SEC 8-K Ex. 99.1
E9   Netflix revenue, Q3 2026                     $12.9 billion              SEC 8-K Ex. 99.1
E10  Intel revenue, Q3 2026                       $16.5 billion              SEC 8-K Ex. 99.1
E11  AMD revenue, Q3 2026                         $13.4 billion              SEC 8-K Ex. 99.1
E12  Uber revenue, Q3 2026                        $14.8 billion              SEC 8-K Ex. 99.1
E13  Palantir revenue, Q3 2026                    $2.313 billion             SEC 8-K Ex. 99.1
E14  Palantir U.S. commercial revenue, Q3 2026    $984 million               SEC 8-K Ex. 99.1
E15  Starbucks net revenues, FQ4 2026             $9.1 billion               SEC 8-K Ex. 99.1
T1   Firecrawl MCP v3: tools per connection       3,000+                     firecrawl.dev/blog
T2   Firecrawl Series B, Nov 2026                 $75 million                firecrawl.dev/blog
T3   #1 on Firecrawl's search-lift leaderboard    Claude Opus 5.5 (79.0)     firecrawl.dev/benchmarks
T4   Questions in that leaderboard                4,703                      firecrawl.dev/benchmarks
T5   AWS AgentCore Web Search API price           $7 per 1,000 queries       aws.amazon.com
T6   AgentCore Web Search API GA date             2026-11-10                 aws.amazon.com
T7   Composio Standard plan                       $18 per seat per month     composio.dev/pricing
T8   Court dismissed Google's amended complaint   2026-11-09                 serpapi.com/blog
T9   Model behind Cursor Voice Chat with Agents   Grok Voice                 cursor.com/changelog
T10  Episodes searchable in Exa Podcasts          3.2 million                exa.ai/blog
T11  Exa 3.0 announcement date                    2026-10-19                 exa.ai/blog
T12  Region where Exa Search launched 2026-11-13  The EU                     exa.ai/blog
T13  Model gemini-2.5-flash routes to on Dec 1    Gemini 3.5 Flash-Lite      ai.google.dev pricing
K1-K10  control: Hubble launch year, symbol for tungsten, author of One Hundred Years of
        Solitude, capital of Australia, IPv6 address bits, HTTP 429, Titan's planet,
        Berlin Wall year, PostgreSQL port, painter of The Night Watch

What does this test not show?

• It is small. 40 news questions and one run per setup. A difference of two or three questions is within noise.
• It only tests Gemini. Other models may search, read and abstain differently.
• It only tests single-fact questions. Multi-step research needs repeated searches and page reads; see Firecrawl's leaderboard for that.
• It does not test other search APIs. We did not run Tavily, Exa, Brave or AWS AgentCore here; for their list prices, see our Tavily alternatives comparison.
• Prices change. Gemini 3.8 Flash token prices double on January 1, 2027, and all prices here are list prices checked on 2026-11-14.

If your agent runs on Gemini and asks a few thousand questions a month, turn on grounding and do not tell the model it may skip the search. If it runs on several models, or you need to control which sources reach the model and what each answer costs, use a web search API and read why APIs should only charge for successful calls.

Written by

Sarah Choy

CEO, API Pick

Sarah Choy is the CEO of API Pick. She writes about building production-ready APIs for AI agents and LLM workflows.