AI Search & AEO

Firecrawl in 2026: Read Your Site the Way an AI Reads It

Every guide to this tool frames it as a developer utility for building AI apps. The more valuable use for a marketing team is the reverse: point it at your own website and read back exactly what a language model can extract from your pages. Whatever is missing from that output is invisible to AI search.

Distk Editorial 14 August 2026 10 min read

Firecrawl converts any web page into clean markdown or structured JSON. Developers use it to feed AI applications. Marketing teams in 2026 should use it for something simpler and more urgent: an AI extractability audit of their own site. Scrape your twenty most commercially important pages, read the markdown, and check whether your prices, specifications, credentials, locations and FAQ answers actually survive the conversion. Anything that does not appear cannot be quoted by an AI answer engine, no matter how good the page looks in a browser. A full audit of a mid-size site fits inside the free tier of 1,000 credits per month. Two things to check before you commit: the core project is AGPL-3.0, which matters if an agency self-hosts it inside a client-facing product, and pointing it at other people's sites carries diligence obligations of its own.

What Is Firecrawl in 2026?

Firecrawl is a hosted service, and also an open source project, that turns a web page into clean text a language model can read. You give it a URL and it returns markdown, JSON, HTML, a screenshot or page metadata, with the navigation, scripts and layout noise stripped away. In 2026 it is billed in credits rather than per seat, and it exposes its work through a small set of named endpoints.

The endpoints matter more than the marketing copy, because each one costs a different number of credits and answers a different question. The table below reproduces the full published list as it stood in August 2026.

EndpointWhat it doesPublished credit cost
ScrapeTurns one page into clean data: markdown, JSON, screenshot and more1 credit per page
CrawlFollows links from a starting URL across a whole site or section1 credit per page
MapReturns a list of links found across a site, for choosing what to scrapeSources disagree, see below
SearchSearches the web and returns full page content from the results2 credits per 10 results
ExtractPulls structured data out of pages against a JSON schema you defineAdditional credits for advanced features
InteractScrapes a page then acts on it: click, scroll, type, wait, press2 credits per browser minute
MonitorWatches pages and reports when they change1 credit per page
AgentAutonomous data gathering, in preview at time of writingDynamic pricing, 5 free runs daily
ParseDocument parsing for PDFs, DOCX and HTML filesNot published on the pricing page

Map deserves a note because the sources are not consistent. The documentation describes it as the fastest way to go from one URL to a map of an entire website, assembled primarily from the sitemap and topped up with search results and previously crawled pages, returning each link with a title and description where available. That documentation states each Map request costs 1 credit regardless of how many URLs come back; the homepage credit table lists Map at 1 credit per page. Check your usage dashboard rather than trusting either.

Output formats in 2026 go well beyond the markdown default. The scrape documentation lists markdown, summary, cleaned HTML, raw HTML, screenshot, links, JSON, images, branding, product, audio, video and query, though an audit needs only the first four. Official SDKs cover Python, Node.js, Go, Rust, Java and Elixir, alongside a CLI and a REST API, with MCP integrations for tools including Cursor, Claude Code and Windsurf.

Why Does Extractability Decide AI Visibility in 2026?

Because an AI answer engine cannot cite a fact it cannot parse. When a model answers a question about your category in 2026, it retrieves pages, converts them to text, and quotes from that text. A price rendered inside a product image, a specification list drawn by a JavaScript widget, a doctor's qualification that only exists as a styled badge: all of these are visible to a human and absent from the text a model receives.

This is the gap that separates traditional SEO from AEO and GEO work. Ranking decides whether a page is retrieved. Extractability decides whether, once retrieved, it can supply the sentence the model needs. A page can win position one and still lose the AI answer, because a competitor stated the number in plain text and you did not.

Most teams treat this as theoretical, discussed in the abstract and never tested. Our pet e-commerce marketing guide for 2026 poses it as an AI search test brands ought to run on themselves. Firecrawl turns that test into a five-minute check with an output you can read.

The one-sentence version

If a fact about your business does not appear as plain text in the markdown output, treat it as a fact your website does not communicate to AI systems in 2026, however prominent it looks on screen.

How Can You Audit Your Own Site's AI Extractability in 2026?

By scraping your own pages and reading what comes back. The workflow is short enough to run in an afternoon and cheap enough to sit inside the free tier. You do not need to write software: the hosted playground accepts a URL and shows you the markdown. The skill involved is editorial judgement about what should be there, not engineering.

  1. Map the site. Run Map on your domain for the URL list. It tells you what exists, which for most sites in 2026 is not what the team believes exists.
  2. Choose the pages that carry commercial weight. Ten to thirty for a first pass: homepage, service or category pages, top product pages, pricing, key location pages, About, and any practitioner profiles.
  3. Scrape each one to markdown. One credit each. Save the output where you can read it beside the live page.
  4. Read the markdown against a checklist of the facts that page is supposed to communicate. Work down the table below.
  5. Log every miss as a page defect, not an SEO task. A missing price is a content bug with a specific fix.
  6. Fix, re-scrape, confirm. This is the step teams skip. One credit tells you whether the fix worked.
  7. Put priority pages on Monitor so a template change does not quietly remove a fact you fixed.

What should appear in the markdown output

The checklist below is the part worth keeping. Each row is a fact type AI systems commonly need in order to cite a business, paired with what its absence from the markdown means. Not every row applies to every page, but a page failing four or five applicable rows is not competing in AI answers in 2026 regardless of its rankings.

Look for in the markdownIf it is missing, it usually means
What the business does, in plain text, within the first 200 wordsThe value proposition lives in a hero image or a slider, so the model has no definition to work from
Prices or price ranges as textPricing is rendered by a script or sits inside an image, and you are absent from every cost comparison
Product specifications as a list or tableSpecs are in a graphic or a tabbed widget the scraper never opened
Credentials, qualifications and years of experience beside the person's nameTrust signals are styled badges rather than text, so the entity has no verifiable attributes
FAQ questions and their answers as readable textThe accordion is collapsed in the DOM or built client-side, so your best AEO asset is invisible
City, service area and contact detailsLocal relevance is only in a map embed or the footer image, which no model reads
Review counts and ratings as numbers in textSocial proof is inside a third-party widget that loads after the page and never reaches the output
Heading hierarchy showing up as markdown levelsHeadings are styled divs, so the page has no structure the model can navigate
Dates: published, updated, or the date a claim applies toNothing tells a model whether your figure is current, and it will prefer a dated source
Tables that survive as tablesComparison data was an image, and comparison queries are where buying decisions get made

Some page types fail far more than others. Practitioner profiles are the worst offenders, which is why the doctor-profile entity block in our pediatric hospital marketing guide for 2026 is written to be extractable rather than merely attractive. Scrape any three profile pages in your sector and you will usually find qualifications, registration numbers and languages spoken all missing from the text.

Extractability is one half of a technical review. Pair this output with a conventional crawl audit of indexation, canonicals and speed, along the lines of the technical audit table in our guide to marketing for IEPF and share recovery firms in 2026. One tells you whether the page can be found, the other whether it can be quoted.

What Else Can a Marketing Team Use It For in 2026?

Three secondary uses justify the subscription once the audit habit is established. All three follow one pattern: they replace a person manually reading and copying from web pages with a scheduled job that returns clean text. None needs a developer beyond initial setup, and all carry the diligence considerations set out further down this page.

Competitor content and messaging monitoring

Crawl a competitor's blog, pricing page or product pages on a schedule and diff the markdown between runs. Monitor is built for exactly this. What you get in 2026 is a factual log of when positioning changed, when a price moved, when a feature was quietly dropped, and which topics a rival is publishing into, rather than an impression formed from occasional browsing.

A knowledge base your AI tools can actually use

Crawl your own site plus a set of trusted external sources into markdown, and you have the corpus for a retrieval system that answers questions from your own material instead of guessing. This is the practical route to grounded internal AI in 2026, and it pairs with the model cost planning in our Gemini 3.7 Flash guide for marketing teams. Clean input is most of the work in any retrieval build.

Structured market research

Extract takes a JSON schema and returns pages as filled fields rather than prose. Point it at a set of category pages with a schema of product name, price, pack size and claim, and you get a spreadsheet instead of a fortnight of copy and paste. It is the right tool for pricing surveys, assortment tracking and competitive feature grids in 2026.

How Does Firecrawl Pricing Work in 2026?

You buy a monthly allowance of credits and each operation consumes some. A page scraped is one credit, a page crawled is one credit, ten search results are two credits, and browser interaction is two credits per minute. Plans also cap concurrency, which is how many requests can run at once, and that limit affects how fast a large crawl finishes rather than what it costs.

PlanPrice per month, billed yearlyCredits per monthConcurrent requests
Free0 USD1,0002
Hobby16 USD5,0005
Standard83 USD100,00050
Growth333 USD500,000100
Scale599 USD1,000,000150
EnterpriseCustomCustom, unlimited pagesCustom

Three published conditions matter to a budget. Scale lists extra credits at 397 US dollars per additional 350,000. Firecrawl states it does not currently offer a pay-per-use plan. And credits do not roll over on self-serve plans, so an allowance sized for an occasional audit is wasted in months you do not run one. Enterprise adds dedicated support, an SLA, bulk discounts, zero-data retention and SSO. The Free tier includes five daily runs of the Agent preview.

Illustration only, not a quote

An extractability audit of a 400 page site: 1 credit to map it, 400 to crawl every page, 40 targeted re-scrapes after fixes, and 25 priority pages monitored weekly for a month at 100 credits. That comes to roughly 541 credits, inside the 1,000 credit Free tier. Paid tiers exist for continuous competitor crawling and research volume, not for auditing your own site. Your figures will differ with page count and re-check frequency.

Pricing figures on this page were checked in August 2026 against the published pricing page. Software pricing moves, and credit definitions move with it, so confirm the current rates and the current per-endpoint credit costs before you commit a budget.

What Does the AGPL Licence Mean for Agencies in 2026?

It means the phrase "it's open source" needs a second look before an agency builds a client-facing service on a self-hosted copy. The repository states that the project is primarily licensed under the GNU Affero General Public License v3.0, and that the SDKs and some UI components are licensed under the MIT License. The repository carries roughly 167.6 thousand stars and publishes a self-hosting guide.

The AGPL is not the same as the permissive MIT or Apache licences most marketers have half-heard about. Its Section 13 is headed "Remote Network Interaction; Use with the GNU General Public License" and opens with this text:

Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software.

Read plainly, that clause turns on modification combined with remote network interaction, which is the shape of an agency that forks the tool, wires it into a client dashboard and charges for access. Whether any specific deployment triggers it is a legal question about your architecture and your modifications. Nothing here is a legal conclusion. Take advice before you build.

Two facts reduce the practical exposure for most readers in 2026. The hosted plans do not raise the question, because you are a customer of a service rather than a distributor of software. And the repository notes that the cloud version at firecrawl.dev includes additional features beyond the open source version, so self-hosting is not a free copy of the hosted product in any case.

What Are the Lawful Use Considerations in 2026?

Readers will point this at competitor websites, so the diligence question deserves a straight treatment. There is no single answer, because what matters is the combination of what you collect, whose data it is, where the subjects are located and what the source site's terms say. The table below separates the uses that are generally low-risk from those that warrant advice before you start.

Generally lower riskGet legal review first
Your own websites and propertiesAnything containing personal data about identifiable people
Publicly available pages containing no personal dataContent behind a login, a paywall or an account agreement
Respecting robots.txt and any stated crawl directivesPlatforms whose terms of service restrict automated access
Conservative rate limits that do not burden the source serverRepublishing or reselling substantial copied content
Reading facts to inform your own original workBulk collection intended to reconstruct a competitor's database

Two regimes are worth naming for Indian and international teams in 2026. Collecting personal data about people in India engages the Digital Personal Data Protection Act 2023, and collecting personal data about people in the European Union engages the GDPR, both of which apply to scraped personal data as much as to data a person typed into your form. Contact names, email addresses and reviewer profiles pulled off competitor sites are personal data.

The practical rule is narrow and easy to hold. Collect facts about businesses, products and prices; do not collect facts about people. Read the current terms of any site you intend to crawl at scale, because terms change. Where a project sits near personal data, put it in front of counsel before the first request. None of this is legal advice.

What Are the Common Mistakes in 2026?

The failures cluster in three places: treating the tool as developer-only when the work is editorial, running the audit once and assuming the result holds, and skipping the licence and lawful use questions until a client asks. Each is cheap to avoid in 2026 and expensive to discover late. The list below covers the eight we see most often.

Key Takeaways for 2026

The short version for anyone deciding whether this belongs in a 2026 marketing stack. The tool is genuinely useful for building AI applications, but its value to a marketing team is diagnostic: it is the cheapest honest answer to the question of what AI systems can actually read on your own website.

The transferable habit has nothing to do with any one tool. In 2026, the only reliable way to know what AI systems can say about your business is to read your website the way they read it: plain text, design removed. That takes an afternoon and a free account, and most teams have never once done it.

Firecrawl in 2026: FAQs

What does Firecrawl actually do for a marketing team in 2026?

It converts a web page into clean text a language model can read, and returns it as markdown, JSON, HTML, a screenshot or page metadata. For a marketing team in 2026 the most useful application is pointing it at your own site, because the markdown it returns is a close proxy for what an AI system can extract from that page. Anything absent from the output is a fact your page is not communicating.

How do you use Firecrawl to audit your own site's AI extractability in 2026?

Map the site to list its URLs, pick the ten to thirty pages that carry commercial weight, scrape each one to markdown, then read the output against a checklist of facts that page is supposed to communicate: what the business does, prices, specifications, credentials, locations, review counts and FAQ answers. Every item you cannot find in the markdown is a fact sitting in an image, a script or an unlabelled layout element. Fix the page, re-scrape, confirm.

What does Firecrawl cost in 2026?

Checked in August 2026, the published plans billed yearly are Free at 0 dollars with 1,000 credits a month and 2 concurrent requests, Hobby at 16 dollars a month with 5,000 credits and 5 concurrent, Standard at 83 dollars with 100,000 credits and 50 concurrent, Growth at 333 dollars with 500,000 credits and 100 concurrent, and Scale at 599 dollars with 1,000,000 credits and 150 concurrent plus 397 dollars per extra 350,000 credits. Enterprise is custom. Firecrawl states it does not currently offer a pay-per-use plan and that credits do not roll over on self-serve plans. Confirm current pricing before you budget.

Is Firecrawl free to use, and what does the open source licence allow in 2026?

There is a free hosted tier of 1,000 credits a month, and the project is also open source. The repository states that the project is primarily licensed under the GNU Affero General Public License v3.0, and that the SDKs and some UI components are licensed under the MIT License. AGPL-3.0 carries a network interaction clause, so an agency that modifies and self-hosts it behind a client-facing service should have counsel review its obligations before launch. The hosted plans do not raise the question at all.

Is it legal to scrape competitor websites in 2026?

It depends on what you collect and where you collect it from, and this is a question for your own lawyer rather than a blog post. Reading publicly available, non-personal pages at a polite rate while respecting robots.txt sits at the low-risk end. Collecting personal data engages India's DPDP Act 2023 and the GDPR for EU data subjects. Logged-in areas, paywalled content and platforms whose terms restrict automated access all warrant legal review before you start.

What does the Map endpoint do, and how many credits does it cost in 2026?

Map takes one URL and returns a list of links found across that website, assembled primarily from the sitemap and supplemented with search results and previously crawled pages, with a title and description for each link where available. It is reconnaissance before a crawl, so you can choose which pages to scrape instead of scraping everything. The sources disagree on cost: the Firecrawl homepage lists Map at 1 credit per page, while the Map documentation states each request consumes 1 credit regardless of how many URLs come back. Verify against your own usage dashboard.

Want to know what AI search can actually say about your business?

Distk runs extractability audits as part of AEO and GEO work for growth teams, founders and agencies across India and international markets. If your pages rank but never get quoted, we can read the site the way the models read it and show you what is missing.

Talk to Distk →