Your personal memory, across sessions, agents, and devices.

Firecrawl: Turn Websites Into LLM-Ready Markdown

memU Team memU Team

Firecrawl is an API that turns web pages into clean Markdown or structured JSON for language models. Developers use it to feed agents and RAG pipelines without writing a scraper per site. It also ships an MCP server, so coding agents such as Claude Code can call it as a tool. Below, we compare it with Parallel's Extract API before going endpoint by endpoint.

What is Firecrawl, in one sentence?

Firecrawl is a web data API that fetches pages or whole websites and returns them as LLM-ready Markdown or schema-shaped JSON.

The Firecrawl docs call Markdown its main, "LLM-ready" output. The project is open source under AGPL-3.0 in the firecrawl/firecrawl repository. The company also runs a hosted API with Python and Node SDKs.

How does Firecrawl compare with Parallel Extract?

The closest alternative is Parallel's Extract API. Both turn a URL into Markdown for an LLM.

Firecrawl Parallel Extract
Input A URL, a whole site (crawl) or a search query A list of URLs plus an objective
Default output Full-page Markdown, with other formats on request Excerpts that match the objective, as Markdown
Full page content Yes, the default Optional, with the full content field
Structured JSON Yes, from a JSON Schema or a prompt Not documented on the Extract page
Site discovery Map and crawl endpoints Separate Search API
Price 1 credit per scrape, 5 with JSON mode. Plans from $0 (1,000 credits) to $749 a month $0.001 per URL ($1 per 1,000)
Self-hosting Yes, AGPL-3.0, with limits Not documented

Sources: Firecrawl's scrape docs and pricing. For Parallel, its Extract quickstart and pricing.

Our reading: Parallel's objective-focused excerpts save tokens when an agent needs a few passages. Firecrawl's per-page credits cost more but return everything on the page.

What do Firecrawl's endpoints do?

The introduction page lists these building blocks:

  • Scrape fetches one URL and returns Markdown or another format you request.
  • Crawl recursively gathers content from an entire site. Map only discovers a site's URLs, so you can pick pages before fetching them.
  • Search runs a web search and returns full page content from the results in one call.
  • Parse converts local files such as PDFs and DOCX into Markdown or JSON.
  • Interact and Browser Sandbox let an agent click and fill forms on a page that needs it.
  • Agent gathers web data autonomously, according to the docs, without you choosing the pages.

A minimal Python scrape that returns Markdown looks like this:

from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")
doc = firecrawl.scrape("https://docs.firecrawl.dev", formats=["markdown"])
print(doc.markdown)

The docs say an API key is optional for basic requests but raises your limits.

How does Firecrawl extract structured JSON?

You ask for the JSON format in a scrape and give it a JSON Schema or a prompt. A schema fixes the output fields. With only a prompt, the model picks the structure, which can change from page to page. The scrape docs show the quick, prompt-only version:

doc = firecrawl.scrape(
    "https://firecrawl.dev",
    formats=[{"type": "json", "prompt": "Extract the company mission from the page."}],
)
print(doc.json)

JSON mode costs 4 extra credits per page, so a JSON scrape is 5 credits against 1 for Markdown. For repeat jobs across many pages, pass a schema.

Scrape results are cached. The default maxAge is two days (172,800,000 ms). Setting it to 0 forces a fresh fetch, which the docs warn is slower and more likely to fail. Cached results still cost 1 credit.

How do you use Firecrawl from a coding agent?

Through its MCP server. The hosted server takes an API key at https://mcp.firecrawl.dev/v2/mcp, or a browser sign-in. A keyless tier covers search, scrape and parse within daily limits. The docs list setup for Claude Code and Codex. ChatGPT and Claude also get one-click connectors. Once connected, your coding agent can read documentation pages or search results as Markdown inside a session.

What does Firecrawl cost, and can you self-host it?

The pricing page listed these monthly plans when we checked in October 2026:

Plan Monthly price Credits per month Concurrent requests
Free $0 1,000 2
Hobby $19 5,000 5
Standard $99 100,000 25
Growth $399 500,000 50
Scale $749 1,000,000 100

Annual billing lowers each paid plan, for example Standard to $83 a month.

Self-hosting is possible with Docker Compose, but the self-host guide lists these gaps. Fire-engine, which handles advanced anti-bot behavior, is not included. Screenshots and page actions are unavailable in the default stack. The Agent and Browser features need Firecrawl Cloud or extra services. LLM extraction needs an OpenAI-compatible provider or Ollama that you connect yourself. The quickstart also runs without API authentication or TLS. You add those before production.

Read the fine print

The prices and credit costs above come from the vendors' own pages. We found no independent benchmark of extraction quality for either, so test both on your own target sites before you commit. AGPL-3.0 also matters if you modify Firecrawl and offer it over a network, so check the license with your legal team.

The bottom line

Use Firecrawl when your agent needs whole sites or schema-shaped JSON, and start on the free tier to measure credits per job. Pick Parallel Extract for known URLs where a few passages answer the question. Plan for Firecrawl Cloud if you need anti-bot handling or screenshots, because the default self-hosted stack leaves them out.

How do you stop agents from re-scraping what they already learned?

Firecrawl caches pages for two days, which saves fetches. It does not keep what the agent concluded from those pages. A coding agent that read an API's docs through Firecrawl on Monday starts Tuesday's session without that knowledge. It scrapes the same pages again and spends credits and context on them. If you switch from Claude Code to Codex, the second agent starts from zero as well.

memU keeps what agents learn as a shared wiki of readable Markdown. A scheduled background task slices new session logs into jobs. The agent distills each job into memory and skill files, for example "this API paginates with a cursor." memU itself makes no LLM calls. It stores and retrieves the Markdown the agent wrote.

A standing instruction in CLAUDE.md or AGENTS.md tells the agent to check memory before it answers. If the answer is already there, it can skip the scrape. Every host reads the same memory backend.

To try the hosted version, which is free and syncs across devices, get an API key from memu.so and send your agent one message:

Read https://memu.pro/SKILL.md, follow its instructions to install and configure memU, API Key is YOUR_MEMU_API_KEY.

The support matrix and self-hosted setup are in the memU GitHub repository.

FAQ

Is Firecrawl open source?

Yes. The main repository is licensed under AGPL-3.0. The self-hosted stack lacks some cloud features, including Fire-engine anti-bot handling and screenshots.

How do I convert a website to Markdown with Firecrawl?

Call the scrape endpoint with the Markdown format for one page, or the crawl endpoint for a whole site. Markdown is Firecrawl's default, LLM-ready output.

How many credits does a Firecrawl scrape cost?

One credit per page. JSON extraction adds 4 credits, for 5 in total. Cached results still cost 1 credit.

What is the difference between Firecrawl and Parallel Extract?

Firecrawl returns full pages and can crawl whole sites. Parallel Extract takes URLs and an objective and returns matching excerpts by default, at $0.001 per URL.

Does Firecrawl work with Claude Code?

Yes. Firecrawl's MCP server has documented setup for Claude Code and other coding agents. The hosted MCP server also has a keyless tier for basic scraping and search.