Connect Firecrawl to collect web content

View as Markdown

Firecrawl turns website content into text and structured data. Connect your account to Mythos to build tools that collect information from the web, organize it, and check it again when it changes.

Firecrawl is an App + chat connector. You can use the same saved connection while building in project chat and in the app you publish. See Connections in Mythos for shared connection settings.

Your app can use Firecrawl to:

  • Extract readable content from a page, including pages that load content with JavaScript.
  • Crawl several pages on a site and collect their content.
  • Search the web and retrieve information from the results.
  • Find URLs on a site before deciding which pages to crawl.
  • Extract selected fields and monitor pages for changes.

It is useful for research tools, directories, content collections, and apps that depend on information published on other websites.

Examples to build

App ideaPrompt to tryWhat it adds
Competitor updates“Build a Firecrawl tracker for these three product pages and show changes since the previous check.”Save page snapshots and compare updates in one view.
Job listings“Collect vacancies from these companies’ careers pages with Firecrawl and highlight newly listed roles.”Bring selected job listings into a searchable feed.
Company profiles“Use Firecrawl to collect public company descriptions, locations, and contact pages from these websites.”Turn website content into structured profiles for review.
Price history“Track the price and availability of these product URLs using Firecrawl.”Keep a history of product changes and show the latest result.
Brand reference“Use Firecrawl to collect the colors, typography, and logo references available on this website.”Organize the available brand information into a reference page.
Research library“Turn a saved URL into readable content with Firecrawl, keep its source link, and let me refresh it.”Create a reusable collection of content with clear sources.

For scheduled checks, include how often to run them and which pages to check. Start with one URL so you can review the result before expanding the collection.

Connect Firecrawl to Mythos

Mythos uses your own Firecrawl account and API key. You will need permission to create App + chat connections in your Mythos workspace. If a suitable connection already exists, use it instead of creating another.

Before you start

Have these ready:

  • A Firecrawl account.
  • An API key for that account.
  • Enough Firecrawl credits for the scraping or crawling you want to run.
  • A page URL you can use for a first example.

Scraping, crawling, searches, extractions, and monitors use your Firecrawl credits. A successful connection check does not mean that future scraping is free.

1. Get your API key

  1. Sign in to Firecrawl and open API Keys.
  2. Create a key with a recognizable name, such as Mythos research.
  3. Copy the key and keep it secure. Firecrawl keys start with fc-.

Paste the key only into the connection form. Do not put it in a chat message or your app’s public files. Firecrawl’s API documentation explains its available features.

2. Add the connection

  1. From the Dashboard, open Connectors and select Firecrawl. If a connection already exists, select Add connection to open a new form.
  2. Enter a Display name, such as Research sources.
  3. Paste your key into API key.
  4. Under Sharing, keep Private, or choose Share with others and select the workspace members who should use it. Share with workspace makes it available throughout the workspace.
  5. Click Connect Firecrawl. Mythos checks the account before saving the connection.

Open your project and choose Chat actions (+) → Project connectors. Select the saved connection and click Use in this project. You can then ask Mythos to build one of the examples above.

Try a small request first:

Use Firecrawl to read https://example.com/about and show me the extracted company name and description, with the source URL.

Review the returned content before asking for a whole-site crawl. Some jobs take time to finish; checking an existing job’s status is different from starting another job.

Usage and billing

Firecrawl bills usage to the account behind your key. Check its billing guide for current credit costs. Mythos does not provide a Firecrawl credit allowance with the connection.

Scraping, mapping, searching, and batch scraping can spend Firecrawl credits without a separate approval question in chat. Starting a crawl, extraction, research agent, or recurring monitor follows the connection’s approval setting. A crawl or monitor can continue spending after it starts, so choose a sensible scope and check frequency.

Limitations

  • The connection cannot automatically replace an expired or revoked API key. Create a new connection when you rotate the key.
  • Linked projects use the saved Firecrawl account. App visitors do not each sign in to their own Firecrawl account through this connection.
  • Website access restrictions can prevent scraping even when the connection works.

If a request fails

If Firecrawl reports insufficient credits, check the balance of the account that owns the key. Adding Mythos credits does not replenish that balance.

For a page that cannot be read, check that the URL is reachable and that Firecrawl supports accessing it. For a crawl that is still running, ask for its status before starting the same crawl again.

Manage the connection

Open Connectors → Firecrawl, then the saved connection. Unlink projects removes access for selected projects. Disconnect stops their access through the connection.

Disconnecting does not stop a crawl or recurring monitor already running in Firecrawl. Stop those jobs separately in Firecrawl if you no longer need them. To change the key, disconnect the old connection, create a new one with the replacement key, and link it to the affected projects. Ask Mythos to set up their app features again before relying on them.