Groq

Firecrawl + Groq: AI-Powered Web Scraping & Data Extraction

Firecrawl is an enterprise-grade web scraping platform that turns any website into clean, AI-ready data. Combined with Groq's fast inference through MCP, you can build intelligent agents that scrape websites, extract structured data, and research the web with natural language instructions.

Key Features:

  • Enterprise Web Scraping: Handles JavaScript, authentication, and anti-bot detection automatically
  • Structured Extraction: Define JSON schemas and get consistent data across sources
  • Web Research: Search the web and synthesize information from multiple pages
  • Site Crawling: Map a site and crawl many pages under it
  • Fast Results: Sub-10 second responses when combined with Groq's inference

Quick Start

1. Install the required packages:

curl
pip install openai python-dotenv

2. Get your API keys:

curl
export GROQ_API_KEY="your-groq-api-key"
export FIRECRAWL_API_KEY="your-firecrawl-api-key"

3. Create your first web scraping agent:

Python
import os
from openai import OpenAI
from openai.types import responses as openai_responses

client = OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY")
)

tools = [
    openai_responses.tool_param.Mcp(
        server_label="firecrawl",
        server_url="https://mcp.firecrawl.dev/v2/mcp",
        headers={"Authorization": f"Bearer {os.getenv('FIRECRAWL_API_KEY')}"},
        type="mcp",
        require_approval="never",
    )
]

response = client.responses.create(
    model="openai/gpt-oss-120b",
    input="Scrape https://console.groq.com/docs/models and provide an overview of available models",
    tools=tools,
    temperature=0.1,
    top_p=0.4,
)

print(response.output_text)

Advanced Examples

Structured Data Extraction

Extract data in specific JSON formats across multiple sources:

Python
response = client.responses.create(
    model="openai/gpt-oss-120b",
    input="""Extract pricing from https://openai.com, https://anthropic.com, https://groq.com
    
    Return JSON:
    {
        "company_name": "string",
        "pricing_plans": [{"plan_name": "string", "price": "string", "features": ["string"]}]
    }""",
    tools=tools,
    temperature=0.1,
)

print(response.output_text)

Multi-Source Web Research

Search the web and synthesize findings across multiple pages:

Python
response = client.responses.create(
    model="openai/gpt-oss-120b",
    input="""Research "latest trends in AI model inference speed and performance":
    1. Recent developments (2024-2025)
    2. Key companies and technologies
    3. Performance benchmarks
    4. Future trends
    
    Provide a comprehensive report with citations.""",
    tools=tools,
    temperature=0.1,
)

print(response.output_text)

Multi-Page Scraping

Map a site, then scrape the pages that matter:

Python
response = client.responses.create(
    model="openai/gpt-oss-120b",
    input="""Map https://console.groq.com/docs, pick the three pages about tool use,
    scrape them, and summarize the key findings from each.""",
    tools=tools,
    temperature=0.1,
)

print(response.output_text)

Available Firecrawl MCP Tools

Firecrawl MCP provides these core tools for web scraping, data extraction, and research. See the Firecrawl MCP tools reference for the full, current list.

ToolDescription
firecrawl_scrapeScrape a single URL as Markdown, or as structured JSON with a schema
firecrawl_mapList the URLs on a site without fetching their content
firecrawl_searchSearch the web and optionally extract content from search results
firecrawl_crawlCrawl many pages under a site and return the results
firecrawl_check_crawl_statusCheck a crawl job by ID
firecrawl_parseParse documents such as PDFs and spreadsheets
firecrawl_agentStart an autonomous research job that returns structured data
firecrawl_agent_statusCheck an agent job and retrieve its results

Use allowed_tools on the MCP tool to limit the model to the tools your application needs.

Challenge: Build an AI-powered competitive intelligence system that monitors competitor websites, extracts key business metrics, and generates automated reports using Firecrawl and Groq!

Additional Resources

For more detailed documentation and resources on building web intelligence applications with Groq and Firecrawl, see:

Was this page helpful?