Price Per TokenPrice Per Token
ScrapeGraph AI Integration Server

ScrapeGraph AI Integration Server

by scrapegraphai

GitHub 58 187 uses Remote
0

About

ScrapeGraph AI is an AI-powered web scraping and data extraction platform that enables intelligent extraction of structured data from websites using natural language prompts. Key capabilities of ScrapeGraph AI: - **AI-Powered Web Scraping**: Extract specific data from web pages using natural language prompts, with intelligent parsing of HTML into structured JSON or custom schemas. - **Markdown Conversion**: Transform entire web pages into clean, structured markdown format for easy consumption. - **Multi-Page Crawling**: SmartCrawler asynchronously crawls websites with configurable depth and page limits to extract data across entire site structures. - **Web Search Integration**: Execute AI-powered web searches and retrieve structured results from across the internet. - **Browser Automation**: Full JavaScript rendering support and infinite scroll handling for dynamic, JavaScript-heavy websites. - **Enterprise Reliability**: Production-grade error handling, timeout management, and async processing for robust performance at scale. The service integrates seamlessly with Claude Desktop, Cursor, and any MCP-compatible client, requiring only an API key from the ScrapeGraph dashboard to get started.

README

ScrapeGraph MCP Server

[](https://opensource.org/licenses/MIT) [](https://www.python.org/downloads/) [](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp)

A production-ready Model Context Protocol (MCP) server that provides seamless integration with the ScrapeGraph AI API. This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability.

[](https://scrapegraphai.com/?utm_source=github&utm_medium=readme&utm_campaign=api_banner&utm_content=api_banner_image)

Table of Contents

  • Key Features
  • Quick Start
  • Available Tools
  • Setup Instructions
  • Remote Server Usage
  • Local Usage
  • Google ADK Integration
  • Example Use Cases
  • Error Handling
  • Common Issues
  • Development
  • Contributing
  • Documentation
  • Technology Stack
  • License
  • Key Features

  • 8 Powerful Tools: From simple markdown conversion to complex multi-page crawling and agentic workflows
  • AI-Powered Extraction: Intelligently extract structured data using natural language prompts
  • Multi-Page Crawling: SmartCrawler supports asynchronous crawling with configurable depth and page limits
  • Infinite Scroll Support: Handle dynamic content loading with configurable scroll counts
  • JavaScript Rendering: Full support for JavaScript-heavy websites
  • Flexible Output Formats: Get results as markdown, structured JSON, or custom schemas
  • Easy Integration: Works seamlessly with Claude Desktop, Cursor, and any MCP-compatible client
  • Enterprise-Ready: Robust error handling, timeout management, and production-tested reliability
  • Simple Deployment: One-command installation via Smithery or manual setup
  • Comprehensive Documentation: Detailed developer docs in .agent/ folder
  • Quick Start

    1. Get Your API Key

    Sign up and get your API key from the ScrapeGraph Dashboard

    2. Install with Smithery (Recommended)

    npx -y @smithery/cli install @ScrapeGraphAI/scrapegraph-mcp --client claude
    

    3. Start Using

    Ask Claude or Cursor:

  • "Convert https://scrapegraphai.com to markdown"
  • "Extract all product prices from this e-commerce page"
  • "Research the latest AI developments and summarize findings"
  • That's it! The server is now available to your AI assistant.

    Available Tools

    The server provides 8 enterprise-ready tools for AI-powered web scraping:

    Core Scraping Tools

    #### 1. markdownify Transform any webpage into clean, structured markdown format.

    markdownify(website_url: str)
    
  • Credits: 2 per request
  • Use case: Quick webpage content extraction in markdown
  • #### 2. smartscraper Leverage AI to extract structured data from any webpage with support for infinite scrolling.

    smartscraper(
        user_prompt: str,
        website_url: str,
        number_of_scrolls: int = None,
        markdown_only: bool = None
    )
    
  • Credits: 10+ (base) + variable based on scrolling
  • Use case: AI-powered data extraction with custom prompts
  • #### 3. searchscraper Execute AI-powered web searches with structured, actionable results.

    searchscraper(
        user_prompt: str,
        num_results: int = None,
        number_of_scrolls: int = None,
        time_range: str = None  # Filter by: past_hour, past_24_hours, past_week, past_month, past_year
    )
    
  • Credits: Variable (3-20 websites × 10 credits)
  • Use case: Multi-source research and data aggregation
  • Time filtering: Use time_range to filter results by recency (e.g., "past_week" for recent results)
  • Advanced Scraping Tools

    #### 4. scrape Basic scraping endpoint to fetch page content with optional heavy JavaScript rendering.

    scrape(website_url: str, render_heavy_js: bool = None)
    
  • Use case: Simple page content fetching with JS rendering support
  • #### 5. sitemap Extract sitemap URLs and structure for any website.

    sitemap(website_url: str)
    
  • Use case: Website structure analysis and URL discovery
  • Multi-Page Crawling

    #### 6. smartcrawler_initiate Initiate intelligent multi-page web crawling (asynchronous operation).

    ```python smartcrawler_initiate( url: str, prompt: str = None, extraction_mode: str = "ai", depth: int = None, max_pages: int = None, same_domain_

    Related MCP Servers

    AI Research Assistant

    AI Research Assistant

    hamid-vakilzadeh

    AI Research Assistant provides comprehensive access to millions of academic papers through the Semantic Scholar and arXiv databases. This MCP server enables AI coding assistants to perform intelligent literature searches, citation network analysis, and paper content extraction without requiring an API key. Key features include: - Advanced paper search with multi-filter support by year ranges, citation thresholds, field of study, and publication type - Title matching with confidence scoring for finding specific papers - Batch operations supporting up to 500 papers per request - Citation analysis and network exploration for understanding research relationships - Full-text PDF extraction from arXiv and Wiley open-access content (Wiley TDM token required for institutional access) - Rate limits of 100 requests per 5 minutes with options to request higher limits through Semantic Scholar

    Web & Search
    12 8
    Linkup

    Linkup

    LinkupPlatform

    Linkup is a real-time web search and content extraction service that enables AI assistants to search the web and retrieve information from trusted sources. It provides source-backed answers with citations, making it ideal for fact-checking, news gathering, and research tasks. Key features of Linkup: - Real-time web search using natural language queries to find current information, news, and data - Page fetching to extract and read content from any webpage URL - Search depth modes: Standard for direct-answer queries and Deep for complex research across multiple sources - Source-backed results with citations and context from relevant, trustworthy websites - JavaScript rendering support for accessing dynamic content on JavaScript-heavy pages

    Web & Search
    2 24
    Math-MCP

    Math-MCP

    EthanHenrickson

    Math-MCP is a computation server that enables Large Language Models (LLMs) to perform accurate numerical calculations through the Model Context Protocol. It provides precise mathematical operations via a simple API to overcome LLM limitations in arithmetic and statistical reasoning. Key features of Math-MCP: - Basic arithmetic operations: addition, subtraction, multiplication, division, modulo, and bulk summation - Statistical analysis functions: mean, median, mode, minimum, and maximum calculations - Rounding utilities: floor, ceiling, and nearest integer rounding - Trigonometric functions: sine, cosine, tangent, and their inverses with degrees and radians conversion support

    Developer Tools
    22 81