August 12, 2019

Screaming Frog SEO Spider

Screaming Frog SEO Spider Icon

Screaming Frog SEO Spider is an advanced desktop SEO auditing tool and website crawler, built by real SEOs with thousands of users worldwide. It allows you to quickly crawl websites’ URLs, resources (links, images, CSS, scripts) and fetch key onsite elements (URL, page title, meta descriptions, headings) to evaluate onsite SEO.

You can view, analyse and filter the crawl data as it’s gathered and updated continuously in the program’s user interface. The program presents the data in tabs by type and allows you to filter for common SEO issues, or slice and dice the data how you see fit by exporting and importing into Excel.

As default, the SEO Spider crawls sites like Googlebot (it obeys allow, disallow directives and wildcard support like Googlebot), but presents its own user-agent ‘Screaming Frog SEO Spider’, which it will obey specific directives for in robots.txt. If there are no directives, it will crawl your site like Googlebot. while still presenting its own UA.

Moreover, The program can connect to the Google Analytics API and fetch user data, such as sessions or bounce rate and conversions, goals, transactions, and revenue for landing pages against the crawl.

The Screaming Frog SEO Spider lite version is free to use for crawling up to 500 URLs in a single crawl (multiple websites allowed) and it does not give you full access to the configuration, saving of crawls, the custom source search or extraction features and Google Analytics integration.

If you wish to unlock the 500 URL crawl limit and all the advanced features listed above you should purchase a license for £149 per year.

Buy Screaming Frog SEO Spider at £149 / year
Buy Now

Key Features:

  • Quick Website Crawler
  • Find Broken Links, Errors & Redirects
  • Analyse Page Titles & Meta Data
  • Discover Duplicate Content
  • Review Robots & Directives
  • Integrate with Google Analytics
  • Google Search Console Integration
  • Bulk export the errors and source URL
  • Generate XML Sitemaps
  • Extract Data with XPath
  • Crawl Configuration
  • Save Crawls & Re-Upload
  • Custom Source Code Search
  • Custom Extraction
  • Google Analytics Integration
  • Search Console Integration
  • Link Metrics Integration
  • Rendering (JavaScript)
  • Custom robots.txt
  • Free Technical Support

Last Update: March 05, 2019

Current Version:

License: Freeware / Shareware

Languages: English

Supported Operating Systems:
Windows XP / Vista / 7 / 8 / 8.1 / 10 (32-Bit, 64-Bit)
Mac OS X 10.7.3 or later

Developer: Screaming Frog Ltd


  • Fix a crash when using Forms Based Authentication at the same time, in two instances.
  • Fix a crash selecting a URL in the main window, for certain types of URL string.
  • Menus vanishing on mouse up on Windows, when used on multiple monitors.
  • Trailing space in meta charset causing page to be read with wrong charset.
  • Fix for API tab configuration buttons, which can be unresponsive.
  • Fix for crash showing open/save dialogs when last used directory has been deleted.
  • Fix for a crash when using AHREFs.
  • Debug check box doesn’t stay ticked.
  • Fonts not anti aliased in Ubuntu.
  • ga:socialActivity has been deprecated and removed to match Googles API changes.
  • Fix for a crash switching to Tree View, after loading in a saved project.
  • Sitemap reading doesn’t extract images.
  • Cookies stored against the wrong URL when using Forms Authentication.
  • Pop ups (authentication/memory etc) while minimised in Windows leaves app unresponsive.
  • SERP Snippet fixes.

Release Notes