Screaming Frog SEO Spider

Screaming Frog SEO Spider is a desktop SEO crawler for Windows, macOS and Linux. It explores a website the way Googlebot does and surfaces, in minutes, everything holding back its rankings: broken links, redirects, missing or duplicate title tags and meta descriptions, blocked pages and thin content. It is the go-to tool for SEO agencies and technical teams to audit a site or secure a migration.

What is Screaming Frog SEO Spider used for?

The tool crawls every accessible URL on a site and sorts them into thematic tabs. At a glance you can see:

• Response codes: 404 pages, 5xx server errors, 301 and 302 redirects, redirect chains and loops.
• SEO tags: missing, too long, too short or duplicate titles and meta descriptions, missing or multiple H1s.
• Indexability: noindex tags, canonicals, pages blocked by robots.txt, orphan pages.
• Internal linking: inlinks per page, crawl depth, anchor text.
• Images: oversized files and missing alt attributes.
• Structured data: presence and validity of JSON-LD.

Free version or paid licence?

The free version of Screaming Frog SEO Spider is limited to 500 URLs per crawl. That is enough for a small business website or a first diagnosis. The paid annual licence removes that limit and unlocks advanced features: JavaScript rendering, custom extraction (XPath, CSS, regex), Google Analytics, Search Console and PageSpeed Insights integrations, scheduled crawls and crawl comparison.

Run your first audit in 5 steps

1. Download and install Screaming Frog SEO Spider from the official website, then open the app.
2. Paste your website URL into the top bar (Spider mode) and click Start.
3. Wait for the crawl to finish: the progress bar reaches 100% and the right-hand panel shows a summary of issues by category.
4. Open the Issues tab: each issue is ranked by priority (high, medium, low) with an explanation and the list of affected URLs.
5. Export the results to CSV or Google Sheets to prioritise fixes with your team.

Practical use cases

Preparing and checking a site migration

Before a redesign, crawl the old site to get the full list of URLs. Once the new site is live, switch to List mode, import that list and check that every old URL 301-redirects to the right new page. This is the method we use at Mazette on every migration to Webflow so no rankings are lost.

Cleaning up titles and meta descriptions

Filter the Page Titles tab on "Duplicate" or "Over 60 characters" to spot the pages to rewrite first in a few seconds.

Strengthening internal linking

The Inlinks column shows how many internal links point to each page. A key page with only one or two inlinks deserves to be better connected to the rest of the site.

Our tips for crawling a Webflow site

• Webflow serves clean, server-rendered HTML: JavaScript rendering is usually not needed, a standard crawl is enough.
• Check that every CMS collection page has a dynamic title and meta description, it is the most common oversight.
• Keep an eye on utility pages and unwanted templates that can end up in the sitemap.

Screaming Frog alternatives

• Sitebulb: another desktop crawler, with more visual reports and more beginner-friendly explanations.
• Semrush Site Audit and Ahrefs Site Audit: cloud-based crawls, no installation, built into a complete SEO suite.
• Lumar: built for very large websites and enterprise teams.

Screaming Frog remains the most complete and flexible option for a precise technical audit, especially when you need custom extractions.

Frequently asked questions

Is Screaming Frog SEO Spider free?

Yes, up to 500 URLs per crawl. Beyond that, and for advanced features, you need an annual licence.

Does Screaming Frog work on Mac?

Yes, the app is available on macOS, Windows and Linux.

Do you need technical skills to use it?

Starting a crawl is simple. Interpreting the results, however, requires solid technical SEO knowledge. If you want a turnkey audit, our SEO team can take care of it.