CrawlDude

The free SEO crawler for your desktop.

Crawl any site and find broken links, redirect chains, missing titles, orphan pages and 50+ other issues. No licence key, no URL limit.

Download

Other platforms

Projects / Acme Outdoor https://acme-outdoor.com/ Crawling
AddressStatusIndexabilityTitleTitle lenInlinksResp. ms
/200IndexableAcme Outdoor | Tents, packs and trail gear421,102182
/tents/200IndexableTents for every trip | Acme Outdoor36844211
/tents/ridgeline-2p200IndexableRidgeline 2P ultralight backpacking tent, 1.2 kg, freestanding6337240
/sale301Non-indexable61248
/packs/summit-40200IndexableSummit 40 | Acme Outdoor2422198
/blog/how-to-pitch-a-tent200Indexable09305
/packs/trailhead-28404Non-indexablePage not found141791
/stores/stockholm200IndexableAcme Outdoor Stockholm2241,420
/account/login—Non-indexableBlocked by robots.txt1,102
/sleeping-bags/200IndexableSleeping bags | Acme Outdoor28790176

What it checks

Every crawl ends with a list of problems, sorted worst first. These are the ones people fix most.

  • Broken links and redirects

    Every 4xx and 5xx, every redirect chain and loop, and the pages that link to them.

  • Titles, meta and headings

    Missing, duplicated, too long or too short. Multiple H1s or none at all.

  • Site health score

    One number from 0 to 100 per crawl, so you can see whether last week's fixes worked.

  • XML sitemap audit

    Orphan pages nothing links to, plus errors and noindex URLs that shouldn't be in the sitemap.

  • Crawl as Googlebot

    Switch user agent to Googlebot, Bingbot or a browser. The crawler then follows that bot's robots.txt rules.

  • Duplicate and thin content

    Pages with the same content, and pages with too few words to rank.

  • Projects and crawl history

    Each site is a project. Every crawl is saved, with its own settings, ready to reopen.

  • CSV export

    Any table, any tab, straight into a spreadsheet or a client report.

How it works

  1. 1

    Download and open CrawlDude.

  2. 2

    Paste your site's URL and press Start.

  3. 3

    Work through the issue list, worst first.

Why CrawlDude

Desktop crawlers usually cost a licence fee and stop at a few hundred URLs until you pay. This one doesn't.

Free, with no URL cap
Crawl ten pages or a hundred thousand. Your computer is the only limit.
Your data stays put
Crawls run and are stored on your computer. Nothing is uploaded, and there is no account.
Open source
MIT licensed. Read the code, report a bug or send a fix on GitHub.
Polite by default
Respects robots.txt and crawl delays, so you won't knock over a client's server.

Download

Latest release on GitHub.

PlatformFileSizeDownload
macOS, Apple SiliconCrawlDude-mac-arm64.dmgDownload
macOS, IntelCrawlDude-mac-x64.dmgDownload
Windows, 64-bitCrawlDude-win-x64.exeDownload
Linux, AppImageCrawlDude-linux-x64.AppImageDownload
Linux, Debian and UbuntuCrawlDude-linux-x64.debDownload

Release notes Older versions

Questions

Something else? Email hello@crawldude.com or open an issue on GitHub.

Is it really free?

Yes. CrawlDude is free and open source under the MIT licence. There is no URL limit, no trial and no paid tier.

Is my data sent anywhere?

No. Crawls run on your computer and are saved there. CrawlDude only talks to the sites you crawl, and to GitHub to check for updates.

Does it render JavaScript?

Not yet. It reads the HTML the server sends, which is also what most search engines see first.

My Mac says CrawlDude can't be opened.

That happens with builds that aren't signed by Apple yet. Right-click CrawlDude in Applications, choose Open, then confirm. You only need to do this once.

Windows says it protected my PC.

Windows SmartScreen warns about new installers it hasn't seen often. Click More info, then Run anyway.

How do I update?

CrawlDude checks for new versions when it starts and updates itself. You can also download the latest installer here at any time.

What do I need to run it?

macOS 11 or later, Windows 10 or later, or a recent 64-bit Linux.