Guide to Use Screaming Frog



Screaming Frog Website SEO Audit ToolThis particular spider is a desktop application you can download and run on your local PC, regardless of platform. It fetches SEO data, including URL, metadata, Schema categories, and more. A spider is a piece of software that crawls around on a website, harvesting data and presenting it to the owner of the spider.

The primary benefit of Screaming Frog’s Spider is the ability to search for and filter various SEO issues. You don’t have to have a deep knowledge of SEO to figure out what is and isn’t done properly; the tool will help filter it for you. It can find bad redirects, Meta refreshes, duplicate pages, missing Metadata, and a whole lot more.

Now, the basic tool is the free version of the tool, which you can download and use for free. However, it limits you in several notable ways. Primarily, you can only crawl 500 URLs with it. I highly recommend, if you have a medium or large-sized site with over 500 URLs you would want to crawl, that you buy the full license. It’s an annual fee of 99 British Pounds, which works out as of this writing to be about $140. Given that it works out to be under $12 per month, most businesses can easily afford it, and it’s well worth the price.


How to Crawl Sites with Screaming Frog:-

The way you use the program is simple. Simply click the URL you want to crawl and go to options. Once there, choose the “crawl all subdomains” option. The program gives you the option to crawl JavaScript, CSS, external links and SWF so you can get a full SEO view of any site. You can choose to uncheck any of these options if you want the site to be crawled faster, or if you only want the page and text elements of a site to be checked.

You can now start the crawl and wait for the results. Please note that the speed at which sites are crawled is not only dependent on the number of options you have checked, but on the speed of your machine as well.

Next step is clicking the internal tab and filtering the results by HTML. Next, click Export. This will give you a complete CSV chart of all the data that has been crawled sorted by each individual HTML page. This info can then be used to identify problems with certain pages and address them to use screaming frog


How to Perform a Link Audit:-

Performing link audits can be difficult because of the sheer number of links in any given website. But performing link audits is easy with Screaming Frog. All you have to do is to check the “all subdomains” option in the Spider menu, but leave all other options unchecked except for links. From there, you’ll be able to choose if you want no-follow links to be crawled as well. You can now start Guide to use Screaming Frogthe crawl and wait for it to be completed. In the advanced menu section, click “all links” and wait for the report to be generated. A report in CSV will then be created of all the links, including information on their locations, anchor text, destinations, directives and other relevant data.


 Performing a Link Audit with Screaming Frog


Links are difficult to audit because they can be difficult to harvest. How many links do you have on a typical page? Couple that with all of your parameters and you have a lot of information you need to gather.

How to do it with the spider…..??

In the spider configuration menu, check all subdomains but uncheck CSS, images, JavaScript, Flash, and any other options you don’t need. Decide if you want to crawl no-followed links and check the boxes accordingly. Initiate the crawl and let it run until it’s finished.

Click the Advanced Report menu and click “All Links” to generate and export a CSV of all of the links it crawls, including their locations, their destinations, their anchor text, their directives, and other data. From here you can export the data, or you can sort it as much as you like. Here are some sorts and actions you can perform.

Click the internal tab and sort by out links. This will show you the pages with the most links on your site. Pages with over 100 links are generally suspect according to Google so you may want to audit those pages to determine why they have so many links, and how you can minimize them.

Click the internal tab and click status code. Any links that show the 404 status code are links that are broken; you will want to fix those links. Links that report a 301 or other redirects may be redirecting to homepages or to harmful pages; check them and determine if they should be removed. This SEO Guide shows you how to use Majestic and Screaming Frog together to find internal linking opportunities.