Test and Report – How to

How to Test and Report

After ScreamingFrog has finished scanning the entire site, it's time to analyze the findings:

NOTE: For each report you'll be asked to export, it is better to choose XLSX format, rather than CSV.

Response Code

Error 4xx:

  • Go to Bulk Export -> response Codes -> Client Error (4xx) inline
  • Save the file.
  • Error 5xx:

  • Go to Bulk Export -> response Codes -> Server Error (5xx) inline
  • Save the file.
  • No response:

  • Go to Bulk Export -> response Codes -> No response inline
  • Save the file.
  • Meta Titles

    Go to 'Page Titles' tab

    Missing Meta Titles:

  • Choose Filter = Missing.
  • If there are records in the filter, the test result is: Fail.
  • If the filter is empty of records, the test result is: Pass.
  • Duplicate Meta Titles:

  • Choose Filter = Duplicate
  • If there are records in the filter, the test result is: Fail.
    Exclusion: If the duplicate headers have a canonical tag, or if pages with a duplicate header are not open to index, that's fine.
    You can check this by scrolling to the right of the rows, over to the Indexability & Indexability Status columns.
    If all of the duplicate pages have only one that is labeled Indexable and the rest are Non-Indexable, that's fine. If the filter is empty of records, the test result is: Pass.
    If the filter is empty of records, the test result is: Pass.
  • If the filter is empty of records - the test result is: Pass.
  • H1

    Go to H1 tab

    Unique H1:

  • Choose filter = Duplicate
  • If there are records in the filter, the test result is: Fail.
    Exclusion: If the duplicate headers have a canonical tag or pages with a duplicate header are not open to index, that's fine.
    You can check this by scrolling to the right of the rows, over to the Indexability & Indexability Status columns.
    If all of the duplicate pages have only one that is labeled Indexable and the rest are Non-Indexable, that's fine. If the filter is empty of records, the test result is: Pass.
  • If the filter is empty of records, the test result is: Pass.
  • Missing H1:

  • Choose filter = Missing
  • If there are records in the filter, the test result is: Fail.
  • If the filter is empty of records, the test result is: Pass.
  • Image size

    Go to "Images" tab

  • Choose filter = Over 100 KB
  • If there are records in the filter, the test result is: Fail.
  • If the filter is empty of records, the test result is: Pass.
  • Self Canonical

    Go to the 'Canonicals' tab

  • Choose filter = Missing
  • If there are records in the filter, the test result is: Fail.
  • If the filter is empty of records, the test result is: Pass.
  • In order to export the data related to self-canonicals, click the 'Export' button on the right.

    Site Speed

    In ScreamingFrog (for the entire site)

    Go to the right hand section of ScreamingFrog to 'Response Times' tab.

    If there are results that are higher than 3 seconds, the test result is: Fail.

    Exclusion: If the number of pages that take longer than 3 seconds to respond are negligible for all site pages, it's okay.

    If the entire site is between 0 and 3, the test result is: Pass.

    In Pingdom tool (for a specific URL):

  • Go to https://tools.pingdom.com/
  • Put the page URL in the text area
  • From the resulting list, choose the nearest server to your audience
  • Click 'Start test'
  • Wait for the scan to run.
  • If there are results higher than 3 seconds, the test result is: Fail.
  • If the entire site is between 0 and 3 seconds, the test result is: Pass.
  • Pagination

  • Go to the archive page of your site (Category page, Tag page, Product category page etc.)
  • Scroll down the page
  • If the site loads more and more items while you keep scrolling the page, you have Infinite Scroll
  • If you need to click on page numbers or "next" and "previous" buttons to browse pages, you have Pagination.
  • All pages are indexable

    Go to the 'Internal' tab

  • Choose filter = Noindex
  • If there are records in the filter, the test result is: Fail.
  • If the filter is empty of records, the test result is: Pass.
  • Google Analytics

  • Open your site in a browser.
  • Right click on the mouse and select 'View Source'
  • Search for: UA-
  • If you find a number in the following style:
  • UA-1234567-1
  • You have Google Analytics. Test result: Pass.
  • If no test result: Fail.
  • Exception: If you embedded GTM in your site (or someone else did) the Google Analytics code probably embedded there and not directly in the code, so you may not find the analytics in the code. If you're using GTM (Google Tag Manager) on your website you need to ask your web developer to check if Google Analytics is implemented as part of GTM
  • Google Search Console

  • Open your site (homepage only) in a browser.
  • Right click on the mouse and select 'View Source'.
  • Search for 'google-site-verification'.
  • If found, then you have Google Search Console. Test result: Pass.
  • If not, the test result is: Fail.
  • Exception: There is a possibility that Google Search Console authentication has been set up differently, either through Google Analytics, a file uploaded to the server, or through Google Tag Manager.
  • If you did not find the search term, you’ll need to check with the site builder to find out whether you have Google Search Console or not.

    Sitemap XML

    In most cases, websites use 'sitemap.xml' as the name for the sitemap file for the website and place it under the root directory of the site.

    In order to find the sitemap file you can try any of the following:

  • Open your site in a browser and add '/sitemap.xml' to your domain (domain.com/sitemap.xml).
  • Open your site in a browser and add '/robots.txt' to your domain (domain.com/robots.txt). See if you can find a line that starts with "sitemap:". If so, the full-path to your sitemap file will be there.
  • If you have access to the Google Search Console account for the website, you can go to 'Index -> Sitemaps' and see if any sitemap files have already been submitted there.
  • If you couldn't find a sitemap file in any of the above, the test result is: Fail.
  • If you found a sitemap file, the test result is: Pass.

  • ScreamingFrog can be used to create a manual sitemap file. To do so, follow the steps below:

    1. Open ScreamingFrog.
    2. Crawl the site.
    3. Click on the Sitemap menu.
    4. Choose XML Sitemap.
    5. Click NEXT (no setting needed).
    6. Save the file.
    7. Upload the file to the ROOT directory on your server.

    Indexed pages

    Go to "internal" tab

  • Choose filter = HTML
  • Check the "Filter Total" at the bottom right
  • Go to google.com
  • Search for your site with "site:" before the domain. example: site:domain.com
  • Look at the Search results and check whether or not the amount of results is close to or equal to the HTML amount you had in ScreamingFrog
  • If no, the test result is: Fail.
  • If yes, the test result is: Pass.
  • SSL

  • Open a browser and type your website's name into the address bar using HTTPS.
  • If you get an error, then you don't have SSL. Test result: Fail.
  • If your site loads nicely, then you do have SSL. Test result: Pass.
  • Open a browser and type your website's name into the address bar using HTTP (without S) and check the address bar line:
    1. If it appears to have HTTPS – your website supports SSL. Test result: Pass.
    2. If it remains HTTP – Test result: Fail.
  • Blog / Article area

    Check if you have a Blog or an articles area on your site.

  • If no, the test result is: Fail.
  • If yes, the test result is: Pass.
  • Duplicate content

  • Visit 1 static page, 1 blog page and 1 product page (if you have any).
  • Take a paragraph from each page and search for it on Google (you can also use a premium service such as Copyscape).
  • If there are results that are not from your site, the test result is: Fail.
  • If your site is the only result, the test result is: Pass.
  • Proper entrance pages

  • Go to the Google Search Console on your site.
  • Go to Search results area.
  • Click Export -> Download Excel.
  • Open Excel.
  • Go to the "Pages" tab.
  • Select all in row A. Copy it (Ctrl +c).
  • Open ScreamingFrog.
  • Go to Mode ->List.
  • Click "Upload".
  • Click paste.
  • Click OK.
  • Wait for crawling to end.
  • Check for 4xx, 5xx & No Response errors (as before).