Link Checking with MOMspider

Tired of having to go through and check all your external links to make sure they still work? We've taken a proven link checker and made the output easier to understand and use.

For each domain we host, we will check your links for free each month. For a slight fee, we will check them more often or check them for domains we do not host.

Combined with our free weekly error log analysis, this will flush out bad links internal to your site, from your site to other sites and from other sites to yours.

NOTES:

• MOMspider does not check internal links(e.g. page.html#internal). It only verifies that the page is there. You must check your internal links by hand.
• If the URL contains a ?(CGI script with a GET), then MOMspider only requests the URL before the ?. This will produce errors if the script requires arguments.

Example Report

Also, the smart_errors.html page, which demonstates bad HTML code and bad links, produces errors as expected.

    For more information on this report and how to use it, see:
    http://www.clarkecomputer.com/services/momspider.html

    This message was automatically generated by MOMspider/1.00 after a
    web traversal on Wed, 01 Sep 1999 16:11:17 

    If you have questions about this report, please forward the report along with
    your question to: momspider@clarkecomputer.com

The following parts of the clarke infostructure may need inspection:

The following pages have problems:
<http://www.clarkecomputer.com/>
    <http://ad.linksynergy.com/fs-bin/show> - redirected to:
	<http://www1.linksynergy.com/fs-bin/show?>
    <http://www.alladvantage.com/go.asp> - redirected to:
	<home.asp?refid=>
<http://www.clarkecomputer.com/competition.html>
    <http://www.Get-A-Host.com/signup/clarkecomp> - redirected to:
	</signup>
<http://www.clarkecomputer.com/links/barnes/scifi.html>
    <http://shop.barnesandnoble.com/booksearch/results.asp> - redirected to:
	</bookSearch/search.asp?userid=26CE10J9JR&mscssid=GH8B4U1NCQSH2NHT0017QJ1R0HLGEMQ6&pcount=0&srefer=&salesurl=Rwww.clarkecomputer.com/links/barnes/scifi.html&emessage=101>
<http://www.clarkecomputer.com/smart_errors.html>
    <http://www.clarkecomputer.com/clarke@clarkecomputer.com> - 404 Not Found
    <http://www.clarkecomputer.com/this_page_isnt_here.html> - 404 Not Found
<http://www.clarkecomputer.com/submit/submit.html>
    <http://ad1.webpromote.com/wplink> - redirected to:
	<http://www.webpromote.com/permission/join.asp>
    <http://ads.smartclicks.com/2/B086860/smartsite> - redirected to:
	<http://www.smartclicks.com>
    <http://ads.smartclicks.com/2/XC0/B086860/clickbar> - redirected to:
	<http://www.smartclicks.com>
    <http://www.submit-it.com/refer> - redirected to:
	<http://submitit.linkexchange.com/system/SIReferral.cfm?>

WebLint Errors:
<http://www.clarkecomputer.com/smart_errors.html>

How to track down and fix the problems

Redirected links - These indicate that a URL has moved and the server was nice enough to give you the new address for the URL. The most common cases are directories without a / on the end of the URL and, on our server which does "spelling correction", spelling or capitalization corrections. Sometimes redirects are used by webmasters when they are changing the structure of a site, want folks to be able to access the pages under the old URLs, but want them to start using the new URLs.

Change this link to where MOMspider tells you that the URL is redirected.

NOTE: DON'T do this for advertising/affiliate links as you will no longer get credit for the referrals if you do. Unfortunately, there is no way for us to automatically determine which are affiliate redirects and which are redirects indicating a relocation of a page.

Also, note that some pages redirect you to a page which redirects you to another, etc. If you copy and paste the URL into your browser, the URL in the Location box on your browser should be the final URL to which the original page was redirected.

Even though you may be tempted to ignore redirects, they are very useful when someone changes a web site's structure and sets up redirects to give you a few months to change your links. Fixing these will also speed up the loading of the page or image because the browser doesn't have to get the redirect notice and then request the real page.

You will see some redirects where all that is changed is a slash(/) appended to the path. This is the proper form for any directory URL's.

404 Not Found - These indicate pages that are no longer, or never were, at that URL. The 404 is just a status code that the web server returns to indicate a URL that is not found. You should either remove the link, or locate the new URL.

To locate where a page has moved to, you can strip off the trailing components of the URL until you find a URL that works(this may be the top of the domain). On that page, look for reference to the page for which you previously had a link. Find the new URL and change your link to the new URL.

If this doesn't work, you can try searching for the page on a search engine.

Now don't you wish they had used a redirect in the first place! :)

WebLint Errors

- These are pages that weblint finds some problem with. Although most warnings are turned off when checking through MOMspider, some still show. By default, most sites are not checked for WebLint errors, but this can be changed by sending an email to momspider@clarkecomputer.com requesting that your pages be checked. The less weblint problems you have in your pages, the more likely it is that your page will show correctly on the different browsers.

To fix, copy the URL that WebLint doesn't like and paste it into: http://www.fortnet.org/cgi-bin/WebLint.pl, overwriting the /www/htdocs/ that is there by default. Submit the form and look at the results. If you have problems understanding why something is considered a problem and the explanation does help, please send an email to: weblint@clarkecomputer.com and I will try to help you understand the problem.

Repeat this for each URL that has a problem.

NOTE: The MOMspider weblint doesn't check for as many problems as the FortNet weblint does, so don't be surprised if you haven't fixed all the weblint warnings, but don't get a complaint about a page during the next run of MOMspider.

If the above doesn't make sense, or you need more explanation of your MOMspider report, then please forward the report along with your question(s) to momspider@clarkecomputer.com.