Site Name:
This should be the name of your web site for this account. For example:
|
 |
|
Gerald's Home Page |
|
|
Acme, Inc. Central |
|
|
Laura's Quake II Central Site |
|
Primary and Secondary URLs:
A Primary URL is your web site's main front page URL. If you are using a redirecting URL
(a URL that just redirects visitors to your true web site URL), do not enter it here.
You should only index URLs that actually contain content. Examples are:
|
|
|
http://www.yourdomain.com/ |
|
|
http://www.somedomain.com/mydirectory/ |
|
Secondary Entrypoint URL should be used for one of the following
reasons:
|
|
|
If you have additional web content on another web server that you would like to
index. Note: do not place mirror site URLs in this section -- otherwise, visitors will
receive duplicate results when they search your site. |
|
|
If you use JavaScript or Java to hyperlink between documents. Our spider
is only capable of finding standard anchor <A HREF> hyperlinks and image maps. In
which case, you would enter several of your site URLs in this field, so that our spider can properly
index them. |
|
|
If you use frames, enter the individual URL of the frames that contain your site
content and navigation bar hyperlinks. This will insure that the spider find all of your
documents/links properly. |
|
Help with Exclusion URLs and Masks:
An Exclusion URL or Exclusion Mask is a way to specify what URLs NOT to include in your
index. This is a useful option if you have data that you do not wish to have crawled,
thus, not showing up in the search results pages. You can enter the entire URL or
enter a URL Mask.
An Exclusion Mask allows you to enter just a part of the URL. If the spider
encounters a hyperlink that contains the Exclusion Mask, then it will skip that URL.
NOTE: Exclusion URLs
and Masks are useful particularly if you do not have access to edit the robots.txt
file. You have the ability to toggle if the spider should recognize robots.txt in the
Indexing Options.
Only enter one Exclusion URL/Mask per line.
Example:
In the example above, the first entry is an entire URL of a directory that will not be
indexed. The second entry instructs the spider not to crawl any document called
private.html regardless
of the directory. The third entry instructs the spider not to crawl any document in
any directory called /private_directory/. |
