Search engines use software to spider the Web and create their databases. These are also sometimes called robots, web crawlers, or worms. Web pages are retrieved and indexed by these search engines. When you enter a query at a search engine, your input is checked against the search engine's keyword indices. The best matches are then returned to you as Web pages of sites and descriptions that correspond to your query.
Each search engine indexes pages differently. When you submit your URL to a search engine, it sends out a spider to index your site and indexes (catalogs) selected information from your web page. What do they index? See the table below:
| What Search Engines Index | |||||
| Search Engine | Alt Tag | Comment | Description | Keyword | Text |
| Alta Vista | Yes | No | Yes | Yes | 100K |
| Excite | No | No | Yes | No | Full-Text |
| HotBot | No | Yes | Yes | Yes | Full-Text |
| InfoSeek | Yes | No | Yes | Yes | Full-Text |
| Lycos | Yes | No | No | No | Full-Text |
| Webcrawler | No | No | Yes | No | Full-Text |
Alta Vista vs Excite: For example, in Alta Vista when their spider indexes your URL, they catalog all ALT text for images, links (hrefs and images), title, description and keyword META tags, the page's URL and up to 100K of text into their database. Excite on the other hand, only indexes your META description tag and the full text of your Web page.
Meta Tags and frames: You may be asking for trouble by using frames. Some browsers still don't support frames, some search engines can not index a site with frames, and some people don't like frames.
- If you still want to use them, Meta tags should be included on individual framed pages and the franeset page. If you only use META tags on the frameset page, your site will not be indexed properly by some search engines.
- The following search engines do not support frames and will not index your site - Excite, Hotbot, Lycos and Webcrawler. Therefore, you should consider creating a no-frames version of your site and register these pages with these search engines
- For additional information about frames read this article at Search Engine Watch: Search Engines And Frames
