| |
Indexer Spider
Indexers work hand in hand with search engine spiders and crawlers and are responsible for processing the pages that are found and downloaded by its spiders. An indexer will process web pages by removing all the HTML tags, discovering & storing web links, removing filter words (to reduce file size), checking for stop words and finally by storing the web pages in a searchable database.
|
|
