about dmoz | help

Top: Computers: Internet: Searching: Search_Engines: Robots

Web robots (also known as crawlers or spiders) are programs that traverse the Web automatically, and which are used by search engines to index the Web, or part of it.

• Open Directory Computers: Internet: E-mail: Spam: Preventing
• Open Directory Computers: Internet: Searching: Search Engines: Sitemaps
• Open Directory Computers: Internet: Web Design and Development: Authoring: Online Tools: robots.txt
• Open Directory Computers: Software: Internet: Site Management: Log Analysis

Copyright © 1998-2007 Netscape