Search | View Active Threads | View New Posts
Current Time: 02:33 AM

 

+ Reply to Thread
Results 1 to 4 of 4
  1. #1

    What is robots.txt?

    What is robots.txt?

  2. #2
    Hi Friends,

    The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

    Dynamic Web Design Company Bangalore | Joomla Web Development Company Bangalore | Php Web Development Company Bangalore

  3. #3
    Join Date
    Jun 2018
    Location
    surat
    Posts
    455
    Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website.

  4. #4
    Join Date
    Feb 2018
    Location
    US
    Posts
    290
    Robots.txt is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  

Sponsors

Above.com Domain Registrar, Marketplace and Revenue Optimizer
Domain Escrow Services
Trellian Direct Search Network

Industry Events

11-14 August 2019
Affiliate Summit East
New York, NY

4-6 March 2019
LeadsCon
Las Vegas

15-18 September 2018
MERGE 2018
Orlando, FL

Poll

How do you invest in domains?
to see the Poll results!
DomainState.com
Domain Tools | Domain Directory | Registrar Stats | Domain Glossary | Industry Events | FAQ | Members | Terms | RSS | Link To Us | Advertise | Contact Us
Other Related Trellian Services:
Above.com Registrar  |  Above.com Marketplace  |  Above.com Domain Manager  |  Free Search Toolbar  |  Free Webpage Builder  |  Keyword Research
Direct Search Navigation  |  Search Engine Submission and SEO Tools
Copyright © 2002 DomainState.com a Trellian Company 2014 Bronze Sponsor - Internet Commerce Association