Results 1 to 9 of 9
  1. #1
    Registered User
    Join Date
    Oct 2012
    Location
    New Delhi
    Posts
    14

    Lightbulb How to Create Robots.txt Files

    How to Create Robots.txt Files?????????????????

    _________________________________
    picnic spots near gurgaon

  2. #2
    Senior Member
    Join Date
    Dec 2011
    Posts
    145
    Hi there. If you want, then here you go:

    http://www.seomoz.org/learn-seo/robotstxt

    Here is different types of way.

    Thanks
    Follow us on Facebook & Twitter

  3. #3
    Registered User
    Join Date
    Sep 2012
    Location
    Delhi,India
    Posts
    23
    Hi priyawilson, robot.txt file creation is very easy and its depend on your need that which code is written.
    For example if you want to disallow all robots to visit your site then use the given code:
    User-agent: *
    Disallow: /

    For more information read http://www.robotstxt.org/robotstxt.html

  4. #4
    Junior Member
    Join Date
    Feb 2012
    Location
    Japanese
    Posts
    21
    Sorry, what do you mean?

  5. #5
    Member
    Join Date
    Aug 2012
    Location
    India
    Posts
    78
    It's not a difficult task to create a robots.txt file. Its so simple to create. Take a look below:

    User-agent:*
    Disallow:

    It is by default setting of a robots.txt file. You have to upload it in root directory of website.

  6. #6

  7. #7
    Senior Member
    Join Date
    Nov 2018
    Posts
    1,693
    Open Notepad, Microsoft Word or any text editor and save the file as 'robots,' all lowercase, making sure to choose . txt as the file type extension (in Word, choose 'Plain Text' ).

  8. #8
    Registered User iastitlesearch's Avatar
    Join Date
    Oct 2019
    Location
    United States
    Posts
    81
    A robots.txt file lives at the root of your site. So, for the site www.example.com, the robots.txt file lives at www.example.com/robots.txt.

    Example:

    # Group 1
    User-agent: Googlebot
    Disallow: /nogooglebot/

    # Group 2
    User-agent: *
    Allow: /

    Sitemap: http://www.example.com/sitemap.xml

  9. #9
    Senior Member
    Join Date
    Aug 2020
    Posts
    1,275
    Open Notepad, Microsoft Word or any text editor and save the file as 'robots,' all lowercase, making sure to choose. txt as the file type extension (in Word, choose 'Plain Text' ).
    Next, add the following two lines of text to your file.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting


Premium Partners:


Visit forums.thewebhostbiz.com: to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.