Broken Link Building In Seo
Jessica Johansson, Jul 7, 2023, Search Engine Optimization
Broken Link Building In Seo by Jessica Johansson
Google Latest Update In Seo by Jessica Johansson
Canonical Tag Usability by Jessica Johansson
Paid Guest Blog Vs Free Guest Blog by Jessica Johansson
Free Blog Posting Vs Paid Blog Posting by Jessica Johansson
Increasing Domain Authority by Jessica Johansson
Online Vs Offline Marketing by Jessica Johansson
Potential Sources For Increasing Web Traffic by Jessica Johansson
Best Social Networking Sites by james jennifer
Improving Small Online Business Through Seo by Jessica Johansson
4:13 PM
▼
▲
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What is Robots.txt?

Discussion in 'Search Engine Optimization' started by Stephen Johnson, Dec 15, 2012.

Page 2 of 2
  1. taylor swift

    taylor swift
    Expand Collapse
    Junior Member

    Joined:
    Jul 30, 2012
    Messages:
    113
    Likes Received:
    1
    thank you buddy
     
    #21 taylor swift, Aug 28, 2013
  2. subin

    subin
    Expand Collapse
    Junior Member

    Joined:
    Aug 25, 2013
    Messages:
    41
    Likes Received:
    8
    Robots.txt is the tool to carwl all weblinks and reach faster in all search engines.
     
    #22 subin, Aug 29, 2013
  3. amansharma

    amansharma
    Expand Collapse
    Junior Member

    Joined:
    Mar 20, 2013
    Messages:
    19
    Likes Received:
    0
    Robots.txt file is a text file which is uploaded on the root of the website. The main task of this file is to instruct search engine crawler that which site they have to visit and which are not.
     
    #23 amansharma, Sep 1, 2013
  4. subin

    subin
    Expand Collapse
    Junior Member

    Joined:
    Aug 25, 2013
    Messages:
    41
    Likes Received:
    8
    Robots.txt is the text file tool to crawl all web pages faster to index them for all top search engines
     
    #24 subin, Sep 2, 2013
  5. Alex_Thompson

    Alex_Thompson
    Expand Collapse
    Junior Member

    Joined:
    Sep 13, 2013
    Messages:
    59
    Likes Received:
    0
    It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want. For instance, if you have two versions of a page (one for viewing in the browser and one for printing), you'd rather have the printing version excluded from crawling, otherwise you risk being imposed a duplicate content penalty.
     
    #25 Alex_Thompson, Sep 16, 2013
  6. subin

    subin
    Expand Collapse
    Junior Member

    Joined:
    Aug 25, 2013
    Messages:
    41
    Likes Received:
    8
    Robots.txt to identify and analysis the websites to crawl faster in reach all search engines.
     
    #26 subin, Sep 16, 2013
  7. pryidevsblog

    pryidevsblog
    Expand Collapse
    Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    110
    Likes Received:
    8
    Technically perfect answer amisha. Answer says a lot words in just two lines.
     
    #27 pryidevsblog, Oct 25, 2013
  8. arunmarvel6

    arunmarvel6
    Expand Collapse
    Junior Member

    Joined:
    Sep 7, 2013
    Messages:
    17
    Likes Received:
    0
    It helps to restrict the particular web pages from crawling of search engines.
     
    #28 arunmarvel6, Oct 29, 2013
  9. Paultours

    Paultours
    Expand Collapse
    Junior Member

    Joined:
    Aug 31, 2013
    Messages:
    12
    Likes Received:
    0
    we make use of the robots text file to tell the search engine about which page we dont want to crawl by the search engine bots.
     
    #29 Paultours, Oct 29, 2013
  10. emma james

    emma james
    Expand Collapse
    Registered Member

    Joined:
    Oct 28, 2013
    Messages:
    11
    Likes Received:
    1
    Robots.txt means it states that which file or a folder should be viewed or which should not.
     
    #30 emma james, Oct 30, 2013
  11. anupam sharma

    anupam sharma
    Expand Collapse
    Junior Member

    Joined:
    Sep 7, 2013
    Messages:
    105
    Likes Received:
    4
    Robot.txt is used in on page so as to hide some coding from the google bots....
     
    #31 anupam sharma, Oct 30, 2013
  12. abhishek sharma

    abhishek sharma
    Expand Collapse
    Registered Member

    Joined:
    Oct 31, 2013
    Messages:
    13
    Likes Received:
    2
    Robot text are used by the programmers a part of coding from the bots of google when the content is crawled by search engine bots
     
    #32 abhishek sharma, Oct 31, 2013
  13. Paultours

    Paultours
    Expand Collapse
    Junior Member

    Joined:
    Aug 31, 2013
    Messages:
    12
    Likes Received:
    0
    Robots.txt is a text file which we upload on our website to tell the various search engines about which part of our website we dont to crawl or index.
    It takes the following format:
    User Agent: Search engine spider
    Disallow: link of our website
     
    #33 Paultours, Nov 16, 2013
  14. tom&jeny

    tom&jeny
    Expand Collapse
    Registered Member

    Joined:
    Nov 14, 2013
    Messages:
    8
    Likes Received:
    0
    what is robots.txt?
    >It is an html tag placed on the source of a webpage which directs search engine spiders which files to crawl on or not.
     
    #34 tom&jeny, Nov 18, 2013
(You must log in or sign up to post here.)
Page 2 of 2

Share This Page