Please upgrade here. These earlier versions are no longer being updated and have security issues.
HackerOne users: Testing against this community violates our program's Terms of Service and will result in your bounty being denied.

Robots.txt - Where do I edit ?

Robots.txt - Where do I edit ?

Comments

  • vrijvlindervrijvlinder Papillon-Sauvage MVP

    ummm, the robots.txt file maybe? this is what google said to me when I asked...

    http://www.robotstxt.org/robotstxt.html

  • R_JR_J Ex-Fanboy Munich Admin

    Use Notepad, not Word ;)

  • i was wanting to know where in FTP I can find robots.txt , looking at root level and all folders can' find it!

  • x00x00 MVP
    edited June 2015

    Try actually bothering to read the search results of google.

    If you took time to actually read you might learn something.

    grep is your friend.

  • I've read it - robots.txt is placed in top level , i cannot find this anywhere in ftp

  • x00x00 MVP
    edited June 2015

    This becuase you are not absorbing the information. Try another search "where to put the robots.txt".

    If it is not there you need to create it.

    Ftp is a tool. It is no in ftp, it is on your server, or not.

    grep is your friend.

  • x00x00 MVP
    edited June 2015

    This could hardly be clearer

    How to create a /robots.txt file
    Where to put it

    The short answer: in the top-level directory of your web server.
    ...

    grep is your friend.

  • R_JR_J Ex-Fanboy Munich Admin

    Read carefully:

    it firsts checks
    There are two important considerations when using /robots.txt:

    That is a hint that there does not need to be such a file. If you cannot find it, and you really searched hard, it is not there. Well, you've learned that it is optional, so it is no problem that it doesn't exist.

    If there is no such file, then create it. If you already know where to put it - great! Everything's fine... B)

Sign In or Register to comment.