Users running a non-download version of Vanilla (pulled from github), on branch release/2019.016 or master from the last 2 weeks should upgrade to release/2019.017 or latest master for security reasons. Downloaded official open sources releases are not affected.

robots.txt on Vanilla clients forums

phreakphreak Vanilla*APP (White Label) & Vanilla*Skins Shop MVP
edited February 2016 in General Banter

Hi Vanilla's,

I found that some of the clients forums of Vanilla do have this Disallows in the robots.txt.

User-agent: * Disallow: /entry/ Disallow: /messages/ Disallow: /profile/comments/ Disallow: /profile/discussions/ Disallow: /search/

Can anyone tell me why to disallow the comments and the discussions through the profile URL? Aren't they indexed already? Wouldn't it be smarter to disallow the whole /profile/ directory.

Just asking. @Adrian, do you know more about it.

Thanx, phreak

  • VanillaAPP | iOS & Android App for Vanilla - White label app for Vanilla Forums OS
  • VanillaSkins | Plugins, Themes, Graphics and Custom Development for Vanilla
Tagged:

Comments

  • AdrianAdrian Wandering Spirit Montreal Vanilla Staff

    @phreak, The only thought I can have is this comment from Todd . Frankly I've never see any value in profile pages being indexed at all, so I normally don't bother in my self-hosted installs. I've never seen any issue from an SEO perspective in doing this in any way.

    Disallow: /profile/

    Sharing is caring

  • phreakphreak Vanilla*APP (White Label) & Vanilla*Skins Shop MVP

    Thanks @Adrian. I will make 2 test cases on a 100.000 UC website and report here. I hope it's only going up. ;)o:)

    • VanillaAPP | iOS & Android App for Vanilla - White label app for Vanilla Forums OS
    • VanillaSkins | Plugins, Themes, Graphics and Custom Development for Vanilla
  • LincLinc Director of Development Detroit Vanilla Staff

    Our concern was that it would be duplicate content to have posts/lists on the profile.

    Adrian
  • phreakphreak Vanilla*APP (White Label) & Vanilla*Skins Shop MVP

    Ok, could you ever verify this or was it solved through proper implementation of the canonical tag and is some kind of leftover from that time (i posted the robots.txt question 2010).

    • VanillaAPP | iOS & Android App for Vanilla - White label app for Vanilla Forums OS
    • VanillaSkins | Plugins, Themes, Graphics and Custom Development for Vanilla
  • LincLinc Director of Development Detroit Vanilla Staff

    @phreak said:
    Ok, could you ever verify this or was it solved through proper implementation of the canonical tag and is some kind of leftover from that time (i posted the robots.txt question 2010).

    No, I don't think anyone has looked at it further. My inclination is that it seems like a correct decision and I don't see value in reversing it.

Sign In or Register to comment.