6

I'm administering EE site with forum. On the forum, there are many redirects in the form of http://domain.com/?URL=http://third-party-site.com/page.html

Recently, Google started returning 404 for my site if the page on third-party site (which I can't control) does nit exist. What should I do to avoid that, except for blocking "?URL" urls in robots.txt (especially having in mind that the links existed always, but started returning errors just recently, EE version has not been updated)

5
  • I'm not posting this as an answer, because I'm just guessing. Have you tried to add rel="nofolow"?
    – Sobral
    Oct 25, 2013 at 10:50
  • @Robson, I did not. Many of the links are from users' signatures on forum, and I don't have contol over formatting in them. The problem is not really that remote URLs are 404, the problems is that they are 404 for MY site.
    – Yuri Salimovskiy
    Oct 25, 2013 at 12:11
  • I understood. Try to add http:domian.com/?URL= to the blocked URLS on robots.txt. Again, I'm just guessing.
    – Sobral
    Oct 25, 2013 at 13:07
  • Thanks, already did that. Wondering whether there is better or more native to EE solution.
    – Yuri Salimovskiy
    Oct 25, 2013 at 14:10
  • We are experiencing a similar issue and it looks like this thread still doesn't have an accepted answer. I'm willing to give Jay F's a try, but @Yuri did you use his approach or a different one? Dec 27, 2013 at 15:04

2 Answers 2

1

You can do this in your htaccess

RewriteCond %{QUERY_STRING} URL [NC]
RewriteRule .* - [R=404,L]

And, EE will return a 404 error code. BUT, EE seems to always return the page that says To proceed to the URL you have requested, click the link below. It won't return your 404 template that you've set in EE (EE seems to give that ?URL parameter precedent over its 404 template processing.)

So, I'd worry that the above is not enough to right things with the Google index, and I'd throw the stronger 410 "Gone" error, instead--plus route it to a static error page outside of EE:

#optional, show a static file as an error document
ErrorDocument 410 /410.html
#
RewriteCond %{QUERY_STRING} URL [NC]
RewriteRule .* - [R=410,L]

EDITED: "Gone" is status code 410 -- corrected, above

I think Google (and other search engines) recognize that people who know how to throw 400 errors aren't kidding, and so those "gone" resources can be removed from the search index right away ;-)


Note that, if you don't use EE for your 404 page--e.g., you instead have ErrorDocument 404 /404.html set in your htaccess pointing to a static page, then the original example of throwing a 404 will totally circumvent EE, and show your static 404 error page.

1
  • I guess I should've asked: do you want the URL redirect to continue working? I was assuming that you wanted to kill the redirect altogether.
    – Jay F
    Nov 27, 2013 at 19:32
0

One option is to add <meta name=”robots” content=”noindex, nofollow”> to the forum header so none of the pages on the forum are indexed. You can do that here:

themes > forum_themes > default > forum_global > meta_tags.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.