6

I'm administering EE site with forum. On the forum, there are many redirects in the form of http://domain.com/?URL=http://third-party-site.com/page.html

Recently, Google started returning 404 for my site if the page on third-party site (which I can't control) does nit exist. What should I do to avoid that, except for blocking "?URL" urls in robots.txt (especially having in mind that the links existed always, but started returning errors just recently, EE version has not been updated)

  • I'm not posting this as an answer, because I'm just guessing. Have you tried to add rel="nofolow"? – Robson Sobral Oct 25 '13 at 10:50
  • @Robson, I did not. Many of the links are from users' signatures on forum, and I don't have contol over formatting in them. The problem is not really that remote URLs are 404, the problems is that they are 404 for MY site. – Yuri Salimovskiy Oct 25 '13 at 12:11
  • I understood. Try to add http:domian.com/?URL= to the blocked URLS on robots.txt. Again, I'm just guessing. – Robson Sobral Oct 25 '13 at 13:07
  • Thanks, already did that. Wondering whether there is better or more native to EE solution. – Yuri Salimovskiy Oct 25 '13 at 14:10
  • We are experiencing a similar issue and it looks like this thread still doesn't have an accepted answer. I'm willing to give Jay F's a try, but @Yuri did you use his approach or a different one? – Adam Christianson Dec 27 '13 at 15:04
1

You can do this in your htaccess

RewriteCond %{QUERY_STRING} URL [NC]
RewriteRule .* - [R=404,L]

And, EE will return a 404 error code. BUT, EE seems to always return the page that says To proceed to the URL you have requested, click the link below. It won't return your 404 template that you've set in EE (EE seems to give that ?URL parameter precedent over its 404 template processing.)

So, I'd worry that the above is not enough to right things with the Google index, and I'd throw the stronger 410 "Gone" error, instead--plus route it to a static error page outside of EE:

#optional, show a static file as an error document
ErrorDocument 410 /410.html
#
RewriteCond %{QUERY_STRING} URL [NC]
RewriteRule .* - [R=410,L]

EDITED: "Gone" is status code 410 -- corrected, above

I think Google (and other search engines) recognize that people who know how to throw 400 errors aren't kidding, and so those "gone" resources can be removed from the search index right away ;-)


Note that, if you don't use EE for your 404 page--e.g., you instead have ErrorDocument 404 /404.html set in your htaccess pointing to a static page, then the original example of throwing a 404 will totally circumvent EE, and show your static 404 error page.

| improve this answer | |
  • I guess I should've asked: do you want the URL redirect to continue working? I was assuming that you wanted to kill the redirect altogether. – Jay F Nov 27 '13 at 19:32
0

One option is to add <meta name=”robots” content=”noindex, nofollow”> to the forum header so none of the pages on the forum are indexed. You can do that here:

themes > forum_themes > default > forum_global > meta_tags.html

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.