1

I have a static site that pulls in pop up content using ajax. I don't want search engines to index those ajax pages. I know there are ways to block pages with

<head>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
</head>

But I don't have that on each of the ajax pages. so wondering what the best approach would be.

1 Answer 1

1

I would think just putting those pages into a robots.txt file would be effective. Just a plain text file that sits in your website's root directory and includes basic site information (e.g. about location of your sitemap etc.). This can include statements instructing Google's robots to ignore specific files and folders - e.g.

Disallow: /ajax/ will tell google to ignore any files in the /ajax directory.

Disallow: /ajax/file.html will tell google to ignore the file /ajax/file.html

Using robots.txt means you can set these exclusions without having to put information into the files themselves, and also exclude directories.

HTH

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.