Is there a way to publish a secure RSS feed that is not indexed by search engines?
We publish an email newsletter and would like to provide an RSS feed of it, but are hesitant to do so because in the newsletter we publish email addresses that we do not want to be indexed online – by the search engines or bots.
Is there a way to generate an RSS feed, secure or otherwise, that is NOT indexed by the search engines, but that could be joined by our subscribers as an option to email?
Observing members:
0
Composing members:
0
4 Answers
@johnpowell That wouldn’t work in this case. robots.txt is just a convention by the major search engines, however, spambots that scrape email addresses off the internet won’t pay attention to that.
The best way to go would be to modify the RSS feed to (automatically) filter out all email addresses. Otherwise, the only way to prevent it is to force your readers to authenticate themselves (as RSS is a pull mechanism, as opposed to email, which is a push mechanism so you know who you’re sending it to), which is possible but terribly annoying and not worth the trouble for your users.
Thank you for the suggestions. I should probably clarify that we are looking at alternate ways to publish a newsletter to subscribers other than email. The newsletter is published daily and in the newsletter are email addresses from people who post, much like one might find with a digest version of an email discussion list. So what we’re looking for a way to do is to be able to get it to subscribers via RSS, and still have the subscribers be able to use the email addresses in it, but prevent them from being picket up by search engines and indexed or hit by bots.
I believe the solution we may be looking for is an interim step which would be some type of obfuscation of the email addresses prior to publishing; however I don’t know enough about RSS to know what would or would not work.
Hmm, obfuscation would indeed be difficult in RSS. What you could do is provide each subscriber with a personalized link that he can only retrieve if he/she is logged in or that is sent to this person by email. Of course, as soon as this user posts this link somewhere on the internet the email addresses are exposed, but I think it’s safe to assume that won’t happen. It would also be less efficient than it could be, seeing as e.g. Google can’t cache the feed once for all users of Google Reader because they all have a different URL.
Also, you would really have to double-check whether Google publicly indexes RSS feeds its users subscribe to, but I really don’t expect them to do that.
Answer this question 
This question is in the General Section. Responses must be helpful and on-topic.
Composing members:
0
