[EP-tech] Re: Robots.txt / OAI

[Tim Brody]

On 10/07/12 17:33, John Salter wrote:
> Just a quick question/comment:
> Does the robots.txt file (e.g. http://demoprints.eprints.org/robots.txt ) mean that compliant bots don't access the OAI-PMH interface http://demoprints.eprints.org/cgi/oai2 ?
>
> Is this an issue?
>
In practise, I don't think OAI bots will look at robots.txt. But I agree 
/oai2 ought to be Allow(ed).

/Tim.