<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I've changed the file to read<br>
<br>
User-agent: *<br>
Allow: /pipermail/zendto/<br>
Disallow: /<br>
<br>
User-agent: ECSbot<br>
Allow: /<br>
<br>
which I think should allow everyone to index the ZendTo list archive
but nothing else, which is what we want.<br>
<br>
Agree?<br>
<br>
Cheers,<br>
Jules.<br>
<br>
<div class="moz-cite-prefix">On 04/10/2012 07:34, Brendon
Baumgartner wrote:<br>
</div>
<blockquote
cite="mid:CAKVBzWGMGMUOGy-MPBs+EiZGCvANY3ikiKavUjkPUg-U04wpcA@mail.gmail.com"
type="cite">I think this should do it. Also, here are the docs
from google.
<div><a moz-do-not-send="true"
href="https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt">https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt</a></div>
<div><br>
<div><br>
</div>
<div>
<div>User-agent: googlebot</div>
<div>Allow: /</div>
<div><br>
</div>
<div>User-agent: ECSbot</div>
<div>Allow: /</div>
<div><br>
</div>
<div>User-agent: *</div>
<div>Disallow: /</div>
<div><br>
</div>
<div>
<br>
</div>
<div><br>
</div>
<br>
<div class="gmail_quote">On Thu, Sep 27, 2012 at 5:04 PM,
Brendon Baumgartner <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:b@brendon.com"
target="_blank">b@brendon.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Can you please allow google to index the list. I see you
have it disabled here:
<div><br>
</div>
<div><a moz-do-not-send="true"
href="http://mailman.ecs.soton.ac.uk/robots.txt"
target="_blank">http://mailman.ecs.soton.ac.uk/robots.txt</a>
</div>
<div><br>
</div>
<div>It would be nice to actually search the mailing list
for past problems/solutions.</div>
<div>Thanks</div>
<span class="HOEnZb"><font color="#888888">
<div>-Brendon</div>
</font></span></blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
ZendTo mailing list
<a class="moz-txt-link-abbreviated" href="mailto:ZendTo@zend.to">ZendTo@zend.to</a>
<a class="moz-txt-link-freetext" href="http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto">http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">Jules
--
Jules Field MEng MBCS CITP CEng
email+iMessage: <a class="moz-txt-link-abbreviated" href="mailto:Jules@ecs.soton.ac.uk">Jules@ecs.soton.ac.uk</a>
Twitter: @JulesFM
Senior Tutor, Postmaster
Electronics and Computer Science
University of Southampton SO17 1BJ, UK
'That is the land of lost content,
I see it shining plain,
The happy highways where I went,
And cannot come again.' - A.E. Houseman
</pre>
</body>
</html>