<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>All I care is the list gets indexed by google. The rest is all you. </div><div><br></div><div>Thanks!<br><br>Sent from my SmartPhone</div>
<div><br>On Oct 4, 2012, at 1:15, Julian Field <<a href="mailto:Jules@ecs.soton.ac.uk">Jules@ecs.soton.ac.uk</a>> wrote:<br><br></div><blockquote type="cite"><div>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
I've changed the file to read<br>
<br>
User-agent: *<br>
Allow: /pipermail/zendto/<br>
Disallow: /<br>
<br>
User-agent: ECSbot<br>
Allow: /<br>
<br>
which I think should allow everyone to index the ZendTo list archive
but nothing else, which is what we want.<br>
<br>
Agree?<br>
<br>
Cheers,<br>
Jules.<br>
<br>
<div class="moz-cite-prefix">On 04/10/2012 07:34, Brendon
Baumgartner wrote:<br>
</div>
<blockquote cite="mid:CAKVBzWGMGMUOGy-MPBs+EiZGCvANY3ikiKavUjkPUg-U04wpcA@mail.gmail.com" type="cite">I think this should do it. Also, here are the docs
from google.
<div><a href="https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt">https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt</a></div>
<div><br>
<div><br>
</div>
<div>
<div>User-agent: googlebot</div>
<div>Allow: /</div>
<div><br>
</div>
<div>User-agent: ECSbot</div>
<div>Allow: /</div>
<div><br>
</div>
<div>User-agent: *</div>
<div>Disallow: /</div>
<div><br>
</div>
<div>
<br>
</div>
<div><br>
</div>
<br>
<div class="gmail_quote">On Thu, Sep 27, 2012 at 5:04 PM,
Brendon Baumgartner <span dir="ltr"><<a href="mailto:b@brendon.com" target="_blank">b@brendon.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Can you please allow google to index the list. I see you
have it disabled here:
<div><br>
</div>
<div><a href="http://mailman.ecs.soton.ac.uk/robots.txt" target="_blank">http://mailman.ecs.soton.ac.uk/robots.txt</a>
</div>
<div><br>
</div>
<div>It would be nice to actually search the mailing list
for past problems/solutions.</div>
<div>Thanks</div>
<span class="HOEnZb"><font color="#888888">
<div>-Brendon</div>
</font></span></blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
ZendTo mailing list
<a class="moz-txt-link-abbreviated" href="mailto:ZendTo@zend.to">ZendTo@zend.to</a>
<a class="moz-txt-link-freetext" href="http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto">http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">Jules
--
Jules Field MEng MBCS CITP CEng
email+iMessage: <a class="moz-txt-link-abbreviated" href="mailto:Jules@ecs.soton.ac.uk">Jules@ecs.soton.ac.uk</a>
Twitter: @JulesFM
Senior Tutor, Postmaster
Electronics and Computer Science
University of Southampton SO17 1BJ, UK
'That is the land of lost content,
I see it shining plain,
The happy highways where I went,
And cannot come again.' - A.E. Houseman
</pre>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>ZendTo mailing list</span><br><span><a href="mailto:ZendTo@zend.to">ZendTo@zend.to</a></span><br><span><a href="http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto">http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto</a></span></div>
</blockquote></body></html>