<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
        {font-family:Verdana;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
        {font-family:Consolas;
        panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
p
        {mso-style-priority:99;
        mso-margin-top-alt:auto;
        margin-right:0cm;
        mso-margin-bottom-alt:auto;
        margin-left:0cm;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
pre
        {mso-style-priority:99;
        mso-style-link:"HTML Preformatted Char";
        margin:0cm;
        margin-bottom:.0001pt;
        font-size:10.0pt;
        font-family:"Courier New";}
span.HTMLPreformattedChar
        {mso-style-name:"HTML Preformatted Char";
        mso-style-priority:99;
        mso-style-link:"HTML Preformatted";
        font-family:Consolas;}
span.EmailStyle20
        {mso-style-type:personal-reply;
        font-family:"Calibri","sans-serif";
        color:#1F497D;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page WordSection1
        {size:612.0pt 792.0pt;
        margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
        {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">thanks for all the thoughts everyone.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I’m looking into getting some throttling setup outside ePrints - since it’s just the occasional highly demanding crawler that seems to be the problem - not
the normal users as far as I can tell.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If it is easy to turn off on-demand regeneration then that sounds worth exploring.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Rich<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk]
<b>On Behalf Of </b>sf2<br>
<b>Sent:</b> 16 December 2014 16:26<br>
<b>To:</b> eprints-tech@ecs.soton.ac.uk<br>
<b>Subject:</b> [EP-tech] Re: load throttling strategy<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="font-family:"Verdana","sans-serif"">I don't reckon the "waiting" strategy is a good idea: 1- there's no indication that server load will get better after waiting for n seconds, 2- if your thread is waiting, it can't take new connections (so
clients will be pilling up).<o:p></o:p></span></p>
<p><span style="font-family:"Verdana","sans-serif"">One strategy (given RG's issues) is to disable the on-demand regeneration of such pages and only generate them offline (via generate_views). Then no problems for the clients since eprints/apache will only
be serving cached pages (cached on-disk that is). And if you really must, set-up Varnish or else in front of your repo...<o:p></o:p></span></p>
<p><span style="font-family:"Verdana","sans-serif"">If a page takes 10mins to regenerate then having it generated on-demand by a client cannot be a good idea ;-) <o:p></o:p></span></p>
<p><span style="font-family:"Verdana","sans-serif"">Also out-of-interest I'd be curious to know of any stats showing that visitors actually use the browse pages (ie. how often/how much). I kinda see the point of having them for crawlers (then just have one
browse view, eg per year) but for users... meh :-)<o:p></o:p></span></p>
<p><span style="font-family:"Verdana","sans-serif"">Seb<o:p></o:p></span></p>
<p><span style="font-family:"Verdana","sans-serif""> <o:p></o:p></span></p>
<p><span style="font-family:"Verdana","sans-serif"">On 16.12.2014 10:11, Ian Stuart wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #1010FF 1.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-bottom:5.0pt">
<pre>On 16/12/14 10:05, Yuri wrote:<o:p></o:p></pre>
<blockquote style="border:none;border-left:solid #1010FF 1.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-family:"Verdana","sans-serif"">The best is to check the system load in the build page plugin/module, wait some seconds, and then go. Is there some documentation somewhere on Eprints strategies on views page rebuilds?<o:p></o:p></span></p>
</blockquote>
<pre>The only thing I'm aware one can do is define the number of days <o:p></o:p></pre>
<pre>view-pages are considered "valid" for, before being automatically rebuilt.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>-- <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Ian Stuart.<o:p></o:p></pre>
<pre>Developer: ORI, RJ-Broker, and OpenDepot.org<o:p></o:p></pre>
<pre>Bibliographics and Multimedia Service Delivery team,<o:p></o:p></pre>
<pre>EDINA,<o:p></o:p></pre>
<pre>The University of Edinburgh.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><a href="http://edina.ac.uk/">http://edina.ac.uk/</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This email was sent via the University of Edinburgh.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The University of Edinburgh is a charitable body, registered in<o:p></o:p></pre>
<pre>Scotland, with registration number SC005336.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>*** Options: <a href="http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech">http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech</a><o:p></o:p></pre>
<pre>*** Archive: <a href="http://www.eprints.org/tech.php/">http://www.eprints.org/tech.php/</a><o:p></o:p></pre>
<pre>*** EPrints community wiki: <a href="http://wiki.eprints.org/">http://wiki.eprints.org/</a><o:p></o:p></pre>
<pre>*** EPrints developers Forum: <a href="http://forum.eprints.org/">http://forum.eprints.org/</a><o:p></o:p></pre>
</blockquote>
<p><span style="font-family:"Verdana","sans-serif""> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana","sans-serif""> <o:p></o:p></span></p>
</div>
</div>
</body>
</html>