<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1811554209;
mso-list-type:hybrid;
mso-list-template-ids:-132233164 134807567 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi George,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">My understanding is that the database max_connections is something that should be tuned in relation to your Apache config (and both should be tuned to the server resources available).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Recently we had been suffering with some performance issues.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We were hitting our Apache MaxClients – but the root cause seems to have been the database:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We were using MyISAM tables (the MySQL default when we implemented EPrints).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The following sequence of events caused us problems:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">Slow SELECT query (possibly from generate_views, but there were a few other possibilities – a paper with >2,500 authors…)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="color:#1F497D"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">Update on table included in above SELECT (whilst it was still running)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="color:#1F497D"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">SELECT on above table<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">MyISAM can handle parallel SELECT queries on the tables without any problem *<b>until</b>* you throw an UPDATE into the equation. MyISAM uses table-level locking, and the UPDATE will wait for the slow SELECT
to complete, and the other SELECTs get queued behind the UPDATE.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">If you have a MySQL session open when you hit a max_connections issue, do a:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">mysql>SHOW PROCESSLIST\G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">You might see a query in the ‘sending’ state (that’s been running for a while) and a lot of ‘waiting for table lock’ entries.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">If you do, you may well be seeing the same issue we had.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We have altered our database tables to use InnoDB (which is the default MySQL engine since 5.5.5), but you’ll need to understand what you’re doing when updating it if you go down this route…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I posted recently on this list about a [dataset]__index tables missing a primary key – I discovered this as part of the planning to move to InnoDB.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">John<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="mso-fareast-language:EN-GB">From:</span></b><span lang="EN-US" style="mso-fareast-language:EN-GB"> eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk]
<b>On Behalf Of </b>George Macgregor<br>
<b>Sent:</b> 15 August 2016 10:20<br>
<b>To:</b> eprints-tech@ecs.soton.ac.uk<br>
<b>Subject:</b> [EP-tech] Too many connections<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hello colleagues.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Occasionally our EPrints suffers the “too many connections” MySQL connection error. In the past when EPrints has suffered this error my assumption has been that there is a query hanging which then has a knock on effect on other queries
as they can’t write to a table. We currently have max_connections set to 300 (which seems like a lot) but I can observe from previous chatter on the list that some go up to 1000. I was therefore wondering what the community considered to be an appropriate
connection limit? Given all the traffic I can observe it is conceivable – however unlikely - that 300 is insufficient. Have others observed something similar?
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks in advance!<o:p></o:p></p>
<p class="MsoNormal"><br>
George<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">--<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">George Macgregor<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Information Services Directorate | University of Strathclyde<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Curran Building | 101 St James Road<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Glasgow G4 ONS<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Tel: 0141 548 3496<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Email: <a href="mailto:george.macgregor@strath.ac.uk">
george.macgregor@strath.ac.uk</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Web: <a href="http://personal.strath.ac.uk/george.macgregor/">
http://personal.strath.ac.uk/george.macgregor/</a> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Twitter: <a href="https://twitter.com/g3om4c">
@g3om4c</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><img border="0" width="16" height="16" style="width:.1666in;height:.1666in" id="Picture_x0020_1" src="cid:image001.png@01D1F6E1.0FB85C30" alt="iD icon"><a href="http://orcid.org/0000-0002-8482-3973">orcid.org/0000-0002-8482-3973</a><br>
--<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;mso-fareast-language:EN-GB">Ensure that your research outputs are eligible for submission in the next REF.
</span></b><span style="font-size:9.0pt;mso-fareast-language:EN-GB">Authors' accepted manuscripts of journal articles and conference proceedings accepted after 01 April 2016 must be deposited in PURE as soon as possible after acceptance for publication – email
<a href="mailto:openaccess@strath.ac.uk">openaccess@strath.ac.uk</a> for further information or visit
<a href="http://www.strath.ac.uk/openaccess">http://www.strath.ac.uk/openaccess</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><a href="https://twitter.com/StrathclydeOA"><span style="mso-fareast-language:EN-GB;text-decoration:none"><img border="0" width="69" height="26" style="width:.7187in;height:.2708in" id="Picture_x0020_2" src="cid:image002.png@01D1F6E1.0FB85C30" alt="twitter logo"></span></a><span style="font-size:9.0pt;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;mso-fareast-language:EN-GB">--<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;mso-fareast-language:EN-GB">The University of Strathclyde is a charitable body, registered in Scotland, with registration number SC015263<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>