<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Hi Lizz</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>With data of this scale, I'd recommend getting it onto the eprints server first (scp and md5sum are your friends), and then attaching from the command line using an API script. That way you have
full control, and can be sure that it won't fail for network reasons. See:</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span><a href="https://github.com/gobfrey/medmus/blob/master/import_tiffs.pl#L60">https://github.com/gobfrey/medmus/blob/master/import_tiffs.pl#L60</a></div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>...for code that attaches a file to an eprint. This script is my solution to a not-too-dissimilar problem, where I was batch creating eprints, and attaching images to the records (there were
lots of them and the tiffs were really big). My first step was to get everything on the server in some directory somewhere.</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>I don't think EPrints has a graceful way of attaching large files. </div>
<div><br>
</div>
<div>--</div>
<div>Adam</div>
<div><br>
</div>
<br>
<div>
<div>On 16 Feb 2016, at 17:57, Lizz Jennings <<a href="mailto:e.jennings@bath.ac.uk">e.jennings@bath.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:571356869;
mso-list-type:hybrid;
mso-list-template-ids:-1395883548 134807569 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m looking at options for importing files with records, and have read quite a few threads and bits of documentation, so have managed to get this to work in a couple of ways:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span style="mso-list:Ignore">1)<span style="font:7.0pt "Times New Roman"">
</span></span><!--[endif]-->Base64 encoding the files and including them in the import file<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span style="mso-list:Ignore">2)<span style="font:7.0pt "Times New Roman"">
</span></span><!--[endif]-->Using the command line interface to import the files by referring to their location<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span style="mso-list:Ignore">3)<span style="font:7.0pt "Times New Roman"">
</span></span><!--[endif]-->Temporarily allowing file imports and importing them by reference in the web interface<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">All of these work until I try and do the same from my normal PC to an instance of EPrints running on a server, at which point the import doesn’t have access to my local files so isn’t able to upload them. I can’t change the access permissions
on the files. Base64 encoding is very much my option of last resort as some of the files this is for are expected to be around 10GB each, so storing a second (larger) copy of them in that format is going to put quite a strain on the storage available to the
researcher.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Before I spend ages trying to get a workable solution, has anyone had a similar issue and found a workaround?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Lizz<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">--<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Lizz Jennings BA MSc ACLIP MCLIP (Revalidated 2015)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Technical Data Officer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">The Library 4.10, University of Bath, Bath, BA2 7AY UK<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="SV" style="mso-fareast-language:EN-GB">Ext. 3570 (External 01225 383570)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><a href="mailto:E.Jennings@bath.ac.uk"><span lang="SV">E.Jennings@bath.ac.uk</span></a></span><u><span lang="SV" style="color:#0563C1;mso-fareast-language:EN-GB"><o:p></o:p></span></u></p>
<p class="MsoNormal">Research Data Management: <a href="http://www.bath.ac.uk/research/data">
http://www.bath.ac.uk/research/data</a><o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"> </span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
*** Options: <a href="http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech">
http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech</a><br>
*** Archive: <a href="http://www.eprints.org/tech.php/">http://www.eprints.org/tech.php/</a><br>
*** EPrints community wiki: <a href="http://wiki.eprints.org/">http://wiki.eprints.org/</a><br>
*** EPrints developers Forum: <a href="http://forum.eprints.org/">http://forum.eprints.org/</a><br>
</blockquote>
</div>
<br>
<font size="1"> <font face="Corbel"> <br />
Jisc is a registered charity (number 1149740) and a company limited by guarantee which is registered in England under Company No. 5747339, VAT No. GB 197 0632 86. Jisc’s registered office is: One Castlepark, Tower Hill, Bristol, BS2 0JA. T 0203 697 5800.<br /><br />
Jisc Services Limited is a wholly owned Jisc subsidiary and a company limited by guarantee which is registered in England under company number 2881024, VAT number GB 197 0632 86. The registered office is: One Castle Park, Tower Hill, Bristol BS2 0JA. T 0203 697 5800.
</font></font></body>
</html>