[EP-tech] Re: Memory usage in 3.2, Sword 1.3 and epdata packages
Ian.Stuart at ed.ac.uk
Fri Jul 12 11:08:40 BST 2013
I also believe that the SWORD 2 implementation of the XML importer
*does* replicate the SWORD 1.3 process: you can send it a complete file
and it will do the CREATE and UPDATE processes in one go.
On 12/07/13 10:51, Tim Brody wrote:
> Correct. In 3.2 the HTTP post is all worked on in memory. In 3.3 XML
> data are streamed and will be written to disk as it arrives.
> On Fri, 2013-07-12 at 08:26 +0100, Ian Stuart wrote:
>> With no real knowledge, and certainly no investigation.... I would
>> suspect the problem is actually with how the base64 files are handled,
>> rather then being an EPrints memory leak per sae.
>> From the SWORD importers I've written, the process seems to be to
>> 1) read in the deposit
>> 2) unpack the deposit (zip into disk space, XML into memory)
>> 3) create the eprint object
>> 4) attach the files
>> 5) write everything out
>> So I would suspect that what's happening is that all your base64 files
>> are created (in memory) from the XML (which is also in memory)
Developer: ORI, RJ-Broker, and OpenDepot.org
Bibliographics and Multimedia Service Delivery team,
The University of Edinburgh.
This email was sent via the University of Edinburgh.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
More information about the Eprints-tech