[EP-tech] Re: Memory usage in 3.2, Sword 1.3 and epdata packages

[Ian Stuart]

I also believe that the SWORD 2 implementation of the XML importer 
*does* replicate the SWORD 1.3 process: you can send it a complete file 
and it will do the CREATE and UPDATE processes in one go.

On 12/07/13 10:51, Tim Brody wrote:
> Correct. In 3.2 the HTTP post is all worked on in memory. In 3.3 XML
> data are streamed and will be written to disk as it arrives.
>
> /Tim.
>
> On Fri, 2013-07-12 at 08:26 +0100, Ian Stuart wrote:
>> With no real knowledge, and certainly no investigation.... I would
>> suspect the problem is actually with how the base64 files are handled,
>> rather then being an EPrints memory leak per sae.
>>
>>   From the SWORD importers I've written, the process seems to be to
>> 1) read in the deposit
>> 2) unpack the deposit (zip into disk space, XML into memory)
>> 3) create the eprint object
>> 4) attach the files
>> 5) write everything out
>>
>> So I would suspect that what's happening is that all your base64 files
>> are created (in memory) from the XML (which is also in memory)


-- 

Ian Stuart.
Developer: ORI, RJ-Broker, and OpenDepot.org
Bibliographics and Multimedia Service Delivery team,
EDINA,
The University of Edinburgh.

http://edina.ac.uk/

This email was sent via the University of Edinburgh.

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.