[EP-tech] OAI2 deleted vs destroyed records
David R Newman
drn at ecs.soton.ac.uk
Thu Apr 15 17:24:57 BST 2021
Hi John,
I will look into whether this can be incorporated into future version of
EPrints.
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Feprints3.4%2Fissues%2F145&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398236442%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=N0Yjl6SBd3QXkw%2FVWAZ6h9UknGkUGVGxtYFv6u%2BxXeA%3D&reserved=0
I think we need something a little bit more sophisticated for general
purposes. So that there is a permission (e.g.
eprint/remove_been_in_archive) that can be set, probably just on an
individual admin user basis, for the very rare cases when privacy or
confidentiality require full removal of a live item. It would also be
useful to make even this 'super' admin user aware that they really do
not want to fully remove a (once) live unless there is no other option.
Regards
Davud Newman
On 15/04/2021 17:00, John Salter via Eprints-tech wrote:
> *CAUTION:* This e-mail originated outside the University of Southampton.
>
> One option to prevent this happening is to alias the
> EPrints::Plugin::Screen::EPrint::Remove plugin with something like
> this - which checks for the datestamp being set - if it is set, then
> the item has been live at some point, and should be retained for
> OAI-PMH incremental harvesting purposes.
>
> ######################################
>
> package EPrints::Plugin::Screen::EPrint::WreoRemove;
>
> our @ISA = ( 'EPrints::Plugin::Screen::EPrint::Remove' );
>
> use strict;
>
> sub can_be_viewed
>
> {
>
> my( $self ) = @_;
>
> return 0 unless $self->could_obtain_eprint_lock;
>
> # 2016-11-01 JLRS Add check for this item ever being live.
>
> return 0 if( $self->{processor}->{eprint}->exists_and_set( "datestamp"
> ) );
>
> return $self->allow( "eprint/remove" );
>
> }
>
> 1;
>
> ######################################
>
> Cheers,
>
> John
>
> *From:*eprints-tech-bounces at ecs.soton.ac.uk
> [mailto:eprints-tech-bounces at ecs.soton.ac.uk] *On Behalf Of
> *Alan.Stiles via Eprints-tech
> *Sent:* 15 April 2021 16:38
> *To:* eprints-tech at ecs.soton.ac.uk
> *Subject:* Re: [EP-tech] OAI2 deleted vs destroyed records
>
> *CAUTION:*This e-mail originated outside the University of Southampton.
>
> Thanks all for the responses – I’ve suggested to our Primo folks that
> they need to set a full re-harvest running, and we’re considering how
> tightly to restrict the destroy option. As you say David, shouldn’t be
> a problem for items that have never been made live, but repository
> admins have been known to use it when they should really be using
> ‘delete’, so removing temptation or confusion may be simpler.
>
> Possibly remove it for everyone and only allow people the option to
> ‘delete’ stuff. Then schedule a job to check items ‘deleted’ within a
> particular timeframe (e.g. between 6 and 7 months ago?) and only
> destroy a record which has never been put live.
>
> Alan
>
> *From: *David R Newman <drn at ecs.soton.ac.uk <mailto:drn at ecs.soton.ac.uk>>
> *Date: *Thursday, 15 April 2021 at 16:27
> *To: *"eprints-tech at ecs.soton.ac.uk
> <mailto:eprints-tech at ecs.soton.ac.uk>" <eprints-tech at ecs.soton.ac.uk
> <mailto:eprints-tech at ecs.soton.ac.uk>>, "Alan. Stiles"
> <alan.stiles at open.ac.uk <mailto:alan.stiles at open.ac.uk>>
> *Subject: *Re: [EP-tech] OAI2 deleted vs destroyed records
>
> CAUTION: This mail comes from outside the University. Please consider
> this before opening attachments, clicking links, or acting on the
> content.
>
> Hi Alan,
>
> Items that have never been live should not appear in any sets. Even
> if they do appear in a set, the individual item request should return
> idDoesNotExist if they have never been live.
>
> I was not sure whether Primo periodically proactively checks
> individual items it has imported to see if their metadata / status has
> changed. From what Iain says, this sounds like it is not the case.
>
> Regards
>
> David Newman
>
> On 15/04/2021 16:16, Alan.Stiles via Eprints-tech wrote:
>
> *CAUTION:*This e-mail originated outside the University of
> Southampton.
>
> Thanks David,
>
> Unfortunately, it looks like historically the destroy option has
> been available to the repository admins alongside the delete
> option, so we have quite a few where that option has been taken,
> though I think mostly they are not records that have ever been
> live, which is probably okay. I have a feeling I might be
> changing the permissions on the destroy option in the near future…
>
> I’ll check with our folks who deal with Primo as to whether they
> can do anything at their end of things with regards to how Primo
> treats those records (if it even sees them – it might not see the
> destroyed record if it’s harvesting a specific set).
>
> Alan
>
> *From: *<eprints-tech-bounces at ecs.soton.ac.uk>
> <mailto:eprints-tech-bounces at ecs.soton.ac.uk> on behalf of
> "eprints-tech at ecs.soton.ac.uk"
> <mailto:eprints-tech at ecs.soton.ac.uk>
> <eprints-tech at ecs.soton.ac.uk> <mailto:eprints-tech at ecs.soton.ac.uk>
> *Reply to: *"eprints-tech at ecs.soton.ac.uk"
> <mailto:eprints-tech at ecs.soton.ac.uk>
> <eprints-tech at ecs.soton.ac.uk>
> <mailto:eprints-tech at ecs.soton.ac.uk>, David R Newman
> <drn at ecs.soton.ac.uk> <mailto:drn at ecs.soton.ac.uk>
> *Date: *Thursday, 15 April 2021 at 16:04
> *To: *"eprints-tech at ecs.soton.ac.uk"
> <mailto:eprints-tech at ecs.soton.ac.uk>
> <eprints-tech at ecs.soton.ac.uk>
> <mailto:eprints-tech at ecs.soton.ac.uk>, "Alan. Stiles"
> <alan.stiles at open.ac.uk> <mailto:alan.stiles at open.ac.uk>
> *Subject: *Re: [EP-tech] OAI2 deleted vs destroyed records
>
> CAUTION: This mail comes from outside the University. Please
> consider this before opening attachments, clicking links, or
> acting on the content.
>
> Hi Alan,
>
> If previously live records have been completely removed rather
> than retired then you can expect bad things to happen. If there
> is a specific privacy issue that means the record cannot even been
> retained in a restricted form (retired), then removing may be the
> only option. However, the need for this should be vanishingly
> small and therefore issues like you describe with Primo should be
> few a far between and may require manual intervention.
>
> The error code idDoesNotExist is deliberately returned by EPrints
> OAI interface when a record is removed, as in effect the record
> never existed. All I can suggest is that Primo should treat
> getting back idDoesNotExist the same as getting back an item that
> is marked as deleted. Obviously, you may want to be a bit more
> careful about what to do when getting back idDoesNotExist, in case
> there is some error in the request that mangles the ID so it
> cannot be found. I have no idea how Primo could be configured to
> do this but as far as I can tell EPrints is behaving as it should;
> reporting completely removed items as not existing whereas retired
> items are reported as 'Deleted'.
>
> Regards
>
> David Newman
>
> On 15/04/2021 15:27, Alan.Stiles via Eprints-tech wrote:
>
> *CAUTION:*This e-mail originated outside the University of
> Southampton.
>
> Hi all,
>
> I feel like there was a discussion about this here a year or
> two ago but I can’t find it now.
>
> Records in our repository that get flagged as deleted show up
> in the OAI feed as ‘Deleted’, but records that get completely
> removed (destroyed) show up as error code ‘idDoesNotExist’.
>
> It appears that Primo (our library search product) isn’t doing
> anything about updating records from the feed (configured
> within Primo to explicitly harvest our repository), at least
> where it’s getting the error response.
>
> Any clues as to whether this is a Primo problem or my problem
> to sort out?
>
> Cheers,
>
> Alan
>
>
>
> *** Options:http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech <http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech>
>
> *** Archive:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398336013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oJ9oXMt95YifZ3Bxe4AlXWL7Ii42MqiF8Mpzcd%2FV0ww%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398336013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oJ9oXMt95YifZ3Bxe4AlXWL7Ii42MqiF8Mpzcd%2FV0ww%3D&reserved=0>
>
> *** EPrints community wiki:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398345970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UdPHLzSowUNmktmUcaWoCMUlOYOMGK%2BoVO4xfkcyyrA%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398345970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UdPHLzSowUNmktmUcaWoCMUlOYOMGK%2BoVO4xfkcyyrA%3D&reserved=0>
>
> *** Options:http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech <http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech>
>
> *** Archive:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sWt2TRepHLUC1ajqR6SixnPWwf5H8Rfya9waDgKN05I%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sWt2TRepHLUC1ajqR6SixnPWwf5H8Rfya9waDgKN05I%3D&reserved=0>
>
> *** EPrints community wiki:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rrVKRYySkSCit6y4q7UXwFUhbmCerthKswE39HzMmnE%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rrVKRYySkSCit6y4q7UXwFUhbmCerthKswE39HzMmnE%3D&reserved=0>
>
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sWt2TRepHLUC1ajqR6SixnPWwf5H8Rfya9waDgKN05I%3D&reserved=0
> *** EPrints community wiki: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rrVKRYySkSCit6y4q7UXwFUhbmCerthKswE39HzMmnE%3D&reserved=0
--
This email has been checked for viruses by AVG.
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.avg.com%2F&data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C3a252901ffff42f1266408d9002b03a1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637541007398355925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zGTWXULVOzg0V3S6i5%2BkpdBg3kekwtMo8CyCzAjOSDU%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20210415/ad6102f3/attachment-0001.html
More information about the Eprints-tech
mailing list