Save Set information for repeated Processes

When setting up a recurring Process that uses a Set with different values each time, Rosetta does not keep a record of the Set values each time it is generated, so that you cannot Rerun the previous Process instances in History. We propose a change to the recording of Set metadata so that each time a Set is used, its values are stored, thus allowing a Historical Process to be rerun at a later date. An example is rolling fixity checks, wherein a different range of SIPs are checked each day using the same Process. By saving the SIP range each time, any Process instance in the History can be rerun if necessary at a later date.

We are looking at supporting the ability to add a new format to our Ingest tool, in conjunction with this we would like that to trigger some type of API call to Rosetta and add the file format as an allowed format as well.

In Rosetta you can add a new file extension in the Submission Format area of the UI. We have provided the option to add a new format in our Ingest tool, So once they do this, we would like this to trigger an some type of API call to Rosetta and add the new file extension as an allowed file extension without having to go into the UI and add the file extension manually in the Submission Format location.

At SLUB, we currently use the IE export feature via API. The webservice is called using an export path that points to an NFS share, which itself is shared between the Rosetta cluster and the Submission Application servers that run ingest and access tasks. One of those tasks is to extract exported IE .tars, restructure them to resemble their original structure as provided by the producer, provide the result to the producer and (this is key) remove the exported .tar, as it is no longer needed. Now, whenever Rosetta exports an IE, the resulting .tar archive is created on that shared NFS share with "dps.exlibris" owner and GID and "0755" privileges. That means that Rosetta's Cleanup-Job could take care of removing those IEs if they were exported to the default location somewhere in the staf work area. However, when requesting exports via API to a non-default location, which we do to provide them for the processing as described above, then there's currently no way to remove these .tars except for manual intervention. Of course, this is not feasible in a fully automated workflow.

We therefore suggest to change the export Webservice to either
a) accept UID and GID that the exported .tar is supposed to have after export, OR
b) set suitable file permissions (e.g. 0660 for files and 0770 for enclosing directories), thus enabling institutions to create a shared group on their ingest servers that is allowed to delete those .tars once the extraction has been completed. We could image having the "exlibris" group (GID 500) on ingest servers as well and making sure that the process user on those ingest servers is a member of the "exlibris" group that exists on the Rosetta cluster by administrative rule, so that the process user can run delete operations on shared NFS storages. Current directory permissions are "0755" for directories of exported IEs and their respective subdirectories and "0644" for files inside these directories, which means "others" read/execute permissions are unnecessary and "group" rwx permissions are insufficient. We explicitely do NOT propose to set "0777" permissions on exported .tars!
c) create a record of all exported .tars even to non-default locations and include these in the Cleanup Job. This, however, is not preferred, as it's deemed more practical to have the ingest server delete .tars when the extraction is complete, because they have sufficient information to determine when a .tar is not needed anymore.

hen running a Publishing Sync Job and looking at the assciated 'View' we get the following information

Mon Jan 07 09:06:40 AEST 2019 INFO sync all configurations for IE publishing
Mon Jan 07 09:06:40 AEST 2019 INFO sync the configurationId: 18981614
Mon Jan 07 09:06:41 AEST 2019 INFO sync the profileId: 18981613
Mon Jan 07 09:06:41 AEST 2019 INFO Handling new IEs
Mon Jan 07 09:06:41 AEST 2019 INFO Handling Removed IEs
Mon Jan 07 09:06:41 AEST 2019 INFO Handling updated IEs
Mon Jan 07 09:06:41 AEST 2019 INFO Finished sync for the profileId: 18981613
Mon Jan 07 09:06:41 AEST 2019 INFO Finished sync for configurationId: 18981614
Mon Jan 07 09:06:41 AEST 2019 INFO sync the configurationId: 20975335
Mon Jan 07 09:06:41 AEST 2019 INFO sync the profileId: 20975337
Mon Jan 07 09:06:41 AEST 2019 INFO Skipping Profile: 20975337 because it is inactive.
Mon Jan 07 09:06:41 AEST 2019 INFO Finished sync for the profileId: 20975337
Mon Jan 07 09:06:41 AEST 2019 INFO sync the profileId: 20975338
Mon Jan 07 09:06:41 AEST 2019 INFO Skipping Profile: 20975338 because it is inactive.
Mon Jan 07 09:06:41 AEST 2019 INFO Finished sync for the profileId: 20975338
Mon Jan 07 09:06:41 AEST 2019 INFO sync the profileId: 20975389
Mon Jan 07 09:06:41 AEST 2019 INFO Handling new IEs

This is Ok but the additional information 'name/label' would be helpful

For example in the above -

Mon Jan 07 09:06:40 AEST 2019 INFO sync the configurationId: 18981614

the informatin would be better dispayed as follows

Mon Jan 07 09:06:40 AEST 2019 INFO sync the configurationId: 18981614 (Handle Publishing)

where 'Handle Publishing' relates to the 'Data Management: Publishing Configuration | Details | General Details | Name'

Another example

Mon Jan 07 09:06:41 AEST 2019 INFO sync the profileId: 20975337

the infomation would be better displayed as

Mon Jan 07 09:06:41 AEST 2019 INFO sync the profileId: 20975337 (Rosetta2Alma)

where Rosetta2Alma is the filed located in 'Data Management: Publishing Configuration | Details | Profile Details | Name'