# SLUB SIP Builder ## Description A CLI tool to create valid [BagIt](https://tools.ietf.org/html/rfc8493) based submission information packages (SIPs) for ingest into the [SLUBArchiv](https://slubarchiv.slub-dresden.de/), the digital long-term preservation archive of the Saxon State and University Library Dresden (SLUB). As a user, you have one of three options to create your SIP: `copy`, `move` or `replace`. NOTE: The use of option `copy` is strongly recommended. The remaining options modify your source directory. Please use these with caution! ## Supported Operating Systems * Debian 10 * Windows 10 ## Requirements * Perl version 5.28 or higher * cpan or cpanm * several CPAN modules such as Archive::BagIt (0.065 or higher) * `sudo cpan -D Archive::BagIt` to check module perl version ## Three ways to install a module * with cpan `cpan Archive::BagIt` * with cpanm `cpanm Archive::BagIt` (NOTE: `cpan App::cpanminus` must be installed. If errors like IO:AIO are present, use `cpanm -i -f Archive::BagIt`). * from source (Linux) 1. Download for example Archive-BagIt-0.072.tar.gz from [metacpan Archive::BagIt](https://metacpan.org/pod/Archive::BagIt) 2. Extract it with `tar zxf Archive-BagIt-0.072.tar.gz` and go to project dir with `cd in Archive-BagIt-0.072` 3. Run `perl Makefile.pl` 4. `(d)gmake` 5. `(d)gmake test` 6. `(d)gmake install` 7. with `cpanm Archive::BagIt` test for the version. Result will be something like `Archive::BagIt is up to date (0.072)`. ## Archive::BagIt module * Sometimes it is offered to update module to a new version use `cpan Archive::BagIt`. If version not changed to a new one, try `cpanm Archive::BagIt` or `perl -MCPAN -e 'install Archive::BagIt'` ## Perl module deinstallation Using cpanm is recommended for an assisted and clean uninstall. * install cpanm with `cpan App::cpanminus`. * remove a Perl module with `cpanm --uninstall MODULE_NAME` ## Installation (Win 10) 1. Check UTF-8 flag at [ENG] `Systemadministration/TimeAndRegion/Region/Administration/AreaPattern` or [DE] `Systemsteuerung/Zeit und Region/Region/Verwaltung/Gebietsschema ändern/`, for using such characters as `äöüÄÖÜß`. Computer will be restarted! 2. Download strawberry perl environment [strawberry-perl-5.30.3.1-64bit-portable](https://strawberryperl.com/releases.html) 3. Extract files from archive 4. Download [Archive::BagIt](https://metacpan.org/pod/Archive::BagIt) module (Example: Archive-BagIt-0.072.tar.gz) 5. Start strawberry perl console `portableshell.bat` 1. Unpack archive `tar zxf Archive-BagIt-0.072.tar.gz` for example in the same dir where strawberryperl extracted 2. Change dir `cd Archive-BagIt-0.072` 3. Generate Makefile `perl Makefile.pl` 4. Run `gmake`, than `gmake install` 5. Check if module installed `cpanm Archive::BagIt`. At this point Archive::BagIt module installed. 6. Download [SLUB_SIP_Builder](https://git.slub-dresden.de/digital-preservation/SLUB_SIP_Builder) as zip 7. Extract files from archive 8. Change console path to extracted SLUB_SIP_Builder 9. Run tests `perl t/slubsipbuilderbagit.t` or `prove.t` and errors will come like 'Can't locate MARC/Record.pm in @INC (you may need to install the `MARC::Record` module) ...' 10. Copy from error module name and install it, repeat step 9 and 10 till test process starts. * `cpanm MARC::Record` * `cpanm XML::XPath` * `cpanm DateTime::Format::ISO8601` * `cpanm Win32::Console::ANSI` 11. All tests must process successfully and you are ready to go with Bag creation. ## Documentation * Project based on BagIt 1.0 specification, visit [RFC8493](https://tools.ietf.org/html/rfc8493) * For valid SIP examples based on operating system please have a look at `export_dir_kitodo/bagit/lin_*` or `export_dir_kitodo/bagit/win_*` * For [valid](https://git.slub-dresden.de/slub-digitalpreservation/submissionapplications4rosetta/-/tree/master/examples/bagit/valid_SLUB_bagits) and [faulty](https://git.slub-dresden.de/slub-digitalpreservation/submissionapplications4rosetta/-/tree/master/examples/bagit/faulty_SLUB_bagits) SLUB BagIt examples take a look at submission application project. * Specification for creating a SLUBArchiv SIP [SLUBArchiv_Produzenten_SIP_Spezifikation.pdf](https://slubarchiv.slub-dresden.de/technische-standards-fuer-die-ablieferung-von-digitalen-dokumenten/) * SLUBArchiv rights coding specification [SLUBArchiv_Produzenten_Rechteauszeichnung.pdf](https://slubarchiv.slub-dresden.de/technische-standards-fuer-die-ablieferung-von-digitalen-dokumenten/) ## Recommendation * Do not name your directories `data` * `marc21.xml` and `rights.xml` file names are reserved in directory `mets`. All same file names will be renamed to `1.xml`, `2.xml` and so on * Use absolute file and directory paths * If you try to execute the program for the first time, than use `save_option` copy ### Explanation ##### Usage ```perl perl bin/slubsipbuilderbagit.pl -help ``` ###### Result ```perl Usage: slubsipbuilderbagit.pl [options] Options: -help brief help message -man full documentation -save_option=<option> payload files save as <copy>, <move>, <replace>, the <copy> option should be prefered -IE_directory=<IE dir> existing IE directory (absolute path!) -SIP_output_path=<target dir> where to put the SIP dir (absolute path!) -ppn=<ppn>|-noppn=<noppn> SWB-PPN or any identifier (uses minimalistic marc21) -SIP_version=<version> mandatory, needs for identification of SIP format, supported <v2020.1> -external_workflow=<workflow> mandatory, should be uniqe workflow name -external_id=<id> mandatory, should be uniqe ID (i.e. a catalog ID), reuse an ID only when updating existing AIP -external_ISIL=<isil> optional, ISIL number of library -external_conservation_flag optional, if set no other "original" still exists -external_value_descr=<text> mandatory, the reason why to archive -rights_version=<version> mandatory, SLUB law managment specification, supported <1.0> -rights_xml=<file> mandatory, path to SLUB rights xml file(absolute path!) -add_meta_file=<file> optional, can be repeated, additional meta files(absolute path!) -add_key_value=<"key:value"> optional, can be repeated, additional key value pairs for bag-info.txt Options: -help Print a brief help message and exits. ``` ##### Full documentation with examples ```perl perl -I lib/ bin/slubsipbuilderbagit.pl -man ``` ###### Result * MUST: Write command in one line. ```perl EXAMPLES Copy (minimalistic) perl bin/slubsipbuilderbagit.pl --save_option=copy --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2 --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1 --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14 --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0 --rights_xml=/metadata/rights/Fallbeispiel-02.xml Copy perl bin/slubsipbuilderbagit.pl --save_option=copy --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2 --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1 --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14 --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0 --rights_xml=/metadata/rights/Fallbeispiel-01.xml --add_meta_file=/metadata/lido.xml --add_meta_file=/metadata/marc21.xml --add_key_value="Author:Lew Nikolajewitsch Tolstoi" --add_key_value="Titel:Krieg und Frieden" --add_key_value="Genre:Roman" Replace (minimalistic) NOT RECOMMENDED! IF FAILS, "IE" MUST BE MANUAL RESTORED TO THE PREVIOUS STATE!!! perl bin/slubsipbuilderbagit.pl --save_option=replace --IE_directory=/IE/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2 --ppn=457035137 --SIP_version=v2020.1 --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14 --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0 --rights_xml=/metadata/rights/Fallbeispiel-01.xml Move (minimalistic) NOT RECOMMENDED! IF FAILS, "IE" MUST BE MANUAL RESTORED TO THE PREVIOUS STATE!!! perl bin/slubsipbuilderbagit.pl --save_option=move --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2 --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1 --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14 --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0 --rights_xml=/metadata/rights/Fallbeispiel-03.xml ``` ##### Run tests * Start script in project directory to run all tests properly. * NOTE: while running tests path `tmp/bagit` will be created. ```perl perl t/slubsipbuilderbagit.t ``` or ```perl prove t ```