# SLUB SIP Builder

## Description
A CLI tool to create valid [BagIt](https://tools.ietf.org/html/rfc8493) based submission information packages (SIPs) for ingest into the [SLUBArchiv](https://slubarchiv.slub-dresden.de/), the digital long-term preservation archive of the Saxon State and University Library Dresden (SLUB).

As a user, you have one of three options to create your SIP: `copy`, `move` or `replace`.  
NOTE: The use of option `copy` is strongly recommended. The remaining options modify your source directory. Please use these with caution!

## Supported Operating Systems
* Debian 10
* Windows 10

## Requirements
* Perl version 5.28 or higher
* cpan or cpanm
* several CPAN modules such as Archive::BagIt (0.065 or higher)
  * `sudo cpan -D Archive::BagIt` to check module perl version

## Three ways to install a module
* with cpan `cpan Archive::BagIt`
* with cpanm `cpanm Archive::BagIt` (NOTE: `cpan App::cpanminus` must be installed. If errors like IO:AIO are present, use `cpanm -i -f Archive::BagIt`).
* from source (Linux)
  1. Download for example Archive-BagIt-0.072.tar.gz from [metacpan Archive::BagIt](https://metacpan.org/pod/Archive::BagIt)
  2. Extract it with `tar zxf Archive-BagIt-0.072.tar.gz` and go to project dir with `cd in Archive-BagIt-0.072`
  3. Run `perl Makefile.pl`
  4. `(d)gmake`
  5. `(d)gmake test`
  6. `(d)gmake install`
  7. with `cpanm Archive::BagIt` test for the version. Result will be something like `Archive::BagIt is up to date (0.072)`.
  
## Archive::BagIt module
* Sometimes it is offered to update module to a new version
use `cpan Archive::BagIt`. If version not changed to a new one, try `cpanm Archive::BagIt` or `perl -MCPAN -e 'install Archive::BagIt'`
  
## Perl module deinstallation
Using cpanm is recommended for an assisted and clean uninstall.
* install cpanm with `cpan App::cpanminus`.
* remove a Perl module with `cpanm --uninstall MODULE_NAME`

## Installation (Win 10)
1. Check UTF-8 flag at [ENG] `Systemadministration/TimeAndRegion/Region/Administration/AreaPattern` or [DE] `Systemsteuerung/Zeit und Region/Region/Verwaltung/Gebietsschema ändern/`, for using such characters as `äöüÄÖÜß`. Computer will be restarted!  
2. Download strawberry perl environment [strawberry-perl-5.30.3.1-64bit-portable](https://strawberryperl.com/releases.html)
3. Extract files from archive
4. Download [Archive::BagIt](https://metacpan.org/pod/Archive::BagIt) module (Example: Archive-BagIt-0.072.tar.gz)
5. Start strawberry perl console `portableshell.bat`
    1. Unpack archive `tar zxf Archive-BagIt-0.072.tar.gz` for example in the same dir where strawberryperl extracted
    2. Change dir `cd Archive-BagIt-0.072`
    3. Generate Makefile `perl Makefile.pl`
    4. Run `gmake`, than `gmake install`
    5. Check if module installed `cpanm Archive::BagIt`. At this point Archive::BagIt module installed. 
6. Download [SLUB_SIP_Builder](https://git.slub-dresden.de/digital-preservation/SLUB_SIP_Builder) as zip
7. Extract files from archive
8. Change console path to extracted SLUB_SIP_Builder
9. Run tests `perl t/slubsipbuilderbagit.t` or `prove.t` and errors will come like 'Can't locate MARC/Record.pm in @INC (you may need to install the `MARC::Record` module) ...'
10. Copy from error module name and install it, repeat step 9 and 10 till test process starts.
    * `cpanm MARC::Record`
    * `cpanm XML::XPath`
    * `cpanm DateTime::Format::ISO8601`
    * `cpanm Win32::Console::ANSI`
11. All tests must process successfully and you are ready to go with Bag creation.

## Documentation
* Project based on BagIt 1.0 specification, visit [RFC8493](https://tools.ietf.org/html/rfc8493) 
* For valid SIP examples based on operating system please have a look at `export_dir_kitodo/bagit/lin_*` or `export_dir_kitodo/bagit/win_*`
* For [valid](https://git.slub-dresden.de/slub-digitalpreservation/submissionapplications4rosetta/-/tree/master/examples/bagit/valid_SLUB_bagits) and [faulty](https://git.slub-dresden.de/slub-digitalpreservation/submissionapplications4rosetta/-/tree/master/examples/bagit/faulty_SLUB_bagits) SLUB BagIt examples take a look at submission application project.
* Specification for creating a SLUBArchiv SIP [SLUBArchiv_Produzenten_SIP_Spezifikation.pdf](https://slubarchiv.slub-dresden.de/technische-standards-fuer-die-ablieferung-von-digitalen-dokumenten/)
* SLUBArchiv rights coding specification [SLUBArchiv_Produzenten_Rechteauszeichnung.pdf](https://slubarchiv.slub-dresden.de/technische-standards-fuer-die-ablieferung-von-digitalen-dokumenten/)

## Recommendation
* Do not name your directories `data`
* `marc21.xml` and `rights.xml` file names are reserved in directory `mets`. All same file names will be renamed to `1.xml`, `2.xml` and so on
* Use absolute file and directory paths
* If you try to execute the program for the first time, than use `save_option` copy

### Explanation
##### Usage
```perl
perl bin/slubsipbuilderbagit.pl -help
```
###### Result
```perl
Usage:
    slubsipbuilderbagit.pl [options]

    Options: -help brief help message -man full documentation

            -save_option=<option>           payload files save as <copy>, <move>, <replace>, the <copy> option should be prefered
            -IE_directory=<IE dir>          existing IE directory (absolute path!)
            -SIP_output_path=<target dir>   where to put the SIP dir (absolute path!)
            -ppn=<ppn>|-noppn=<noppn>       SWB-PPN or any identifier (uses minimalistic marc21)

            -SIP_version=<version>          mandatory, needs for identification of SIP format, supported <v2020.1>
            -external_workflow=<workflow>   mandatory, should be uniqe workflow name
            -external_id=<id>               mandatory, should be uniqe ID (i.e. a catalog ID), reuse an ID only when updating existing AIP
            -external_ISIL=<isil>           optional, ISIL number of library
            -external_conservation_flag     optional, if set no other "original" still exists
            -external_value_descr=<text>    mandatory, the reason why to archive
            -rights_version=<version>       mandatory, SLUB law managment specification, supported <1.0>

            -rights_xml=<file>              mandatory, path to SLUB rights xml file(absolute path!)
            -add_meta_file=<file>           optional, can be repeated, additional meta files(absolute path!)
            -add_key_value=<"key:value">    optional, can be repeated, additional key value pairs for bag-info.txt

Options:
    -help   Print a brief help message and exits.

```

##### Full documentation with examples
```perl
perl -I lib/ bin/slubsipbuilderbagit.pl -man
```
###### Result
* MUST: Write command in one line.
```perl
EXAMPLES
  Copy    (minimalistic)
    perl bin/slubsipbuilderbagit.pl --save_option=copy
    --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1
    --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-02.xml

  Copy
    perl bin/slubsipbuilderbagit.pl --save_option=copy
    --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1
    --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-01.xml
    --add_meta_file=/metadata/lido.xml --add_meta_file=/metadata/marc21.xml
    --add_key_value="Author:Lew Nikolajewitsch Tolstoi"
    --add_key_value="Titel:Krieg und Frieden" --add_key_value="Genre:Roman"

  Replace (minimalistic)
    NOT RECOMMENDED! IF FAILS, "IE" MUST BE MANUAL RESTORED TO THE PREVIOUS
    STATE!!!

    perl bin/slubsipbuilderbagit.pl --save_option=replace
    --IE_directory=/IE/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --ppn=457035137 --SIP_version=v2020.1 --external_id=10008
    --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-01.xml

  Move    (minimalistic)
    NOT RECOMMENDED! IF FAILS, "IE" MUST BE MANUAL RESTORED TO THE PREVIOUS
    STATE!!!

    perl bin/slubsipbuilderbagit.pl --save_option=move
    --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1
    --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-03.xml
```

##### Run tests
* Start script in project directory to run all tests properly.
* NOTE: while running tests path `tmp/bagit` will be created.
```perl
perl t/slubsipbuilderbagit.t
```
or
```perl
prove t
```