Skip to content
Snippets Groups Projects

SLUB SIP Builder

Description

A CLI tool to create valid BagIt based submission information packages (SIPs) for ingest into the SLUBArchiv, the digital long-term preservation archive of the Saxon State and University Library Dresden (SLUB).

As a user, you have one of three options to create your SIP: copy, move or replace.
NOTE: The use of option copy is strongly recommended. The remaining options modify your source directory. Please use these with caution!

Supported Operating Systems

  • Debian 10
  • Windows 10

Requirements

  • Perl version 5.28 or higher
  • cpan or cpanm
  • several CPAN modules such as Archive::BagIt (0.065 or higher)
    • sudo cpan -D Archive::BagIt to check module perl version

Three ways to install a module

  • with cpan cpan Archive::BagIt
  • with cpanm cpanm Archive::BagIt (NOTE: cpan App::cpanminus must be installed. If errors like IO:AIO are present, use cpanm -i -f Archive::BagIt).
  • from source (Linux)
    1. Download for example Archive-BagIt-0.072.tar.gz from metacpan Archive::BagIt
    2. Extract it with tar zxf Archive-BagIt-0.072.tar.gz and go to project dir with cd in Archive-BagIt-0.072
    3. Run perl Makefile.pl
    4. (d)gmake
    5. (d)gmake test
    6. (d)gmake install
    7. with cpanm Archive::BagIt test for the version. Result will be something like Archive::BagIt is up to date (0.072).

Archive::BagIt module

  • Sometimes it is offered to update module to a new version use cpan Archive::BagIt. If version not changed to a new one, try cpanm Archive::BagIt or perl -MCPAN -e 'install Archive::BagIt'

Perl module deinstallation

Using cpanm is recommended for an assisted and clean uninstall.

  • install cpanm with cpan App::cpanminus.
  • remove a Perl module with cpanm --uninstall MODULE_NAME

Installation (Win 10)

  1. Check UTF-8 flag at [ENG] Systemadministration/TimeAndRegion/Region/Administration/AreaPattern or [DE] Systemsteuerung/Zeit und Region/Region/Verwaltung/Gebietsschema ändern/, for using such characters as äöüÄÖÜß. Computer will be restarted!
  2. Download strawberry perl environment strawberry-perl-5.30.3.1-64bit-portable
  3. Extract files from archive
  4. Download Archive::BagIt module (Example: Archive-BagIt-0.072.tar.gz)
  5. Start strawberry perl console portableshell.bat
    1. Unpack archive tar zxf Archive-BagIt-0.072.tar.gz for example in the same dir where strawberryperl extracted
    2. Change dir cd Archive-BagIt-0.072
    3. Generate Makefile perl Makefile.pl
    4. Run gmake, than gmake install
    5. Check if module installed cpanm Archive::BagIt. At this point Archive::BagIt module installed.
  6. Download SLUB_SIP_Builder as zip
  7. Extract files from archive
  8. Change console path to extracted SLUB_SIP_Builder
  9. Run tests perl t/slubsipbuilderbagit.t or prove.t and errors will come like 'Can't locate MARC/Record.pm in @INC (you may need to install the MARC::Record module) ...'
  10. Copy from error module name and install it, repeat step 9 and 10 till test process starts.
    • cpanm MARC::Record
    • cpanm XML::XPath
    • cpanm DateTime::Format::ISO8601
  11. All tests must process successfully and you are ready to go with Bag creation
  12. If colors on console not visible, than add line use Win32::Console::ANSI in file SLUB_SIP-Builder/bin/slubsipbuilderbagit.pl above use Term::ANSIColor;
    • cpanm Win32::Console::ANSI

Documentation

Recommendation

  • Do not name your directories data
  • marc21.xml and rights.xml file names are reserved in directory mets. All same file names will be renamed to 1.xml, 2.xml and so on
  • Use absolute file and directory paths
  • If you try to execute the program for the first time, than use save_option copy

Explanation

Usage
perl bin/slubsipbuilderbagit.pl -help
Result
Usage:
    slubsipbuilderbagit.pl [options]

    Options: -help brief help message -man full documentation

            -save_option=<option>           payload files save as <copy>, <move>, <replace>, the <copy> option should be prefered
            -IE_directory=<IE dir>          existing IE directory (absolute path!)
            -SIP_output_path=<target dir>   where to put the SIP dir (absolute path!)
            -ppn=<ppn>|-noppn=<noppn>       SWB-PPN or any identifier (uses minimalistic marc21)

            -SIP_version=<version>          mandatory, needs for identification of SIP format, supported <v2020.1>
            -external_workflow=<workflow>   mandatory, should be uniqe workflow name
            -external_id=<id>               mandatory, should be uniqe ID (i.e. a catalog ID), reuse an ID only when updating existing AIP
            -external_ISIL=<isil>           optional, ISIL number of library
            -external_conservation_flag     optional, if set no other "original" still exists
            -external_value_descr=<text>    mandatory, the reason why to archive
            -rights_version=<version>       mandatory, SLUB law managment specification, supported <1.0>

            -rights_xml=<file>              mandatory, path to SLUB rights xml file(absolute path!)
            -add_meta_file=<file>           optional, can be repeated, additional meta files(absolute path!)
            -add_key_value=<"key:value">    optional, can be repeated, additional key value pairs for bag-info.txt

Options:
    -help   Print a brief help message and exits.
Full documentation with examples
perl -I lib/ bin/slubsipbuilderbagit.pl -man
Result
  • MUST: Write command in one line.
EXAMPLES
  Copy    (minimalistic)
    perl bin/slubsipbuilderbagit.pl --save_option=copy
    --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1
    --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-02.xml

  Copy
    perl bin/slubsipbuilderbagit.pl --save_option=copy
    --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1
    --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-01.xml
    --add_meta_file=/metadata/lido.xml --add_meta_file=/metadata/marc21.xml
    --add_key_value="Author:Lew Nikolajewitsch Tolstoi"
    --add_key_value="Titel:Krieg und Frieden" --add_key_value="Genre:Roman"

  Replace (minimalistic)
    NOT RECOMMENDED! IF FAILS, "IE" MUST BE MANUAL RESTORED TO THE PREVIOUS
    STATE!!!

    perl bin/slubsipbuilderbagit.pl --save_option=replace
    --IE_directory=/IE/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --ppn=457035137 --SIP_version=v2020.1 --external_id=10008
    --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-01.xml

  Move    (minimalistic)
    NOT RECOMMENDED! IF FAILS, "IE" MUST BE MANUAL RESTORED TO THE PREVIOUS
    STATE!!!

    perl bin/slubsipbuilderbagit.pl --save_option=move
    --IE_directory=/IE/git/SLUB_SIP_Builder/export_dir_kitodo/bagit/test2
    --SIP_output_path=/output_sips --ppn=457035137 --SIP_version=v2020.1
    --external_id=10008 --external_workflow=kitodo --external_ISIL=DE-14
    --external_value_descr="Gesetzlicher Auftrag" --rights_version=1.0
    --rights_xml=/metadata/rights/Fallbeispiel-03.xml
Run tests
  • Start script in project directory to run all tests properly.
  • NOTE: while running tests path tmp/bagit will be created.
perl t/slubsipbuilderbagit.t

or

prove t