| ... | ... | @@ -24,7 +24,7 @@ If the message returned is "Python 3.10.0" or a higher number, this is a suffici |
|
|
|
|
|
|
|
If you forgot to tick the box during installation, or Python has already been installed without it having been ticked, follow the instructions given in this document [How_to_add_Python_to_PATH_manually.docx](uploads/40f39542559fcd537ef4bdc73c7df772/How_to_add_Python_to_PATH_manually.docx).
|
|
|
|
|
|
|
|
To test whether the installation of Perl has been successful, open a command window (type “cmd” in the field where it states “Type here to search” in the bottom-left of the desktop) and at the prompt in the command window that appears, type “python --version”. If the message returned is "Python 3.10.0” or a higher number, Python has been installed.
|
|
|
|
To test whether the installation of Python has been successful, open a command window (type “cmd” in the field where it states “Type here to search” in the bottom-left of the desktop) and at the prompt in the command window that appears, type “python --version”. If the message returned is "Python 3.10.0” or a higher number, Python has been installed successfully.
|
|
|
|
|
|
|
|
## 2. Install the Toolkit
|
|
|
|
|
| ... | ... | @@ -69,9 +69,9 @@ The example command lines will be like those shown below. The command lines will |
|
|
|
|
|
|
|
Execute the script that validates that the project csv file.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_project_csv.pl Z:\GFS\UKLSE\EX1\ZT01 default
|
|
|
|
python H:\lse_digital_toolkit\gfs_validate_project_csv.py Z:\GFS\UKLSE\EX1\ZT01 default
|
|
|
|
|
|
|
|
Note that the script creates a “logs” folder within the project code folder, and that a log file is created within the folder by the execution of the script.
|
|
|
|
Note that the script creates a “logs” folder within the project-code folder, and that a log file is created within the folder by the execution of the script.
|
|
|
|
|
|
|
|
The output in the command window after the script has been executed should be similar to that indicated in this screenshot:
|
|
|
|
|
| ... | ... | @@ -79,7 +79,7 @@ The output in the command window after the script has been executed should be si |
|
|
|
|
|
|
|
Execute the script that validates the tranche csv file.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_tranche_csv.pl Z:\GFS\UKLSE\EX1\ZT01\001 default
|
|
|
|
python H:\lse_digital_toolkit\gfs_validate_tranche_csv.py Z:\GFS\UKLSE\EX1\ZT01\001 default
|
|
|
|
|
|
|
|
Note that the script creates a “logs” folder within the tranche number folder (folder 001), and that a log file has been created within the folder by the execution of the script.
|
|
|
|
|
| ... | ... | @@ -89,7 +89,7 @@ The output in the command window after the script has been executed should be si |
|
|
|
|
|
|
|
Execute the script that validates the tranche folder structure.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_tranche_folder.pl y Z:\GFS\UKLSE\EX1\EH01\001 jpg alto text pdf
|
|
|
|
python H:\lse_digital_toolkit\gfs_validate_tranche_folder.py y Z:\GFS\UKLSE\EX1\EH01\001 jpg alto text pdf
|
|
|
|
|
|
|
|
The output in the command window after the script has been executed should be similar to that indicated in this screenshot:
|
|
|
|
|
| ... | ... | @@ -127,9 +127,9 @@ This section is mandatory. |
|
|
|
|
|
|
|
For example, if your chosen organisation code is _XXXXX_, and you have created a folder of that name, and within that folder, you have renamed the _EX1_ department code folder from the copied hierarchy, to _YY1_, you need to rename the _UKLSE_organisation.csv_ file, so that its name is _XXXXX_organisation.csv_, and rename the _UKLSE_EX1_department.csv_ file, that resides in the _YY1_ folder, so that its name is _XXXXX_YY1_department.csv_.
|
|
|
|
|
|
|
|
Similarly, if you have renamed the _ZT01_ project code folder from the copied hierarchy to _ZZ01_ you need to rename the _UKLSE_EX1_ZT01_project.csv_ file, so that its name is _XXXXX_YY1_ZZ1_project.csv_.
|
|
|
|
Similarly, if you have renamed the _ZT01_ project code folder from the copied hierarchy to _ZZ01_ you need to rename the _UKLSE_EX1_ZT01_project.csv_ file, so that its name is _XXXXX_YY1_ZZ01_project.csv_.
|
|
|
|
|
|
|
|
It is also necessary to rename the the _UKLSE_EX1_ZT01_001_tranche.csv_ file so that its name is equivalent to _XXXXX_YY1_ZZ1_001_tranche.csv_
|
|
|
|
It is also necessary to rename the the _UKLSE_EX1_ZT01_001_tranche.csv_ file so that its name is equivalent to _XXXXX_YY1_ZZ01_001_tranche.csv_
|
|
|
|
|
|
|
|
Continuing with the same renaming logic, edit the equivalent of the file named _XXXXX_organisation.csv_ file, and change the content of the gfs.departmentCode and gfs.dpartmentName columns in row 2 from _EX1_ and “Example department for LSE Digital Toolkit users” to your equivalent of _YY1_ and to a corresponding department name.
|
|
|
|
|
| ... | ... | @@ -146,7 +146,7 @@ _XXXXX_YY1_ZZ01_project.csv_ that resides in the equivalent of the “_ZZ01_” |
|
|
|
|
|
|
|
Execute the script that validates your project csv file.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_project_csv.pl Z:\GFS\\\<organisation code>\\\<department code>\\\<project code> default
|
|
|
|
perl H:\lse_digital_toolkit\gfs_validate_project_csv.py Z:\GFS\\\<organisation code>\\\<department code>\\\<project code> default
|
|
|
|
|
|
|
|
## 8. Create s new tranche folder structure that reflects your content
|
|
|
|
|
| ... | ... | @@ -160,17 +160,17 @@ The numerical content of the “gfs.parentFolder” column represents a grouping |
|
|
|
|
|
|
|
Keep things as simple as possible for the purposes of this exercise.
|
|
|
|
|
|
|
|
After each change, run the gfs_validate_tranche_csv.pl script to check that the tranche csv file still passes validation.
|
|
|
|
After each change, run the gfs_validate_tranche_csv.py script to check that the tranche csv file still passes validation.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_tranche_csv.pl Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 default
|
|
|
|
python H:\lse_digital_toolkit\gfs_validate_tranche_csv.py Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 default
|
|
|
|
|
|
|
|
Once the tranche csv file is complete and validates correctly, decide which folder-types should be created within the tranche (tif, jpg, text, mp4, etc) and run the “gfs_create_tranche_folder.pl” script, quoting the folder-types on the command line parameter list, rather than those indicated in the example below, and substituting in your own codes in the GFS path.
|
|
|
|
Once the tranche csv file is complete and validates correctly, decide which folder-types should be created within the tranche (tif, jpg, text, mp4, etc) and run the “gfs_create_tranche_folder.py” script, quoting the folder-types on the command line parameter list, rather than those indicated in the example below, and substituting in your own codes in the GFS path.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_create_tranche_folder.pl Z:\GFS\\\<organization code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
python H:\lse_digital_toolkit\gfs_create_tranche_folder.py Z:\GFS\\\<organization code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
|
|
|
|
If it runs without error, you will observe that a folder structure has been created beneath the tranche folder, and that it reflects the content of the tranche csv file.
|
|
|
|
|
|
|
|
If the script reports that the folder-type was not recognised, open the gfs_folder_type_list.csv file in a suitable spreadsheet editor (such as Excel), and insert a new row for each of the folder-types that were not recognised into the alphabetical listing. Enter values in both the "gfs.folderName" and "gfs.fileNameExtension" columns, and then try running the gfs_create_tranche_folder.pl script again.
|
|
|
|
If the script reports that the folder-type was not recognised, open the gfs_folder_type_info.csv file in a suitable spreadsheet editor (such as Excel), and insert a new row for each of the folder-types that were not recognised into the alphabetical listing. Enter values in both the "gfs.folderName" and "gfs.fileNameExtension" columns, and then try running the gfs_create_tranche_folder.pl script again.
|
|
|
|
|
|
|
|
## 9. Populate the new tranche folder with asset files
|
|
|
|
|
| ... | ... | @@ -186,7 +186,7 @@ This section is mandatory. |
|
|
|
|
|
|
|
Run the folder validation script as follows, substituting in your chosen folder-types rather than those shown below:
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_tranche_folder.pl n Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
python H:\lse_digital_toolkit\gfs_validate_tranche_folder.py n Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
|
|
|
|
View the screen output or script log file to ensure that the outcome meets with expectations.
|
|
|
|
|
| ... | ... | @@ -194,9 +194,9 @@ View the screen output or script log file to ensure that the outcome meets with |
|
|
|
|
|
|
|
This section is not mandatory.
|
|
|
|
|
|
|
|
All the scripts indicated in this "Getting started with the toolkit" guide will still work if the these files are not renamed to correspond with their location in the new tranche. See the [Script Groups](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/Script-groups) section to find out which scripts will not, function when the files do not abide by the GFS file-naming convention. There are some minimum filename requirements for files to abide by in order that be processed by the scripts. These requirements are stated in the [The Generic Folder Structure (GFS)](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/LSE-Digital-Toolkit#the-generic-folder-structure-gfs) section.
|
|
|
|
All the scripts indicated in this "Getting started with the toolkit" guide will still work if the these files are not renamed to correspond with their location in the new tranche. See the [Script Groups](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/Script-groups) section to find out which scripts will not function when the files do not abide by the GFS file-naming convention. There are some minimum filename requirements for files to abide by in order that be processed by the scripts. These requirements are stated in the [The Generic Folder Structure (GFS)](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/LSE-Digital-Toolkit#the-generic-folder-structure-gfs) section.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_rename_tranche_files.pl Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
python H:\lse_digital_toolkit\gfs_rename_tranche_files.py Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
|
|
|
|
If that script runs with no errors, all the files in the specified folders will have been renamed.
|
|
|
|
|
| ... | ... | @@ -204,17 +204,17 @@ If that script runs with no errors, all the files in the specified folders will |
|
|
|
|
|
|
|
This section is only mandatory if the instructions in section 11 were followed.
|
|
|
|
|
|
|
|
Run the gfs_validate_tranche_folder.pl script again, but this time with the first parameter set to “y”, rather than “n”. This instructs the script to check that the asset files abide by the GFS naming convention, and that the sequence numbers in the filenames in each folder-type are continuous.
|
|
|
|
Run the gfs_validate_tranche_folder.py script again, but this time with the first parameter set to “y”, rather than “n”. This instructs the script to check that the asset files abide by the GFS naming convention, and that the sequence numbers in the filenames in each folder-type are continuous.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_validate_tranche_folder.pl y Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
python H:\lse_digital_toolkit\gfs_validate_tranche_folder.py y Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001 jpg text alto pdf
|
|
|
|
|
|
|
|
## 13. Create a package suitable for upload to Arkivum’s Digital Preservation Platform (Perpetua)
|
|
|
|
|
|
|
|
This section is not mandatory.
|
|
|
|
|
|
|
|
If you have installed 7-Zip, and made any necessary edits to the path for the 7z.exe executable file in the two relevant scripts, run the script “gfs_create_arkivum_upload.pl” to create the upload package.
|
|
|
|
Run the script “gfs_create_arkivum_upload.py” to create the upload package.
|
|
|
|
|
|
|
|
perl H:\LSE_TK_PERL\gfs_create_arkivum_upload.pl y y "Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001" preservation_and_access default pdf
|
|
|
|
python H:\lse_digital_toolkit\gfs_create_arkivum_upload.py y y "Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\001" preservation_and_access default pdf
|
|
|
|
|
|
|
|
The start of the output to the command window when the script is executed should look similar to that indicated in this screenshot:
|
|
|
|
|
| ... | ... | @@ -224,17 +224,17 @@ The end of the output to the command window when the script is executed should l |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
In between, there will be output produced by the repeated execution of the 7z.exe file. Each time, the 7-Zip output is preceded by a line output of the form:
|
|
|
|
There will be output of the form indicated below produced for each child folder:
|
|
|
|
|
|
|
|
INFO: Processing parent folder: x child folder y
|
|
|
|
|
|
|
|
The upload-package-container-folder will be created automatically within the project folder. The upload-package-container-folder will have the following path:
|
|
|
|
|
|
|
|
Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\arkivum_v5_<organisation code>_<department code>_<project code>_001_preservation_and_access
|
|
|
|
Z:\GFS\\\<organisation code>\\\<department code>\\\<project code>\arkivum_v6_<organisation code>_<department code>_<project code>_001_preservation_and_access
|
|
|
|
|
|
|
|
Within this folder can be found a zip file:
|
|
|
|
|
|
|
|
"arkivum_v5_<organisation code>_<department code>_<project code>_001_preservation_and_access_upload.zip"
|
|
|
|
"arkivum_v6_<organisation code>_<department code>_<project code>_001_preservation_and_access_upload.zip"
|
|
|
|
|
|
|
|
This is the zip file that can be uploaded to Perpetua.
|
|
|
|
|
| ... | ... | @@ -242,13 +242,13 @@ As "preservation_and_access" was specified on the command line, before uploading |
|
|
|
|
|
|
|
If using the default configuration, the slug will take the form "<organisation code>-<department code><project code>". Using the value that was automatically created in the metadata.csv as a slug will ensure that the naming conventions present in the records of the AtoM module are consistent with those in the GFS.
|
|
|
|
|
|
|
|
For further information about slug creation see the [The Generic Folder Structure (GFS)](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/LSE-Digital-Toolkit#the-generic-folder-structure-gfs) section.
|
|
|
|
For further information about slug creation see the [The Generic Folder Structure (GFS)](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/LSE-Digital-Toolkit#the-generic-folder-structure-gfs) section.
|
|
|
|
|
|
|
|
The way in which the tranche in the example GFS displays when uploaded to Perpetua can be seen [here](https://lse-atom.arkivum.net/uklse-ex1zt01).
|
|
|
|
|
|
|
|
A successful upload of this zip file will only occur if the content of the metadata.csv file matches the way in which the user's Perpetua instance has been configured. It will be necessary to contact Arkivum Technical Support to ensure that this is the case.
|
|
|
|
|
|
|
|
The composition of the metadata.csv file can be controlled by editing the “gfs_arkivum_column_header_info.csv” configuration file. Details of how to configure the toolkit can be found in the [Configuration](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/Configuration) section.
|
|
|
|
The composition of the metadata.csv file can be controlled by editing the “gfs_arkivum_column_header_info.csv” configuration file. Details of how to configure the toolkit can be found in the [Configuration](https://itsagit.lse.ac.uk/nub/lse_digital_toolkit/-/wikis/Configuration) section.
|
|
|
|
|
|
|
|
## 14. Tidying up your GFS
|
|
|
|
|
| ... | ... | @@ -258,10 +258,10 @@ The “UKLSE” folder, and everything beneath it, can now be deleted, so that t |
|
|
|
|
|
|
|
## 15 . What next?
|
|
|
|
|
|
|
|
After some more experimentation with the toolkit scripts (the instructions for each script can be found at the top by opening the scripts with a text editor such as Notepad), it would be appropriate to consult the [Configuration](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/Configuration) section in order to decide how best to set up the GFS so that it meets the requirements of your organisation. You can then add more departments, projects and tranches as required.
|
|
|
|
After some more experimentation with the toolkit scripts (the instructions for each script can be found at the top by opening the scripts with a text editor such as Notepad), it would be appropriate to consult the [Configuration](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/Configuration) section in order to decide how best to set up the GFS so that it meets the requirements of your organisation. You can then add more departments, projects and tranches as required.
|
|
|
|
|
|
|
|
If you upload to Arkivum's Perpetua, once you are confident that your version of the GFS is well founded, contact Arkivum Technical Support to ask them to configure your system so that upload files produced by the gfs_create_arkivum_upload.pl script can be processed.
|
|
|
|
If you upload to Arkivum's Perpetua, once you are confident that your version of the GFS is well founded, contact Arkivum Technical Support to ask them to configure your system so that upload files produced by the gfs_create_arkivum_upload.py script can be processed.
|
|
|
|
|
|
|
|
The [Workflows](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/LSE-Digital-Toolkit#workflows) section indicates how some scripts can be used in conjunction with others to achieve particular outcomes.
|
|
|
|
The [Workflows](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/LSE-Digital-Toolkit#workflows) section indicates how some scripts can be used in conjunction with others to achieve particular outcomes.
|
|
|
|
|
|
|
|
[Return to documentation home page](https://git.lse.ac.uk/bywell/lse-digital-toolkit-perl-version/-/wikis/LSE-Digital-Toolkit) |
|
|
\ No newline at end of file |
|
|
|
[Return to documentation home page](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/LSE-Digital-Toolkit) |
|
|
\ No newline at end of file |
