| ... | @@ -2,65 +2,89 @@ |
... | @@ -2,65 +2,89 @@ |
|
|
|
|
|
|
|
## The full list of toolkit scripts:
|
|
## The full list of toolkit scripts:
|
|
|
|
|
|
|
|
|
- gfs_batch_validate_checksum.bat
|
|
|
|
|
|
|
|
Runs the gfs_validate_tranche_checksum.py script against multiple tranches to validate file fixity
|
|
|
|
|
|
|
|
- gfs_copy_asset_folder_to_target.py
|
|
|
|
|
|
|
|
Copies the files in the asset-folders specified on the command line to folders that are created by the
|
|
|
|
script with names that reflect their location within the GFS within a folder that is also specified
|
|
|
|
on the command line
|
|
|
|
|
|
|
- gfs_copy_tranche_files_to_folder.py
|
|
- gfs_copy_tranche_files_to_folder.py
|
|
|
|
|
|
|
|
Copies the files in the folder-types specified on the command line from a tranche to a single folder
|
|
Copies the asset-files in the asset-folders specified on the command line from a tranche to a single folder
|
|
|
|
|
|
|
|
- gfs_copy_tranche_to_tranche.py
|
|
- gfs_copy_tranche_to_tranche.py
|
|
|
|
|
|
|
|
Copies the files in the folder-types specified on the command line from one tranche to an identical
|
|
Copies the asset-files in the asset-folders specified on the command line from one tranche to an identical
|
|
|
(but empty) set of folder-types in an identical tranche in a separate GFS
|
|
(but empty) set of asset-folders in an identical tranche in a separate GFS
|
|
|
|
|
|
|
|
- gfs_copy_folder_type_to_target.py
|
|
|
|
|
|
|
|
|
|
Copies the files in the folder-types specified on the command line to folders that are created by the
|
|
|
|
|
script with names that reflect their location within the GFS within a folder that is also specified
|
|
|
|
|
on the command line
|
|
|
|
|
|
|
|
|
|
- gfs_create_arkivum_upload.py
|
|
- gfs_create_arkivum_upload.py
|
|
|
|
|
|
|
|
Creates an folder structure suitable for upload to Arkivum's Perpetua, which is a digital preservation
|
|
Creates a BagIt folder suitable for upload to Arkivum, which is a digital preservation
|
|
|
and dissemination platform
|
|
and dissemination platform
|
|
|
|
|
|
|
|
- gfs_create_or_delete_tranche_folder_type.py
|
|
- gfs_create_or_delete_tranche_asset_folder.py
|
|
|
|
|
|
|
|
Creates or deletes the folder-types specified on the command line within the child-folders of the
|
|
Creates or deletes the asset-folders specified on the command line within the child-folders of the
|
|
|
specified tranche
|
|
specified tranche
|
|
|
|
|
|
|
|
|
- gfs_create_or_delete_tranche_checksum_folder.py
|
|
|
|
|
|
|
|
Creates or deletes the checksum-folders specified on the command line within the child-folders of the
|
|
|
|
specified tranche
|
|
|
|
|
|
|
|
- gfs_create_tranche_checksum_file.py
|
|
|
|
|
|
|
|
Creates and populates checksum-manifest-files within the checksum-folders that are specified on the command
|
|
|
|
line of the specified tranche
|
|
|
|
|
|
|
- gfs_create_tranche_folder.py
|
|
- gfs_create_tranche_folder.py
|
|
|
-
|
|
|
|
|
Creates a tranche folder structure from the tranche csv file with the folder-types that are specified
|
|
Creates a tranche folder structure from the information provided in the tranche csv file with the
|
|
|
on the command line
|
|
asset-folders that are specified on the command line
|
|
|
|
|
|
|
|
- gfs_distribute_arkivum_export_to_tranche.py
|
|
- gfs_distribute_arkivum_export_to_tranche.py
|
|
|
|
|
|
|
|
Repopulates the folder-types specified on the command line for a tranche from a BagIt
|
|
Repopulates the asset-folders specified on the command line for a tranche from a BagIt
|
|
|
folder structure that has been downloaded from Arkivum's Perpetua via the "Bulk Export" option
|
|
folder that has been exported from Arkivum's Perpetua via the "Bulk Export" option
|
|
|
|
|
|
|
|
- gfs_distribute_files_to_tranche.py
|
|
- gfs_distribute_files_to_tranche.py
|
|
|
|
|
|
|
|
Populates the folder-types specified on the command line for a tranche from files that abide by the
|
|
Populates the asset-folders specified on the command line for a tranche from asset-files that abide by
|
|
|
GFS file-naming convention that are located in a single folder
|
|
the GFS file-naming convention that are located in a single folder
|
|
|
|
|
|
|
|
- gfs_generate_checksum.py
|
|
|
|
|
|
|
|
Reports a checksum generated for an asset-file whose path is specified on the command line, there by enabling
|
|
|
|
the checksum to be compared with a checksum of an asset-file listed in a checksum-manifest-file
|
|
|
|
|
|
|
|
- gfs_migrate_tranche_folder.py
|
|
- gfs_migrate_tranche_folder.py
|
|
|
|
|
|
|
|
Populates the folder-types specified on the command line for a tranche from the folders of a legacy
|
|
Populates the asset-folders specified on the command line for a tranche from the asset-files in the
|
|
|
collection
|
|
folders of a legacy collection
|
|
|
|
|
|
|
|
- gfs_rename_tranche_files.py
|
|
- gfs_rename_tranche_files.py
|
|
|
|
|
|
|
|
Renames the files within the folder-types specified on the command line for a tranche so that the
|
|
Renames the asset-files within the asset-folders specified on the command line for a tranche so that the
|
|
|
files abide by the GFS file-naming convention
|
|
asset-files abide by the GFS file-naming convention
|
|
|
|
|
|
|
|
- gfs_rename_tranche_folder_type.py
|
|
- gfs_rename_tranche_asset_folder.py
|
|
|
|
|
|
|
|
Renames a folder-type within the child-folders of a tranche
|
|
Renames an asset-folder within the child-folders of a tranche
|
|
|
|
|
|
|
|
- gfs_validate_project_csv.py
|
|
- gfs_validate_project_csv.py
|
|
|
|
|
|
|
|
Validates that a project csv file contains the correct columns, that running number sequences are
|
|
Validates that a project csv file contains the correct columns, that running number sequences are
|
|
|
valid, and that mandatory fields have been completed
|
|
valid, and that mandatory fields have been completed
|
|
|
|
|
|
|
|
|
- gfs_validate_tranche_checksum.py
|
|
|
|
|
|
|
|
Validates that the checksums for the asset-files created in memory by the script match the checksums
|
|
|
|
recorded in the checksum-manifest-file for the asset-folders specified on the command line for a tranche
|
|
|
|
|
|
|
- gfs_validate_tranche_csv.py
|
|
- gfs_validate_tranche_csv.py
|
|
|
|
|
|
|
|
Validates that a tranche csv file contains the correct columns, that running number sequences are
|
|
Validates that a tranche csv file contains the correct columns, that running number sequences are
|
| ... | @@ -68,9 +92,9 @@ |
... | @@ -68,9 +92,9 @@ |
|
|
|
|
|
|
|
- gfs_validate_tranche_folder.py
|
|
- gfs_validate_tranche_folder.py
|
|
|
|
|
|
|
|
Validates that all the folder-types specified on the command line contain the expected file content
|
|
Validates that all the asset-folders specified on the command line contain the expected asset-file content
|
|
|
in terms of the composition of the filenames, the sequence number component of the filename (if it
|
|
in terms of the composition of the filenames, the sequence number component of the asset-file name (if it
|
|
|
has one) and reports the total numbers of files and the total disk space the files occupy
|
|
has one) and reports the total numbers of asset-files and the total disk space that the asset-files occupy
|
|
|
|
|
|
|
|
|
|
|
|
|
More detailed instructions for running each of the scripts are contained in the script header and can be viewed by opening a script with a text editor such as Notepad.
|
|
More detailed instructions for running each of the scripts are contained in the script header and can be viewed by opening a script with a text editor such as Notepad.
|
| ... | @@ -80,13 +104,13 @@ More detailed instructions for running each of the scripts are contained in the |
... | @@ -80,13 +104,13 @@ More detailed instructions for running each of the scripts are contained in the |
|
|
- gfs_validate_project_csv.py
|
|
- gfs_validate_project_csv.py
|
|
|
- gfs_validate_tranche_csv.py
|
|
- gfs_validate_tranche_csv.py
|
|
|
|
|
|
|
|
## Scripts that are specific to Arkivum’s Digital Preservation Platform (Perpetua)
|
|
## Scripts that are specific to Arkivum’s Digital Preservation Platform
|
|
|
- gfs_create_arkivum_upload.py
|
|
- gfs_create_arkivum_upload.py
|
|
|
- gfs_distribute_arkivum_export_to_tranche.py
|
|
- gfs_distribute_arkivum_export_to_tranche.py
|
|
|
|
|
|
|
|
## Scripts that can be used by the Digitisation Provider to aid workflow
|
|
## Scripts that can be used by the Digitisation Provider to aid workflow
|
|
|
- gfs_validate_tranche_folder.py
|
|
- gfs_validate_tranche_folder.py
|
|
|
- gfs_copy_folder_type_to_target.py
|
|
- gfs_copy_asset_folder_to_target.py
|
|
|
- gfs_distribute_files_to_tranche.py
|
|
- gfs_distribute_files_to_tranche.py
|
|
|
|
|
|
|
|
## Scripts that have "start row" and "stop row" command line parameters
|
|
## Scripts that have "start row" and "stop row" command line parameters
|
| ... | @@ -97,5 +121,23 @@ More detailed instructions for running each of the scripts are contained in the |
... | @@ -97,5 +121,23 @@ More detailed instructions for running each of the scripts are contained in the |
|
|
|
|
|
|
|
- gfs_distribute_files_to_tranche.py
|
|
- gfs_distribute_files_to_tranche.py
|
|
|
|
|
|
|
|
|
## Scripts whose sole purpose relates to file fixity checking
|
|
|
|
- gfs_batch_validate_checksum.bat
|
|
|
|
- gfs_create_or_delete_tranche_checksum_folder.py
|
|
|
|
- gfs_create_tranche_checksum_file.py
|
|
|
|
- gfs_generate_checksum.py
|
|
|
|
- gfs_validate_tranche_checksum.py
|
|
|
|
|
|
|
|
## Scripts that have some file fixity checking in their processing
|
|
|
|
- gfs_create_arkivum_upload.py
|
|
|
|
- gfs_distribute_arkivum_export_to_tranche.py
|
|
|
|
- gfs_copy_tranche_files_to_folder.py
|
|
|
|
- gfs_copy_tranche_to_tranche.py
|
|
|
|
|
|
|
|
## Scripts that validate some aspect of the GFS, either the csv files, or the content of the folders
|
|
|
|
- gfs_validate_project_csv.py
|
|
|
|
- gfs_validate_tranche_checksum.py
|
|
|
|
- gfs_validate_tranche_csv.py
|
|
|
|
- gfs_validate_tranche_folder.py
|
|
|
|
|
|
|
|
[Return to documentation home page](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/LSE-Digital-Toolkit) |
|
[Return to documentation home page](https://itsagit.lse.ac.uk/hub/lse_digital_toolkit/-/wikis/LSE-Digital-Toolkit) |
