renku storage
Manage an external storage.
Commands and options
renku storage
Manage an external storage.
renku storage [OPTIONS] COMMAND [ARGS]...
check
Check if large files are committed to Git history.
renku storage check [OPTIONS]
Options
- --all
Include all branches.
clean
Remove files from lfs cache/turn them back into pointer files.
renku storage clean [OPTIONS] PATHS...
Arguments
- PATHS
Required argument(s)
migrate
Migrate large files committed to git by moving them to LFS.
renku storage migrate [OPTIONS] [PATHS]...
Options
- -a, --all
Migrate all large files not in git LFS.
Arguments
- PATHS
Optional argument(s)
pull
Pull the specified paths from external storage.
renku storage pull [OPTIONS] PATHS...
Arguments
- PATHS
Required argument(s)
Pulling files from git LFS
LFS works by checking small pointer files into git and saving the actual contents of a file in LFS. If instead of your file content, you see something like this, it means the file is stored in git LFS and its contents are not currently available locally (they are not pulled):
version https://git-lfs.github.com/spec/v1
oid sha256:42b5c7fb2acd54f6d3cd930f18fee3bdcb20598764ca93bdfb38d7989c054bcf
size 12
You can manually pull contents of file(s) you want with:
$ renku storage pull file1 file2
Removing local content of files stored in git LFS
If you want to restore a file back to its pointer file state, for instance to free up space locally, you can run:
$ renku storage clean file1 file2
This removes any data cached locally for files tracked in in git LFS.
Migrate large files to git LFS
If you accidentally checked a large file into git or are moving a non-LFS renku repo to git LFS, you can use the following command to migrate the files to LFS:
$ renku storage migrate --all
This will move all files that are not excluded by .renkulfsignore into git LFS.
Note
Recent versions of Git LFS don’t support filtering files based on their size. Therefore, Renku ignores lfs_threshold config value when migrating files to LFS using this command.
To only migrate specific files, you can also pass their paths to the command like:
$ renku storage migrate big_file other_big_file