renku storage

Manage an external storage.

Commands and options

renku storage

Manage an external storage.

renku storage [OPTIONS] COMMAND [ARGS]...

check

Check if large files are committed to Git history.

renku storage check [OPTIONS]

Options

--all

Include all branches.

clean

Remove files from lfs cache/turn them back into pointer files.

renku storage clean [OPTIONS] PATHS...

Arguments

PATHS

Required argument(s)

migrate

Migrate large files committed to git by moving them to LFS.

renku storage migrate [OPTIONS] [PATHS]...

Options

-a, --all

Migrate all large files not in git LFS.

Arguments

PATHS

Optional argument(s)

pull

Pull the specified paths from external storage.

renku storage pull [OPTIONS] PATHS...

Arguments

PATHS

Required argument(s)

Pulling files from git LFS

LFS works by checking small pointer files into git and saving the actual contents of a file in LFS. If instead of your file content, you see something like this, it means the file is stored in git LFS and its contents are not currently available locally (they are not pulled):

version https://git-lfs.github.com/spec/v1
oid sha256:42b5c7fb2acd54f6d3cd930f18fee3bdcb20598764ca93bdfb38d7989c054bcf
size 12

You can manually pull contents of file(s) you want with:

$ renku storage pull file1 file2

Removing local content of files stored in git LFS

If you want to restore a file back to its pointer file state, for instance to free up space locally, you can run:

$ renku storage clean file1 file2

This removes any data cached locally for files tracked in in git LFS.

Migrate large files to git LFS

If you accidentally checked a large file into git or are moving a non-LFS renku repo to git LFS, you can use the following command to migrate the files to LFS:

$ renku storage migrate --all

This will move all files that are not excluded by .renkulfsignore into git LFS.

Note

Recent versions of Git LFS don’t support filtering files based on their size. Therefore, Renku ignores lfs_threshold config value when migrating files to LFS using this command.

To only migrate specific files, you can also pass their paths to the command like:

$ renku storage migrate big_file other_big_file