Convert an Existing Project to a Renku project

So, at this point you have gone through Get Started on RenkuLab and learned how Renku can help your data analysis be more reproducible and collaborative. The following guide will help you migrate your existing projects into Renku.

Practical Renku Usage & Migration Tips

In addition to supplementing your research with reproducibility bits, Renku aims to help you use current best practices for data analysis workflows in an unobtrusive way. Therefore, when you initialize a new Renku project in either the command-line interface or through the web platform, a template set of directories and files will be created for you. These files deal with setting up renku, docker, software dependencies, and organizational structure parts of your project. For the full specification of files and directories present in the provided templates read Templates in Renku.

Note

Although Renku provides this default template, it is fully customizable– as long as you keep the hidden .renku directory, you should be able to modify this filesystem to fit your own project structure.

In the case that you are:

  • developing on renkulab.io,

  • running a local Renku Docker container, or

  • will later import a local Renku project to renkulab,

you will want to keep some pieces from the Dockerfile and gitlab-ci.yml generated by the renku init call in order to properly build the image to spawn the JupyterLab and/or Rstudio sessions. Check out Templates in Renku for more information.

You can also create Renku projects that serve as templates that you and others can fork or use as cookiecutters.

Create an account on Renku & set up GitLab SSH key

In order to upload your project into Renku’s web platform and the associated GitLab, you will need to create an account on renkulab.io. There are several login options; pick the one most convenient for you.

When you create a Renku account, a GitLab account gets created automatically for you. On this GitLab account (https://renkulab.io/gitlab), you will need to set up an SSH key to be able to clone and push code from your local machine. To do this:

  • visit http://renkulab.io/gitlab (while logged into renkulab.io)

  • click the icon in the top righthand corner

  • click Settings from the dropdown

  • in the lefthand column, click SSH Keys

  • follow the instructions on this page

Renku-ize your project with Renku CLI

In order to Renku-ize your project, you’ll need the Renku command-line interface (CLI). You can find the quick installation instructions here.

If your project is not yet under version control (i.e. you haven’t called git init), you can do the following (replace my_project with the name of your existing project):

$ cd my_project
$ renku init

If your project is already under version control, first make sure your git status is clean, and inside the top level of your project run:

$ renku init --force

As noted in the above section on Practical Renku Usage & Migration Tips, when you renku init’d, a number of directories, files, and dotfiles were added and committed in the top level of your directory.

Now your project is ready to be sent to the web platform!

Push your project to RenkuLab

Before you can push your project to Renku you need to create a GitLab repository to store the data and code at Renku’s GitLab deployment. Renku will adopt the GitLab settings that exist in your repository when it comes to access and security. For example, making your GitLab repository public also makes the related Renku project public. In addition, granting access to specific users is also done through GitLab. When you create the repository in GitLab you can select the namespace that the project will belong to (this can be your personal namespace or a group) and the project (i.e. repository) name. The instructions below show how you can use the namespace and project name to push your code into the newly created project.

If your project does not have a remote yet (check git remote), you can use origin:

$ git remote add origin git@renkulab.io:<namespace>/<project-name>.git
$ git push origin master

If there is already a remote:

$ git remote add renku git@renkulab.io:<namespace>/<project-name>.git
$ git push renku master

View your project on renkulab.io

If everything went smoothly, you should be able to view your project at https://renkulab.io/gitlab/<namespace>/<project-name>. When you push code to this repo, the CI/CD pipeline will run according to settings in the .gitlab-ci.yml file, rebuilding your docker image as specified by the Dockerfile in the repo. If your build fails, see Session Basics.

Warning

If you already had a Dockerfile and/or .gitlab-ci.yml file, keep a close eye on the logs to make sure you’re getting expected behavior.

Your project should also now appear in Your Projects list on the front page of your logged-in Renku home page and the Projects tab from the top of the page.

Utilize renkulab.io features

At this point you can also continue development as usual by starting up a JupyterLab or RStudio session from the Renku web platform (if you don’t remember how to do this, you can check out Ways to do data science in Renku).

To make full use of the Renku reproducibility features, you will want to check back to the tutorial for how to Add data to your project, and renku run and renku rerun Build a reproducible workflow.

For Renku collaboration features, you can check out Collaborate on RenkuLab.