Plugin Support

Renku has several plugin hooks that can be used to add additional metadata and commands to the Renku CLI.

The following hooks are currently available:

Runtime Plugins

Runtime plugins are supported using the pluggy library.

Runtime plugins can be created as Python packages that contain the respective entry point definition in their setup.py file, like so:

from setuptools import setup

setup(
    ...
    entry_points={"renku": ["name_of_plugin = myproject.pluginmodule"]},
    ...
)

where myproject.pluginmodule points to a Renku hookimpl e.g.:

from renku.core.plugin import hookimpl

@hookimpl
def plugin_hook_implementation(param1, param2):
    ...

renku run hooks

Plugin hooks for renku run customization.

renku.core.plugin.run.activity_annotations(activity)[source]

Plugin Hook to add Annotation entry list to a Activity.

Run when creating an activity from a renku run.

Parameters

activity – An Activity object to get annotations for.

Returns

A list of renku.domain_model.provenance.annotation.Annotation objects.

renku.core.plugin.run.plan_annotations(plan)[source]

Plugin Hook to add Annotation entry list to a Plan.

Run when a Plan is created by renku run.

Parameters

plan – A Plan object to get annotations for.

Returns

A list of renku.domain_model.provenance.annotation.Annotation objects.

renku.core.plugin.run.pre_run(tool)[source]

Plugin Hook that gets called at the start of a renku run call.

Can be used to setup plugins that get executed during the run.

Parameters

tool (PlanFactory) – The plan factory that captured the run.

This repository contains an implementation of an activity annotation plugin.

CLI Plugins

Command-line interface plugins are supported using the click-plugins <https://github.com/click-contrib/click-plugins> library.

As in case the runtime plugins, command-line plugins can be created as Python packages that contain the respective entry point definition in their setup.py file, like so:

from setuptools import setup

setup(
    ...
    entry_points={"renku.cli_plugins": ["mycmd = myproject.pluginmodule:mycmd"]},
    ...
)

where myproject.pluginmodule:mycmd points to a click command e.g.:

import click

@click.command()
def mycmd():
    ...

An example implementation of such plugin is available here.

Workflow Converter Plugins

Additional workflow converters can be implemented by extending renku.domain_model.workflow.converters.IWorkflowConverter. By default renku provides a CWL converter plugins that is used when exporting a workflow:

$ renku workflow export --format cwl <my_workflow>
class renku.domain_model.workflow.converters.IWorkflowConverter[source]

Abstract class for converting Plan to a workflow format.

abstract workflow_convert(workflow, basedir, output, output_format)[source]

Converts a single workflow step to a desired workflow format.

abstract workflow_format()[source]

Supported workflow description formats.

We created a dummy implementation of such a converter plugin.

Workflow Provider Plugins

Additional workflow providers can be implemented by extending renku.domain_model.workflow.provider.IWorkflowProvider. See Implementing a workflow provider for more information.

class renku.domain_model.workflow.provider.IWorkflowProvider[source]

Abstract class for executing Plan.

abstract workflow_execute(dag, basedir, config)[source]

Executes a given AbstractPlan using the provider.

Returns

A list of output paths that were generated by this workflow.

abstract workflow_provider()[source]

Supported workflow description formats.

Returns

A tuple of the provider itself and the workflow executor backends name.

Return type

Tuple[IWorkflowProvider,str]