Renku Workflow

Renku uses PROV-O and its own Renku ontology to represent workflows.

plan

Represent run templates.

class renku.domain_model.workflow.plan.AbstractPlan(*args, **kwargs)

Bases: renku.infrastructure.persistent.Persistent, abc.ABC

Abstract base class for all plans.

assign_new_id()

Assign a new UUID.

This is required only when there is another plan which is exactly the same except the parameters’ list.

delete(*, when=datetime.datetime(2022, 11, 10, 14, 10, 1, tzinfo=datetime.timezone(datetime.timedelta(0), 'UTC')))

Mark a plan as deleted.

NOTE: Don’t call this function for deleting plans since it doesn’t delete the whole plan derivatives chain. Use renku.core.workflow.plan::remove_plan instead.

property deleted

True if plan is deleted.

derive()

Create a new AbstractPlan that is derived from self.

find_parameter(parameter)

Find if a parameter exists on this plan.

find_parameter_workflow(parameter)

Return the workflow a parameter belongs to.

static generate_id(uuid=None)

Generate an identifier for Plan.

is_derivation()

Return if an AbstractPlan has correct derived_from.

resolve_direct_reference(reference)

Resolve a direct parameter reference.

resolve_mapping_path(mapping_path)

Resolve a mapping path to its reference parameter.

static validate_name(name)

Check a name for invalid characters.

class renku.domain_model.workflow.plan.Plan(*args, **kwargs)

Bases: renku.domain_model.workflow.plan.AbstractPlan

Represent a renku run execution template.

assign_new_id()

Assign a new UUID.

This is required only when there is another plan which is exactly the same except the parameters’ list.

copy()

Create a copy of this plan.

Required where a plan is used several times in a workflow but we need to set different values on them.

derive()

Create a new Plan that is derived from self.

find_parameter(parameter)

Find if a parameter exists on this plan.

find_parameter_workflow(parameter)

Return the workflow a parameter belongs to.

get_field_by_id(id)

Return an in Input/Output/Parameter by its id.

get_parameter_by_id(parameter_id)

Get a parameter on this plan by id.

get_parameter_path(parameter)

Get the path to a parameter inside this plan.

is_derivation()

Return if an Plan has correct derived_from.

is_similar_to(other)

Return true if plan has the same inputs/outputs/arguments as another plan.

property keywords_csv

Comma-separated list of keywords associated with workflow.

resolve_direct_reference(reference)

Resolve a direct parameter reference.

resolve_mapping_path(mapping_path)

Resolve a mapping path to its reference parameter.

set_parameters_from_strings(params_strings)

Set parameters by parsing parameters strings.

to_argv(with_streams=False)

Convert a Plan into argv list.

class renku.domain_model.workflow.plan.PlanDetailsJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: marshmallow.schema.Schema

Serialize a plan to a response object.

composite_plan

Represent a group of run templates.

class renku.domain_model.workflow.composite_plan.CompositePlan(*args, **kwargs)

Bases: renku.domain_model.workflow.plan.AbstractPlan

A plan containing child plans.

add_link(source, sinks)

Validate and add a ParameterLink.

add_mapping(mapping)

Add a mapping to this run.

derive()

Create a new CompositePlan that is derived from self.

find_link_by_target(target)

Find a link on this or a child workflow that has target as a sink.

find_parameter(parameter)

Check if a parameter exists on this run or one of its children.

find_parameter_workflow(parameter)

Return the workflow a parameter belongs to.

get_parameter_by_id(parameter_id)

Get a parameter on this plan by id.

get_parameter_path(parameter)

Get the path to a parameter inside this plan.

is_derivation()

Return if an CompositePlan has correct derived_from.

map_all_inputs()

Map all unmapped inputs from child steps to the parent.

map_all_outputs()

Map all unmapped outputs from child steps to the parent.

map_all_parameters()

Map all unmapped parameters from child steps to the parent.

resolve_direct_reference(reference)

Resolve a direct parameter reference.

resolve_mapping_path(mapping_path)

Resolve a mapping path to its reference parameter.

set_links_from_strings(link_strings)

Set links between parameters of child steps.

set_mapping_defaults(default_strings)

Set default value based on a default specification string.

set_mapping_descriptions(mapping_descriptions)

Set descriptions for mappings.

set_mappings_from_strings(mapping_strings)

Set mappings by parsing mapping strings.

parameter

Classes to represent inputs/outputs/parameters in a Plan.

class renku.domain_model.workflow.parameter.CommandInput(*, default_value=None, description=None, id, mapped_to=None, name=None, position=None, prefix=None, encoding_format=None, derived_from=None, postfix=None)

Bases: renku.domain_model.workflow.parameter.CommandParameterBase

An input to a command.

derive(plan_id)

Create a new CommandInput that is derived from self.

static generate_id(plan_id, position=None, postfix=None)

Generate an id for CommandInput.

to_stream_representation()

Input stream representation.

class renku.domain_model.workflow.parameter.CommandOutput(*, create_folder=False, default_value=None, description=None, id, mapped_to=None, name=None, position=None, prefix=None, encoding_format=None, derived_from=None, postfix=None)

Bases: renku.domain_model.workflow.parameter.CommandParameterBase

An output from a command.

derive(plan_id)

Create a new CommandOutput that is derived from self.

static generate_id(plan_id, position=None, postfix=None)

Generate an id for CommandOutput.

to_stream_representation()

Input stream representation.

class renku.domain_model.workflow.parameter.CommandParameter(*, default_value=None, description=None, id, name=None, position=None, prefix=None, derived_from=None, postfix=None)

Bases: renku.domain_model.workflow.parameter.CommandParameterBase

An argument to a command that is neither input nor output.

derive(plan_id)

Create a new CommandParameter that is derived from self.

static generate_id(plan_id, position=None, postfix=None)

Generate an id for CommandParameter.

class renku.domain_model.workflow.parameter.CommandParameterBase(*, default_value, description, id, name, position=None, prefix=None, derived_from=None, postfix=None)

Bases: object

Represents a parameter for a Plan.

property actual_value

Get the actual value to be used for execution.

property actual_value_set

Whether the actual_value on this parameter has been set at least once.

abstract derive(plan_id)

Create a new command parameter from self.

property role

Return a unique role for this parameter within its Plan.

to_argv()

String representation (sames as cmd argument).

class renku.domain_model.workflow.parameter.MappedIOStream(*, id=None, stream_type)

Bases: object

Represents an IO stream (stdin, stdout, stderr).

static generate_id(stream_type)

Generate an id for parameters.

class renku.domain_model.workflow.parameter.ParameterLink(source, sinks, id)

Bases: object

A link between a source and one or more sink parameters.

apply()

Apply source value to sinks.

static generate_id(plan_id)

Generate an id for parameters.

class renku.domain_model.workflow.parameter.ParameterMapping(*, default_value, description=None, id, name=None, mapped_parameters, **kwargs)

Bases: renku.domain_model.workflow.parameter.CommandParameterBase

A mapping of child parameter(s) to a parent CompositePlan.

property actual_value

Get the actual value to be used for execution.

derive(plan_id)

Create a new CommandParameter that is derived from self.

static generate_id(plan_id, position=None, postfix=None)

Generate an id for CommandOutput.

property leaf_parameters

Return leaf (non-Mapping) parameters contained by this Mapping.

to_stream_representation()

Input stream representation.

Renku Workflow Conversion

Renku allows conversion of tracked workflows to runnable workflows in supported tools (Currently CWL)

CWL

Converter for workflows to CWL.

class renku.core.workflow.converters.cwl.CWLExporter

Converts a CompositePlan or a Plan to CWL format.

workflow_convert(workflow, basedir, output, output_format)

Converts the specified workflow to CWL format.

workflow_format()

Workflow format name.