Advanced workflows / sub-workflows
==================================

Advanced workflows can be created by combining multiple limited-purpose
workflows.

Sub-workflows are integrated in the general graph of their parent workflow
as WorkRequests of type ``workflow``.

From a user interface perspective, sub-workflows are typically hidden as a
single step in the visual representation of the parent's workflow.

Cooperation between workflows is defined at the level of workflows.
Individual work requests should not concern themselves with this; they
are designed to take inputs using lookups and produce output artifacts
that are linked to the work request.

Sub-workflow coordination takes place through the workflow's internal
collection (which is shared among all sub-workflows of the same root
workflow), providing a mechanism for some work requests to declare that they
will provide certain kinds of artifacts which may then be required by work
requests in other sub-workflows.

On the providing side, workflows use the
:ref:`action-update-collection-with-artifacts` event reaction to add
relevant output artifacts from work requests to the internal collection, and
create :ref:`promises <bare-data-promise>` to indicate to other workflows
that they have done so.  Providing workflows choose item names in the
internal collection; it is the responsibility of workflow designers to
ensure that they do not clash, and workflows that provide output artifacts
have a optional ``prefix`` field in their task data to allow multiple
instances of the same workflow to cooperate under the same root workflow.

On the requiring side, workflows look up the names of artifacts they require
in the internal collection; each of those lookups may return nothing, or a
promise including a work request ID, or an artifact that already exists, and
they may use that to determine which child work requests they create.  They
use :ref:`lookups <lookup-syntax>` in their child work requests to refer to
items in the internal collection (e.g.
``internal@collections/name:build-amd64``), and add corresponding
dependencies on work requests that promise to provide those items.

Sub-workflows may depend on other steps within the root workflow while still
being fully populated in advance of being able to run.  A workflow that
needs more information before being able to populate child work requests
should normally depend on the work requests that will provide the
information it needs; failing that, it should use :ref:`workflow callbacks
<workflow-callback>` to run the workflow orchestrator again when it is
ready.  (For example, a workflow that creates a source package and then
builds it may not know which work requests it needs to create until it has
created the source package and can look at its ``Architecture`` field.)

Workflows should not normally have dependencies, since that means that their
orchestrators cannot run and populate the work request graph in advance.
The exception is where the workflow orchestrator itself needs some
information from some of its input artifacts in order to work out which
child work requests to create; in such cases the workflow itself should have
dependencies that mean the orchestrator does not run until that information
is available.  Otherwise, it is better for workflows to create child work
requests that have whatever dependencies are needed.
