Flow45 concepts

Workflow

A workflow is a template, sometimes we say a workflow template. It is a set of tasks, with dependencies amongst themselves. It can be seen as a directed graph, where the tasks are the vertices. If task B depends on task A, then we have an edge from A to B. In a worklfow graph, cycles are not allowed. We have directed acyclic graphs, DAGs. A workflow can also be seen as function composition, where each task is a function that takes the results of other functions as input arguments and produces a result.

Workflow Instantiation

In many business process management systems, orchestration tools, and workflow managers, workflows can be instantiated. A workflow instance has task instances. If you think about workflows in terms of function composition, then instantiating a workflow is like calling the composite function. Workflows can be instantiated based on a schedule, or, they can be instantiated (called, triggered) by a process making a function call. Flow45 is a bit different. In Flow45, you don't instantiate workflows as a whole. Rather, you instantiate tasks.

Task instantiation

Tasks can be instantiated
  1. based on a schedule of their own,
  2. based on the schedule of a single upstream task, or
  3. based on the unique id of an input argument of a function call.
In the latter two cases, if you repeat the same patteren, then it feels the same as intantiating a worklfow. You trigger the root of the DAG. Or, you call the innermost function of one big composite function. But, there can be multiple tasks within the same workflow that are on a schedule; the same schedule, or even a different one. A workflow in Flow45 is a set of tasks, with optional dependencies between them. Tasks that do have dependencies between them must be part of the same workflow. A task that is on a schedule must have a start date.

Workflow evolution

Tasks are allowed to evolve a bit. And so are workflows. Tasks can be moved from one workflow to another. Team A team in Flow45 is associated with
  1. A team service account, used to authenticate and authorise Flow45 API calls
  2. Functions that the team manage
  3. Data types managed by the team

Data types

Data types are struct types. Data types are used in function signatures. They define the format of a function results, or of a function its input parameter. Data types can be evolved in a backward compatible manner. When developers change a data type, upon deployment, it is checked that the change is backward compatible, and that the developers have bumped the version by one. If task instances are undone, and then bumped (cleared), historical output results of upstream task instances are again provided as input to the function picking up the cleared task instance. If the data types of those input parameters have been evolved in a backwards compatible manner, then the function will still handle those historical values correctly. If data types are evolved in a non-backwards compatible manner, they have to be given a different name, and the old definition should not be removed.

Functions

Functions are just a name and a signature. Functions are owned by a single team. The team also owns the process that will look for task instance with this function name. Data types of a function its parameters and result can be owned by the same team as well. If a team wants to evolve a function, it may add optional fields to the output data type (evolve the result data type in a backwards compatible way). In this case, they should first evolve the output data type. Any other functions, possibly owned by other teams, should then bind the input parameters of their functions to the new version of the data type explicitly, signaling their function will behave correctly with the new function. It will still also handle the older versions correctly, due to the backward compatibility agreement. Now, the upstream team can bind the result of their function to the new data type. Even now, their process can still produce the old data type. Finally, they change their process, emitting the new data type from now on for new task instances.
Created
Sat 24 Feb 2024 20:16:21 CET
Updated
Sat 24 Feb 2024 20:16:21 CET