Pipeline tokens. More...
#include <tpie/exception.h>#include <tpie/pipelining/exception.h>#include <tpie/pipelining/predeclare.h>#include <tpie/pipelining/container.h>#include <tpie/types.h>#include <tpie/tpie_assert.h>#include <map>#include <vector>#include <iostream>#include <boost/intrusive_ptr.hpp>#include <boost/optional.hpp>#include <unordered_map>#include <unordered_set>Go to the source code of this file.
Classes | |
| class | tpie::pipelining::bits::node_map |
| struct | tpie::pipelining::bits::node_map::pipe_base_forward_t |
| class | tpie::pipelining::node_token |
Namespaces | |
| tpie | |
| pipelining/factory_base.h Base class of pipelining factories | |
| tpie::pipelining | |
| TPIE pipelining framework. | |
Enumerations | |
| enum | node_relation { pushes, pulls, depends, no_forward_depends, memory_share_depends } |
Pipeline tokens.
A pipeline consists of several nodes. Each node either produces, transforms or consumes items. One node may push items to another node, and it may pull items from another node, and it may depend implicitly on the execution of another node. For instance, to reverse an item stream using two nodes, one node will write items to a file_stream, and the other will read them in backwards. Thus, the second node depends on the first, but it does not directly push to or pull from it.
To a pipeline graph we associate two different directed edge sets.
The item flow graph is a directed acyclic graph, and edges go from producer towards consumer, regardless of push/pull kind.
The actor graph is a directed graph where edges go from actors, so a node has an edge to another node if the corresponding node either pushes to or pulls from the other corresponding node.
The item flow graph is useful for transitive dependency resolution and execution order decision. The actor graph is useful for presenting the pipeline flow to the user graphically.
Since nodes are copyable, we cannot store node pointers limitlessly, as pointers will change while the pipeline is being constructed. Instead, we associate to each node a token (numeric id) that is copied with the node. The node_token class signals the mapping from numeric ids to node pointers to a node_map.
However, we do not want a global map from ids to node pointers, as an application may construct many pipelines throughout its lifetime. To mitigate this problem, each node_map is local to a pipeline, and each node_token knows (directly or indirectly) which node_map currently holds the mapping of its id to its node.
When we need to connect one node to another in the pipeline graphs, we need the two corresponding node_tokens to share the same node_map. When we merge two node_maps, the mappings in one are copied to the other, and one node_map remembers that it has been usurped by another node_map. This corresponds to the set representative in a union-find data structure, and we implement union-find merge by rank. We use Boost smart pointers to deallocate node_maps when they are no longer needed.
Definition in file tokens.h.