Transformers in Parallel

A stream is a flow of data represented by connections in the workspace. A key concept in FME is the ability to have multiple parallel streams within a workspace.

Multiple Streams

Multiple streams are useful when a user needs to process the same data but in many different ways. A workspace author can turn one stream into several, or combine several streams of data into one, as required:

Here an author is creating three data streams, each of which is processed separately then combined back into a single stream.

Creating Multiple Streams

Creating multiple data streams can occur in a number of ways. Sometimes a transformer with multiple output ports (a Tester transformer is a good illustration of this) will divide (or filter) data with several possible output streams:

Here data is divided into two streams, one of which is not connected to anything.

Additionally, a full stream of data can be duplicated by simply making multiple connections out of a single output port. In effect it creates a set of data for each connection:

Here FME reads 8 features but, because there are multiple connections, creates multiple copies of the data.

Bringing Together Multiple Streams

When multiple streams are connected to the same input port no merging takes place. The data is simply accumulated into a single stream. This is often called a union.

Here, three streams of data converge into a writer feature type:

No merging takes place; the data simply accumulates into 12 distinct features in the output dataset. Think of it as three pipes of water emptying into a single contained.

To carry out actual merging of data requires a specific transformer such as the FeatureMerger or FeatureJoiner.