Translation and Transformation Statements

The main body of the FME log concerns the translation and transformation of data. This section often appears confusing to new users, until they understand how FME operates.

Firstly, there can be several parallel streams of processing in a workspace. If you are not specifically setting the order of these streams, the order of the log will be similarly indecisive.

Secondly, there is the issue of transformer type.

As you might know, some transformers operate on a feature-by-feature basis, while others work on a group of features at a time.

When a series of transformers are all feature-based, FME works by pushing features through on an individual basis, not as a group. For example, the first feature is read and then processed by each transformer in turn. Then the next feature is read and processed by each transformer, and so on.

This makes for a log file that is harder to understand, because you don’t see one specific entry for each Reader or transformer in turn. Nor do you see a message for each feature. Instead, a cumulative time is calculated and output at regular time-based intervals.

Key Messages

There are several key messages that may appear in a log that signify specific events in a translation.

Emptying factory pipeline

This message signifies that reading of data is at an end. Because of how FME processes data, this message may appear near the beginning of a translation or near the end. But after this point the source datasets are closed and only transformation and/or writing will take place.

By checking the timestamp for this message you can get an approximate value for how long the workspace took to read the source data.

Unexpected Input Remover

This message denotes where features read by a Reader are tested against the available feature types (and merge filters) to see if they will be allowed to pass into the workspace.

This message may indicate a performance issue as it highlights potentially unnecessary work, where features are being read but not used in the translation.

ResourceManager: Optimizing Memory Usage. Please wait...

This message alerts you to the fact that the memory management limits mentioned in the configuration part of the log have been reached, and that FME is having to work around the issue, possibly by caching data to disk.

If you see this message frequently, then it’s time to either redesign your translation or switch to a 64-bit setup. It is a key indicator that a lack of memory is affecting performance.