Translation and Transformation Statements
The main body of the FME log concerns the translation and transformation of data. This section often appears confusing to new users, until they understand how FME operates.
Firstly, there can be several parallel streams of processing in a workspace. If you are not specifically setting the order of these streams, the order of the log will be similarly indecisive.
Secondly, there is the issue of transformer type.
As you might know, some transformers operate on a feature-by-feature basis, while others work on a group of features at a time.
When a series of transformers are all feature-based, FME works by pushing features through on an individual basis, not as a group. For example, the first feature is read and then processed by each transformer in turn. Then the next feature is read and processed by each transformer, and so on.
This makes for a log file that is harder to understand, because you don’t see one specific entry for each reader or transformer in turn. Nor do you see a message for each feature. Instead, a cumulative time is calculated and output at regular time-based intervals.
| NEW |
|
2017 introduced updated formats and transformers that work with new technology that we call Feature Tables.
Feature Tables are for vector or tabular data, but resemble raster or point cloud data in how they operate. Instead of reading and processing one feature at a time, as before, data is handled more in "chunks". In general, things should "just work", as they did before, and performance will improve by itself , but it might make interpreting the log window a little bit more difficult than before. So far, only one or two formats use this technology (CSV, MapInfo Extended); but expect to notice more of an impact as we use it throughout the product. |
Key Messages
There are several key messages that may appear in a log that signify specific events in a translation.
Emptying factory pipeline
This message signifies that reading of data is at an end. Because of how FME processes data, this message may appear near the beginning of a translation or near the end. But after this point the source datasets are closed and only transformation and/or writing will take place.
By checking the timestamp for this message you can get an approximate value for how long the workspace took to read the source data.
Unexpected Input Remover
This message denotes where features read by a reader are tested against the available feature types (and merge filters) to see if they will be allowed to pass into the workspace.
This message may indicate a performance issue as it highlights potentially unnecessary work, where features are being read but not used in the translation.
ResourceManager: Optimizing Memory Usage. Please wait...
This message alerts you to the fact that some reorganization of memory usage is going on. For example, maybe the memory management limits mentioned in the configuration part of the log have been reached, and FME is having to work around the issue.
If you see this message frequently, then it’s time to either redesign your translation or switch to a 64-bit setup. It is a key indicator that a lack of memory is affecting performance.
| Miss Vector says… |
|
How would you find the Emptying Factory Pipeline setting in the log window?
1. Looking for the color: it is the only message highlighted in red 2. Looking for the message type: it is the first message of type STATS 3. By using the log search tools 4. By opening Tools > FME Options to turn off all other messages |