Improving Writer Performance
There are various ways to speed up writing data. Compared to reading, tuning the underlying systems is a more important improvement, whereas the number of features is less important as it's much harder to write extra data unintentionally.
File System Improvements
If you are writing to a file system then make sure the disk is fast and responsive – use solid state drives – and that the operating system is not busy writing other files at the same time, as the latter could cause a significant bottleneck.
Also check if you are using RAID as some configurations need multiple writes and can definitely slow things down.
Database Improvements
If you are writing to a database, then existing indexes and joins can cause the process to be slower than expected. In many cases it will be quicker to drop an index, write the data, and then recreate the index.
More information on database performance with FME comes in a later section.
Multiple Writers
Perhaps the most important technique for improving Writer performance involves the scenario where a workspace has multiple Writers.
In short, it’s important to get the writers into the optimum order, to ensure that the Writer that is to receive the largest amount of data is written first.
The reasoning behind this is that the first Writer in a workspace starts to write data as soon as it is received. Other writers cache theirs until they are ready to start writing.
Therefore, if the largest amount of data is written immediately, lesser amounts of data have to be written to, and stored in, a cache.
This can improve performance tremendously, particularly when the translation is especially unbalanced; for example one million features go to one Writer, and only ten features go to another.
| Jake Speedie says... |
| Think of it like an airport. It’s more efficient when you load the busiest flights first, because it empties the terminal waiting areas quicker. For more information see this FME Evangelist article. |
Setting Writer Order
There are two ways to affect the order that writing occurs in.
Firstly each Writer is listed in the Navigator window in Workbench and can be re-ordered by moving them up and down in the list in the Navigator window:
The first Writer in the list is the one that is executed first, therefore it should be the one to receive the most data.
The second method is to use a workspace parameter called Order Writers By:
This parameter can be left to Position in Workbench Navigator in which case the order of Writers as defined in the Navigator takes priority. Alternatively it can be set to First Feature Written. In that case the Writer that receives the first feature will be the first to start writing data.
| Miss Vector says… |
Given this screenshot, which should we make the first Writer in this workspace?
1. A 2. B 3. C 4. Don't know! |