Assessing Writer Performance

As with Readers, you can't improve the performance of a Writer unless you can first assess how well it is already performing. But assessing the speed of writing has the same complexity as reading: FME starts writing data as soon as it becomes available, and doesn’t necessarily wait until processing is done.

False Writer Performance

Take the previous workspace, which read a set of address points and found their nearest neighbor, but now has a Writer too:

According to the log file, we find that the output dataset is open for writing before the source dataset has even finished being read!

Opened Shape File C:\FMEData2015\Output\PostalAddress.shp for output
Opened DBF File 'C:\FMEData2015\Output\PostalAddress.dbf' for output
DBF Writer: Writing DBF File using character encoding 'ANSI'
Reading source feature # 5000
Reading source feature # 7500
Reading source feature # 10000
Reading source feature # 12500

If you examined this log you would get quite a false representation of how well the Writer is performing.

True Writer Performance

So, how can we assess Writer performance? Plainly this is different to Readers. If you isolate the Writers by disabling everything else, there won't be any data to write!

However, it can be done with a two-step process. Firstly you would add a Recorder transformer – after all other transformation has taken place – to preserve the data in FFS format at the moment it is about to be written:

Now replace the Recorder with a Player transformer – to re-read the preserved FFS data – and disable everything else up to that point:

Now the data will be played back into the workspace, and is followed up by being written to the output.

In this example the "Emptying Factory Pipeline" message appeared after 7.2 seconds and the translation took 8.9 seconds in total, indicating 1.7 seconds was spent writing data.