Configuration and Setup Information
Configuration and setup messages in the FME log tell us vital information about FME's version and configuration, the system resources and how FME intends to use them, and what system paths are being used.
For example, here you can see which version of FME is being used (FME2017, 32-bit), its license type (floating), and the machine name (Training2017). If you do have multiple FME versions installed, here’s where you can check to ensure the correct one is running.
Further on we can see that the system has over 176GB of free space and 4 GB of virtual memory (the maximum possible on 32-bit). We can also see what operating system we are running FME on (Windows Server 10) and what the current language and encoding settings are:
Later in the log is important information about the system resources and FME’s memory management:
In this case there is a limit of 4GB memory per process, indicative of 32-bit processing, but then the machine only has 4GB of total memory anyway. The following numbers indicate how FME will manage memory resources. It will use 2.4GB of memory and then it will start to release memory by caching features to disk. This caching will stop once memory use is less than about 1.8GB.
This way FME will perform to its potential automatically, while not taking so much memory that the system may fail or other processes on the system would suffer.
64-bit FME gives a slightly different set of messages, like so:
| NEW |
|
The reporting of these numbers has been updated slightly in FME2017.
With 32-bit FME the numbers reported include "address space". Address space is a theoretical limit of the amount of memory FME could use, regardless of available physical memory and disk space. In 32-bit it cannot go higher than 4GB, so it's a realistic number to use. With 64-bit FME, address space is an impractically high number (128TB in Windows 8.1 or later) that has no practical value (other failures would occur long before that number is breached). So instead of address space, on 64-bit FME the log reports "Virtual Memory", a more realistic number that combines physical memory with a portion of disk called Swap Space. |
Temporary Folder
For performance tuning one of the most important parts of the log reports the temporary folder being used. When physical memory resources become low FME starts to cache data to disk and swap that data in and out of physical memory as required.
The temporary folder is where it writes data to, so there are two important considerations for that folder.
Firstly it's important to ensure this folder does have enough temporary disk space for the translation being carried out. Depending on the workspace, the transformations being carried out, and other processes happening on the computer; temporary disk space requirements may be many times greater than the size of the original dataset.
Secondly it's useful if the disk being written to is both fast and unused by any other process. It will not, for example, help performance to share the temporary disk with the operating system; additionally, a solid-state disk is much faster than a traditional hard drive.
| Jake Speedie says… |
|
The comparative benefits of RAM vs SSD vs HDD are hard to quantify. Do a web search if you don't believe me; people quote RAM as being 4x as fast, 20x as fast, 100,000x as fast!
In reality it depends on so many factors. But in general, use as much memory as you can to avoid caching, and use SSD so that - where caching/swapping is unavoidable - it runs as fast as possible. |