Exercise 6 | Authoring Workspace Chains |
Data | Voting Divisions (GML (Geography Markup Language)) Addresses (Esri Geodatabase (File Geodb API)) |
Overall Goal | Create workspaces to: - process voting divisions - to assign voting divisions to addresses - to chain the previous two translations together |
Demonstrates | Authoring workspace chains |
Start Workspace | None |
End Workspace | C:\FMEData2017\Workspaces\ServerAuthoring\Basics-Ex6-CompleteA.fmw C:\FMEData2017\Workspaces\ServerAuthoring\Basics-Ex6-CompleteB.fmw C:\FMEData2017\Workspaces\ServerAuthoring\Basics-Ex6-CompleteMaster.fmw |
You're a technical analyst in the GIS department of your local city. You have plenty of experience using FME Desktop, and your department has just purchased FME Server.
A municipal election is about to happen and Elections Interopolis have provided a dataset of new voting divisions in GML format. Your first task today is to create a workspace to translate these voting divisions to a SpatiaLite database format for use within the city, and write the data to a resources folder on FME Server so that everyone can use it.
Coincidentally, the planning department heard of this update and has asked you to assign voting division IDs to each of the records in the city's address database, for use in election planning.
You realize that you can chain these two translations together to execute consecutively under a master workspace. So in all you have three workspaces to create!
1) Start FME Workbench
Start FME Workbench and generate a translation with these parameters:
Reader Format | GML (Geography Markup Language) |
Reader Dataset | C:\FMEData2017\Data\Elections\ElectionVoting.gml |
Writer Format | SpatiaLite |
Writer Dataset |
The Writer dataset can be left empty for now. When prompted, leave both source feature types (layers) selected.
2) Create Resources
We'll handle the input and output of this workspace using the resources folders on FME Server. So, log in to the FME Server web interface and navigate to the Resources page.
If you carried out exercise 5, then you should already have a folder Resources\Data\Election\Input containing the source data used in the workspace.
If not, create that set of folders and subfolders. Upload the source GML dataset to the Input folder (you should upload both the .gml and .xsd files):
3) Edit Workspace to use Resources
Back in FME Workbench look in the Navigator window under User Parameters for the two existing published parameters called SourceDataset_GML and DestDataset_SPATIALITE. Click on each in turn and press the delete key to delete them.
Next locate the parameters for the GML source dataset and SpatiaLite destination dataset. Update the parameters to read as follows:
GML Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Input\ElectionVoting.gml |
SpatiaLite Writer | $(FME_SHAREDRESOURCE_DATA)\Election\Output\VotingDivisions.sl3 |
One final tweak: change the Writer parameter Overwrite Existing Database to Yes
4) Save, Publish, and Run Workspace
Save the workspace (to something like Basics-Ex6-CompleteA.fmw) and remember the filename: it will be important later. Publish the workspace to FME Server. It should be registered with the Job Submitter service.
Locate the workspace in the Server web interface and run it to make sure it runs to completion. The evidence of success will be the log and an sl3 file in the resources folder.
Select the sl3 dataset and click the button to download the file. This is important; we'll need the file to set up our next workspace.
Save the file to the Elections folder, so you will remember where it is; i.e. C:\FMEData2017\Data\Elections\VotingDivisions.sl3
5) Generate Workspace
That was the first workspace in our project. Now for the second.
Open Workbench if necessary and generate a new workspace with these parameters:
Reader Format | Esri Geodatabase (File Geodb Open API) |
Reader Dataset | C:\FMEData2017\Data\Addresses\Addresses.gdb |
Writer Format | Esri Geodatabase (File Geodb Open API) |
Writer Dataset | C:\FMEData2017\Output\Training\NewAddresses.gdb |
When prompted, leave both source feature types (tables) selected.
6) Add Reader
To assign voting divisions we need to have that data in our workspace. So, select Readers > Reader from the menubar and add a reader to read the downloaded VotingDivisions SpatiaLite database:
Reader Format | SpatiaLite |
Reader Dataset | C:\FMEData2017\Data\Elections\VotingDivisions.sl3 |
NB: If you can't find that sl3 file, go back to step 4 and make sure you downloaded the result of the first workspace.
When prompted, select only the source feature type (table) votingdivisions.
7) Add Transformer
Now let's add a transformer to assign voting divisions to each address. Place a PointOnAreaOverlayer transformer into the workspace. Connect it as follows:
- Delete Connection: Geodatabase:PostalAddress > Geodatabase:PostalAddress
- Add Connection: Geodatabase:PostalAddress > PointOnAreaOverlayer:Point
- Add Connection: SpatiaLite:votingdivisions > PointOnAreaOverlayer:Area
- Add Connection: PointOnAreaOverlayer:Point > Geodatabase:PostalAddress
8) Edit Writer Schema
That transformer will copy the division attribute on to each address, but that attribute won't be written unless we also add it to the output schema.
So, inspect the parameters for the writer feature type PostalAddress. In the User Attributes tab add a new attribute called division (of type int):
division is case-sensitive, since we want it to match what is coming in from the votingdivisions table.
9) Test Run Workspace
Before we start adjusting the dataset paths for use on FME Server, run the workspace to ensure it produces the correct output; i.e. that each address now has a division attribute.
10) Create Resources
We'll also handle the input and output of this workspace using the resources folders on FME Server.
Firstly, we can upload a File Geodatabase as a folder/file only if we're using the Chrome web browser. Just in case you aren't, locate the source Geodatabase in your file system and compress it into a single zip file:
Next, upload the zipped address file to the Resources folder on FME Server:
11) Edit Workspace to use Resources
Back in FME Workbench look in the Navigator window under User Parameters for the three existing published parameters called SourceDataset_FILEGDB, SourceDataset_SPATIALITE, and DestDataset_SPATIALITE. Click on each in turn and press the delete key to delete them.
Next locate the parameters for the Geodatabase source dataset, SpatiaLite source dataset, and Geodatabase destination dataset. Update the parameters to read as follows:
Geodatabase Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Input\Addresses.gdb.zip |
SpatiaLite Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Output\VotingDivisions.sl3 |
Geodatabase Writer | $(FME_SHAREDRESOURCE_DATA)\Election\Output\NewAddresses.gdb.zip |
One final tweak: change the Writer parameter Overwrite Geodatabase to Yes
12) Save, Publish, and Run Workspace
Save the workspace (to something like Basics-Ex6-CompleteB.fmw) and remember the filename: it will be important later. Publish the workspace to FME Server. It should be registered with the Job Submitter service.
Locate the workspace in the Server web interface and run it to make sure it runs to completion. The evidence of success will be the log and a zipped geodatabase file in the resources folder.
You may wish to download the newly created dataset to inspect it and make sure the output is correct.
13) Create Workspace
That was the second workspace in our project. Now for the third and final workspace. This workspace will be used to chain the previous two workspaces. It is going to be the master, with the two prior workspaces as children.
So, open Workbench and start with an empty canvas. Place a Creator transformer followed by two FMEServerJobSubmitter transformers:
14) Set Parameters
Inspect the parameters for the first of the FMEServerJobSubmitter transformers.
Firstly select your FME Server connection. Then select the Training repository and the first of the two prior workspaces (the one that converted election divisions from GML to SpatiaLite).
Finally set Wait for Server Job to Complete to Yes. If we didn't do this then the second job submitter transformer would run before the first had finished!
Below is an area where we can set the parameters for the translation. However, since there are no published parameters, we don't need to worry about that.
Click OK to close the dialog.
Now repeat the same process for the second FMEServerJobSubmitter, this time selecting the second workspace (the one that did the overlay of addresses on divisions).
15) Save, Publish, and Run Workspace
Save the workspace and publish it to FME Server. It should be registered with the Job Submitter service.
Locate the workspace in the Server web interface and run it to make sure it runs to completion. It will run each of the two child workspaces in turn.
Don't worry that FME reports zero features written. That only refers to the master workspace (not the child workspaces). Evidence of success will be the log, and new output files (sl3,gdb) in the resources folder:
Notice that the date/timestamps will be very similar for the two datasets; the VotingDivisions.sl3 file should be created first and then NewAddresses.gdb.zip shortly after.
Sister Intuitive says... |
As already mentioned there are many ways to set up chains and this is just one of them. The drawback of writing data to a fixed location (like here) is that someone might change the first workspace to write data to a different location, causing the second workspace to fail. |
CONGRATULATIONS |
By completing this exercise you have learned how to:
|