Exercise 5 | Daily Database Updates: Using Resources |
Data | Neighborhoods (KML) Election Voting (GML) |
Overall Goal | Create a workspace to read and process departmental data and publish it to FME Server |
Demonstrates | Uploading data to a resources folder and authoring a workspace to make use of it |
Start Workspace | C:\FMEData2017\Workspaces\ServerAuthoring\Basics-Ex5-Begin.fmw |
End Workspace | C:\FMEData2017\Workspaces\ServerAuthoring\Basics-Ex5-Complete.fmw |
For the exercises in this chapter, you are a technical analyst in the GIS department of your local city.
You have already (Exercise 4) created a workspace to carry out a translation, and published it to FME Server; both with data and using data uploaded temporarily.
However, such data management tools are not particularly suited to a long term project, so the task here is to upgrade the workspaces to use datasets stored in a Resources folder. There we can store source data and write destination data.
1) Open FME Server Web Interface
Log in to the FME Server web interface using an administrator account (such as admin/admin). Click Resources on the menubar to navigate to the resources management pages.
2) Create Folder
In most cases data should be stored under the Data folder, so click on Data in the Resources dialog to open that folder. To avoid mixing datasets our data should go into its own subfolder. So click on the New Folder button and create a folder called Election:
Next click on the Election folder and within there create subfolders called Input and Output.
3) Upload Source Datasets
Browse to the Input folder and click the upload button. Upload the source datasets for the current translation:
Reader Datasets | C:\FMEData2017\Data\Elections\ElectionVoting.gml C:\FMEData2017\Data\Elections\ElectionVoting.xsd C:\FMEData2017\Data\Boundaries\VancouverNeighborhoods.kml |
So we now have both source datasets and a folder to write the output data to.
4) Add Writer
Up until now all of our workspaces have had only a NULL (dummy) writer. Now we know about Resources we can add a proper writer and point its output to the Resources Output folder.
So, open the workspace listed above in FME Workbench and then select Writers > Add Writer on the menubar and set up a new writer with the following parameters:
Writer Format | Esri Geodatabase (File Geodb Open API) |
Writer Dataset | C:\FMEData2017\Output\Training\DepartmentData.gdb |
Feature Class or Table Definition | None (Advanced) |
.1 UPDATE |
Note that Esri Geodatabase format was renamed from *File Geodb API* to *File Geodb **Open** API* in FME 2017.1! |
The reason we want to add no feature types is that we can move the existing ones from the NULL writer. So when you click OK the workspace will look no different, but there will be a new writer in the Navigator window:
5) Move Feature Types
Inspect the parameters dialog for each writer feature type in turn. For each type move it from the NULL writer to the FILEGDB writer, like so:
This will expose a number of extra parameters. The key one to set is Geometry. For the Neighborhoods they should be set to geodb_polygon:
For the VotingPlaces feature type the Geometry parameter should be set to geodb_point.
Now the two feature types belong to the Geodatabase writer, and the NULL writer can be deleted from the Navigator window if you wish.
6) Set Geodatabase Parameter
One (very quick) last thing to change: locate the Geodatabase writer in the Navigator window and expand its list of parameters. Double-click the parameter labelled Overwrite Existing Geodatabase and set it to Yes:
This ensures we aren't continually adding data to the same dataset if we run the workspace more than once.
7) Run Workspace
Test run the workspace in FME Desktop. Inspect the output. You should find the output is a Geodatabase containing seven tables (the Neighborhoods table and a separate table for each set of voting places).
8) Publish and Run Workspace
Publish the workspace to FME Server. Be sure not to check the button to upload any data. Register the workspace against the Job Submitter service as usual.
Return to the FME Server web interface. Locate the workspace under the Run Workspace dialog. Notice how the dataset paths are all hard-coded to the original file locations:
Obviously this will be of no use if the Server does not have access to those files. However, because we already uploaded them to the Resources folders we can use those files.
So, for each file, click the browse button, browse to the appropriate subfolder in the Resources folder, and select/set the file location. For the Geodatabase output location you'll need to type the file name manually:
$(FME_SHAREDRESOURCE_DATA)/Election/Output/DepartmentData.gdb
Remember to remove any existing references to the incorrect files:
Now when the workspace is completed a Geodatabase file should appear in the folder Resources\Data\Election\Output:
9) Apply FME Server Parameter
Although the workspace ran correctly, and used the data in the resources folder, that's only because we selected that data at run time. It is not a permanent feature of the workspace.
It would be much better if the workspace was programmed to look into the resources folders automatically.
So, return to the workspace in FME Workbench.
If we do set the workspace to read from the resources folders, we don't want to give users the chance to change that. So in the Navigator window locate the three parameters for source and destination datasets and delete them:
10) Set Source/Destination Parameters
Now, in turn, locate the source and destination dataset parameters for the two readers and one writer. Double-click each in turn and change them to:
KML Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Input\VancouverNeighborhoods.kml |
GML Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Input\ElectionVoting.gml |
Geodatabase Writer | $(FME_SHAREDRESOURCE_DATA)\Election\Output\DepartmentData.gdb |
Save the workspace and publish it back to FME Server.
Sister Intuitive says... |
This time you won't be able to test-run the workspace in FME Workbench, because it won't recognize the shared resource parameter. Only FME Server will return a value for that parameter. |
11) Re-Run Workspace
Now run the workspace on FME Server. Be sure to use the Job Submitter service (not Data Download) so the output is written to the required file. This time you will not be prompted with a parameter to select the source (or destination) datasets, but they will be used from the resources folders just the same.
CONGRATULATIONS |
By completing this exercise you have learned how to:
|