Exercise 2.2 | Daily Database Updates: Using Resources |
Data | Neighborhoods (KML) Election Voting (GML) |
Overall Goal | Create a workspace to read and process departmental data and publish it to FME Server |
Demonstrates | Uploading data to a resources folder and authoring a workspace to make use of it |
Start Workspace | C:\FMEData2018\Workspaces\ServerAuthoring\DataHandling-Ex2-Begin.fmw |
End Workspace | C:\FMEData2018\Workspaces\ServerAuthoring\DataHandling-Ex2-Complete.fmw |
You have already (Exercise 2.1) created a workspace to carry out a translation, and published it to FME Server; both with data and using data uploaded temporarily.
However, such data management tools are not particularly suited to a long-term project, so the task here is to upgrade the workspaces to use datasets stored in a Resources folder. There we can store source data and write destination data.
1) Open FME Server Web Interface
Log in to the FME Server web interface using an administrator account (such as admin/admin). Click Resources on the menu bar to navigate to the resources management pages.
2) Create Folder
In most cases data should be stored under the Data folder, so click on Data in the Resources dialog to open that folder. To avoid mixing datasets, our data should go into its own subfolder. So click on the New Folder button and create a folder called Election:
Next click on the Election folder and within there create new subfolders called Input and Output:
3) Upload Source Datasets
Browse to the Input folder and click the Upload button and then select Files. Upload the source datasets for the current translation:
Reader Datasets | C:\FMEData2018\Data\Elections\ElectionVoting.gml C:\FMEData2018\Data\Elections\ElectionVoting.xsd C:\FMEData2018\Data\Boundaries\VancouverNeighborhoods.kml |
So we now have both source datasets and a folder to write the output data to.
4) Add Writer
Up until now all of our workspaces have had only a NULL (dummy) writer. Now we know about Resources we can add a proper writer and point its output to the Resources Output folder.
So, open the starting workspace listed above in FME Workbench and then select Writers > Add Writer on the menubar and set up a new writer with the following parameters:
Writer Format | Esri Geodatabase (File Geodb Open API) |
Writer Dataset | C:\FMEData2018\Output\Training\DepartmentData.gdb |
Feature Class or Table Definition | None (Advanced) |
The reason we want to add no feature types is that we can move the existing ones from the NULL writer. So when you click OK the workspace will look no different, but there will be a new writer in the Navigator window:
5) Move Feature Types
Inspect the parameters dialog for each writer feature type in turn. For each type move it from the NULL writer to the FILE GDB writer, like so:
This will expose a number of extra parameters. The key one to set is Geometry. For the Neighborhoods they should be set to geodb_polygon:
For the VotingPlaces feature type, the Geometry parameter should be set to geodb_point.
Now the two feature types belong to the Geodatabase writer, and the NULL writer can be deleted from the Navigator window if you wish.
6) Set Geodatabase Parameter
One (very quick) last thing to change: locate the Geodatabase writer in the Navigator window and expand its list of parameters. Double-click the parameter labeled Overwrite Existing Geodatabase and set it to Yes:
This ensures we aren't continually adding data to the same dataset if we run the workspace more than once.
7) Run Workspace
Test run the workspace in FME Desktop. Inspect the output. You should find the output is a Geodatabase containing seven tables (the Neighborhoods table and a separate table for each set of voting places).
8) Publish and Run Workspace
Publish the workspace to FME Server. Be sure not to check the button to upload any data. Register the workspace against the Job Submitter service as usual.
Return to the FME Server web interface. Locate the workspace under the Run Workspace dialog. Notice how the dataset paths are all hard-coded to the original file locations:
Obviously, this will be of no use if the Server does not have access to those files. However, because we already uploaded them to the Resources folders, we can use those files.
So, for each file, click the browse button, browse to the appropriate subfolder in the Resources folder, and select/set the file location. For the Geodatabase output location you'll need to type the file name manually:
$(FME_SHAREDRESOURCE_DATA)/Election/Output/DepartmentData.gdb
Remember to remove any existing references to the incorrect files:
Now when the workspace is run, a completed Geodatabase file should appear in the folder Resources\Data\Election\Output:
9) Apply FME Server Parameter
Although the workspace ran correctly, and used the data in the resources folder, that's only because we selected that data at runtime. It is not a permanent feature of the workspace.
It would be much better if the workspace was programmed to look into the resources folders automatically.
So, return to the workspace in FME Workbench.
If we do set the workspace to read from the resources folders, we don't want to give users the chance to change that. So in the Navigator window locate the three parameters for source and destination datasets and delete them:
10) Set Source/Destination Parameters
Now, in turn, locate the source and destination dataset parameters for the two readers and one writer. Double-click each in turn and change them to:
KML Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Input\VancouverNeighborhoods.kml |
GML Reader | $(FME_SHAREDRESOURCE_DATA)\Election\Input\ElectionVoting.gml |
Geodatabase Writer | $(FME_SHAREDRESOURCE_DATA)\Election\Output\DepartmentData.gdb |
Save the workspace and publish it back to FME Server.
TIP |
If you try running this workspace in FME Workbench now that you've updated the dataset parameters to use the FME Server Shared Resource parameter, you will get an error because Workbench won't recognize that FME Server parameter.
If you still want to be able to test this workspace in Workbench, you can set a value for the FME_SHAREDRESOURCE_DATA parameter. Configure it to point to a local file path that matches the folder structure used on FME Server. The workspace will still run on FME Server as Server will automatically override any value for that parameter and replace it with the correct path. |
11) Re-Run Workspace
Now run the workspace on FME Server. This time you will not be prompted with a parameter to select the source (or destination) datasets, but they will be used from the resources folders just the same.
CONGRATULATIONS |
By completing this exercise you have learned how to:
|