Exercise 2.1 | Daily Database Updates: Publishing Data |
Data | Neighborhoods (KML) Election Voting (GML) |
Overall Goal | Create a workspace to read and process departmental data and publish it to FME Server |
Demonstrates | Publishing source data and uploading temporary datasets |
Start Workspace | C:\FMEData2018\Workspaces\ServerAuthoring\DataHandling-Ex1-Begin.fmw |
End Workspace | C:\FMEData2018\Workspaces\ServerAuthoring\DataHandling-Ex1-Complete.fmw |
For the exercises in this chapter, you are a technical analyst in the GIS department of your local city.
You have already (Exercises 1.1, 1.2, and 1.3) created a workspace to carry out a translation, published it to FME Server, ran it to confirm it works, shared the repository, and set the workspace to run on a schedule.
Now you have a task to create a new workspace. One of the datasets it uses is the same as in the previous exercise, so we will try to have this second workspace use the data belonging to the first.
Sister Intuitive says... |
If you have lots of experience with FME Workbench - and if your instructor agrees - simply open the end workspace listed in the header above and skip to step 7 |
1) Inspect Source Data
The first task in creating any new workspace is to inspect the source data, so let's do that. Use the FME Data Inspector to open these two datasets:
Reader Format | GML (Geography Markup Language) |
Reader Dataset | C:\FMEData2018\Data\Elections\ElectionVoting.gml |
You can turn off the layer of VotingDivisions. All we are interested in for this exercise are the point features designated as VotingPlaces:
Map tiles by Stamen Design, under CC-BY-3.0. Data by OpenStreetMap, under CC-BY-SA.
2) Create Workspace
Open the starting workspace listed above.
You might notice that it's a copy of our previous workspace since the requirements for this workspace are so similar. If you do choose to carry on working in that workspace, be sure to save it under a different name - otherwise the data we will publish will not work for this exercise.
The workspace looks like this:
3) Remove Firehalls
For this workspace we need to process election data instead of FireHalls, so firstly delete the writer feature type for the FireHalls, and then the reader feature type.
When you delete the reader feature type, you will be asked if you wish to delete the entire reader. We could reuse it but, for the sake of simplicity, click yes.
The workspace now looks like this:
4) Add VotingPlaces
Now select Readers > Add Reader to start adding a reader to the workspace. When prompted, enter the following details for the VotingPlaces data:
Reader Format | GML (Geography Markup Language) |
Reader Dataset | C:\FMEData2018\Data\Elections\ElectionVoting.gml |
Click OK to add the Reader to the workspace. When prompted only select the VotingPlaces feature type, not VotingDivisions:
5) Add VotingPlaces to Writer
To add VotingPlaces to the writer, right-click the newly placed reader feature type and choose Duplicate on 'NULL':
There will now be a reader and writer feature type for the VotingPlaces dataset:
Change the connections to pass the VotingPlaces data through the Clipper transformer just as the FireHalls used to be:
6) Set VotingPlaces Feature Type Name
Finally, as with the FireHalls, let's set the Feature Type Name for the VotingPlaces writer feature type.
Inspect its parameters and under Feature Type Name either enter:
VotingPlaces-@Value(NeighborhoodName)
...or click the dropdown and use the text editor dialog to enter that value. This will cause voting places in each different neighborhood to be written to a separate table/layer.
Save the workspace. As already mentioned, make sure it has a different name than the first project.
7) Publish to Server
Publish the workspace to FME Server. This time you can simply choose the previously created FME Server connection, rather than having to enter parameters all over again.
For the repository select the previously created Training repository and enter a name for the workspace if it doesn't already have one.
This time, instead of simply checking the box to upload all the data files, click the Select Files button:
This dialog lists the files we are about to publish to the repository with the workspace. Technically the VancouverNeighborhoods dataset was already published to the repository with the previous workspace, but it's not very good practice to try and re-use data this way (even though we could) so place a check mark against all files and click OK:
In the final dialog of the publishing wizard, once again choose the Job Submitter as the web service to register the workspace against.
8) Examine Files
If you have access to the FME Server computer itself, open a file browser and browse to the location that repository data is stored. Here it is C:\ProgramData\Safe Software\FME Server\repositories\Training:
You'll see that each workspace is saved to a separate folder. If you inspect the contents of a folder, you'll see the uploaded datasets within it.
This is how a workspace has access to files published with it. It can also, with some manual effort, access files stored with another workspace in the same repository.
9) Run Workspace
Log in to FME Server and then locate and run the workspace. In the Run dialog notice that the published parameters denoting the source data include an FME environment variable, FME_MF_DIR:
This variable tells FME to look in the same folder as the workspace for the source data files. As you can see, it isn't particularly user-friendly to handle data in this way, even though the workspace will run just fine.
9) Upload Temporary Data
Now let's pretend that the layer of VotingPlaces data has changed in some way. You can simulate that by simply opening a file browser and making a copy of the GML file.
For example, rename C:\FMEData2018\Data\Elections\ElectionVoting.gml to NewElectionVoting.gml
NB: You don't also have to copy ElectionVoting.xsd - it's okay to use that schema file for the new GML dataset.*
Now, in the FME Server web interface, log out of the admin account and log in as a user (user/user).
So, as a user, we wish to run the workspace with the new data. We can't publish the data because the user account doesn't have permission to write to that repository; and in any case, since the workspace hasn't changed in any way, we shouldn't have to go through the publishing process.
So, click Run Workspace and select the newly published workspace in the Training repository. However, to use the new dataset, click the browse button to the right of the Source GML prompt:
In the dialog that opens, click the Temporary Uploads tab and then on the Upload File button:
Select both the files NewElectionVoting.gml and ElectionVoting.xsd and click Open to upload them. Now - back in the prior dialog - click the X button to deselect the XSD file:
The file needs to exist, but it doesn't need to be selected. Now click OK and then click the Run button.
The workspace will now run to completion using the uploaded dataset.
However - and this is the important part - this was only a temporary upload. The workspace can be re-run immediately, and the data will still appear in the temporary upload section, but it is not a permanent solution. The data is likely to be cleaned up automatically within 24 hours.
CONGRATULATIONS |
By completing this exercise you have learned how to:
|