Exercise 6 | Shape Dataset Processing |
Data | Contour Datasets (Esri Shapefile) |
Overall Goal | Trigger notification for new files |
Demonstrates | Processing Directory Watch notifications |
Start Workspace | None |
End Workspace | C:\FMEData2016\Workspaces\ServerAuthoring\Notifications-Ex6-Complete.fmw |
Having carried out various notification tasks, your mind goes back to a Directory Watch project you started but didn't finish. It involved watching for Shape datasets representing contours. Let's see if we can put that start to a good use.
Miss Vector says... |
This exercise continues where exercise 1 left off. You must have completed exercise 1 to carry out this exercise. |
1) Create Workspace
Start Workbench and begin with an empty workspace.
Select Readers > Add Reader from the menubar. When prompted set the parameters as follows:
Reader Format | Text File |
Reader Dataset | C:\FMEData2016\readme.txt |
Reader Parameters | Read Whole File at Once: Yes |
As you know by now, it doesn't really matter what text file we use as the source, it won't be used anyway. It will be replaced by the content of the incoming message.
2) Add JSONFlattener
Now add a JSONFlattener transformer to the workspace, after the Text File Reader:
Open the parameters dialog. In there select text_line_data as the source of the JSON content. Under Attributes to Expose manually enter dirwatch_publisher_content
Add Logger transformers after the JSONFlattener
3) Publish to FME Server
Publish the workspace to FME Server, registering it under the Job Submitter service.
4) Create Subscription
Now log in to the FME Server web interface and navigate to Manage > Notifications.
Click on the Subscriptions tab and click New to create a new Subscription.
Call the subscription "Process Incoming Shape". Subscribe to the topic ShapeIncomingFile
Now set the protocol to Workspace and select the workspace uploaded in the previous step. Under Notification Message Mapping, select the published parameter for the Text File Reader source dataset.
Click OK to create the Subscription.
5) Test Topic
Locate the source Shape datasets in C:\FMEData2016\Data\ElevationModel\Contours - select a set of Shape files (.dbf, .prj, .shp, .shx) and create a zip file out of them (as you did in Exercise 1).
Copy the zip file into the newly created Resources folder. You can do this through the file system (by copying the file to C:\ProgramData\Safe Software\FME Server\resources\data\ShapeContours) or using the web interface.
6) Check Results
Now Navigate to Manage > Jobs. Under completed jobs should be the workspace you just caused to run. View or download the log file and look for the logged feature. You should find it has an attribute containing JSON, and a number of attributes extracted from the JSON.
dirwatch_publisher_action | CREATE |
dirwatch_publisher_content | ENTRY_CREATE C:\ProgramData\Safe Software\FME Server\resources\data\ShapeContours\K11.zip |
dirwatch_publisher_path | C:\ProgramData\Safe Software\FME Server\resources\data\ShapeContours\K11.zip |
So now we know what the data looks like and can process it accordingly.
7) Edit JSONFlattener Transformer
Back in FME Workbench open the JSONFlattener transformer parameters. Under Attribute to Expose add the attribute dirwatch_publisher_path
8) Add FeatureReader Transformer
Now add a FeatureReader transformer in place of the Loggers:
This is a transformer that will let us read the contents of the dataset. Open the parameters dialog. Set:
Reader Format | Esri Shapefile |
Reader Dataset | Select Attribute Value > dirwatch_publisher_path |
Output Port | Single Output Port |
Click OK to close the dialog. You may receive a warning message, but it can be ignored.
9) Add Writer
Having read the data from Shape, we can now add it to our corporate database.
Select Writers > Add Writer from the menubar. When prompted set the parameters as follows:
Writer Format | SpatiaLite |
Writer Dataset | C:\FMEData2016\Output\Contours.sl3 |
Writer Parameters | Overwrite Existing Database: No Drop Existing Tables: No |
Add Feature Types | Table Definition: Manual |
In the new feature type that is created, simply change the name to contours:
Click OK to close the dialog and connect the new feature type to the FeatureReader transformer's <Generic> output port:
10) Republish Workspace
Publish the workspace back to FME Server
11) Adjust Subscription
Navigate to Manage > Notifications and open the Process Incoming Shape Subscription for editing. The settings should now include one for the output database. Change it to write the database in the Resources folder:
Don't - for fairly obvious reasons - write it back to the same folder as the incoming Shape data!
12) Test Solution
Now test the solution by putting more zipped Shape data into the directory watch folder. You will find that each dataset put into the folder is added to the SpatiaLite database:
CONGRATULATIONS |
By completing this exercise you have learned how to:
|