Exercise 6 Shape Dataset Processing
Data Contour Datasets (Esri Shapefile)
Overall Goal Trigger notification for new files
Demonstrates Processing Directory Watch notifications
Start Workspace None
End Workspace C:\FMEData2016\Workspaces\ServerAuthoring\Notifications-Ex6-Complete.fmw

Having carried out various notification tasks, your mind goes back to a Directory Watch project you started but didn't finish. It involved watching for Shape datasets representing contours. Let's see if we can put that start to a good use.


Miss Vector says...
This exercise continues where exercise 1 left off. You must have completed exercise 1 to carry out this exercise.


1) Create Workspace
Start Workbench and begin with an empty workspace.

Select Readers > Add Reader from the menubar. When prompted set the parameters as follows:

Reader Format Text File
Reader Dataset C:\FMEData2016\readme.txt
Reader Parameters Read Whole File at Once: Yes

As you know by now, it doesn't really matter what text file we use as the source, it won't be used anyway. It will be replaced by the content of the incoming message.


2) Add JSONFlattener
Now add a JSONFlattener transformer to the workspace, after the Text File Reader:

Open the parameters dialog. In there select text_line_data as the source of the JSON content. Under Attributes to Expose manually enter dirwatch_publisher_content

Add Logger transformers after the JSONFlattener


3) Publish to FME Server
Publish the workspace to FME Server, registering it under the Job Submitter service.


4) Create Subscription
Now log in to the FME Server web interface and navigate to Manage > Notifications.

Click on the Subscriptions tab and click New to create a new Subscription.

Call the subscription "Process Incoming Shape". Subscribe to the topic ShapeIncomingFile

Now set the protocol to Workspace and select the workspace uploaded in the previous step. Under Notification Message Mapping, select the published parameter for the Text File Reader source dataset.

Click OK to create the Subscription.


5) Test Topic
Locate the source Shape datasets in C:\FMEData2016\Data\ElevationModel\Contours - select a set of Shape files (.dbf, .prj, .shp, .shx) and create a zip file out of them (as you did in Exercise 1).

Copy the zip file into the newly created Resources folder. You can do this through the file system (by copying the file to C:\ProgramData\Safe Software\FME Server\resources\data\ShapeContours) or using the web interface.


6) Check Results
Now Navigate to Manage > Jobs. Under completed jobs should be the workspace you just caused to run. View or download the log file and look for the logged feature. You should find it has an attribute containing JSON, and a number of attributes extracted from the JSON.

dirwatch_publisher_actionCREATE
dirwatch_publisher_contentENTRY_CREATE C:\ProgramData\Safe Software\FME Server\resources\data\ShapeContours\K11.zip
dirwatch_publisher_pathC:\ProgramData\Safe Software\FME Server\resources\data\ShapeContours\K11.zip

So now we know what the data looks like and can process it accordingly.


7) Edit JSONFlattener Transformer
Back in FME Workbench open the JSONFlattener transformer parameters. Under Attribute to Expose add the attribute dirwatch_publisher_path


8) Add FeatureReader Transformer
Now add a FeatureReader transformer in place of the Loggers:

This is a transformer that will let us read the contents of the dataset. Open the parameters dialog. Set:

Reader FormatEsri Shapefile
Reader DatasetSelect Attribute Value > dirwatch_publisher_path
Output PortSingle Output Port

Click OK to close the dialog. You may receive a warning message, but it can be ignored.


9) Add Writer
Having read the data from Shape, we can now add it to our corporate database.

Select Writers > Add Writer from the menubar. When prompted set the parameters as follows:

Writer Format SpatiaLite
Writer Dataset C:\FMEData2016\Output\Contours.sl3
Writer Parameters Overwrite Existing Database: No
Drop Existing Tables: No
Add Feature Types Table Definition: Manual

In the new feature type that is created, simply change the name to contours:

Click OK to close the dialog and connect the new feature type to the FeatureReader transformer's <Generic> output port:


10) Republish Workspace
Publish the workspace back to FME Server


11) Adjust Subscription
Navigate to Manage > Notifications and open the Process Incoming Shape Subscription for editing. The settings should now include one for the output database. Change it to write the database in the Resources folder:

Don't - for fairly obvious reasons - write it back to the same folder as the incoming Shape data!


12) Test Solution
Now test the solution by putting more zipped Shape data into the directory watch folder. You will find that each dataset put into the folder is added to the SpatiaLite database:


CONGRATULATIONS
By completing this exercise you have learned how to:
  • Use Loggers to identify what JSON attributes an incoming message provides
  • Use a FeatureReader transformer to read the data added to a Directory Watch folder

results matching ""

    No results matching ""