Data Integration Scenario
Data City of Vancouver Open Data (see table below)
Overall Goal Use your FME skills to integrate data from several sources to answer a question.
Demonstrates Content Transformation, Structural Transformation, Schema Editing
Start Workspace C:\FMEData2018\Workspaces\DataIntegration\ScenarioBegin.fmw
End Workspace Instructors can request access to the final workspaces here.

Introduction

In the previous exercises you have learned how to translate and transform data using FME. Now let's put that knowledge to use! As we covered in the Lecture section, data integration is a key requirement for modern organizations. In this exercise you will develop a workspace that solves a data integration problem in a city government scenario.

You will:

  • inspect your data
  • combine your datasets
  • query your data to answer a question or solve a problem
  • answer lab questions about the process of integrating data

This exercise will provide an example of integrating three datasets into a central database. After that you can follow similar steps to choose one additional dataset to answer another question. If you have trouble carrying out the steps you can refer to the Additional Procedures section, FME Workbench Help (also available online in our Documentation) or the community Knowledge Center.

Please continue to refer to the Lab Questions throughout, as some of the questions refer to the scenario.

Exercise Goals

  • Integrate data into a single database using multiple feature types on a single writer
  • Use transformers to create the desired schema in the destination database.
  • Merge data using an Overlayer transformer
  • Use ChartGenerator to make analyze integrated data
  • Select another dataset and add it to your workspace to answer a question.

Inspect Data

To start off, inspect the data you have been given. There is a wide variety of topics and formats available. Following the instructions from Exercise 3, open some of these files in the FME Data Inspector to see what attributes they have.

For the walkthrough we will be using the following datasets:

  • Bike Paths
  • Neighborhoods
  • Public Art

More data is available from the City of Vancouver Open Data catalogue.

Topic Format Location (FMEData2018\Data...)
Addresses Esri Geodatabase Addresses\Addresses.gdb
Building Footprints SpatialLite Database or AutoCAD DWG Engineering\BuildingFootprints\building_footprints.sl3 or C:\FMEData2018\Data\Parcels\BuildingFootprints.dwg
Bike Paths Esri Shapefile Transportation\Cycling\BikePaths*.shp
Business Licenses Excel Spreadsheet Planning\BusinessLicenses.xlsx
City Grid Geography Markup Language Boundaries\CityGrid.gml
City-owned Properties Esri Shapefile Parcels\CityProperties\CityProperties.shp
Coastal Zones Geography Markup Language Boundaries\CoastalZones\CoastalZones.gml
Community Resources Esri Geodatabase CommunityMapping\CommunityMap.gdb
Crime Comma Separated Values Emergency\Crime.csv
Drinking Fountains Comma-separated Values Engineering\DrinkingFountains.csv
Election Results Excel Spreadsheet Elections\ElectionResults.xls
Election Voting Districts Geography Markup Language Elections\ElectionVoting.gml
Elevation Digital Elevation Model ElevationModel\DEM-Full.dem
Firehalls with Zones Geography Markup Language Emergency\FireHallsWithZones.gml
Forward Sortation Areas (for postcode) Esri Shapefile Addresses\ForwardSortationAreas.shp
Land Boundary Esri Shapefile Boundaries\LandBoundary\VancouverLandBoundary.shp
Neighborhoods Google Keyhole Markup Language Boundaries\VancouverNeighborhoods.kml
Orthophotos (scale corrected aerial photographs) TIFF Orthophotos*.tif
Parcels AutoCAD DWG Parcels\Parcels.dwg
Parking Meters MapInfo TAB Transportation\Parking\Meters.tab
Parking Tickets Comma-separated Values Transportation\Parking\ParkingTickets.csv
Parks MapInfo TAB Parks\Parks.tab
Planning Restrictions (Business Improvement Areas, Historic Areas, Noise Control Areas, and View Cones) OCG Geopackage Planning\PlanningRestrictions.gpkg
Point Cloud (LiDAR) LAS PointClouds\CoV*.laz
Public Art Excel Spreadsheet Culture\PublicArt.xlsx
Public Trees Comma-separated Values Parks\Parks.tab
Roads AutoCAD DWG or Microstation DGN Transportation\CompleteRoads.dwg or RoadsDGN.dgn
Street Lighting AutoCAD DWG and Microstation DGN Engineering\StreetLighting.dwg and StreetLightingDGN.dgn
Traffic Signals AutoCAD DWG and Microstation DGN Engineering\TrafficSignals.dwg and TrafficSignalsDGN.dgn
Zoning MapInfo TAB Zoning\Zones.tab
Zoning Descriptions Adobe PDF Zoning\ZoneDescriptions.pdf

results matching ""

    No results matching ""