Exercise 12 Creating a Job Queue
Data None
Overall Goal To demonstrate how the FME REST API can be used to manage job queues in FME Server
Demonstrates How to create a queue
Workspace austinDownload.fmw. Stored in the Samples Repository

NEW
Previously, job management was configured through engine tags. This has been deprecated and now we use job queues. However, the REST API endpoints are the same but manage queues now.

Create a Job Queue

The following call allows you to create a new queue for a job. The following tag created will be called High_Priority. Any jobs with this tag will be sent to FMETRAINING_Engine1. You may need to add a tag to multiple engines.

FMETRAINING_Engine1 is specific to the training machines


1) Enter the following URL and Headers into Postman

POST http://<yourServerHost>/fmerest/v3/transformations/jobroutes/tags

Headers:

  • Content-Type: application/x-www-form-urlencoded

  • Accept: application/json

  • Authorization: fmetoken token= <yourServerHost>


2) Switch to the Body tab. Click raw and paste the following information

engines=FMETRAINING_Engine1&name=High_priority&priority=1


Additional Parameters

Below are some additional parameters you may use for future calls.

Parameter Description
description Description of the tag
engines Engine assignment for the tag
name Required- Unique name of the tag to create
repositories Repository assignments for the tag
priority Priority for the tag. Priority values must be integers between 1 and 10.


3) Click Send and Review the Response from the FME Server.

You should receive a blank response with a status code of 201 Created.


Assigning an Engine to the Job Queue


4) Enter the Following URL and Headers into Postman

POST http://<yourServerHost>/fmerest/v3/transformations/jobroutes/tags/High_priority/engines

Headers:

  • Content-Type: application/x-www-form-urlencoded

  • Accept: application/json

  • Authorization: fmetoken token= <yourServerHost>


5) Switch to the Body tab. Click raw and paste the following information:

engines=FMETRAINING_Engine1


6) Click Send and Review the Response from the FME Server.

Assigning a Repository to the Job Queue

In the next stage of this exercise, we are taking the job queue we just created and assigning it to a repository. This ensures that every time a job in the repository is run it is assigned to this queue.


7) Enter the Following URL and Headers into Postman

Request

POST http://<yourServerHost>/fmerest/v3/transformations/jobroutes/tags/High_priority/repositories

Headers:

  • Content-Type: application/x-www-form-urlencoded

  • Accept: application/json

  • Authorization: fmetoken token= <yourServerHost>


8) Switch to the Body tab. Click raw and paste the following information:

repositories=RESTTraining


Additional Parameters

Below are some additional parameters you may use for future calls.

Parameter Description
repositories Repository assignments for the queue
tag Name of the job queue


9) Click Send and Review the Response from the FME Server.

You should receive the following response.

204- Success. The assigned repositories were assigned.


10) Check the FME Server to see if the Repository was successfully assigned

Go to your FME Server and Click Repositories.

To the right of the repository name, you should see a drop down list of potential queues. The High_Priority queue should be assigned.


Submit a Job and Include the Tag in the Request

When you submit a job. You may specify the queue in the request, under the TMDirectives. The TMDirectives specify how the job should be run by the server. Below is the full list of parameters.

WARNING
Please note that the priority has been deprecated and queues will be honored instead.
TMDirectives {
rtc (boolean, optional): Runs a job until it is explicitly canceled. The job will run again regardless of whether the job completed successfully, failed, or the server crashed or was shut down.,

ttc (integer, optional): Time (in seconds) elapsed for a running job before it's canceled. The minimum value is 1 second, values less than 1 second are ignored.,

ttl (integer, optional): Time to live in the job queue (in seconds),
description (string, optional): Description of the request,

priority (integer, optional): The priority of the job. Priority values must be integers between 1 and 200. If a request's priority value is less than 1, greater than 200, or is not specified, then FME Server sets it to 100.,

tag (string, optional): The job routing tag for the request
}

Below is an example of what the TMDirectives can look like:

"TMDirectives": {
    "rtc": false,
    "ttc": 60,
    "description": "This is my description",
    "tag": "linux",
    "ttl": 60
  },

Now, we are going to use a Submit call and alter the parameters to include the job queue (also called a tag in the REST API) previously created.


11) Enter the Following URL and Headers into Postman

POST http://<yourServerHost>/fmerest/v3/transformations/submit/Samples/austinDownload.fmw

Headers:

  • Content-Type: application/json

  • Accept: application/json

  • Authorization: fmetoken token= <yourServerHost>


12) Switch to the Body tab. Click raw and paste the following information

{
  "publishedParameters": [
    {
      "name": "MAXY",
      "value": "42"
    },
    {
      "name": "THEMES",
      "value": [
        "airports",
        "cenart"
      ]
    }
  ],
  "TMDirectives": {
    "tag": "High_priority"

  }
}


13) Click Send and Review the Response from the FME Server.

{

    "id": 32

}

Please Note. The job ID that is returned by the Server will be different based on how many jobs the FME Server has completed.

Now we can check the job to see if the job ran successfully under the High_priority tag.


14) Enter the Following URL and Headers into Postman

GET http://<yourServerHost>/fmerest/v3/transformations/jobs/id/<yourJobID>

Headers:

  • Accept: application/json

  • Authorization: fmetoken token=<yourToken>


15) Click Send and Review the Response from the FME Server

In the response, you can find the Engine that the call was submitted to and the parameters of the call.

As you can see here, the job was submitted under the High_priority tag. However, the priority demonstrated was still -1. The tag will override this priority, so it was still submitted with a priority of 1, it is just not displayed in the FME Server.


CONGRATULATIONS
By completing this exercise you have learned how to:
  • Create a job queue using the REST API
  • Assign an engine to a job queue using the REST API
  • Assigning a repository to a job queue using the REST API
  • Run a job using the REST API and assign a queue within the call

results matching ""

    No results matching ""