Exercise 12 | Creating a Job Queue |
Data | None |
Overall Goal | To demonstrate how the FME REST API can be used to manage job queues in FME Server |
Demonstrates | How to create a queue |
Workspace | austinDownload.fmw. Stored in the Samples Repository |
NEW |
Previously, job management was configured through engine tags. This has been deprecated and now we use job queues. However, the REST API endpoints are the same but manage queues now. |
Create a Job Queue
The following call allows you to create a new queue for a job. The following tag created will be called High_Priority. Any jobs with this tag will be sent to FMETRAINING_Engine1. You may need to add a tag to multiple engines.
FMETRAINING_Engine1 is specific to the training machines
1) Enter the following URL and Headers into Postman
POST | http://<yourServerHost>/fmerest/v3/transformations/jobroutes/tags |
---|
Headers:
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Authorization: fmetoken token= <yourServerHost>
2) Switch to the Body tab. Click raw and paste the following information
engines=FMETRAINING_Engine1&name=High_priority&priority=1
Additional Parameters
Below are some additional parameters you may use for future calls.
Parameter | Description |
---|---|
description | Description of the tag |
engines | Engine assignment for the tag |
name | Required- Unique name of the tag to create |
repositories | Repository assignments for the tag |
priority | Priority for the tag. Priority values must be integers between 1 and 10. |
3) Click Send and Review the Response from the FME Server.
You should receive a blank response with a status code of 201 Created.
Assigning an Engine to the Job Queue
4) Enter the Following URL and Headers into Postman
POST | http://<yourServerHost>/fmerest/v3/transformations/jobroutes/tags/High_priority/engines |
---|
Headers:
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Authorization: fmetoken token= <yourServerHost>
5) Switch to the Body tab. Click raw and paste the following information:
engines=FMETRAINING_Engine1
6) Click Send and Review the Response from the FME Server.
Assigning a Repository to the Job Queue
In the next stage of this exercise, we are taking the job queue we just created and assigning it to a repository. This ensures that every time a job in the repository is run it is assigned to this queue.
7) Enter the Following URL and Headers into Postman
Request
POST | http://<yourServerHost>/fmerest/v3/transformations/jobroutes/tags/High_priority/repositories |
---|
Headers:
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Authorization: fmetoken token= <yourServerHost>
8) Switch to the Body tab. Click raw and paste the following information:
repositories=RESTTraining
Additional Parameters
Below are some additional parameters you may use for future calls.
Parameter | Description |
---|---|
repositories | Repository assignments for the queue |
tag | Name of the job queue |
9) Click Send and Review the Response from the FME Server.
You should receive the following response.
204- Success. The assigned repositories were assigned.
10) Check the FME Server to see if the Repository was successfully assigned
Go to your FME Server and Click Repositories.
To the right of the repository name, you should see a drop down list of potential queues. The High_Priority queue should be assigned.
Submit a Job and Include the Tag in the Request
When you submit a job. You may specify the queue in the request, under the TMDirectives. The TMDirectives specify how the job should be run by the server. Below is the full list of parameters.
WARNING |
Please note that the priority has been deprecated and queues will be honored instead. |
TMDirectives {
rtc (boolean, optional): Runs a job until it is explicitly canceled. The job will run again regardless of whether the job completed successfully, failed, or the server crashed or was shut down.,
ttc (integer, optional): Time (in seconds) elapsed for a running job before it's canceled. The minimum value is 1 second, values less than 1 second are ignored.,
ttl (integer, optional): Time to live in the job queue (in seconds),
description (string, optional): Description of the request,
priority (integer, optional): The priority of the job. Priority values must be integers between 1 and 200. If a request's priority value is less than 1, greater than 200, or is not specified, then FME Server sets it to 100.,
tag (string, optional): The job routing tag for the request
}
Below is an example of what the TMDirectives can look like:
"TMDirectives": {
"rtc": false,
"ttc": 60,
"description": "This is my description",
"tag": "linux",
"ttl": 60
},
Now, we are going to use a Submit call and alter the parameters to include the job queue (also called a tag in the REST API) previously created.
11) Enter the Following URL and Headers into Postman
POST | http://<yourServerHost>/fmerest/v3/transformations/submit/Samples/austinDownload.fmw |
---|
Headers:
Content-Type: application/json
Accept: application/json
Authorization: fmetoken token= <yourServerHost>
12) Switch to the Body tab. Click raw and paste the following information
{
"publishedParameters": [
{
"name": "MAXY",
"value": "42"
},
{
"name": "THEMES",
"value": [
"airports",
"cenart"
]
}
],
"TMDirectives": {
"tag": "High_priority"
}
}
13) Click Send and Review the Response from the FME Server.
{
"id": 32
}
Please Note. The job ID that is returned by the Server will be different based on how many jobs the FME Server has completed.
Now we can check the job to see if the job ran successfully under the High_priority tag.
14) Enter the Following URL and Headers into Postman
GET | http://<yourServerHost>/fmerest/v3/transformations/jobs/id/<yourJobID> |
---|
Headers:
Accept: application/json
Authorization: fmetoken token=<yourToken>
15) Click Send and Review the Response from the FME Server
In the response, you can find the Engine that the call was submitted to and the parameters of the call.
As you can see here, the job was submitted under the High_priority tag. However, the priority demonstrated was still -1. The tag will override this priority, so it was still submitted with a priority of 1, it is just not displayed in the FME Server.
CONGRATULATIONS |
By completing this exercise you have learned how to:
|