Controlling Resource Usage: Retain Exclusive vs. Step Limit
Description
This article describes various ways to control the usage of your resources. Through various methods, you can ensure that only a single job runs on a resource, or all steps for a particular job are run on the same resource in a pool.
Solution
Described here are a few ways to control the use of your resources. These can be combined to achieve specific controls based on the machine, or the jobs and steps that are executed.
Step Limit
The simplest way to limit the number of steps that can run on any resource at one time is through the use of the Step Limit option on the Resource definition itself. Setting the Step limit to one ensures that only one Step can run at a time on an agent. However, this will restrict you from potentially running parallel processes on a particular agent, and given the high capacity of machines nowadays, most are capable of running multiple steps at a time. If you need to ensure that a particular agent is used exclusively for a single step at all times, set the Step Limit to 1. Similarly, if you wish to restrict an agent machine to running only three parallel steps at a time, then set it to 3.
This is done on the Cloud tab, when creating or editing a resource:
Retain Exclusive
Using the Retain Exclusive option in your steps ensures that the resource used for that step is only used for that single step or the job, prohibiting other steps or jobs from running on that resource until this one completes.
The screenshot below illustrates 4 selections for the Retain Exclusive option: None, Job, Step, and Call. None obviously does no retaining of the resource, the number of jobs or steps that can be run in parallel are limited by the Step Limit explained above.
The Job Exclusive holds this resource for the length of this job, or until the resource is released, whichever occurs first.
The Step Exclusive holds this resource only for as long as this Step runs.
The Call Exclusive holds this resource for the duration of the procedure calling this step.
An important aspect about Retain Exclusive is that this setting cannot bump out jobs that are already running on the agent. So any step defined to have an Exclusive setting will wait until all other steps assigned to that agent have completed their work. In other words, only once the resource is truly "unused" will the exclusive-step get a chance to claim the resource.
When a resource is marked Exclusive, this fact is not currently displayed in the GUI. As a result, one edge-case situation that could arise is the following: It is quite likely the job with the exclusive resource could have other jobsteps which rely on other resources. This means it's possible that the Exclusive resource would be locked to the job, yet have periods of time where it is not actually running any jobsteps. Whereas an active resource will have jobsteps to display in the GUI which suggest that it's a "busy" resource, in the scenario above, this list could be empty. This may lead someone to surmise that the resource should be available for their procedure to be using yet be confused as to why their jobstep is in a "waiting for resource" state. Eventually, the resource will become available (once formally freed, or the job completes), which will allow the waiting jobstep to move proceed. This situation can be confirmed by reviewing the commander.log files.
Save the Resource as a Property
The Resource field on a Step can contain the name of a resource, the name of a pool, or it can be used with a property, which is described here.
If you wish to run all of a job's steps on a single server (such as: your checkout, build, unit tests, and so on) but you dont want to hold that server exclusively for your job, you can allow this. By getting a resource assigned to your job, and saving the name of that resource into a property, you can use the property as your resource for all of your following steps.
In the procedure's first step, select your resource – this could be a single resource or from a pool defined as the Resource on the step. For the Step Command, set the resource name into a job parameter:
ectool setProperty /myJob/resourceName --value "$[/myJobStep/assignedResourceName]"
In subsequent steps that you want to occur all on the same agent, the resource is:
$[/myJob/resourceName]
This is important to keep in mind when you are using a pool and running a multi-step procedure, such as a build, which will check out and build the code in different steps, but which need to occur on the same machine. If you do not use a method such as this, or selecting Retain Exclusive for the Job, you could check the code out on one machine in the pool, and try to build on another machine in the pool.
An Edge Case
As a sample of how these features can be utilized, one customer needed to run Selenium tests on an agent. The requirement was that only one Selenium test could be running on that agent at one time. The moment a second Selenium test starts on the same agent it would take away screen control from the first Selenium test and as a result the first test would hang or timeout. However, you could still run other non-Selenium commands on that agent. Procedures were designed to run multiple Selenium tests and there were instances when multiple tests started running in parallel on a single agent.
The solution implemented was the creation of two resources for the same agent: One was called win7-vm and the other was called win7-vm_selenium. The win7-vm_selenium resource was configured with the step limit of 1, but no step limit was set for win7-vm resource. This way, whenever a Selenium test was run on win7-vm_selenium, no other Selenium tests were run (since the step limit was 1), but other non-Selenium tests could run in parallel on the same environment on the second agent.
It is possible, and sometimes effective, to create multiple resources for the same agent.
Conclusion
There are many ways to control the usage of your resources, and by combining these methods, you can easily ensure that you use your resources effectively. As an example, by setting the Step Limit for a Resource to 3, and then using a Property to save the Resource name, you could be running up to 3 parallel jobs on one agent machine. In a separate scenario, if you know you will be running a CPU intensive task, rather than rely on step-limits, you might choose to set the Retain Exclusive field to the Job level (rather than step level), thereby ensuring that no other jobs can utilize that resource while your Job is running.
Comments