HTCondor - Revision history

Vacin: /* Interactive submission */

2026-08-06T12:49:34Z

Interactive submission

Vacin: /* Requests */

2026-08-06T12:38:39Z

Requests

Vacin: /* Executable, input, arguments, outputs, errors and logs */

2026-08-06T12:35:52Z

Executable, input, arguments, outputs, errors and logs

Vacin: /* Quick start */

2026-08-06T12:29:38Z

Quick start

Vacin: /* Quick start */

2026-08-06T12:29:01Z

Quick start

Vacin: /* Basic batch concepts */

2026-08-06T12:23:58Z

Basic batch concepts

Vacin: /* Introduction */

2026-08-06T12:22:53Z

Introduction

Cacosta: /* Accounting Group and +experiment option */

2026-06-16T05:18:50Z

Accounting Group and +experiment option

Cacosta: /* Accounting Group */

2026-06-16T04:53:55Z

Accounting Group

Cacosta: /* The htcondor command */

2026-03-27T13:26:05Z

The htcondor command

@@ Line 750: / Line 750: @@
 == Interactive submission ==
-As it happens with Torque/Maui, HTCondor also has the possibility to submit interactive jobs using the option "-i" or "-interactive".
+HTCondor has the possibility to submit interactive jobs using the option "-i" or "-interactive".
 <pre>

← Older revision		Revision as of 12:38, 6 August 2026
Line 353:		Line 353:


−	You can access any of these machines directly by using the request_gpus option. However, please note that jobs running on gpu06-gpu12 will be in preemptible mode. This means that if a higher-priority job (from Jupyter or one of the priority groups) requires the resources, your job may be terminated and re-queued. The ~~execptions~~ are <b>gpu05, gpu13-gpu15</b>, where submitted jobs will not be preempted by any other job once they are running.	+	You can access any of these machines directly by using the request_gpus option. However, please note that jobs running on gpu06-gpu12 will be in preemptible mode. This means that if a higher-priority job (from Jupyter or one of the priority groups) requires the resources, your job may be terminated and re-queued. The exceptions are <b>gpu05, gpu13-gpu15</b>, where submitted jobs will not be preempted by any other job once they are running.

	Example of submit file to run in any possible gpu:		Example of submit file to run in any possible gpu:

@@ Line 132: / Line 132: @@
 <pre>
 $ cat test-736.0.log
-(736.000.000) 03/25 10:00:22 Job submitted from host: <193.109.174.82:9618?addrs=193.109.174.82-9618&noUDP&sock=961738_da40_3>
+(736.000.000) 03/25 10:00:22 Job submitted from host: <**** addrs=****&noUDP&sock=961738_da40_3>
 ...
-(736.000.000) 03/25 10:00:40 Job executing on host: <192.168.101.48:9618?addrs=192.168.101.48-9618&noUDP&sock=14755_47fd_3>
+(736.000.000) 03/25 10:00:40 Job executing on host: <****addrs=****&noUDP&sock=14755_47fd_3>
 ...
 (736.000.000) 03/25 10:00:49 Image size of job updated: 972
@@ Line 162: / Line 162: @@
 $ condor_wait -status log-62.0.log
 .0.0 submitted
-.0.0 executing on host <192.168.100.29:9618?addrs=192.168.100.29-9618+[--1]-9618&noUDP&sock=73457_f878_3>
+.0.0 executing on host <****addrs=****+[--1]-9618&noUDP&sock=73457_f878_3>
 .0.0 completed
 All jobs done.The command '''condor_wait''' is used to track the information in log file. This command will be explained in next sections.
@@ Line 209: / Line 209: @@
 $ condor_q 740
--- Schedd: submit01.pic.es : <193.109.174.82:9618?... @ 03/25/19 11:27:30
+-- Schedd: submit01.pic.es : <**** ... @ 03/25/19 11:27:30
 OWNER   BATCH_NAME    SUBMITTED   DONE   RUN    IDLE  TOTAL JOB_IDS
 testuser MyJobs       3/25 11:27      _      _      2      4 740.0-1
@@ Line 222: / Line 222: @@
 $ condor_q -nobatch 740
--- Schedd: submit01.pic.es : <193.109.174.82:9618?... @ 03/25/19 11:27:33
+-- Schedd: submit01.pic.es : <**** ?... @ 03/25/19 11:27:33
   ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
 .0   testuser         3/25 11:27   0+00:00:00 I  0    0.0 test.sh -c 1 -t 60

@@ Line 75: / Line 75: @@
--- Schedd: submit01.pic.es : <193.109.174.82:9618?... @ 02/25/19 13:07:52
+-- Schedd: submit01.pic.es : <****... @ 02/25/19 13:07:52
 OWNER   BATCH_NAME    SUBMITTED   DONE   RUN    IDLE  TOTAL JOB_IDS
 testuser ID: 281      2/25 13:07      _      _      1      1 281.0
@@ Line 92: / Line 92: @@
--- Schedd: submit01.pic.es : <****?... @ 02/25/19 13:10:10
+-- Schedd: submit01.pic.es : <****... @ 02/25/19 13:10:10
   ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
 .0   testuser         2/25 13:09   0+00:00:00 I  0    0.0 test.sh -c 1 -t 60

@@ Line 32: / Line 32: @@
 Before taking a deeper view in all the elements of the job submission, we will show you the basic commands for a quick start guide to HTcondor.
-In our old Torque/Maui environment, the user would log into a machine, prepare the input and submit jobs to a queue using qsub command. Now, in a very similar way, the user logs into the User Interface, prepares a submit file, and then creates and inserts jobs into the queue using a condor_submit command.
+The user logs into the User Interface, prepares a submit file, and then creates and inserts jobs into the queue using a condor_submit command.
 ''Since November 2019, all the users can submit directly their jobs from the user interfaces.''
@@ Line 92: / Line 92: @@
--- Schedd: submit01.pic.es : <193.109.174.82:9618?... @ 02/25/19 13:10:10
+-- Schedd: submit01.pic.es : <****?... @ 02/25/19 13:10:10
   ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
 .0   testuser         2/25 13:09   0+00:00:00 I  0    0.0 test.sh -c 1 -t 60

← Older revision		Revision as of 12:23, 6 August 2026
Line 18:		Line 18:
	HTCondor does not work as other batch systems where you submit your job to a differentiated queue that has some specifications. It employs the language of ClassAds (the same concept of classified advertisements) in order to match workload requests and resources. In other words, the jobs and the machines have their particular attributes (number of CPUs, memory, etc.) and the central manager of HTcondor does the matchmaking between these attributes.		HTCondor does not work as other batch systems where you submit your job to a differentiated queue that has some specifications. It employs the language of ClassAds (the same concept of classified advertisements) in order to match workload requests and resources. In other words, the jobs and the machines have their particular attributes (number of CPUs, memory, etc.) and the central manager of HTcondor does the matchmaking between these attributes.

−	Furthermore~~, similarly to Torque/Maui~~, there is the concept of fair-share which aims at ensuring that all groups and users are provided resources as needed in correspondence to their respective quota (e.g. the Atlas T2 quota equals to 9% of our resources). The fair-share concept implies that your jobs and the jobs of your experiment will have a greater priority while they are agreed at or below the share, if you are consuming more resources than your share, then the next job with more priority should belong to another experiment.	+	Furthermore, there is the concept of fair-share which aims at ensuring that all groups and users are provided resources as needed in correspondence to their respective quota (e.g. the Atlas T2 quota equals to 9% of our resources). The fair-share concept implies that your jobs and the jobs of your experiment will have a greater priority while they are agreed at or below the share, if you are consuming more resources than your share, then the next job with more priority should belong to another experiment.

@@ Line 1: / Line 1: @@
 = Introduction =
-At PIC HTCondor is the current production batch system replacing the old Torque/Maui environment.
+At PIC HTCondor is the current production batch system.
 The aim of this document is to show how to submit jobs to the HTCondor infrastructure to all of the non-grid users of the PIC batch system. In other words, this document is a guide to submit local jobs to the HTCondor infrastructure at PIC.
-This User Guide begins with a presentation of the batch system concepts in HTCondor in comparison with the old Torque/Maui ones. Then, there is a Quick Start section focused on the minimum knowledge needed to submit a job in an HTCondor pool. The remaining sections try to give a deeper approach in how to submit, monitor and remove jobs in PIC HTCondor pool.
+This User Guide begins with a presentation of the batch system concepts in HTCondor. Then, there is a Quick Start section focused on the minimum knowledge needed to submit a job in an HTCondor pool. The remaining sections try to give a deeper approach in how to submit, monitor and remove jobs in PIC HTCondor pool.
 Furthermore, you can access to the HTCondor User Guide slides and a git repository with several examples.

@@ Line 729: / Line 729: @@
 </pre>
-Using the option +experiment="virgo", the job will have the share of the virgo experiment and will also be accounted for in our records as a virgo job. You can request a job for any group you belong to using the +experiment= knob. If you request a group you do not belong to or one that does not exist, you will not be able to submit the job.
+Using the option +experiment="virgo", the job will have the share of the virgo experiment and will also be accounted for in our records as a virgo job. You can request a job for any group you belong to using the +experiment= knob. If you request a group you do not belong to or one that does not exist, you will not be able to submit the job:
 Furthermore, the automatic assignment of the Accounting Groups is also done for those groups that have dedicated User Interfaces (such as mic.magic and at3). In that case, the Accounting Group is determined according to the User Interface's group owner. In other words, the main group of the user does not matter; if a job is submitted from the mic.magic machine, the AccountingGroup will belong to magic.

@@ Line 711: / Line 711: @@
 You can find a (rare) situation where the executable is not present in your user interface but you know for sure that is present in the WNs. Take into account that HTCondor always looks for your executable before submitting your job unless you explicitly add transfer_executable = false in your submit file.
-== Accounting Group ==
+== Accounting Group and +experiment option ==
-The priority of your job is calculated depending on the Accounting Group you belong to. The common users do not have to worry about the Accounting Group, as it will be automatically taken considering the primary group.
+The priority of your job is calculated depending on the Accounting Group you belong to. Regular users do not have to worry about the Accounting Group, as it will be automatically taken considering their primary group.
-Anyway, if you are in two groups and need to change your Accounting Group for any submission, you can add +experiment="experiment" option in your submit file. Thus, for instance, there is one user that has main group vip and secondary group virgo and want to submit a job that only accounts to virgo.
+However, if you are in different groups and need to change your Accounting Group for a specific submission, you can add the +experiment="experiment" option in your submit file. For instance, if a user has vip as a main group and virgo as a secondary group, and wants to submit a job that only accounts to the virgo group:
 <pre>
@@ Line 729: / Line 729: @@
 </pre>
-Using the option +experiment="virgo" the job will have the share of the virgo experiment and will also be accounted in our records as a virgo job.
+Using the option +experiment="virgo", the job will have the share of the virgo experiment and will also be accounted for in our records as a virgo job. You can request a job for any group you belong to using the +experiment= knob. If you request a group you do not belong to or one that does not exist, you will not be able to submit the job.
-The local experiments available now at PIC are:
+Furthermore, the automatic assignment of the Accounting Groups is also done for those groups that have dedicated User Interfaces (such as mic.magic and at3). In that case, the Accounting Group is determined according to the User Interface's group owner. In other words, the main group of the user does not matter; if a job is submitted from the mic.magic machine, the AccountingGroup will belong to magic.
 == Priority per user basis ==

← Older revision		Revision as of 13:26, 27 March 2026
Line 1,443:		Line 1,443:

	The goal of the htcondor command is to consolidate the functionality of multiple legacy commands into a single, intuitive interface. HTCondor developers have insisted that the old commands (like condor_submit, condor_q, etc.) are never going to disappear. However, you might find the way this new command interacts with the system and presents information to be quite useful and user-friendly.		The goal of the htcondor command is to consolidate the functionality of multiple legacy commands into a single, intuitive interface. HTCondor developers have insisted that the old commands (like condor_submit, condor_q, etc.) are never going to disappear. However, you might find the way this new command interacts with the system and presents information to be quite useful and user-friendly.
		+
		+	Although the command contains different subcommands, as a standard user, the three main subcommands are <b>job</b>, <b>eventlog</b>, and <b>dag</b>.

	== Managing jobs ==		== Managing jobs ==