gwcelery.tasks.condor module

Submit and monitor HTCondor jobs [1].

Notes

Internally, we use the XML condor log format [2] for easier parsing.

References

exception gwcelery.tasks.condor.JobAborted[source]

Bases: Exception

Raised if an HTCondor job was aborted (e.g. by condor_rm).

exception gwcelery.tasks.condor.JobRunning[source]

Bases: Exception

Raised if an HTCondor job is still running.

exception gwcelery.tasks.condor.JobFailed(returncode, cmd, output=None, stderr=None)[source]

Bases: CalledProcessError

Raised if an HTCondor job fails.

(task)gwcelery.tasks.condor.submit(submit_file, log=None)[source]

Submit a job using HTCondor.

Parameters:
  • submit_file (str) – Path of the submit file.

  • log (str) – Used internally to track job state. Caller should not set.

Raises:
  • JobAborted – If the job was aborted (e.g. by running condor_rm).

  • JobFailed – If the job terminates and returns a nonzero exit code.

  • JobRunning – If the job is still running. Causes the task to be re-queued until the job is complete.

Example

>>> submit.s('example.sub',
...          accounting_group='ligo.dev.o3.cbc.explore.test')
(task)gwcelery.tasks.condor.check_output(args, log=None, error=None, output=None, **kwargs)[source]

Call a process using HTCondor.

Call an external process using HTCondor, in a manner patterned after subprocess.check_output(). If successful, returns its output on stdout. On failure, raise an exception.

Parameters:
  • args (list) – Command line arguments, as if passed to subprocess.check_call().

  • log (str) – Used internally to track job state. Caller should not set.

  • error (str) – Used internally to track job state. Caller should not set.

  • output (str) – Used internally to track job state. Caller should not set.

  • **kwargs – Extra submit description file commands. See the documentation for condor_submit for possible values.

Returns:

Captured output from command.

Return type:

str

Raises:
  • JobAborted – If the job was aborted (e.g. by running condor_rm).

  • JobFailed – If the job terminates and returns a nonzero exit code.

  • JobRunning – If the job is still running. Causes the task to be re-queued until the job is complete.

Example

>>> check_output.s(['sleep', '10'],
...                accounting_group='ligo.dev.o3.cbc.explore.test')