SubmitToDeadline is a rop node that submits Houdini jobs to Deadline managed renderfarms.
SubmitToDeadline is self contained in a python houdini digital asset, handles all sort of dependencies automatically. It has the same features and functionality of proprietary solutions seen at leading VFX studios.
Link to the video overview of submitToDeadline
Requirements:
It requires Thinkbox’s Deadline 6.1 or onwards and Houdini 12 or onwards. DeadlineClient should be installed on the user’s machine. Alternatively the environment variable DEADLINE_PATH should set up instead.
Please note Thinkbox’s Deadline is a commercial software which comes with a free-license mode up to two slaves.
How to use:
Put down a submitToDeadline rop, plug rops to it, press submit. Done.
Each rop node plugged to submitToDeadline is submitted as a deadline job.
Complex job dependencies are created by recursively navigating all input connections to a given rop and building deadline job dependencies accordingly. Bypassing of rop nodes is supported. All most common rop nodes are supported (currently excluding the fetch rop).
Initial configuration:
Setting the configuration tab is a one-off operation. Usually all is required are the mantra and houdini pools names of your Deadline installation. In some studios custom environment variables are expected to be set on the renderslave. In such case add any required variable to the extraVar section in the form of $UPPERCASEVAR i.e. $JOB Once completed save a preset and create permanent defaults. This way parameters set will be preserved every time a submitToDeadline rop is created.
Link to the initial configuration video
General Features:
- Matches features of in-house Houdini submitters seen at big visual effects facilities.
- it’s been designed to solve real production needs by a veteran of the industry.
- Proven to increase productivity.
- Successfully proven production experience at an award winning VFX studio.
Specific Features:
- Self contained Houdini Digital Asset.
- Automatic handle of all sort of dependencies.
- Solid algorythm to traverse networks and build accurate dependencies.
- All common rops are supported including Alembic Redshift Arnold.
- Dependencies on a per-job basis.
- Mantra rops are submitted as two jobs (Ifd and Mantra).
- Mantra sub-job dependencies on a per-frame basis.
- Ability to specify custom deadline plugins.
Please note Simultools and submitToDeadline are not affiliated to Thinkbox.
Main
Batch Jobs
Priority
Priority assigned in Deadline to all submitted Houdini Batch jobs
Chunk size
Number of frames to be grouped in a single task
Use Machine List
Enables whitelisting or blacklisting or renderslaves
Type
Choose between using the List attribute to blacklist or whitelist nodes
List
list of names of renderslaves separated by comma
Get List of Slaves
Prints the list of available slaves
Render Jobs
Priority
Priority assigned in Deadline to all submitted Mantra jobs
Chunk size
Number of frames to be grouped in a single task
Use Machine List
Enables whitelisting or blacklisting or renderslaves
Type
Choose between using the List attribute to blacklist or whitelist nodes
List
list of names of renderslaves separated by comma
Get List of Slaves
Prints the list of available slaves
Configuration
Pools Groups Limits
Batch Jobs
Pool
Pool assigned in Deadline to all submitted Houdini Batch jobs. Usually this is the most important parameter to set. Generally speaking there should be two pools setup by your administrator/IT/you, a Houdini Batch and a Mantra pool, but your mileage may vary
Secondary Pool
Optional Secondary pool assigned in Deadline to all submitted Houdini Batch jobs
Group
Group assigned in Deadline to all submitted Houdini Batch jobs. For more informations check the Deadline documentation
Limit Groups
Limit Groups assigned in Deadline to all submitted Houdini Batch jobs. For more informations check the Deadline documentation
Concurrent Tasks
Number of tasks to run on a single renderslave at the same time
Machines Limit
Maximum number of slaves that can render the batch jobs simulatenously (0=no limit)
Render Jobs
Pool
Pool assigned in Deadline to all submitted Mantra jobs. Usually this is the most important parameter to set. Generally speaking there should be two pools setup by your administrator/IT/you, a Houdini Batch and a Mantra pool, but your mileage may vary
Secondary Pool
Optional Secondary pool assigned in Deadline to all submitted Houdini Mantra jobs
Group
Group assigned in Deadline to all submitted Mantra jobs. For more informations check the Deadline documentation
Limit Groups
Limit Groups assigned in Deadline to all submitted Houdini Mantra jobs. For more informations check the Deadline documentation
Concurrent Tasks
Number of tasks to run on a single renderslave at the same time
Machines Limit
Maximum number of slaves that can render the mantra jobs simulatenously (0=no limit)
Refresh
Queries Deadline again to retrieve up to date lists of pools and groups.
Extra Variables and Extra Infos
extraVar1..6
Extra environment variables passed to the renderslave running all submitted jobs, at execution time. Check with your administrator/IT/You which enviroment variables you may need to forward to your renderslave. Usual values might be $HOME $USER $HOUDINIVER $JOB etc. Wildcards can be used to pass multiple variables at once (i.e. $HOU* will pass all environment variables beginning with HOU)
extraInfo1..4
Extra values passed to job. For more informations check the Deadline documentation / your administrator/IT/You. Values could be job name. Wildcards can be used to pass multiple variables at once (i.e. $HOU* will pass all environment variables beginning with HOU)
Advanced
Department
Fills the department field for all jobs submitted to Deadline
Temp Env Var
Usually set to $TEMP, it points to the environment variable pointing to the temp folder. It is used to store .log and temp submission files
Render Hip Folder
The parameter lets you change the relative path to the render hip location, the location where SubmitToDeadline stores render hip files (the hip files which are executed on the farm) Tip
setting ‘Render Hip Folder’ to . and disabling ‘Append timestap to the render Hip File’ will make jobs pointing to the current hip file instead.
Very important, the default behaviour is for SubmitToDeadline to create a folder one level up and next to your current $HIP folder called renderhip. It is recommended to store render hipfiles in this manner to ensure that relative paths in your scene beginning with $HIP will evaluate correctly A common example is when having in your scene relative paths such as $HIP/../geo/blablabla
Append timestamp to the render Hip File
It appends the time stamp to the copy of the render hip file stored in Render Hip Folder ( default is ../renderhip ). This is useful to create unique hipfiles when submitting jobs to the farm.
Save Per-Rop copy of Hip File
It saves an extra copy of the render hip file inside the output folder of each rop, inside a subfolder called hip i.e. $HIP/../geo/myrop/hip/testSim3_myrop.171048.hip
Include Environment
It gathers the whole environment and loads it onto the slave.
Use Job Environment Only
Use Job Environment Only
Render as Batch job
Render jobs are forced as Batch jobs instead of being split into two jobs (Scene translation and Render). This only affects Mantra renders as Arnold and Redshift renders are currently Batch jobs only. Also this is forced enabled on Limited commercial licenses (Indie and Apprentice).
Verbose Output
Prints more detailed informations
Comment
Fills the jobs’ comment fields with the text written here
Submit Suspended
It submits jobs suspended. Dependencies are maintained so only job without incoming dependencies are submitted as suspended
Submit
Runs the submission process