Problem:

Configuring a submit node to submit workload from. If done incorrectly, can gerate errors such as:

ERROR: cannot establish connection - Operation now in progress (hostname:port=:42559) 

Unable to communicate with <TORQUE_HOSTNAME>(TORQUE_IP_ADDRESS) 
Cannot connect to specified server host '<TORQUE_HOSTNAME>'

Solution:

Make sure you have added the login nodes in qmgr on your Torque server :

qmgr -c "set server submit_hosts += <YOUR_SUBMIT_NODE>"

Also, make sure the submit node has the right hostname on the submit node in /var/spool/torque/server_name

If you are using Moab to submit as well, make sure your have the following in the submit nodes' /opt/moab/etc directory:
moab.cfg (You only need 1 entry, that is SCHEDCFG...)
.moab.key (from the moab head node)

The errors described above were caused by IPV6 being enabled on the submit node, and not on the headnode. Torque was sending requests out the interface card using IPV6 which the torque server never received