Actually, latency was the other thing I worry about, because I definitely feel it in some instances. In fact, that's the more frustrating aspect of those blocks. If the chain were one long line of effects (or the equivalent of that in the grid without the send/return blocks) I feel like we could cut down on latency. It all adds up.
The send and return blocks do not add any additional latency. You can run them in parallel to a row of shunts and there is no phase cancelation between the two. Don't confuse CPU load with latency. The two are not the same.