A question about Compressor and codecs..

So, a while back I discovered job-chaining in Compressor and it seriously enhanced my workday. I’ve been thinking more about this and I’m trying to understand it.
When I produce a video, I need a high resolution master to archive, and then 3 versions for a media server that are half the frame rate and 3 different frame sizes. My high res master is typically a ProRes 422 file. The 3 smaller files I produce are H.264 movs. These videos are usually 5 minutes or less.

If I take that ProRes file and try to convert it straight over into those 3 different H.264 files, it takes about one million years.

If I take that ProRes file into Compressor and do all the frame resizing and frame rate changes to convert it to another ProRes file, then take that resized ProRes and convert it to the 3 H.264 files, it takes less than 10 minutes.

So what is going on here is that the heavy lifting of Frame Controls within Compressor is happening separate from the heavy lifting of crunching the bitrate. The ProRes to ProRes processing doesn’t touch the frame size/rate. The smaller ProRes to H.264 is only the compression with no Frame Controls.

Can someone explain the codec voodoo science behind why this works exactly? Is changing from ProRes to H.264 with frame controls on just too much for Compressor? Or is it the nature of both codecs – does the way they both work mixed with all these parameters I’m putting on it just take forever in any encoding program? It makes sense to me in a way (obviously, I just tried it one day because it seemed like it made sense to try it). My guess is that the calculations become too complex between resizing, adjusting the frame rate, reordering frames and adjusting the keyframs, and crunching the bit rate overall for Compressor to handle it all effectively, but when they are broken apart, the calculations are straightforward.

Anybody want to explain this to me?