If you want encrypted transport, then all you need is a parallel hardware crypto accelerator so you do not bottleneck on slow serial CPU encryption.
If you want to keep it off the memory bus, then all you need is a hardware copy/DMA engine so you do not bottleneck on slow CPU serial memcpy().
Doing a whole new bespoke network protocol in hardware seems like overkill if you are only going for 800 Gb/s.