Use RTP/RTCP fds on the RTP structure, don't bother storing them.