Don't count as outgoing until we actually send the new INVITE