Fix segfault when building peer based on IP