Church members reportedly relocate the church building after the pastor's wife declines to serve them tea at his home