I suppose so long as you are walking by sight rather than faith, and you acknowledge one another by the outward dead man rather than by the Spirit of Jesus Christ, and you believe the church is a building and not the spiritual body of God’s Children in the Holy Spirit, these things will be a continual, never-ending dilemma for you.
On the other hand, if you ARE walking by faith in God alone, and you ARE quite aware you are a spiritual saint and your old man is dead & crucified with Christ, and you KNOW the church is not made with hands, you don’t have to lose a wink of sleep over the vain babblings and endless chatter that never seems to end about man's opinions on doctrine.