A particular pastor or elder may have gifts and abilities that may set him head and shoulders above others, and it is not wrong to recognize him as such. But his gifting does not give him overall authority over the others. Regards of his maturity, experience and gifting, he still has only one vote on the board of elders.
Paul knew this, and he told the Corinthians to follow him as he follows Christ. Also, he told them that others may not recognize him as an apostle, but he is an apostle to them because he was the one who brought them to Christ and he was their spiritual father.
I listened to my old pastor during the three years, as a 21-24 year old under his ministry, because he spent time in prayer for me, and was faithful to God's Word in the things he taught me. It is right for us to have mentors in our lives, who stand in the gap with God for us, and who are not afraid to correct us when we need it. My old pastor had authority over me, not because he demanded it, but his prayers and faithfulness to Christ gave him the right as long as I was a member of his church.