Speaking in tongues was the evidence that the Holy Ghost had arrived inside the individuals.
Peter specifically stated what the people were hearing initially at Pentecost is what Joel prophesied about; the pouring out of God's Spirit: "THIS IS THAT" Tongues was proof of the experience having occurred.
"And they were all filled with the Holy Ghost, and began to speak with other tongues, as the Spirit gave them utterance....
And they were all amazed, and were in doubt, saying one to another, What meaneth this?
Others mocking said, These men are full of new wine.
But Peter, standing up with the eleven, lifted up his voice, and said unto them, Ye men of Judaea, and all ye that dwell at Jerusalem, be this known unto you, and hearken to my words:
For these are not drunken, as ye suppose, seeing it is but the third hour of the day.
But THIS IS THAT which was spoken by the prophet Joel;
And it shall come to pass in the last days, saith God, I will pour out of my Spirit upon all flesh: and your sons and your daughters shall prophesy, and your young men shall see visions, and your old men shall dream dreams:
And on my servants and on my handmaidens I will pour out in those days of my Spirit; and they shall prophesy:" Acts 2:12-18
This Jesus hath God raised up, whereof we all are witnesses.
Therefore being by the right hand of God exalted, and having received of the Father the promise of the Holy Ghost, he hath shed forth this, which ye now see and hear." Acts 2:12-18, 32-33