I'm going to go to Genesis now as you know this was an account of Moses given by GOD and Moses was born Jewish and raised Egyptian
Genesis 16:7 The angel of the Lord found her by a spring of water in the wilderness, the spring on the way to Shur. 8 And he said, “Hagar, servant of Sarai, where have you come from and where are you going?” She said, “I am fleeing from my mistress Sarai.” 9 The angel of the Lord said to her, “Return to your mistress and submit to her.” 10 The angel of the Lordalso said to her, “I will surely multiply your offspring so that they cannot be numbered for multitude.” 11 And the angel of the Lord said to her,
“Behold, you are pregnant
and shall bear a son.
You shall call his name Ishmael,[
c]
because the Lord has listened to your affliction.
12 He shall be a wild donkey of a man,
his hand against everyone
and everyone's hand against him,
and he shall dwell over against all his kinsmen.”
13 So she called the name of the Lord who spoke to her, “You are a God of seeing,”[
d] for she said, “Truly here I have seen him who looks after me.”[
e]
I ask you this how did the writer of Genesis know that Ishmael's descendants offspring would be multiplied so much they could not be numbered
Genesis 21:9 But Sarah saw the son of Hagar the Egyptian, whom she had borne to Abraham, laughing.[
b] 10 So she said to Abraham, “Cast out this slave woman with her son, for the son of this slave woman shall not be heir with my son Isaac.” 11 And the thing was very displeasing to Abraham on account of his son. 12 But God said to Abraham, “Be not displeased because of the boy and because of your slave woman. Whatever Sarah says to you, do as she tells you, for through Isaac shall your offspring be named. 13 And I will make a nation of the son of the slave woman also, because he is your offspring.”
Genesis 25:23 And the Lord said to her,
“Two nations are in your womb,
and two peoples from within you[
c] shall be divided;
the one shall be stronger than the other,
the older shall serve the younger.”
Genesis 28:8 So when Esau saw that the Canaanite women did not please Isaac his father, 9 Esau went to Ishmael and took as his wife, besides the wives he had, Mahalath the daughter of Ishmael, Abraham's son, the sister of Nebaioth.
other than these particular scriptures not much was ever known about nation that GOD promised Ishmael until Muslims, making these scriptures somewhat prophetic. If you chose to call people who GOD promised to make a nation and did worshippers of a false GOD that's your choice; I on the other hand think GOD has his plans for all his people.