Good point .The falling away would seem to have occurred at Pentecost. As a sign promised against the unbelieving Jew . I believe Jews as Amorites and Hittites, Abram's lineage were used to represent the whole of mankind as his people in that way.
The Holy Spirit at Pentecost was bringing prophets from all the nations of the world preaching the gospel as promised in Joel. Mocking them with stammering lips a demonstration to the whole world exposing unbelief in the hearts of men from all nations .And the power of the gospel our new tongue to those who do believe.
Falling away or falling backward seems to be a common metaphor used through the bible to indicate his judgment that was violated in the garden .
In Revelation 21 in that description of the wife of Christ the church. The tribe of Dan in the book of Revelation is missing from the description of the bride. Dan, a word that means "God is my judge" is used to reveal those who do not fall away or fall back. . the bride of Christ. They fall forward. And in respect to new testament saints the apostle Judah is left out of the 12 set aside for that purpose, as a witness to the name Dan
To fall away is to crucify Christ over and over exposing him to public shame as if God could not finish it in one three day work.
Genesis 49:16-17 King James Version (KJV Dan shall judge his people, as one of the tribes of Israel. Dan shall be a serpent by the way, an adder in the path, that biteth the horse heels, so that his rider shall fall backward
Above a serpent or adder is used to represent the effects of false doctrine coming from a brood of vipers representing the kings and fathers (Pharisees with Sadducees) as the oral tradition of men. That by their tradition make the word of God without effect. .The letter of the law as all things written in the law and prophets judges them, again signified as Dan, as the witnesses of God .
Judas is used to represent the same false prophecy as commandments of men.
John 18:4-6 King James Version (KJV) Jesus therefore, knowing all things that should come upon him, went forth, and said unto them, Whom seek ye?They answered him, Jesus of Nazareth. Jesus saith unto them, I am he. And Judas also, which betrayed him, stood with them. As soon then as he had said unto them, I am he, they went backward, and fell to the ground.
The faithless Jews were used to represent unbelieving mankind as a warning to those who mock God as a sign against them . .At Pentecost. He brings prophecy as tongues through all the nations and they who would have nothing to do line upon line here a little and there a little (sola scriptura) which is comparing the spiritual understanding to the same, faith to faith. They who have tasted of the good things to come will not hear and believe. . they fall back or fall away .
For precept must be upon precept, precept upon precept; line upon line, line upon line; here a little, and there a little: For with stammering lips and another tongue will he speak to this people. To whom he said, This is the rest wherewith ye may cause the weary to rest; and this is the refreshing: yet they would not hear. But the word of the Lord was unto them precept upon precept, precept upon precept; line upon line, line upon line; here a little, and there a little; that they might go, and fall backward, and be broken, and snared, and taken. Wherefore hear the word of the Lord, ye scornful men, that rule this people which is in Jerusalem. Isaiah 28:10-14