I had family member that used to go to a church with a famous leader, and they would always tell me this person left that person left, and they couldnt figure out why. and the church leader would always say "they fell away" and insinuate they were "faithless". a few years later my family members left because they found out the church leader ws crooked, like very very bad. Im sure he said the same about them when they left.
Not saying this is your experience, I have no clue of your experience, but my point remains, people can leave a church and not fall away. Moving, finding one they deem better, learning more truth, etc.
John was in the truth more than any "church" today, so people leaving their ekklesia was much different than leaving a church today.
I tried to fully explain, hope I worded it well.