Are you referencing this?
[SUP]51 [/SUP]Then, behold, the veil of the temple was torn in two from top to bottom; and the earth quaked, and the rocks were split, [SUP]52 [/SUP]and the graves were opened; and many bodies of the saints who had fallen asleep were raised; [SUP]53 [/SUP]and coming out of the graves after His resurrection, they went into the holy city and appeared to many.
I see no mention of them being in glorified bodies nor any mention of them ascending into Heaven with Christ. In fact, I find no other mention of these people being with Christ either here or later during the 40 days prior to His ascension. Further, and apparently, only those who were in the vicinity of Jerusalem were raised up. Notice also that it says "many of the bodies" and not "all of the bodies?" Another interesting point to make is that the graves opened during the earthquake which accompanied His death but they did not come out of the ground until 3 days later, after His resurrection. So, the graves were opened for three days before they came out, if I'm reading this correctly.
Again, I find no verse that says they "left with Jesus" as you put it, or ascended with Him. Can you please point that passage out to me as perhaps I overlooked it. Thanks buddy.