Since we are focused on the Old vs the New Covenant, that would be singular. At the same time, God made a covenant with the children of those who perished in the wilderness (for unbelief) regarding their occupancy of the land, and how they would be driven out for their disobedience. That is why we have covenant(s) in Rom 9:4.
Now therefore hearken, O Israel, unto the statutes and unto the judgments, which I teach you, for to do them, that ye may live, and go in and possess the land which the LORD God of your fathers giveth you. (Deut 4:1)
THE OLD COVENANT AFTER THE EXODUS
And he declared unto you his covenant, [singular] which he commanded you to perform, even ten commandments; and he wrote them upon two tables of stone. (Deut 4:13)
THE COVENANT WITH THOSE WHO SURVIVED THE WILDERNESS
The LORD made not this covenant with our fathers, but with us, even us, who are all of us here alive this day... But as for thee, stand thou here by me, and I will speak unto thee all the commandments, and the statutes, and the judgments, which thou shalt teach them, that they may do them in the land which I give them to possess it.
(Deut 5:3,31)