Hmmmm...?
Tough question, and one that I ask myself from time to time.
On the one hand, we all face the potential danger of "leaving our first love" (Rev. 2:4) or of somehow losing the insatiable desire that we once had for the Lord and his presence.
Personally, I see this type of insatiable desire in the woman in the Song of Solomon.
For example:
Song of Solomon chapter 3
[1] By night on my bed I sought him whom my soul loves: I sought him, but I found him not.
[2] I will rise now, and go about the city in the streets, and in the broad ways I will seek him whom my soul loves: I sought him, but I found him not.
[3] The watchmen that go about the city found me: to whom I said, Saw ye him whom my soul loves?
[4] It was but a little that I passed from them, but I found him whom my soul loves: I held him, and would not let him go, until I had brought him into my mother's house, and into the chamber of her that conceived me.
On the other hand, however, if we truly love God, then one outworking of that love is going to be in how we relate to other people in our lives. Some of those relationships include how we relate to a spouse or potential spouse, how we relate to our friends, how we relate to brothers and sisters in Christ, and even how we relate to our enemies. In other words, it's not all about just locking ourselves up in a cloister somewhere where we only seek God continually. Again, our relationship with him ought to ultimately greatly impact our relationships with others as well.
Just yesterday, a friend of mine on this forum asked me privately if I considered that Satan might be exploiting a weakness in me in relation to my desire to be with a woman. I told her that I honestly didn't believe that Satan was my problem, but that, as weird as this may initially sound, "love" is my problem. In other words, if we're truly spending the type of quality time with God that you're concerned about, then we should be getting more and more conformed to his image over time, and God is love. With such being the case, we should ever be becoming more loving ourselves, and, again, that love isn't just going to manifest itself towards God, but towards other people as well.
I guess what I'm trying to say to you is don't fall into the trap of feeling condemned if you have a need for human fellowship. God is interested in how we relate to others, and it's okay to have other relationships alongside our relationships with God...as long as they're not evil in his sight.
I hope that this makes some sense.