Well, the fact that you feel conviction about something that went wrong in the past- that isn't bad. But if you are feeling guilt or maybe low self esteem for days at a time- that's probably not from god.Need a clarification on what you said. Does that heaviness comes from God so that i won't go back there or is it from evil spirit ?
You've got to identify what that heaviness is.
If you did any kind of fornication- fornication is particularly difficult to shake because it can take the genuine love for that person, and use it to remind you of sin for the rest of your life. But how you respond to that reminder is what makes the difference.