What to do in this situation? (details inside)

It sounds as though you have known this person for a long time. I would think if romantic feelings were going to happen for you, they would have appeared by now. You can’t force it. If he wants more from this relationship than you can give, or are ever likely to be able to give, you have to be straight with him. Sadly you may lose his friendship, at least for a while. He might need time to heal and some people are never able to go back to ‘just being friends’.

First, tell him how you are feeling now. That you do not feel anything romantically for him. But tell him you are willing to give the relationship a try.

Then do.

If nothing develops, then you break it off by saying, “I’ve really tried, John, but I just don’t have any feelings for you romantically. Can we go back to being friends? (He may or may not be willing to. You have to honor his feelings on that issue.)

Basically you are not ready for a Romantic relationship, with him yet.
Perhaps your intuition is telling you to wait on this one.

You need to be honest with him. Love has to be a two way street.

Talk to him.
“you were friends, he got a girlfriend and didn’t talk” “then he came back,...but you don’t feel the same”
There is not much information to go on, but it sounds like each of you are just kind of using each other out of convenience. Decide what it is you want or feel and then make a decision. Stick with it.

Chances are that you are both just using each other for comfort and sex. There is a good chance his attachment is only because he thinks that is what he needs to do to keep you happy and “involved” romantically.
Talk to him, be frank and let him know how you feel and see where it goes from there.

You didn’t chat with your friend when he was sparing time with someone else. This sign seems like you have soft corner for love to the guy. Firstly you need to make yourself clear whether you want him as a friend or more than friend. Because if you are not clear, then you are wasting his life.