It would completely depend upon the actual relationship I had with the guy. Where I work, there are several males, with whom I’m also friends with outside of work, so with those people it wouldn’t be weird or awkward. Even one of those dudes is my boss, so no problem there. If it were someone I didn’t know well, or only knew as a work colleague (rather than also being a friend outside of work) it might be a bit awkward, but the way you’ve described it, it seems like a fairly normal situation, since the guy was just getting cleaned up to go to the airport, and a shirtless male is not the same thing as a shirtless female.
I probably would have said something like, “Ooops, sorry. I didn’t mean to intrude.” Even though you weren’t intruding. Just said in a calm fashion, as if you’ve accidentally bumped into someone, or stepped on their toe.
In my case, if this guy had been one person in particular. I probably would have shoved him back into his office, with me following, shutting and locking the door and ravishing him on the floor : P But that’s just me.
Back to you. I don’t think you should have told him in an e-mail (or at all) that you’ll never knock on his door again. By saying that, you turned a non-situation into an awkward situation. He might now think that he has inadvertently offended you, or that you might put forth a sexual harassment complaint against him, because it sounds like you were shocked and upset. I don’t think that YOU feel that you were sexually harassed, but your comment to him might make HIM think that you do feel that way.
I think that you kind of like this guy, and he was pretty cute, and the view of his naked chest was unexpected and you kind of over-reacted rather than taking the chance that you had, in making a light hearted joke/comment to him at the time.
Do you think that there will be a time in the near future that you might need to knock on his door again? If so, knock and casually ask, “Are you decent?” Then smile at him when he opens the door. Don’t build this up in your mind as this BIG situation.