I don’t think the solution is to stop loving him.
He sounds confused. (Have you told him that aside from this comment, you two already have what you would otherwise constitute as being ‘a relationship’?)
“I have found him taking more solo trips than trips with me which was starting to make me wonder where I stood in his life. When I had confronted him on that, I think it scared him off.”
Can you go into this bit in a bit more detail for us? xx