That's why I don't ship him with anyone who doesn't have a few hundred on them minimum.
So far, that's pretty much left him Romana and other Time Ladies.
When it comes to shipping The Doctor, (though I really don't ship him with anyone), I really see his relationships as the same thing that I see with the idea of other immortals, like Vampires. In the case of Rose, she gets a version of him that ages like she does. The Rose x Ten!Two is the equivalent of the immortal giving up their powers to live with a mortal, but in this case it's also a clone. While I see him often being interested an occasionally infatuated with other Companions, I don't think that he honestly sees himself as someone who settles down or has "a certain someone" because he's too dynamic to have any long-term romantic relationship with a human.
Hundreds of years of age, regenerations, and everything that he's been through with that puts him as a very lone creature in the universe as it stands, so while the nature of his relationship with his companions might change in the degree of how romantic it is or isn't, I always think that he loves humans because they burn so brightly and so briefly, and they make him better and brighter (more on why he doesn't/shouldn't travel alone). It's the reason that he puts a lot into his relationship with River, despite it being so strange and difficult (the opposite of his companions) because she understands things that his companions don't, and she's like him in the very same ways that they're not.
As for 11/Clara, I think that she is genuinely rather infatuated with him in a slightly romantic sort of way, and he probably cares for her a bit that way, especially after the long period of solitude and not (really) having River around anymore. However, I think that when she dove through his Timestream and scattered herself all across his life to undo the damage that The Great Intelligence attempted to cause, that it's going to change how they interact, the same way that he changed as soon as he was able to figure out for sure that she wasn't a trap. I think that that "mysterious unknown" that she was attracted to is going to exit with Eleven, and that as Twelve, it'll feel like the two of them are best of friends discovering new things together, since they know each other so well, but that the element of romance is going to shift into a close, platonic friendship.
(I wrote a lot more about this than I intended to).
On a sorta unrelated note, I'm interested to see how John Hurt's Doctor and Twelve fit into this:
http://xsoldier.tumblr.com/post/57400697032/tempus-aeterna-shh-im-wondering
X 
People will ship it.
