It is not at all easy to be virtuous. Especially if you want to please all these various philosophers. Your case would be much like the father, son and donkey. If the father rides the donkey, people would scoff at him for letting the young chap suffer the walking; if the son rides the donkey, others would deride the son for not respecting the age of the father; if both ride the donkey, up will spring the PETA lot screaming at the injustice meted out to the donkey by overloading it; and if the father and son choose to carry the donkey, the DONKEY objects by kicking them senseless. In other words, there is no pleasing everyone with your virtuousness.
Tiru, not to be outdone by his fellow philosophers, is equally as demanding if not more so. Here he goes with
Enbi ladhanai veyilpolak kaayumae anbiladhanai aram - Tirukkural
Like the Sun scorches boneless beings like worms, virtue scorches those lacking in love - Loose Translation
Boneless beings, here, is merely a description of the sort of beings that cannot stand extreme heat. In other words, it is not their bonelessness that makes them suffer but their inability to regulate their body temperatures because they are cold-blooded.
There is a difficulty in understanding why Virtue should scorch a person lacking in love...as in, is it not possible for a person to behave virtuously even though he feels no love for those around him? To behave as a person of integrity...does it also need you to love? Is it not actually a fact that to be impartial can be a virtue for a ruler, say, and a ruler who does not love would be better because he has no biases? Why, then, should virtue scorch such a person?
The problem with virtue IS that almost everything that you classify as virtuous behavior would, in some form or other, translate to either actively benefitting or, at the very least, no hurting those around you. In other words, the measure of virtue lies in the impact your actions have on other people.
Take that impartial, unbiased actions that you may need to take; actions that could well prove to be detrimental to those close to you because they were needed to be taken in the interests of the larger good. NOW, if you are the sort of person overflowing with love, and your love for those around you is visible...THEN, even though you cost them benefits, they COULD still be friendly with you. (Unlikely, you say, and I'd be inclined to agree. But 'unlikely' is not necessarily 'impossible'.) More to the point, others who are unaffected and more likely to see things without emotions are likely to see things your way and give credit for your virtuous actions. IF, however, you are lacking in love and KNOWN to be so...how quickly you'd alienate everyone in your vicinity (IF he can do THAT to his son-in-law....). THEN your virtuous decisions will readily be taken as vices and, yes, you'll be scorched as a result.
The problem with being seen as a 'cold fish' is that, even when you benefit people by your choice, it'd either be seen as accidental OR as some devious plot to take advantage. AND if you hurt people, even by accident, it will invariably be taken to be a deliberate act. After all, you're seen as someone who does not care for anyone at all.
The other problem IS that, absent love, it is actually very tough to be truly virtuous. Justice, they say, needs to be tempered with mercy. To paraphrase and elaborate on the idea, you cannot live life off a rule book. A rule book is a guide but, in each given circumstance, you need to apply it with compassion and empathy. AND I am yet to see anyone who is capable of compasstion and empathy being incapable of love. So, when your so-called virtue is void of compassion, it's actually no virtue at all. Your charity becomes a self-serving exercise where you look down on the recipients; your help undermines the pride of those who receive it; and your entire life full of acts that can only be dry entries in a ledger book of virtues without really possessing any virtue.
So, yes, sans love you are a 'cold fish' and, thus, scorched by the Sun of virtue!