This kind of charge reveals a good deal about the personality of the people who make it; to impute such motives to another man is to imply you're harboring them yourself.