Perhaps there is some confusion about the meaning here, as the usage of the word 'notorious' is not quite the most common usage.
From wiktionary: Notorious:
Adjective
notorious (comparative more notorious, superlative most notorious)
[edit] Synonyms
.The doctor is not 'notorious' from having done something bad.
Broadly speaking the doctor is famous to the world at large because of his humanitarian activity. This is the 'at large'. But up close, in person, people avoid close contact with him because of his contact with lepers. And so this is like being notorious, even though it is not the normal meaning.
' Sanders reflected, and no doubt the reverse was true'
It appears that the character thinks that the reverse may be true, that in some cases, a person may have some activity or profession that makes the person like-able in person, but to the world at large, they may be held in some disrespect. Some lawyers, or finance or bank people might be in this category. Some people have negative attitudes about lawyers, but may have a very good friend that is a lawyer.