I believe there were many fans that were just too glad to see him gone when he left the show for good, so they did not want to attract negative attention from them.
Personally, I liked Wes, but then again, the super-genius characters and those who have trouble understanding human emotion are often my favorites in their respective shows:
Scotty (somewhat, he is a miracle worker after all) and Spock,
Wesley and Data,
Bashir and Odo,
did not watch VOY enough to form an opinion,
Trip and T'Pol.