I like most of the unpopular Castlevania games, of the ones I've played at least.
I played through Haunted Castle via the Japanese PS2 release and turned all the lives into health. I didn't find it especially difficult, and it was fun, even if it is an admittedly sloppy game.
All three NES games are classics. People who whine about the "terrible" controls can plug off. It may be the worst of the three, but Simon's Quest is still good.
I actually like Circle of the Moon more than Harmony of Dissonance and Aria of Sorrow. The latter two are just too derivative of SotN. Circle of the Moon has more originality than the other two GBA games, and I like that it puts up something of a challenge.
Legacy of Darkness is the best 3D Castlevania game I've played thus far (and I suspect it'll remain my favorite of the 3D ones). Haven't played the first N64 Castlevania yet.
Dracula XX is no Rondo, but shouldn't be compared to Rondo so much to begin with, since it's basically an entirely different game. It's not among the best of the classic games, but it's still fun.
Castlevania Adventure is straight up good. I'll never understand why people talk as if it's garbage. Yes, you move slow. Big deal. The controls are tight, the level design is solid, the music is great, and the difficulty is healthy but fair and surmountable.
As for the unpopular ones where I agree with the consensus, Castlevania Legends deserves its reputation. I like the characters, story, and music just fine, but the gameplay is junk. The worst is how the game handles spawn points. You scroll that point off screen by one damn pixel and back and there's the enemy again. I hate that kind of nonsense. I'm also pretty meh on Lament of Innocence.