Nostalgia has nothing to do with it.
CV4 brought real innovations to game play. Many of which had never been done before in a game much less a Castlevania.
Rondo brought a map system, alternate character and anime cinemas. Dracula's Curse brought alternate paths and characters. Bloodlines borrowed from CV4 and Dracula's curse. None of these games expanded on the way you actually play the game like CV4 did.
CV4 wins in innovation. It wins in graphics IMO. And it wins in music IMO. And the presentation of the game is incredible on top of everything else.
Nostalgia. Right.
The keyword being IMO. The only expansion of gameplay CV4 brought was multi-direcional whipping (which made the game easy as fuck, because it's overpowered) and better air control/jump on stairs.
I'm actually glad that multi-directional whipping was dropped for subsequent games. Make the subweapons relevant while trying to design the stages again, instead of using the whip to kill EVERYTHING, even bosses.
It was in some ways a downgrade from CVIII. It had some cool stages here and there, but I think it's one of the easiest classic CVs (it's not a complete cakewalk, though).
It was the first CV game of many people. Nostalgia IS a factor when you're trying to choose your favourite game.