Simon's Quest is the only other Castlevania game besides IV that I played when I was really little. It wasn't until Castlevania for the N64 that I got really crazy into the series. Dunno why, since I'd played Symphony of the Night like a million times since it came out, and I certainly love(d) that game a lot more than CV64. Yet, CV64 is the game that made me decide to play beyond SOTN and CV4.
About Simon's Quest... I barely played it, and I was maybe around five or six at the time. Maybe I was even younger and had actually played it before CV4, I'm not sure. A friend of my brother's had brought it over and I got to watch. Tried playing a bit, but they had gotten rather far in the game, so it was a lot tougher than the beginning and I kept dying at those parts. It was loads of fun, though, just experiencing it, despite all the dying.
Eventually they made it to Dracula, and I thought it was epic beyond all get-out watching Simon walk through those huge gray hallways with that great music playing, and then how freaky and massive Dracula looked (and, again, the great music). Totally blew my mind. BLEW MY MIND!