I never really approached any Castlevania game with dislike (so there are quite a few "games I liked at first but soon grew to dislike") but Castlevania 64 was highly dissatisfying and awkward the first few times, which encompassed only the first few levels.
There was no music (blasphemy in the Castlevania world) and when there was it was overly simple and unmelodic (equal or worse blasphemy), the camera was mentally challenged, the platforming was tedious rather than fun, Bloody Tears was present for all of about 5 seconds (which is not only blasphemy but a giant tease), and the 3D was sometimes disorienting and confusing.
Then I discovered the Villa and would have been elated if the whole rest of the game took place in a larger version of that stage. Then I discovered the Castle Center and felt the designers were getting a LOT closer to "what Castlevania should be", again finding myself wanting that stage to be the rest of the game. Over time, I grew to accept the faults and differences and just enjoy the game - but not without feeling it wasn't a Castlevania game.
A few years later, I was introducing Castlevania to my then-girlfriend game by game, and of course we eventually got around to 64. This time through, after pointing out some of my issues with it, I experienced not only some nostalgia for the better parts but also a sense of the game through the eyes of a Castlevania fan and a horror movie buff (those being hers) and really, really enjoyed myself.
A few years later, I lost my N64, due to not labeling my "going to storage" box as such and having it get thrown out instead - along with several other systems. I wasn't incredibly worried since most of my games were on the Gamecube or Virtual Console...but then I remembered the Castlevania games and downloaded some emulation stuff for the PC. For the first stage or two, I had some kind of schoolgirlish glee over having my lost games reproduced so beautifully without the real hardware. The glee managed to go on for a while, at which point I realized I was 5 stages in and the emulation accuracy novelty had worn off - I was actually enjoying the game A LOT. More than I even thought I would. Part of it was nostalgia, part was the accuracy, part was the horror-movie-esque vibe, part was that I'd recently experienced Curse of Darkness for the first time and was glad to not be playing that again, and part was knowing that even years later, the game was still one of a kind and would probably never be duplicated with spiritual successors or even released on Virtual Console - this was it, the one and only. And I was damn proud to have experienced it when it was current, in its 1999 context, and without any expectations built up from years of MetroidVanias and series plot messes - in 1999, I'd had zero exposure to the online fan community and wouldn't even play Symphony of the Night for two or three more years yet. My last Castlevania was Dracula XX. Point being, the nostalgia I felt emulating wasn't just a "glimpse into the past" "retro" kind of thing, or nostalgia for that period in my life, but a "this thing will only ever happen once and I was there when it happened" feeling.
Now it's one of my favorite Castlevania games and I'm of the opinion that it was the only 3D 'vania to "get it right", in spite of its camera with a single-digit IQ. I've found I even prefer 64 to LoD - not just because of level layouts or music versions, but right down to the inferior graphics and clunkier camera. There's just something about that specific combination of elements that brings me all of the above feelings where even Reinhardt's quest in LoD just doesn't as much. I actually bought LoS on Day 1 (soundtrack CD and all) because Cox's advertising gave the impression it was essentially Castlevania 64 with modern graphics and much more classic Castlevania feel; then I played LoS for half an hour and felt thoroughly let down and betrayed because it -wasn't- like 64. At all. (Bear in mind I'd also just had my first experience with God of War not even 48 hours before my first experience with LoS, driving the betrayal deeper.) It's been said that LoS "alienated" old fans - compared to the expectation of a new CV64, LoS felt -extremely- alien to me. But I digress.
That, I'd say, is basically the extreme of "dismissed at first but grew to like" - I went from being among the "this isn't a Castlevania game" crowd to "this is the best and only true 3D Castlevania" over the course of 10 years.