I'd say the reason retro franchises generally had some variety in their sequels because of a few reasons:
1.) They didn't want to offer something so similar that consumers would go "I just have the first one, why bother?"
2.) They want to experiment with what else they can do with their set up, and since the original was successful, it'll at least sell modestly abd they get to see if fans like it even better, to see a direction to take for the future
3.) Reach more people that may have avoided the previous game due to uninterest in genre. Add a few elements they like, BOOM you have two possible markets
4.) Get more attention; Something straying from the formula will attract some poeple like "Really?", and if someone reads about RPGs, they'll see something like Zelda 2, they read about adventure games and platformers, they see Zelda 2 pop up again.
5.) Make a game look like it has more, make it look more ambitious, make it more packed, like Castlevania combining it's gameplay with RPG elements to make Simon's Quest