Portrait of Ruin has to be it, as there is so much in that game that made me so sad.
1. The Storyline. Or rather, how the storyline was presented. The 'anime' fashion in which it was told killed what could have been pretty dramatic and deep. (The little bit in sister's mode could have just been presented in a flashback or something during the game.)
2. The bosses where too easy, though I still love the final boss. (Until anime witch #49 said something about how their team work was great however, failed in their attempts. Everytime she spoke, I wanted her to die.)
3. Good lord, I hated the artwork. (And I actually like anime if done well.)
4. The rooms kinda got boring and the second set of portraits where a let down. Kinda felt ripped off.
5. I liked the idea of switching between to characters and such but I felt like it could have been put to better use in the game than it was.