SFA 3 Max aka SF Alpha 3 Double Upper - more characters added from regular Upper (which is hidden on the PS2 Anthology and has to be unlocked, and then is still hidden unless you hold select over Alpha 3 while pressing start). Yes the PSP is ass with fighting games, but this is the best fighter you can get on a handheld as far as I'm concerned.
Capcom Puzzle World - 3 puzzle games on this, Buster Bros, Buster Buddies, and Super Puzzle Fighter II Turbo, with extra modes found only in the Dreamcast version (2 other variants of Puzzle Fighter modes, one like Tetris, another like Columns).
Powerstone Collection - if you like those games, you'll love this. Wish I had picked up all 3 I just listed when I saw them, regrets out the wahoo now as I don't see them anymore locally.
Gradius Collection/Salamander Portable/Parodius Portable/Twinbee Portable - all 4 of these are neat Konami shmup collections, though the Paro games infuriated me enough to get the console versions. Songs were replaced (at least 6 over the 5 games on Paro Portable) with lamer songs, still the shooting is solid.
Gokumakaimura Kai - yes you have UGnG, this is the second revision that does away with the fetch quest, brings you back to 2 hits and dead, has 2 quests (UGnG sorta had that too), and few checkpoints. If you die, you go back to a checkpoint ala UGnG's Ultimate Mode. Continues are also limited, some say though lives are plentiful to get.
Darkstalkers Chaos Tower - yes fighting games and ass again, and this is one I don't own either (would love to), this is basically I believe a revamp of Vampire Savior/VS2. I think almost every character is in too, but I could be wrong.
Darius Burst - this one never came to the US despite getting a ERSB rating. It's the latest in the Darius title, but sadly it's going price is still around, or a little higher than when it came out a few years ago. Taito didn't disappoint with this one either.