He's been asleep for like, what, 1000 years? Not even in the castle itself, but in some church. And when he gets back out he's all crusty and old and depowered. Castle has a life of it's own, it was made through a pact with a demon, and probably is a demon itself. probably considers him weak now, or unfit to be the master. Dracula rules through fear. He is extremely strong, and everyone knows it. Even the Castle. Considering you have to regain your lost powers, the Castle is not afraid of you, and thinks you are a weakling now, even if the subordinates themselves still respect you.
There's also the story elements we don't know about yet. Satan's involved again, so his influence could be relevant there.
TL;DR, if you watched the video you would have heard, the Castle-posessed monster(Is it a Vampire warrior?) says "You're not our lord anymore"