Nope, it would have been Whedon saying "Get his ass out of here and give me an actor I can work with, not one who wants to do my damn job!".
Or something to that effect. Besides, I liked the new interpretation of Banner. Finally we have moved completely away from the Bill Bixby TV Series and made the comics version of the Hulk. And it was awesomeness. I don't expect it will happen, but I would love Whedon to do the next Hulk film, with the Leader as the villain.