A pity, but it makes sense since IM2 seemed to be teetering on Spider-Man 3 territory with the sheer amount of stuff going on in there. That perhaps hints at a lack of control for Favreau, although it's to his credit that he pulled a salvageable film out of that many plotlines. In fact, I think there was more going on in IM2 than Spidey 3; I count Vanko, Fury, Tony's Dad, Tony's Palladium Poisoning, Hammer, Rhodey, Pepper's subplot, Making a New Element, and the Black Widow, at least. That's an awful lot of 'main' characters and plots to juggle, and it still came out pretty darn coherent.
That said, I think it's wise that he maybe backs out now if he's a little dubious. The Marvel Movieverse they're generating has been pretty cute so far, with the nods and stuff to other characters and whatever, but it is a very dangerous game Marvel Studios is playing. By tying everything together, one film hedges its bets on another, and you run the risk of people not seeing one film and being confused by another. If one film flops, does that damage the entire line? Having to tie characters down in their own films to make up the continuity of the main Avengers team films, possibly, also strikes me as a little dubious. So if Favreau thinks that Marvel doesn't quite know how to pull this together, it's probably best that he back out now rather than be tied down to a film where he doesn't have much to do.