In my oppinion the animators don't have to follow the manga strictly, Manga and anime aren't the same. What I'm trying to say is that an anime just can't follow the manga strictly due to various reasons (eg the storyline becoming too long or too short). I do think animes should follow the mangas concept and feel though, eg a serious manga turning into some unserious comedy is totaly out of place. The best, imho, would be the manga artist working together with the animators to ensure a similiar feeling.

Also fillers are not necessarily a bad thing as long as they dont strive to far away from the original concept. (I hate unserious fillers)