Just got back from seeing this:
Admittedly, except for the first Iron Man, I haven't seen any of the other films leading up to this, though watching this, I didn't feel like there was a lot that would absolutely confuse someone who hadn't really seen many of the precursors to this, so don't bee too discouraged in you're going in blind.
Though I thought the action was good, the dialogue was the real highlight of the film for me (It reminded me of Cabin in the Woods, also written by Whedon), and both of those films really made me laugh.
All of the main performances were good, and I felt the in-universe creative differences amongst the characters did not feel forced, so that really helped IMO.
My only minor gripe is that the villain (Loki), at least to me, just did not seem that threatening. (Though the part where the Hulk smashes him was great)
Overall, I'd give the Avengers a 4 out of 5. Between the last two movies I've seen in theaters, I'd say I still liked Cabin in the Woods more, though that's mainly because, as established, I'm more of a horror person than an action person, though this is still a good popcorn flick.
Also, with the Amazing Spider-Man and Dark Knight Rises later this summer, this may be the first year I end up checking out all of the main superhero flicks released.