In my experience, these games explicitly trade away the benefits of trust-based gaming with familiar partners (like many of the sublime moments of brilliance that arise purely as a reflection of creative individuals riffing off of each other in the pursuit of art) for a rules system that provides a mediocre but functional simulation of familiarity. Because these games do what it says on the tin you can weed out people with unreasonably selfish demands upfront and subsume reasonable demands without having to stop and address each one (because the rules just tell you your demand isn’t met and you move on.)

I, myself, find medium-to-long-term play with friends (i.e. individuals I trust creatively and emotionally) significantly more interesting and therefore find that ultra-restrictive games are less useful than broader games that leave room for riffing.

But narrative focus should be driven by the interests of the characters in my opinion and thus having a character decide to do something wacky that could push a carefully constructed story off the rails is just something you have to roll with as a GM. To be sure, the consequences of ignoring what to you as the GM was the major focus of the game should be felt, but if players get hooked into a different aspect of your game world, then kudos for designing or using a game world that is interesting enough to get them into different facets of it, not just the aspect that’s thrown as the players as the “narrative focus”.

To your comment about rulesets, I agree to an extent. You use the ruleset that works best for the forms of games that your group wants to play, but I’d argue that if you’re using a ruleset that is so inflexible that you can’t do something with it to surprise your characters then that’s a shame. And if your group is so petty that if you as the GM “break” the rules to allow for elements of narrative and they get upset because of that, then you need a better group. Hell, 4E D&D codified mook rules (“minions”) in part so that you could have battles against lots of mooks but also so that if the DM wants to dramatically kill off some NPC character, the players can justify it as “Oh, he must have been like a minion”, likely because someone had the same scenario in 2E D&D and their players said “No way! He must have had at least 20 hit points! There’s no way a dagger could have killed him!”. Anyway, now I’m rambling.

The indie community often talks about how Golden-Ruling, multiple agendas, fiat and such are *the* sources of pain. But if you actually observe poorly functioning games in the wild, that’s not what’s going on. Instead, you have people working very hard to *adhere* to procedures in the text (rules-lawyering) and mostly stick to a common agenda, but they have little true empathy for one another. This is what leads to shouting matches in teenage D&D games. When I’ve observed certain indie games in action (though not all of them end up like this) at conventions, the shouting is replaced by stony silence and joyless, perfunctory decision making.

To continue with your example – if I’d gone out to see Avatar expecting it to have the narrative quality of, say, The Word For World Is Forest, of course I would have been disappointed. It’s a fucking action flick and a two-hours effects demo, and on that basis it is untouchable. I went in with reasonable expectations and I came out satisfied.

You pick the set of surprises you feel comfortable with beforehand. If you go out to see a movie, or you read a book, you come at it with certain expectation of what it will and will not provide.