Apart from the change to PTP_CHDK_CallFunction I'm not sure I really see much difference between the two approaches. One puts the code in Lua although this could be refined (see below), the other puts it in PTP - does the distinction really make a difference?
The distinction is the Lua part is completely artificial. There is no use for it in Lua, it is, as mweerden said, just a way of calling a function by name, so you can pretend that the protocol hasn't changed. This is not helpful, because anything that actually *uses* it will be affected by changes just the same as a protocol change. It just makes the whole thing more opaque and complicated.
The compatibility problems are the same, you are just making them less centrally documented. Imagine a third party wants to implement live view (gphoto actually picked up ewavrs old code, so this isn't just theoretical. Hopefully once we've got this stabilized we can get them to support the trunk version instead) If it's in the protocol, then they only have to update when the protocol changes, which is visible, centralized and has a well defined way to check protocol compatibility. If it's defined elsewhere, then (as you discuss with mweerden in the following posts) you have to add versioning to that elsewhere, which makes things more complicated.
That's why I'm arguing for putting it in the protocol. Once you get the protocol right it shouldn't need to change too often.
To me that is (almost) like saying HTTP is part of TCP.
Right, if you want to define another protocol for live view, that runs on top of CHDK PTP, but IMO that's not warranted.
that's also why I wouldn't be a fan of your monolithic PTP_CHDK_GetStreamingData. It only takes one of the flags to change in meaning to destroy backwards compatibility.
Then don't change the flags unless you really need to, deprecate them, add new ones, and only drop them when you are forced to make a protocol change for other reasons. If you use up all 128 bits of params to PTP_CHDK_GetStreamingData, you can always add PTP_CHDK_GetStreamingDataEx if you really want to

The same thing happens if change the flags in your system, it just less clearly defined.
Centralizing other things that a client would also want to poll also has real value. If getting the palette and frame buffer separately hurts frame rate, then polling script status and live view will do the same.
There are other ways you could set it up rather than using flags, but honestly I don't see us using more than a couple ints worth.