I noticed a few things from looking at the pictures and the log file above. First, the exposure didn't change on the second shot. The reason is that the 2nd shot happened immediately, with no delay. The Lua script didn't calculate and store the adjusted tv value until the 2nd shot was already in progress. This is probably also happening on the shots where the shutter time exceeded the interval time, and the delay goes to 0. The exposure changes were so small, it's not possible to tell, but I'm pretty sure it was happening all the time. That means I would be 2 shots behind in setting exposure instead of one.
One reason is that the shot_number was being incremented AFTER the delay, instead of between the call to build_histogram and the start of the delay loop. shot_number++ signals the Lua script that the shot_meters are valid, so it can set the new TV values. But at this high shooting speed, it's a race between the camera and the script, and the script was losing. With the shot_number++ set up this way, the problem was probably happening after every shot.
I'll start by adding a minimum 20 msec delay AFTER shot_number++, and before returning and letting the camera shoot the next picture. If that's not enough time, I can add a longer delay that can be aborted by the Lua script after it sets the new TV values.
=========
I also figured out what's happening when going from light to dark areas rapidly, like taking a picture of a bright monitor at night, and then at a dark area of the room. Going from bright picture to dark picture is no problem. The shot_meter correctly estimates the exposure change needed. The first shot is dark, and the next one is exposed properly. With a really dark area, there might be a 3rd shot needed to zero in on the correct exposure, but usually not.
Going from dark to bright is a different story. The first shot of the computer monitor taken with the dark exposure is totally white, all pixels blown, as expected. To the shot meter, totally white is only 1 f/stop above the 50% exposure level. I return 2 f/stops below that, so the maximum correction the meter will make downward from a totally blown white shot is 3 f/stops. It works, but it takes 3 or 4 pictures, 3 f/stops less each picture, to get the right exposure. It's interesting to watch the details appear out of white picture a little more each shot, but it's not the desired result.
I need to add an extra correction based on how far above 50% the linear pixel average is. A completely blown picture might need 7 or 8 f/stops less exposure, instead of the 3 maximum. I can probably design an experimental lua script that collect this data systematically, and computes an adjustment factor for the various over exposure levels. I guess I'll have to take a few thousand more pictures of my blank wall! Fun.