Common logging module - LUA Scripting - CHDK Forum

Common logging module

  • 5 Replies
  • 5083 Views
*

Offline c_joerg

  • *****
  • 1248
Common logging module
« on: 11 / April / 2018, 08:20:05 »
Advertisements
I also wanted to understand more from lua scripts. First, I wanted to deal with the data logging. I copied the most important things from rawopint. I hope that I was allowed to do so.

What I'm wondering, should not you put these functions into a generic module so everyone could use it?
M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd

*

Offline reyalp

  • ******
  • 14080
Re: Common logging module
« Reply #1 on: 12 / April / 2018, 00:17:46 »
I also wanted to understand more from lua scripts. First, I wanted to deal with the data logging. I copied the most important things from rawopint. I hope that I was allowed to do so.
Yes, you are welcome to. I released under GPL, so technically a script that includes it should also be GPL, but this isn't something I'm going to worry about.
Quote
What I'm wondering, should not you put these functions into a generic module so everyone could use it?
That was my original plan (and I may still get around to it someday), but I found putting all the "modules" inline in the script makes it easier to distribute and avoids compatibility issues. The down side is copy/pasting it into multiple scripts if I update.

Some general information on the log module. c_joerg probably already knows all this but maybe useful for anyone else reading:
The idea is to generate a csv with one row per short, but be able to record values wherever they are naturally available in code.

The columns and some other options are defined using log:init.

Values are recorded with log:set{colname=value,...}
Unset values are just logged as empty cells, so there's no harm if something isn't set on a particular code path.
If a value is set more than once before the row is written, the last value is used.

log:set takes care of things like exposure values nicely, but it's also useful to log some free-form text for diagnostic messages. This is done with logdesc, which acts like printf and puts the result in the desc column delimited /. logdesc isn't strictly part of the log module, log:text_logger is used to create functions like this bound to a particular column.

The values are stored in a lua table until log:write() is called.

One thing to watch out for is that the module current doesn't escape values written to the log, so values should not include , or "

There are two special column types: function and table, defined in init using the 'funcs' and 'tables' sections.
function columns get their value from a function at when a row is written. This is useful for things like temperature or free memory.

Table columns append values to table each time set is called. When write is called, all the values are output to the cell, delimited by the delimiter defined in init. So far I've only used this for the desc column.
Don't forget what the H stands for.

*

Offline c_joerg

  • *****
  • 1248
Re: Common logging module
« Reply #2 on: 13 / April / 2018, 04:25:23 »
One thing to watch out for is that the module current doesn't escape values written to the log, so values should not include , or "
Thanks for the hint and the detailed description.

I also find it interesting because the whole thing works on my PC in Notepad ++ in the CHDK LUA local.
M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd

*

Offline reyalp

  • ******
  • 14080
Re: Common logging module
« Reply #3 on: 13 / April / 2018, 15:51:04 »
I also find it interesting because the whole thing works on my PC in Notepad ++ in the CHDK LUA local.
I've tried to keep the "modules" in rawopint, fixedint etc as isolated as possible. log is probably the only one that doesn't depend on CHDK functions, but the others should be relatively easy to re-use in CHDK scripts.

Another thing I try to do in those scripts is isolate the menu values from main script logic. They often need adjusting (i.e. table values to apex96, seconds to ms etc), so it's nice to have a clearly defined separation even if it bloats the script a bit. My convention is to always prefix the menu values with ui_ and not use them directly in the modules
Don't forget what the H stands for.


*

Offline c_joerg

  • *****
  • 1248
Re: Common logging module
« Reply #4 on: 14 / April / 2018, 01:54:09 »
I've tried to keep the "modules" in rawopint, fixedint etc as isolated as possible. log is probably the only one that doesn't depend on CHDK functions, but the others should be relatively easy to re-use in CHDK scripts.
Once you have dealt with it then you can understand it quite well.
The advantage with Notepad ++ in the CHDK LUA local for me is that I can also use it when I am traveling and have no chance to connect a camera.
Another thing I try to do in those scripts is isolate the menu values from main script logic.
I like the way you program. Everything is very structured and I try to keep a little bit to it. 

I am not sure if we have already talked about the following problem.
I'm trying a script right now. After each recording, I make a log: write (). Sometimes the camera crashes. But if I took 10 shots and then crashes the camera, then there is always only the first line in the log file.
I work in parallel with print and print_screen(get_day_seconds()/10). With print, however, all information is always up to the crash in the file.
M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd

*

Offline reyalp

  • ******
  • 14080
Re: Common logging module
« Reply #5 on: 14 / April / 2018, 03:50:04 »
I'm trying a script right now. After each recording, I make a log: write (). Sometimes the camera crashes. But if I took 10 shots and then crashes the camera, then there is always only the first line in the log file.
By default, the log is buffered. log:write() calls the camera fwrite, without closing the file or flushing.

if you use buffer_mode='sync' the init options, it will open and close the file for each write.
Don't forget what the H stands for.

 

Related Topics