supplierdeeply

Suggestion for improvement of benchmark - gui_bench.c

  • 7 Replies
  • 1488 Views
*

Offline whim

  • ******
  • 2013
  • A495/590/620/630 ixus70/115/220/230/300/870 S95
  • Publish
    Suggestion for improvement of benchmark - gui_bench.c
    « on: 01 / March / 2008, 09:25:40 »
    Advertisements
    Reading in gui_bench.c , I noticed the loop overheads in the benchmark loops were not
    compensated for, so I decided to do that and check if it made much difference.
    Results on my A620:

                            trunk310+mod                trunk310
                            ------------------          ------------------ 
    Screen Write      6923 Kb/s  82 FPS        5346 Kb/s  63 FPS
               Read     27692 Kb/s 328 FPS     14025 Kb/s 166 FPS

    Memory Write         40705 Kb/s              40206 Kb/s
                Read          56987 Kb/s              56013 Kb/s

    In the card writing benchmarks differences were smaller than the 'natural' variation.
    (BTW, the first card bench always rises by ~50% if I run the bench a second time)

    As you can see, the differences are significant, especially for the screen benches, so I
    suggest inclusion in a future build.

    Changes are only in function gui_bench_run(), like this
    Code: (c) [Select]
    static void gui_bench_run() {
        register long t;
        register int i, n;
        register int *buf;
        register char *scr;
        int s;
        int x=0x55;
        char c, c1;
        long ts, tm, td;
     
        scr = vid_get_bitmap_fb();
        s = vid_get_bitmap_width() * vid_get_bitmap_height();

        // first calc bench looping overheads

        t = get_tick_count();
        for (c=0; c<64; ++c)
            for (i=0; i<s; ++i) ; // empty loop
        t = get_tick_count() - t;
        ts = t; // looping overhead screen bench

        t = get_tick_count();
        for (n=0; n<1024; ++n)
            for (i=0; i<0x10000/4; ++i) ; // empty loop
        t = get_tick_count() - t;
        tm = t; // looping overhead mem bench

        t = get_tick_count();
        for (n=0; n<256; ++n) ; // empty loop
        t = get_tick_count() - t;
        td = t; // looping overhead disk bench

        // end overhead calc, now do the bench ...

        bench_to_draw = 2;

        bench.screen_output_bps = 0;
        gui_bench_draw();
        t = get_tick_count();
        for (c=0; c<64; ++c)
            for (i=0; i<s; ++i)
                scr[i] = c;
        t = get_tick_count() - t;
        t -= ts; // subtract looping overhead
        bench.screen_output_bps = s*64*100 / (t/10);
        bench_to_draw = 1;
       
        bench.screen_input_bps = 0;
        gui_bench_draw();
        scr = vid_get_viewport_fb();
        s = vid_get_bitmap_width()*2 * vid_get_viewport_height();
        t = get_tick_count();
        // using same looping, char i.o. register int ...
        for (c1=0; c1<64; ++c1)
            for (i=0; i<s; ++i)
                c = scr[i];
        t = get_tick_count() - t;
        // .. so we can reuse 'ts'
        t -= ts;
        bench.screen_input_bps = s*64*100 / (t/10);
        bench_to_draw = 2;

        buf = malloc(0x10000);
        if (buf) {
            bench.memory_write_bps = 0;
            gui_bench_draw();
            t = get_tick_count();
            for (n=0; n<1024; ++n)
                for (i=0; i<0x10000/4; ++i)
                    buf[i] = x;
            t = get_tick_count() - t;
            t -= tm; // subtract looping overhead
            bench.memory_write_bps = 0x10000*100 / (t/10) * 1024;
            bench_to_draw = 2;

            bench.memory_read_bps = 0;
            gui_bench_draw();
            t = get_tick_count();
            for (n=0; n<1024; ++n)
                for (i=0; i<0x10000/4; ++i)
                    x = buf[i];
            t = get_tick_count() - t;
            t -= tm; // subtract looping overhead
            bench.memory_read_bps = 0x10000*100 / (t/10) * 1024;
            bench_to_draw = 2;
        }

        x = open("A/BENCH.TMP", O_WRONLY|O_CREAT, 0777);
        if (x>=0) {
            bench.disk_write_raw_bps = 0;
            gui_bench_draw();
            t = get_tick_count();
            s=write(x, hook_raw_image_addr(), hook_raw_size());
            t = get_tick_count() - t;
            close(x);
            bench.disk_write_raw_bps = s*100 / (t/10);
            bench_to_draw = 2;
        }

        x = open("A/BENCH.TMP", O_WRONLY|O_CREAT, 0777);
        if (x>=0) {
            bench.disk_write_mem_bps = 0;
            gui_bench_draw();
            t = get_tick_count();
            s=write(x, (void*)0x10000, 0xC00000);
            t = get_tick_count() - t;
            close(x);
            bench.disk_write_mem_bps = s*100 / (t/10);
            bench_to_draw = 2;
        }

        if (buf) {
            x = open("A/BENCH.TMP", O_WRONLY|O_CREAT, 0777);
            if (x>=0) {
                bench.disk_write_buf_bps = 0;
                gui_bench_draw();
                s = 0;
                t = get_tick_count();
                for (n=0; n<256; ++n)
                    s+=write(x, buf, 0x10000);
                t = get_tick_count() - t;
                t -= td; // subtract looping overhead
                close(x);
                bench.disk_write_buf_bps = s*100 / (t/10);
                bench_to_draw = 2;
            }

            x = open("A/BENCH.TMP", O_RDONLY, 0777);
            if (x>=0) {
                bench.disk_read_buf_bps = 0;
                gui_bench_draw();
                s = 0;
                t = get_tick_count();
                for (n=0; n<256; ++n)
                    s+=read(x, buf, 0x10000);
                t = get_tick_count() - t;
                t -= td; // subtract looping overhead
                close(x);
                bench.disk_read_buf_bps = s*100 / (t/10);
                bench_to_draw = 2;
            }
            free(buf);
        }
        remove("A/BENCH.TMP");
           
        gui_bench_draw();
    }


    wim

    *

    Offline GrAnd

    • ****
    • 916
    • [A610, S3IS]
      • CHDK
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #1 on: 01 / March / 2008, 10:22:58 »
    Thank you for the deep investigation.
    Originally, benchmarks were written to have a way to compare different cameras/SD-cards. Therefore, the absolute values were not calibrated properly. And actually, screen and memory tests, which are the most inaccurate, are not so interesting. For disk tests, there is no significant difference.
    But I think, we should not modify benchmark tests for backward compatibility reason.
    CHDK Developer.

    *

    Offline adcz

    • ***
    • 151
    • Canon S5 | Before Thou Smiteth Me, Readeth Sig
      • OSFlower
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #2 on: 07 / June / 2008, 16:06:54 »
    BTW, guys, let's all post our benchmarks for our cameras! I wanna see how my compares <<
    For my S5 (I also posted this on Wikipedia):
    Benchmarks by the benchmark feature:

    [edit] Screen

        * Write: 5346 Kb/s 63 FPS (I lost to whim's 620 w/mod but tied with normal :))
        * Read: 13965 Kb/s 55 FPS (lost again :()

    [edit] Memory

        * Write: 39960 Kb/s (lost again...ARG!)
        * Read: 56496 Kb/s (sorta tied)

    [edit] Flash Card

    This will depend on the type of card you use ( tested with a 4 gig Class 6 )

        * Write (RAM): 6688 Kb/s
        * Write (Mem): 6237 Kb/s
        * Write (64k): 4762 Kb/s
        * Read (64k): 7185 Kb/s

    Hmm...though the S5 would be more powerful then a tiny 620 with a Digic II processor (Canon promised that performance improved on Digic III D:)

    Btw, what is this mod you speak of O.o
    Quote
    trunk310+mod 
    Does it make the cam faster?! ADZ wants a fast cam :P

    *

    Offline PhyrePhoX

    • *****
    • 2253
    • make RAW not WAR
      • PhyreWorX
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #3 on: 07 / June / 2008, 16:18:43 »
    uhm, this thread was not about the actual results of benchmarks. please don't hijack threads like these. you could have easily opened a new thread. it seems i'm constantly bashing you, i'm sorry, but it all boils down to the fact that a) you are the new guy and you dont seem to have read the wiki and the forum enough and b) i'm the evil forum moderator :D

    p.s: there is a huge wiki page dedicated to benchmarks, it can be found here: Benchmarks - CHDK Wiki there you should post your results.
    Quote
    For my S5 (I also posted this on Wikipedia):
    - what page do you mean by that, somewhere in the wiki (wikia) or really somewhere in the wikipedia?

    and no, you cant make your cam any faster... the "mod" can be found in the first post, the "quote" that actually is sourcecode.


    *

    Offline adcz

    • ***
    • 151
    • Canon S5 | Before Thou Smiteth Me, Readeth Sig
      • OSFlower
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #4 on: 07 / June / 2008, 16:35:58 »
    uhm, this thread was not about the actual results of benchmarks. please don't hijack threads like these. you could have easily opened a new thread. it seems i'm constantly bashing you, i'm sorry, but it all boils down to the fact that a) you are the new guy and you dont seem to have read the wiki and the forum enough and b) i'm the evil forum moderator :D
    It's alright :) Better than secretly hating me anyways :P About the threads, I though before you said that I am opening up too many new threads (so I stopped). I just searched "Benchmarks" and this one seemed most related. So I posted here.

    Quote
    p.s: there is a huge wiki page dedicated to benchmarks, it can be found here: Benchmarks - CHDK Wiki there you should post your results.
    Quote
    For my S5 (I also posted this on Wikipedia):
    - what page do you mean by that, somewhere in the wiki (wikia) or really somewhere in the wikipedia?
    Ooooh! Cool! Thanks :)  And I posed the benchmarks on the Canon S5 Wikipedia page.

    Quote
    and no, you cant make your cam any faster... the "mod" can be found in the first post, the "quote" that actually is sourcecode.
    Aww :( Oh well...it's not like I am going to playing Crysis on it :P

    *

    Offline DataGhost

    • ****
    • 314
    • EOS 40D, S5IS
      • DataGhost.com
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #5 on: 07 / June / 2008, 16:46:32 »
    This is again one of those nice situations where you generalize your case to *everyone*. The benchmark is mainly to compare different memory cards in cameras, because memory cards to make a big difference in speeds. The benchmark results for one specific memory card aren't particularly useful as they are probably very different for other cards and filesystems.

    Apart from that, I still have to investigate some issues (or someone else) but they are low on my priority list, so I haven't really looked at them. The main concern is cache... As you can see in this image showing some benchmark results, the measured speeds change drastically when changing the ORDER of the tests. This suggests the benchmark is actually quite flawed, unless I made a mistake somewhere.

    *

    Offline adcz

    • ***
    • 151
    • Canon S5 | Before Thou Smiteth Me, Readeth Sig
      • OSFlower
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #6 on: 07 / June / 2008, 16:49:49 »
    This is again one of those nice situations where you generalize your case to *everyone*. The benchmark is mainly to compare different memory cards in cameras, because memory cards to make a big difference in speeds. The benchmark results for one specific memory card aren't particularly useful as they are probably very different for other cards and filesystems.
    But screen/memory write should be the same (they're not supposed to access the card at all are they??)

    Quote
    Apart from that, I still have to investigate some issues (or someone else) but they are low on my priority list, so I haven't really looked at them. The main concern is cache... As you can see in this image showing some benchmark results, the measured speeds change drastically when changing the ORDER of the tests. This suggests the benchmark is actually quite flawed, unless I made a mistake somewhere.
    OK :) I can't wait till the latest benchmarking tool comes out. My S5 waits for redemption :)

    *

    Offline PhyrePhoX

    • *****
    • 2253
    • make RAW not WAR
      • PhyreWorX
  • Publish
    Re: Suggestion for improvement of benchmark - gui_bench.c
    « Reply #7 on: 07 / June / 2008, 16:53:29 »
    no, you are not opening too many threads. you have to make a distinction here: you open too many threads that contain information that already exists, this - again - is directly in relation to you being new and not having read anough in the forum here.
    post as many threads as you want, as long as you try to post NEW information (after searching first to avoid posting of doublettes).
    i don't hate you, why should i? it is just my job "teaching you some manners", forum-technically speaking. i'm pretty sure you will turn out a fine "chdk-aficionado" that can channel his energy in a helpful manner :)
    p.s.: moved this part of the thread to a new one.

    edit: haha, did NOT move thread somewhere else, as DG posted here something ONTOPIC :)


     

    Related Topics