How to average a 3D array into a 2D array in the fastest way?

[Kevin Price]

Not sure I'm entirely clear on what you're dealing with.
I'm going to suppose that your X,Y, and Z are just integer indices into a 3D matrix, and the value found at (X,Y,Z) is your measurement.  For conversation's sake, I'll imagine this as a cubic grid of temperature probes in a fluid of interest.
If I'm on the right track, then each value of index Z represents a different X-Y plane.  It sounds like you want to average over Z, i.e., find an average X-Y plane.
If so, then at least half the battle is making sure you build your 3D matrix properly, so a single auto-indexed For loop will index on the Z dimension, extracting each of the XY planes.  You can then easily use a Shift Register to sum up your X-Y planes and then divide by count after the loop for an average.
Then again, maybe that's not even close to what you're after.  If not, can you post again with more detail?
-Kevin P.

[Kevin Price]

***@CARYA wrote:Another tactic might be to flatten the 3d array to one 1d array. If you do this correct (depending on how the 3d array is build up)...It might be faster, but the only way to find out is to try it out.

I *think* that the 3d build-up that makes Wiebe's idea work would be different than the one that makes auto-indexing work.  A speed trial (including any gyrations required to perform each type of build-up) would be a good idea.   That said, I expect his other ideas might well prove to be faster yet.  Here they are again for emphasis:


Have you thought about avoiding the 3d array? Or averaging the data each time you add a sample (by keeping the last average and the number of samples, you can add a new sample to the average without averaging all the data)?

-Kevin P.

[wiebe@CARYA]

 
Hi,
 
This VI runs in about 0.11 - 0.15 sec. on my ancient 800 MHz PII. Should do much better on your PC (hopefully you'll have a 3 GHz dual core for this fancy graphics stuff).
 
The indices of all arrays can be rearanged, without speed penalty, to suit your needs.
 
Regards,
 
Wiebe.


3D Array Average.vi:
http://forums.ni.com/attachments/ni/170/212350/1/3D Array Average.vi

[Matt W]

If speed is still an issue here's a faster/simpler version of what wiebe posted. It's about 3 times faster for me on Labview 8.2.<img src="Loading Image..."> Message Edited by Matt W on 10-27-2006 03:16 PMMessage Edited by Matt W on 10-27-2006 03:16 PM


3D Array Average 2.vi:
http://forums.ni.com/attachments/ni/170/212698/1/3D Array Average 2.vi


3D Array Average.PNG:
http://forums.ni.com/attachments/ni/170/212698/3/3D Array Average.PNG

[arvmenon]

Fellas, Thank you very much for this awesome post.Arvind.

[altenbach]

All other things being equal, I would go with the solution with the least amount of code. (=mine ;))
&nbsp;
Here are three solutions (Wiebe (slightly modified), Matt, and mine) benchmarked. Mine uses the fewest amount of code and is comparable to Matt's solution. Simple code is easier to debug. :D
&nbsp;
Wiebe: Your benchmark is completely meaningless, because the entire calculation is folded into a constant and calculated at compile time exactly once and then&nbsp;never again. You need at least one of the dimensions as a control. Also, the indicator belongs outside the timing frame, because you don't want to possibly partially measure the updating of the indicator.
&nbsp;


3D_Array_AverageMOD2.vi:
http://forums.ni.com/attachments/ni/170/343942/1/3D_Array_AverageMOD2.vi

[Wiebe@CARYA]

Altenbach,

I'm pretty sure the benchmark was done before LabVIEW did any constant
folding... So, IMHO, it's valid, although doing the benchmark again in LV
8.x would be meaningless (don't tell me you did it anyway?).

Regards,

Wiebe.

[altenbach]

***@CARYA wrote:I'm pretty sure the benchmark was done before LabVIEW did any constantfolding... So, IMHO, it's valid, although doing the benchmark again in LV8.x would be meaningless (don't tell me you did it anyway?).


Constant folding was done well before LabVIEW 8.0 and the extend and details varied a bit between versions. One thing that got introduced in LabVIEW 8.2 was the capability to show where folding occurs (<a href="http://zone.ni.com/devzone/cda/pub/p/id/347" target="_blank">fuzzy wires, fuzzy structures</a>, see image). I have this always enabled, it is of great help analyzing code. ;)
&nbsp;
<img src="Loading Image...">
&nbsp;
I no longer have any pre-8.0 version installed, but it would be easy to check if there is any timing difference between a version with all diagram constants and one with some of them changed to controls.
Using a constant of&nbsp;100 on the outer loop gives 0ms and changed to a control gives about 188ms in LabVIEW 8.2.1.
&nbsp;
There are some weird things between Matts and my version, for example mine slows down if I swap the inputs to the ADD node or add an innocent explicit ToDBL conversion right before the "add"&nbsp;to avoid the coercion dot. This is one of the very&nbsp;few cases I encountered where an implicit conversion is significantly faster than an explicit conversion(!!!).&nbsp;It almost seems that the implicit conversion can avoid the extra buffer allocation for the DBL array right before the add node. (I still need one in the "0" case to enforce DBL type, but that's cheap because it occurs only once).
&nbsp;
Matts solution is 50% slower in some runs for no obvious reason at all. I haven't really studied all that....
&nbsp;
Message Edited by altenbach on 07-25-2008 08:26 AM


Fuzzylicious.png:
http://forums.ni.com/attachments/ni/170/344047/1/Fuzzylicious.png

[Wiebe@CARYA]

Post by altenbach
Constant folding was done well before LabVIEW 8.0 and the extend and

details varied a bit between versions. One thing that got introduced in
LabVIEW 8.2 was the capability to show where folding occurs (<a
href="http://zone.ni.com/devzone/cda/pub/p/id/347" target="_blank">fuzzy
wires, fuzzy structures</a>, see image). I have this always enabled, it is
of great help analyzing code. ;)

I thought it was very primitive before 8. I never noticed it (any speed
increase caused by it)...

Post by altenbach
I no longer have any pre-8.0 version installed, but it would be easy to

check if there is any timing difference between a version with all diagram
constants and one with some of them changed to controls.

I might try that one day...

Post by altenbach
Using a constant of&nbsp;100 on the outer loop gives 0ms and changed to a

control gives about 188ms in LabVIEW 8.2.1.

Post by altenbach
There are some weird things between Matts and my version, for example mine

slows down if I swap the inputs to the ADD node or add an innocent explicit
ToDBL conversion right before the "add"&nbsp;to avoid the coercion dot. This
is one of the very&nbsp;few cases I encountered where an implicit conversion
is significantly faster than an explicit conversion(!!!).&nbsp;It almost
seems that the implicit conversion can avoid the extra buffer allocation for
the DBL array right before the add node. (I still need one in the "0" case
to enforce DBL type, but that's cheap because it occurs only once).

The speed of the explicit conversion weird indeed...

I've also noticed that the order in which the tests are performed matters.
At my PC, the last case is always a bit faster, even if all test execute the
same case (or copies of the same code).


One other thing that amazes me is that avoiding the case doesn't make much
difference. I tried this. Get the first 2D element. To make the entire array
0, I used the growable math function, with a dbl 0, the array, and the
inverted array (0+2d-2d). This results in a 2d array of dbls, with value 0.
The case can go now, and we have a nice array to initialise the shift
register. You would expect that this is faster, at least when the for loop
is executed often enough. But I can't even measure any difference! Probably
the add inside the loop is much much more significant then the case...

Have a good weekend,

Wiebe.

[Wiebe@CARYA]

I'd use your solution as well, Altenbach.

I couldn't resist to give it a few more tries...

The only improvement that makes a difference it this (it saves about 5%).
Don't devide by N, but multiply by 1/N... But it's such a small margin, that
even when using the PerformaceCounter, the noise is almost higher then the
signal and the order of the test cases makes a difference.

Regards,

Wiebe.

[arvmenon]

Hey, I dont mean to butt my head into your little exchange here, but I just have to say that altenbach's code is just ingenious.