Re: Trouble with Math::NumberCruncher (fwd)

Hi Nick, 
    I cannot tell you how ashamed I am. This should've been absolutely
    obvious to me (I also did graduate studies in Statistics, after which I
    switched to Economics, and Finance.) I guess it's TGIF feeling, or just
    workload, but my mind didn't perhaps work, when I failed to find this
    simple reason! 

    Thanks very much, 

    -tir



On Fri, 15 Mar 2002 Nick_D_Montpetit@eFunds.com wrote:

> 
> Hi Tirthankar,
> 
> I knew my Graduate work in Statistics would come in handy at some point...
> :-)  Without looking at the documentation for Math:NumberCruncher, I can
> fairly confidently say that both you and Math:NumberCruncher are right, but
> you are trying to calculate different things.  In the following
> explanation, I'll use the statistical language loosely, because you'll want
> to consult a statistics textbook (or possibly the documentation for
> Math:NumberCruncher) for the details.
> 
> If the numbers 0..20 represent a sample of 21 independent random variables
> from a given population (or distribution), the estimated variance is indeed
> 38.5.  Keep in mind that this is an estimate of the true variance, which
> presumably you don't know because you just have a sample of all possible
> values.
> 
> If the numbers 0..20 represent every possible value in the population, and
> each value is equally likely, then the actual variance - which can be
> calculated because you know each value and its probability - is 36.6666666,
> and this is the value returned by Math:NumberCruncher.
> 
> So one answer is an (unbiased) estimate of the true variance and the other
> is a calculation of the true variance, and which one you use depends on
> whether you have a sample or the entire population.  In most cases (where
> samples are involved) you'll want to use the first calculation.
> 
> This relationship is true: (n-1)/n*(sample variance estimate) = (population
> variance calculation) so that
> 
> (20/21)* 38.5 = 36.66667
> 
> Hope this helps,
> -Nick
> 
> 
>                                                                                                                    
>                     "Tirthankar                                                                                    
>                     C. Patnaik"          To:     beginners@perl.org                                                
>                     <tir@igidr.ac        cc:                                                                       
>                     .in>                 Subject:     Trouble with Math::NumberCruncher                            
>                                                                                                                    
>                     03/15/02                                                                                       
>                     06:23 AM                                                                                       
>                                                                                                                    
>                                                                                                                    
> 
> 
> 
> This might seem trivial, but it has me stumped.
> 
> The variance of numbers 0..20 is 38.5.
> Then why does Math:NumberCruncher show it as 36.6667?
> 
> Here's a sample prog, and it's output. Here test04.dat is a file that has
> numbers 0 to 20, on separate lines.
> #-------------------start----------------
> #!/usr/bin/perl -w
> use Math::NumberCruncher;
> # Program  to test Math::NumberCruncher
> 
> # Get a file from STDIN, print the variance, and the std. deviation.
> 
> @array = <>;
> map {chomp} @array;
> 
> $variance = Math::NumberCruncher::Variance(\@array);
> $stddev = Math::NumberCruncher::StandardDeviation(\@array);
> $sd2 = $stddev**2;
> 
> print scalar(@array), "\n";
> 
> printf "Var=$variance, S.D=$stddev S.D^2 = $sd2\n";
> 
> exit;
> #----------------end---------------------
> 
> And the output is:
> 
> 21
> Var=36.6666666666667, S.D=6.05530070819498 S.D^2 = 36.6666666666667
> 
> 
> What could be possibly wrong? Even the standard value is wrong.
> 
> 
> TIA,
> -tir
> 
> 
> --
>  Tirthankar, IGIDR.
>  +91-22-8400919 x275 (r), x593(o), x542(CFL).
>  http://www.igidr.ac.in/~tir
> 
>   LEGACY, n.  A gift from one who is legging it out of this vale of
>   tears.
> 
> 
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> 
> 
> 
> 
> 
> 


-- 
 Tirthankar, IGIDR. 
 +91-22-8400919 x275 (r), x593(o), x542(CFL). 
 http://www.igidr.ac.in/~tir

  NON-COMBATANT, n.  A dead Quaker.

• Re: Trouble with Math::NumberCruncher (fwd) by Tirthankar C. Patnaik