Regression analysis:
In “Statistical Methods” has defined Regression as, “The measure of the average relationship between two or more variables in terms of the original units of the data”.
Morris Hamburg has defined Regression Analysis as, “The method by which estimates are made of the values of a variable from knowledge of the values of one or more other variables and to the measurement of the errors involved in this estimation process”.
Correlation analysis attempts to study the relationship between the two variables x and y. Regression analysis attempts to predict the average x for a given y. in Regression it is attempted to quantify the dependence of one variable on the other. Example; There are two variables x and y. y depends on x. the dependence is expressed in the form of the following equation:
Y=a+bx
Regression analysis used to estimate the values of the dependent variables from the values of the independent variables.
Regression analysis uses to get a measure of the error involved while using the regression line as a basis for estimation.
Regression coefficient is used to calculate correlation coefficient. The square of correlation coefficient measures the degree of association of correlation that prevails between the given two variables.
Correlation coefficient measures the degree of co variability between the given variables x and y. the regression analysis uses to study the ‘nature of relationship’ between the given variables; the value of one variable is predicted based on another.
Correlation analysis does not study cause and effect relationship of the given variables. It is not stated that one variable is the cause and other the effect. In regression analysis one variable is taken as dependent and another independent. Here there is possibility to study the cause and effect relationship.
In statistics, regression analysis examines the relation of a dependent variable (response variable) to specified independent variables (explanatory variables). The mathematical model of their relationship is the regression equation. The dependent variable is modeled as a random variable because of uncertainty as to its value, given only the value of each independent variable. A regression equation contains estimates of one or more hypothesized regression parameters ("constants"). These estimates are constructed using data for the variables, such as from a sample. The estimates measure the relationship between the dependent variable and each of the independent variables. They also allow estimating the value of the dependent variable for a given value of each respective independent variable.
Uses of regression include curve fitting, prediction (including forecasting of time-series data), modeling of causal relationships, and testing scientific hypotheses about relationships between variables.
Showing posts with label Statistics for Management. Show all posts
Showing posts with label Statistics for Management. Show all posts
Wednesday, January 16, 2008
Briefly explain the concepts of Correlation in your own words
Types of Correlation are given below:
a. Positive of Negative
b. Simple, Partial and Multiple
c. Linear and Non-Linear
a. Positive Correlation: Both the variables (X and Y) will vary in the same direction. If variable X increases, variable Y also will increase; if variable X decreases, variable Y also will decrease.
Negative Correlation: The given variables will vary in opposite direction. If on variable increases, other variable will decrease.
b. Simple, Partial and Multiple Correlations: In simple correlation, relationship between two variables are studies. In partial and multiple correlations three or more variables are studies. Three or more variables are simultaneously studied in multiple correlation. In partial correlation more than two variables are studied, but the effect on one variable is kept constant and relationship between other two variables is studied.
c. Linear and Non-Linear Correlation: It depends upon the constance of the ration of change between the variables. In linear correlation the percentage change in one variable will be equal to the percentage change in another variable. It is not so in non-linear correlation.
Method of Studying Correlation::
The various methods of studying correlation are given below.
a. Scatter Diagram Method.
b. Graphic Method
c. Karl Pearson’s Coefficient of Correlation
d. Concurrent Deviation Method
e. Method of Least Squares
In probability theory and statistics, correlation, also called correlation coefficient, indicates the strength and direction of a linear relationship between two random variables. In general statistical usage, correlation or co-relation refers to the departure of two variables from independence. In this broad sense there are several coefficients, measuring the degree of correlation, adapted to the nature of data.
A number of different coefficients are used for different situations. The best known is the Pearson product-moment correlation coefficient, which is obtained by dividing the covariance of the two variables by the product of their standard deviations.
a. Positive of Negative
b. Simple, Partial and Multiple
c. Linear and Non-Linear
a. Positive Correlation: Both the variables (X and Y) will vary in the same direction. If variable X increases, variable Y also will increase; if variable X decreases, variable Y also will decrease.
Negative Correlation: The given variables will vary in opposite direction. If on variable increases, other variable will decrease.
b. Simple, Partial and Multiple Correlations: In simple correlation, relationship between two variables are studies. In partial and multiple correlations three or more variables are studies. Three or more variables are simultaneously studied in multiple correlation. In partial correlation more than two variables are studied, but the effect on one variable is kept constant and relationship between other two variables is studied.
c. Linear and Non-Linear Correlation: It depends upon the constance of the ration of change between the variables. In linear correlation the percentage change in one variable will be equal to the percentage change in another variable. It is not so in non-linear correlation.
Method of Studying Correlation::
The various methods of studying correlation are given below.
a. Scatter Diagram Method.
b. Graphic Method
c. Karl Pearson’s Coefficient of Correlation
d. Concurrent Deviation Method
e. Method of Least Squares
In probability theory and statistics, correlation, also called correlation coefficient, indicates the strength and direction of a linear relationship between two random variables. In general statistical usage, correlation or co-relation refers to the departure of two variables from independence. In this broad sense there are several coefficients, measuring the degree of correlation, adapted to the nature of data.
A number of different coefficients are used for different situations. The best known is the Pearson product-moment correlation coefficient, which is obtained by dividing the covariance of the two variables by the product of their standard deviations.
Write down the properties of Chi-square distribution
Ans. Properties of χ2 distribution:
1. Mean of χ2 distribution = Degrees of freedom = V
2. SD of χ2 distribution = √2V
3. Median of χ2 distribution divides the area of the curve into two equal parts, each part being 0.5
4. Mode of χ2 distribution is equal to degrees of freedom less 2 i.e., V-2
5. χ2 values are always positively skewed.
6. χ2 values increase with the increase in the DF, there is a new χ2 distribution with every increase in the no. of degrees of freedom.
7. The lowest value of χ2 is zero and the highest is infinity i.e. 0 < χ2 <
8. When two chi-squares γ 2\1 and χ2/2 are independent following χ2 distribution with n1 and n2 degrees of freedom, their sum χ2/2 +χ2/2 will follow χ2 distribution with n1 + n2 degrees of freedom.
9. When V>30, √2χ2 - √2V -1 approximately follows the standard normal distribution.
Chi-Square Distribution:
This article is about the mathematics of the chi-square distribution. For its uses in statistics, see Chi-Square Test.
Explain Type I and Type II errors in testing of Hypothesis
Types of hypothesis:
A proposition may take the form of asserting a causal relationship (such as "A causes B"). A proposition often (but not necessarily) involves an assertion of causation. For example, if a particular independent variable changes, then a certain dependent variable also changes. This formulation, also known as an "If and Then" statement, applies whether or not a proposition asserts a direct cause-and-effect relationship.
A hypothesis about possible correlation does not stipulate the cause and effect per se, only stating that "A is related to B". Investigators may have more difficulty in verifying causal relationships than other correlations, because quite commonly intervening variables also become involved, possibly giving rise to the appearance of a possibly direct cause-and-effect relationship, but which (upon further investigation) turn out to have some other, more direct causal factor not mentioned in the proposition. Also, a mere observation of a change in one variable, when correlated with a change in another variable, can actually mistake the effect for the cause, and vice-versa (i.e., potentially get the hypothesized cause and effect backwards).
Empirical hypotheses that experimenters have repeatedly verified may become sufficiently dependable that, at some point in time, they become considered as "proven". Some people may succumb to the temptation to term such hypotheses "laws", but they would do so mistakenly, since by definition a hypothesis explains and a law describes (for example, a law can state: "Matter can neither be created or destroyed, only changed in form"). More accurately, one could refer to repeatedly verified hypotheses simply as "adequately verified", or as "dependable".
Statistics features a rather more general concept of a hypothesis: this involves making assertions about the probability distributions or likelihoods of events.
Statisticians use two kinds of hypothesis: first, the null hypothesis or H0; secondly, the alternative hypothesis or H1. To give the simplest non-trivial example, one might formulate two hypotheses about tossing a coin:
• H0: coin-tossing operates "fairly" (equally likely to fall "Heads" or "Tails")
• H1: coin-tossing operates in a biased manner to give a 90% probability of falling "Heads"
No finite sequence of results could utterly falsify either hypothesis. However, various statistical approaches (such as Bayesian statistics and classical statistics (i.e. t-tests)) can quantify the strong intuition that H1 appears much less likely than H0 if, in 1,000 tosses, 495 came out "Heads" — and much more likely if 895 came out "Heads". In more complex sciences, researchers generally evaluate experiments statistically rather than as simple verifications or falsifications.
A hypothesis (from Greek ὑπόθεσις) consists either of a suggested explanation for a phenomenon or of a reasoned proposal suggesting a possible correlation between multiple phenomena. The term derives from the Greek, hypotithenai meaning "to put under" or "to suppose." The scientific method requires that one can test a scientific hypothesis. Scientists generally base such hypotheses on previous observations or on extensions of scientific theories.
Type I and Type II errors:
Rejecting a null hypothesis when it is true is called a Type I error, and its probability ( Which, as we have seen, is also the significance level of the test) is symbolized α (alpha). Alternatively, accepting a null hypothesis when it is false is called a Type II error, and its probability is symbolized β (beta). There is a trade-off between these two errors: the probability of making one type of error can be reduced only if we are willing to increase the probability of making the other type or error. With an acceptance region this small, we will rarely accept a null hypothesis when it is not true, but as a cost of being this sure, we will often reject a null hypothesis when it is true.
Put another way, in order to get a low β, we will have to put up with a high α. To deal with this trade-off in personal and professional situations, decision makers decide the appropriate level of significance by examining the costs or penalties attached to both types of errors.
When Type I error is preferred:
Suppose that making a Type I error (rejecting a null hypothesis when it is true) involves the time and trouble of reworking a batch of chemicals that should have been accepted. At the same time, making a Type II error (accepting a null hypothesis when it is false) means talking a chance that an entire group of users of this chemical compound will be poisoned. Obviously, the management of this company will prefer a Type I error to a Type II error and, as a result, will set very high levels of significance in its testing to get low β s.
When Type II error is preferred:
Suppose, on the other hand, that making a Type I error involves disassembling an entire engine at the factory, but making a Type II error involves relatively inexpensive warranty repairs by the dealers. Then the manufacturer is more likely to prefer a Type II error and will set lower significance levels in its testing.
A proposition may take the form of asserting a causal relationship (such as "A causes B"). A proposition often (but not necessarily) involves an assertion of causation. For example, if a particular independent variable changes, then a certain dependent variable also changes. This formulation, also known as an "If and Then" statement, applies whether or not a proposition asserts a direct cause-and-effect relationship.
A hypothesis about possible correlation does not stipulate the cause and effect per se, only stating that "A is related to B". Investigators may have more difficulty in verifying causal relationships than other correlations, because quite commonly intervening variables also become involved, possibly giving rise to the appearance of a possibly direct cause-and-effect relationship, but which (upon further investigation) turn out to have some other, more direct causal factor not mentioned in the proposition. Also, a mere observation of a change in one variable, when correlated with a change in another variable, can actually mistake the effect for the cause, and vice-versa (i.e., potentially get the hypothesized cause and effect backwards).
Empirical hypotheses that experimenters have repeatedly verified may become sufficiently dependable that, at some point in time, they become considered as "proven". Some people may succumb to the temptation to term such hypotheses "laws", but they would do so mistakenly, since by definition a hypothesis explains and a law describes (for example, a law can state: "Matter can neither be created or destroyed, only changed in form"). More accurately, one could refer to repeatedly verified hypotheses simply as "adequately verified", or as "dependable".
Statistics features a rather more general concept of a hypothesis: this involves making assertions about the probability distributions or likelihoods of events.
Statisticians use two kinds of hypothesis: first, the null hypothesis or H0; secondly, the alternative hypothesis or H1. To give the simplest non-trivial example, one might formulate two hypotheses about tossing a coin:
• H0: coin-tossing operates "fairly" (equally likely to fall "Heads" or "Tails")
• H1: coin-tossing operates in a biased manner to give a 90% probability of falling "Heads"
No finite sequence of results could utterly falsify either hypothesis. However, various statistical approaches (such as Bayesian statistics and classical statistics (i.e. t-tests)) can quantify the strong intuition that H1 appears much less likely than H0 if, in 1,000 tosses, 495 came out "Heads" — and much more likely if 895 came out "Heads". In more complex sciences, researchers generally evaluate experiments statistically rather than as simple verifications or falsifications.
A hypothesis (from Greek ὑπόθεσις) consists either of a suggested explanation for a phenomenon or of a reasoned proposal suggesting a possible correlation between multiple phenomena. The term derives from the Greek, hypotithenai meaning "to put under" or "to suppose." The scientific method requires that one can test a scientific hypothesis. Scientists generally base such hypotheses on previous observations or on extensions of scientific theories.
Type I and Type II errors:
Rejecting a null hypothesis when it is true is called a Type I error, and its probability ( Which, as we have seen, is also the significance level of the test) is symbolized α (alpha). Alternatively, accepting a null hypothesis when it is false is called a Type II error, and its probability is symbolized β (beta). There is a trade-off between these two errors: the probability of making one type of error can be reduced only if we are willing to increase the probability of making the other type or error. With an acceptance region this small, we will rarely accept a null hypothesis when it is not true, but as a cost of being this sure, we will often reject a null hypothesis when it is true.
Put another way, in order to get a low β, we will have to put up with a high α. To deal with this trade-off in personal and professional situations, decision makers decide the appropriate level of significance by examining the costs or penalties attached to both types of errors.
When Type I error is preferred:
Suppose that making a Type I error (rejecting a null hypothesis when it is true) involves the time and trouble of reworking a batch of chemicals that should have been accepted. At the same time, making a Type II error (accepting a null hypothesis when it is false) means talking a chance that an entire group of users of this chemical compound will be poisoned. Obviously, the management of this company will prefer a Type I error to a Type II error and, as a result, will set very high levels of significance in its testing to get low β s.
When Type II error is preferred:
Suppose, on the other hand, that making a Type I error involves disassembling an entire engine at the factory, but making a Type II error involves relatively inexpensive warranty repairs by the dealers. Then the manufacturer is more likely to prefer a Type II error and will set lower significance levels in its testing.
Write short notes on Interval estimates
Interval estimation:
The purpose of gathering samples is to learn more about a population. We can compute this information from the sample data as either point estimates, An Interval estimate describes a range of values within which a population parameter is likely to lie.
Interval estimation:
In statistics, interval estimation is the use of sample data to calculate an interval of possible (or probable) values of an unknown population parameter. The most prevalent forms of interval estimation are confidence intervals (a frequentist method) and credible intervals (a Bayesian method).
* Behrens-Fisher problem
* fiducial inference
Other common interval estimation are
* Tolerance interval
* Prediction interval - used mainly in Regression Analysis.
The purpose of gathering samples is to learn more about a population. We can compute this information from the sample data as either point estimates, An Interval estimate describes a range of values within which a population parameter is likely to lie.
Interval estimation:
In statistics, interval estimation is the use of sample data to calculate an interval of possible (or probable) values of an unknown population parameter. The most prevalent forms of interval estimation are confidence intervals (a frequentist method) and credible intervals (a Bayesian method).
* Behrens-Fisher problem
* fiducial inference
Other common interval estimation are
* Tolerance interval
* Prediction interval - used mainly in Regression Analysis.
What is the criteria of a good estimator, explain the points in your own word
There are two number of estimates about a population:
1. A point estimate and
2. An interval estimate:
Estimator:
In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. Many different estimators are possible for any given parameter. Some criterion is used to choose between the estimators, although it is often the case that a criterion cannot be used to clearly pick one estimator over another. To estimate a parameter of interest (e.g., a population mean, a binomial proportion, a difference between two population means, or a ratio of two population standard deviation), the usual procedure is as follows:
1. Select a random sample from the population of interest.
2. Calculate the point estimate of the parameter.
3. Calculate a measure of its variability, often a confidence interval.
4. Associate with this estimate a measure of variability.
There are two types of estimators: Point Estimators and Interval Estimators.
1. A point Estimate: is a single number that is used to estimate an unknown population parameter. A point estimate is often insufficient, because it is either right or wrong. If you are told only that her point estimate of enrollment is wrong, you do not know how wrong it is, and you cannot be certain of the estimate’s reliability. If you learn that is off by only 10 students, you would accept 350 students as a good estimate of future enrollment.
Point estimation:
In statistics, point estimation involves the use of sample data to calculate a single value (known as a statistic) which is to serve as a "best guess" for an unknown (fixed or random) population parameter.
More formally, it is the application of a point estimator to the data.
Point estimation should be contrasted with Bayesian methods of estimation, where the goal is usually to compute (perhaps to an approximation) the posterior distributions of parameters and other quantities of interest. The contrast here is between estimating a single point (point estimation), versus estimating a weighted set of points (a probability density function).
2. An interval estimator: is a range of values used to estimate a population parameter. It indicates the error in two ways: by the parameter lying within that range.
Interval estimation:
In statistics, interval estimation is the use of sample data to calculate an interval of possible (or probable) values of an unknown population parameter. The most prevalent forms of interval estimation are confidence intervals (a frequentist method) and credible intervals (a Bayesian method).
* Behrens-Fisher problem
* fiducial inference
Other common interval estimation are
* Tolerance interval
* Prediction interval - used mainly in Regression Analysis
Criteria of a good estimator:
1. Unbiasedness: This is a desirable property for a good estimator to have. The term unbiasedness refers to the fact that a sample mean is an unbiased estimator of a population mean because the mean of the sampling distribution of sample means taken from the same population is equal to the population mean itself. We can say that a statistic is an unbiased estimator if, on average, it tends to assume values that ate the same extent as it tends to assume values that are below the population parameter being estimated.
2. Efficiency: Another desirable property of a good estimator is that it be efficient. Efficiency refers to the size of the standard error of the statistic. If we compare two statistics from a sample of the same size and try to decide which one is the more efficient estimator, we would pick the statistic that has the smaller standard error, or standard deviation of the sampling distribution. Suppose we choose a sample of a given size and must decide whether to use the sample mean or the sample median to estimate the population mean. If we calculate the standard error of the sample mean and find it to be 1.05 and then calculate the standard error of the sample median and find it to be 1.6, we would say that the sample mean is a more efficient estimator of the population mean be cause its standard error is smaller. It makes sense that an estimator with a smaller standard error (with less variation) will have more chance of producing an estimate nearer to the population parameter under consideration.
3. Consistency: A statistic is a consistent estimator of a population parameter if as the sample size increases, it becomes almost certain that the value of the statistic comes very close to the value of the population parameter. If an estimator is consistent, it becomes more reliable with large samples. Thus, if you are wondering whether to increase the sample size to get more information about a population parameter, find out first whether your statistic is a consistent estimator. If it is not, you will waste time and money by taking larger samples.
Consistency:
A Consistent Estimator is an estimator that converges in probability to the quantity being estimated as the sample size grows without bound.
An estimator tn (where n is the sample size) is a consistent estimator for parameter θ if and only if, for all ε > 0, no matter how small, we have
\lim_{n\to\infty}\Pr\left\{ \left| t_n-\theta\right|<\epsilon \right\}=1.
It is called strongly consistent, if it converges almost surely to the true value.
4. Sufficiency: An estimator is sufficient if it makes so much use of the information in the sample that no other estimator could extract from the sample additional information about the population parameter being estimated.
Efficiency:
Main article: Efficiency (statistics)
The quality of an estimator is generally judged by its mean squared error.
However, occasionally one chooses the unbiased estimator with the lowest variance. Efficient estimators are those that have the lowest possible variance among all unbiased estimators. In some cases, a biased estimator may have a uniformly smaller mean squared error than does any unbiased estimator, so one should not make too much of this concept. For that and other reasons, it is sometimes preferable not to limit oneself to unbiased estimators; see estimator bias. Concerning such "best unbiased estimators", see also Cramér-Rao bound, Gauss-Markov theorem, Lehmann-Scheffé theorem, Rao-Blackwell theorem.
1. A point estimate and
2. An interval estimate:
Estimator:
In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. Many different estimators are possible for any given parameter. Some criterion is used to choose between the estimators, although it is often the case that a criterion cannot be used to clearly pick one estimator over another. To estimate a parameter of interest (e.g., a population mean, a binomial proportion, a difference between two population means, or a ratio of two population standard deviation), the usual procedure is as follows:
1. Select a random sample from the population of interest.
2. Calculate the point estimate of the parameter.
3. Calculate a measure of its variability, often a confidence interval.
4. Associate with this estimate a measure of variability.
There are two types of estimators: Point Estimators and Interval Estimators.
1. A point Estimate: is a single number that is used to estimate an unknown population parameter. A point estimate is often insufficient, because it is either right or wrong. If you are told only that her point estimate of enrollment is wrong, you do not know how wrong it is, and you cannot be certain of the estimate’s reliability. If you learn that is off by only 10 students, you would accept 350 students as a good estimate of future enrollment.
Point estimation:
In statistics, point estimation involves the use of sample data to calculate a single value (known as a statistic) which is to serve as a "best guess" for an unknown (fixed or random) population parameter.
More formally, it is the application of a point estimator to the data.
Point estimation should be contrasted with Bayesian methods of estimation, where the goal is usually to compute (perhaps to an approximation) the posterior distributions of parameters and other quantities of interest. The contrast here is between estimating a single point (point estimation), versus estimating a weighted set of points (a probability density function).
2. An interval estimator: is a range of values used to estimate a population parameter. It indicates the error in two ways: by the parameter lying within that range.
Interval estimation:
In statistics, interval estimation is the use of sample data to calculate an interval of possible (or probable) values of an unknown population parameter. The most prevalent forms of interval estimation are confidence intervals (a frequentist method) and credible intervals (a Bayesian method).
* Behrens-Fisher problem
* fiducial inference
Other common interval estimation are
* Tolerance interval
* Prediction interval - used mainly in Regression Analysis
Criteria of a good estimator:
1. Unbiasedness: This is a desirable property for a good estimator to have. The term unbiasedness refers to the fact that a sample mean is an unbiased estimator of a population mean because the mean of the sampling distribution of sample means taken from the same population is equal to the population mean itself. We can say that a statistic is an unbiased estimator if, on average, it tends to assume values that ate the same extent as it tends to assume values that are below the population parameter being estimated.
2. Efficiency: Another desirable property of a good estimator is that it be efficient. Efficiency refers to the size of the standard error of the statistic. If we compare two statistics from a sample of the same size and try to decide which one is the more efficient estimator, we would pick the statistic that has the smaller standard error, or standard deviation of the sampling distribution. Suppose we choose a sample of a given size and must decide whether to use the sample mean or the sample median to estimate the population mean. If we calculate the standard error of the sample mean and find it to be 1.05 and then calculate the standard error of the sample median and find it to be 1.6, we would say that the sample mean is a more efficient estimator of the population mean be cause its standard error is smaller. It makes sense that an estimator with a smaller standard error (with less variation) will have more chance of producing an estimate nearer to the population parameter under consideration.
3. Consistency: A statistic is a consistent estimator of a population parameter if as the sample size increases, it becomes almost certain that the value of the statistic comes very close to the value of the population parameter. If an estimator is consistent, it becomes more reliable with large samples. Thus, if you are wondering whether to increase the sample size to get more information about a population parameter, find out first whether your statistic is a consistent estimator. If it is not, you will waste time and money by taking larger samples.
Consistency:
A Consistent Estimator is an estimator that converges in probability to the quantity being estimated as the sample size grows without bound.
An estimator tn (where n is the sample size) is a consistent estimator for parameter θ if and only if, for all ε > 0, no matter how small, we have
\lim_{n\to\infty}\Pr\left\{ \left| t_n-\theta\right|<\epsilon \right\}=1.
It is called strongly consistent, if it converges almost surely to the true value.
4. Sufficiency: An estimator is sufficient if it makes so much use of the information in the sample that no other estimator could extract from the sample additional information about the population parameter being estimated.
Efficiency:
Main article: Efficiency (statistics)
The quality of an estimator is generally judged by its mean squared error.
However, occasionally one chooses the unbiased estimator with the lowest variance. Efficient estimators are those that have the lowest possible variance among all unbiased estimators. In some cases, a biased estimator may have a uniformly smaller mean squared error than does any unbiased estimator, so one should not make too much of this concept. For that and other reasons, it is sometimes preferable not to limit oneself to unbiased estimators; see estimator bias. Concerning such "best unbiased estimators", see also Cramér-Rao bound, Gauss-Markov theorem, Lehmann-Scheffé theorem, Rao-Blackwell theorem.
State Central Limit Theorem. Explain in your own words
A Central Limit Theorem is any of a set of weak-convergence results in probability theory. They all express the fact that any sum of many independent and identically-distributed random variables will tend to be distributed according to a particular "attractor distribution". The most important and famous result is called The Central Limit Theorem which states that if the sum of the variables has a finite variance, then it will be approximately normally distributed (i.e., following a Gaussian distribution).
Since many real processes yield distributions with finite variance, this explains the ubiquity of the normal probability distribution.
Several generalizations for finite variance exist which do not require identical distribution but incorporate some condition which guarantees that none of the variables exert a much larger influence than the others. Two such conditions are the Lindeberg condition and the Lyapunov condition. Other generalizations even allow some "weak" dependence of the random variables. Also, a generalization due to Gnedenko and Kolmogorov states that the sum of a number of independent random variables with power-law tail distributions decreasing as 1/|x|α+1 with 0 < α < href="http://en.wikipedia.org/wiki/L%C3%A9vy_skew_alpha-stable_distribution" title="Lévy skew alpha-stable distribution">Lévy distribution as the number of variables grows. This article will only be concerned with the central limit theorem as it applies to distributions with finite variance.
History:
“The central limit theorem has an interesting history. The first version of this theorem was postulated by the French-born English mathematician Abraham de Moivre, who, in a remarkable article published in 1733, used the normal distribution to approximate the distribution of the number of heads resulting from many tosses of a fair coin. This finding was far ahead of its time, and was nearly forgotten until the famous French mathematician Pierre-Simon Laplace rescued it from obscurity in his monumental work Théorie Analytique des Probabilités, which was published in 1812. Laplace expanded De Moivre's finding by approximating the binomial distribution with the normal distribution. But as with De Moivre, Laplace's finding received little attention in his own time. It was not until the nineteenth century was at an end that the importance of the central limit theorem was discerned, when, in 1901, Russian mathematician Aleksandr Lyapunov defined it in general terms and proved precisely how it worked mathematically. Nowadays, the central limit theorem is considered to be the unofficial sovereign of probability theory. ”
See Bernstein (1945) for a historical discussion focusing on the work of Pafnuty Chebyshev and his students Andrey Markov and Aleksandr Lyapunov that led to the first proofs of the C.L.T. in a general setting.
Proof of the central limit theorem:
For a theorem of such fundamental importance to statistics and applied probability, the central limit theorem has a remarkably simple proof using characteristic functions. It is similar to the proof of a (weak) law of large numbers. For any random variable, Y, with zero mean and unit variance (var(Y) = 1), the characteristic function of Y is, by Taylor's theorem,
\varphi_Y(t) = 1 - {t^2 \over 2} + o(t^2), \quad t \rightarrow 0
Where o (t2 ) is "little o notation" for some function of t that goes to zero more rapidly than t2. Letting Yi be (Xi − μ)/σ, the standardised value of Xi, it is easy to see that the standardised mean of the observations X1, X2, ..., Xn is just
Z_n = \frac{n\overline{X}_n-n\mu}{\sigma\sqrt{n}} = \sum_{i=1}^n {Y_i \over \sqrt{n}}.
By simple properties of characteristic functions, the characteristic function of Zn is
\left[\varphi_Y\left({t \over \sqrt{n}}\right)\right]^n = \left[ 1 - {t^2 \over 2n} + o\left({t^2 \over n}\right) \right]^n \, \rightarrow \, e^{-t^2/2}, \quad n \rightarrow \infty.
But, this limit is just the characteristic function of a standard normal distribution, N(0,1), and the central limit theorem follows from the Lévy continuity theorem, which confirms that the convergence of characteristic functions implies convergence in distribution.
Since many real processes yield distributions with finite variance, this explains the ubiquity of the normal probability distribution.
Several generalizations for finite variance exist which do not require identical distribution but incorporate some condition which guarantees that none of the variables exert a much larger influence than the others. Two such conditions are the Lindeberg condition and the Lyapunov condition. Other generalizations even allow some "weak" dependence of the random variables. Also, a generalization due to Gnedenko and Kolmogorov states that the sum of a number of independent random variables with power-law tail distributions decreasing as 1/|x|α+1 with 0 < α < href="http://en.wikipedia.org/wiki/L%C3%A9vy_skew_alpha-stable_distribution" title="Lévy skew alpha-stable distribution">Lévy distribution as the number of variables grows. This article will only be concerned with the central limit theorem as it applies to distributions with finite variance.
History:
“The central limit theorem has an interesting history. The first version of this theorem was postulated by the French-born English mathematician Abraham de Moivre, who, in a remarkable article published in 1733, used the normal distribution to approximate the distribution of the number of heads resulting from many tosses of a fair coin. This finding was far ahead of its time, and was nearly forgotten until the famous French mathematician Pierre-Simon Laplace rescued it from obscurity in his monumental work Théorie Analytique des Probabilités, which was published in 1812. Laplace expanded De Moivre's finding by approximating the binomial distribution with the normal distribution. But as with De Moivre, Laplace's finding received little attention in his own time. It was not until the nineteenth century was at an end that the importance of the central limit theorem was discerned, when, in 1901, Russian mathematician Aleksandr Lyapunov defined it in general terms and proved precisely how it worked mathematically. Nowadays, the central limit theorem is considered to be the unofficial sovereign of probability theory. ”
See Bernstein (1945) for a historical discussion focusing on the work of Pafnuty Chebyshev and his students Andrey Markov and Aleksandr Lyapunov that led to the first proofs of the C.L.T. in a general setting.
Proof of the central limit theorem:
For a theorem of such fundamental importance to statistics and applied probability, the central limit theorem has a remarkably simple proof using characteristic functions. It is similar to the proof of a (weak) law of large numbers. For any random variable, Y, with zero mean and unit variance (var(Y) = 1), the characteristic function of Y is, by Taylor's theorem,
\varphi_Y(t) = 1 - {t^2 \over 2} + o(t^2), \quad t \rightarrow 0
Where o (t2 ) is "little o notation" for some function of t that goes to zero more rapidly than t2. Letting Yi be (Xi − μ)/σ, the standardised value of Xi, it is easy to see that the standardised mean of the observations X1, X2, ..., Xn is just
Z_n = \frac{n\overline{X}_n-n\mu}{\sigma\sqrt{n}} = \sum_{i=1}^n {Y_i \over \sqrt{n}}.
By simple properties of characteristic functions, the characteristic function of Zn is
\left[\varphi_Y\left({t \over \sqrt{n}}\right)\right]^n = \left[ 1 - {t^2 \over 2n} + o\left({t^2 \over n}\right) \right]^n \, \rightarrow \, e^{-t^2/2}, \quad n \rightarrow \infty.
But, this limit is just the characteristic function of a standard normal distribution, N(0,1), and the central limit theorem follows from the Lévy continuity theorem, which confirms that the convergence of characteristic functions implies convergence in distribution.
Write short notes on Simple Random Sampling
In statistics, a Simple Random Sample is a group of subjects (a sample) chosen from a larger group (a population). Each subject from the population is chosen randomly and entirely by chance, such that each subject has the same probability of being chosen at any stage during the sampling process. This process and technique is known as Simple Random Sampling, and should not be confused with Random Sampling.
In small populations such sampling is typically done "without replacement", i.e., one deliberately avoids choosing any member of the population more than once. An unbiased random selection of subjects is important so that in the long run, the sample represents the population. However, this does not guarantee that a particular sample is a perfect representation of the population. Simple random sampling merely allows one to draw externally valid conclusions about the entire population based on the sample. Although simple random sampling can be conducted with replacement instead, this is less common and would normally be described more fully as simple random sampling with replacement.
Conceptually, simple random sampling is the simplest of the probability sampling techniques. It requires a complete sampling frame, which may not be available or feasible to construct for large populations. Even if a complete frame is available, more efficient approaches may be possible if other useful information is available about the units in the population.
Advantages are that it is free of classification error, and it requires minimum advance knowledge of the population. It best suits situations where not much information is available about the population and data collection can be efficiently conducted on randomly distributed items. If these conditions are not true, stratified sampling or cluster sampling may be a better choice.
Simple Random Sampling:
Simple Random Sampling selects samples by methods that allow each possible sample to have an equal probability of being picked and each item in the entire population to have an equal chance of being included in the sample.
Suppose we have a population of four students in a seminar and we want samples of two students at a time for interviewing purposes. The table below illustrates all of the possible combinations of samples of two students in a population size of four, the probability of each sample being picked, and the probability that each student will be in a sample.
Discuss briefly the Continuous Probability Distribution
The Normal Distribution/ Continuous Probability, also called the Gaussian distribution, is an important family of continuous probability distributions, applicable in many fields. Each member of the family may be defined by two parameters, location and scale: the mean ("average", μ) and variance ("variability", σ2), respectively. The Standard Normal Distribution is the normal distribution with a mean of zero and a variance of one (the green curves in the plots to the right). Carl Friedrich Gauss became associated with this set of distributions when he analyzed astronomical data using them [1], and defined the equation of its probability density function. It is often called the bell curve because the graph of its probability density resembles a bell.
The importance of the normal distribution as a model of quantitative phenomena in the natural and behavioral sciences is due to the central limit theorem. Many psychological measurements and physical phenomena (like noise) can be approximated well by the normal distribution. While the mechanisms underlying these phenomena are often unknown, the use of the normal model can be theoretically justified by assuming that many small, independent effects are additively contributing to each observation.
The normal distribution also arises in many areas of statistics. For example, the sampling distribution of the sample mean is approximately normal, even if the distribution of the population from which the sample is taken is not normal. In addition, the normal distribution maximizes information entropy among all distributions with known mean and variance, which makes it the natural choice of underlying distribution for data summarized in terms of sample mean and variance. The normal distribution is the most widely used family of distributions in statistics and many statistical tests are based on the assumption of normality. In probability theory, normal distributions arise as the limiting distributions of several continuous and discrete families of distributions.
So far in this subsection, we have been concerned with discrete probability distributions (Binomial and Poisson). In this section, we shall turn to cases in which the variable can take on any value within a given range and in which the probability distribution is continuous.
A very important continuous probability distribution is the normal distribution. Several mathematicians were instrumental in its development, including the eighteenth-century mathematician-astronomer Karl Gauss. In honour of his work, the normal probability distribution is often called the Gaussian distribution.
There are two basic reasons why the normal distribution occupies such a prominent place in statistics. First, it has some properties that make it applicable to a great many situations in which it is necessary to make inferences by taking samples. In subsequent units, we will find that the normal distribution is a useful sampling distribution. Second, the normal distribution comes close to fitting the actual observed frequency distributions of many phenomena, including human characteristics (weights, heights, and IQs), outputs from physical processes (dimensions and yields), and other measures of interest to managers in both the public and private sectors.
Write short notes on Bernoulli distribution. Also write the use of Bernoulli process
In probability theory and statistics, the Bernoulli distribution, named after Swiss scientist Jakob Bernoulli, is a discrete probability distribution, which takes value 1 with success probability p and value 0 with failure probability q = 1 − p. So if X is a random variable with this distribution, we have:
Definition:
A Bernoulli process is a discrete-time stochastic process consisting of a finite or infinite sequence of independent random variables X1, X2, X3,..., such that
- For each i, the value of Xi is either 0 or 1;
- For all values of i, the probability that Xi = 1 is the same number p.
In other words, a Bernoulli process is a sequence of independent identically distributed Bernoulli trials. The two possible values of each Xi are often called "success" and "failure", so that, when expressed as a number, 0 or 1, the value is said to be the number of successes on the ith "trial". The individual success/failure variables Xi are also called Bernoulli trials.
Random variables associated with the Bernoulli process include:
- The number of successes in the first n trials; this has a binomial distribution;
- The number of trials needed to get r successes; this has a negative binomial distribution.
- The number of trials needed to get one success; this has a geometric distribution, which is a special case of the negative binomial distribution.
The problem of determining the process, given only a limited sample of Bernoulli trials, is known as the problem of checking if a coin is fair.
Bernoulli distribution:
One widely used probability distribution of a discrete random variable is the binomial distribution. It describes a variety of processes of interest to managers. The binomial distribution describes discrete, not continuous, data, resulting from an experiment known as a Bernoulli process, after the seventeenth-century Swiss mathematician Jacob Bernoulli. The tossing of a fair coin a fixed number of times is a Bernoulli process, and the outcomes of such tosses can be represented by the binomial probability distribution. The success or failure of interviewees on an aptitude test may also be described by a Bernoulli process. On the other hand, the frequency distribution of the lives of fluorescent lights in a factory would be measured on a continuous scale of hours and would not quality as a binomial distribution.
Use of the Bernoulli process:
We can use the outcomes of a fixed number of tosses of a fair coin as an example of a Bernoulli process. We can describe this process as follows:
1. Each trial (each toss, in this case) has only two possible outcomes: heads or tails, yes or no, success or failure.
2. The probability of the outcome of any trial (toss) remains fixed over time with a fair coin, the probability of heads remains 0.5 for each toss regardless of the number of times the coin is tossed.
3. The trials are statistically independent; that is, the outcome of one toss does not affect the outcome of any other toss.
Briefly explain relative frequency of occurrence in your own words
Relative Frequency of Occurrence:-
In statistics the frequency of an event i is the number ni of times the event occurred in the experiment or the study. These frequencies are often graphically represented in histograms.
We speak of absolute frequencies, when the counts ni themselves are given and of (relative) frequencies, when those are normalized by the total number of events:
f_i = \frac{n_i}{N} = \frac{n_i}{\sum_i n_i}
Taking the fi for all i and tabulating or plotting them leads to a frequency distribution.
The Relative Frequency Density of occurrence of an event is the relative frequency of i divided by the size of the bin used to classify i.
For example: If the lower extreme of the class you are measuring the density of is 15 and the upper extreme of the class you are measuring is 30, given a relative frequency of 0.0625, you would calculate the frequency density for this class to be:
Rel.freq / (Upper Extreme of Class - Lower Extreme of Class) = Density
0.0625 / (30 - 15) = 0.0625 / 15 = 0.0041666.. That is: 0.00417 to 5 S.F.
In biology, relative frequency is the occurrence of a single gene in a specific species that makes up a gene pool.
The Limiting Relative Frequency of an event over a long series of trials is the conceptual foundation of the frequency interpretation of probability. In this framework, it is assumed that as the length of the series increases without bound, the fraction of the experiments in which we observe the event will stabilize. This interpretation is often contrasted with Bayesian probability.
Frequency is the measurement of the number of occurrences of a repeated event per unit of time. It is also defined as the rate of change of phase of a sinusoidal waveform.
Relative Frequency of Occurrence:-
Question like : “What is the probability that I will live to be 85 ? or “ what are the chances that I will below one of my stereo speakers if I turn my 200 –watt amplifier up to wide open?” or “what is the probability that the location of a new paper plant on the river near our town will cause a substantial fish kill?. We quickly see that we may not be able to state in advance, without experimentation, what these probabilities are. Other approaches may be more useful.
In the 1800s, British statisticians, interested in a theoretical foundation for calculating risk of losses in life insurance and commercial insurance, began defining probabilities from statistical data collected on births and deaths. Today, this approach is called The relative frequency of Occurrence. It defines probability as either:
1. The observed relative frequency of an event in a very large number of trials, or
2. The proportion of times that an event occurs in the long run when conditions are stable.
This method used the relative frequencies of past occurrences as probabilities. We determine how often something has happened in the past and use that figure to predict the probability that it will happen again in the future.
For example. Suppose an insurance company knows from past actuarial data that of all males 40 years old, about 60 out or every 100,000 will die within a 1 – year period. Using this method, the company estimates the probability of death of that age group as:
___60____ or 0.0006
1000
A second characteristic of probabilities established by the relative frequency of occurrence method can be shown by tossing one of our fair coins 300 times. Here we can see that although the proportion of heads was far from 0.5 in the first 100 tosses, it seemed to stabilize and approach 0.5 as the number of tosses increased.
In statistical language, we would say that the relative frequency becomes stable as the number of tosses becomes large (if we are tossing the coin under uniform conditions). Thus, when we use the relative frequency approach to establish probabilities, our probability figure will gain accuracy as we increase the number of observations. Of course, this improved accuracy is not free; although more tosses of our coin will produce a more accurate probability of head occurring, we must bear the time and the cost of additional observations.
One difficulty with the relative frequency approach is that people often use it without evaluating a sufficient number of outcomes. If you heard someone say, “My aunt and uncle got the flu this year, and they are both over 65, so everyone in that age bracket will probably get the flu, “you would know that your friend did not base his assumptions on enough evidence. His observations were insufficient data for establishing a relative frequency of occurrence probability.
In statistics the frequency of an event i is the number ni of times the event occurred in the experiment or the study. These frequencies are often graphically represented in histograms.
We speak of absolute frequencies, when the counts ni themselves are given and of (relative) frequencies, when those are normalized by the total number of events:
f_i = \frac{n_i}{N} = \frac{n_i}{\sum_i n_i}
Taking the fi for all i and tabulating or plotting them leads to a frequency distribution.
The Relative Frequency Density of occurrence of an event is the relative frequency of i divided by the size of the bin used to classify i.
For example: If the lower extreme of the class you are measuring the density of is 15 and the upper extreme of the class you are measuring is 30, given a relative frequency of 0.0625, you would calculate the frequency density for this class to be:
Rel.freq / (Upper Extreme of Class - Lower Extreme of Class) = Density
0.0625 / (30 - 15) = 0.0625 / 15 = 0.0041666.. That is: 0.00417 to 5 S.F.
In biology, relative frequency is the occurrence of a single gene in a specific species that makes up a gene pool.
The Limiting Relative Frequency of an event over a long series of trials is the conceptual foundation of the frequency interpretation of probability. In this framework, it is assumed that as the length of the series increases without bound, the fraction of the experiments in which we observe the event will stabilize. This interpretation is often contrasted with Bayesian probability.
Frequency is the measurement of the number of occurrences of a repeated event per unit of time. It is also defined as the rate of change of phase of a sinusoidal waveform.
Relative Frequency of Occurrence:-
Question like : “What is the probability that I will live to be 85 ? or “ what are the chances that I will below one of my stereo speakers if I turn my 200 –watt amplifier up to wide open?” or “what is the probability that the location of a new paper plant on the river near our town will cause a substantial fish kill?. We quickly see that we may not be able to state in advance, without experimentation, what these probabilities are. Other approaches may be more useful.
In the 1800s, British statisticians, interested in a theoretical foundation for calculating risk of losses in life insurance and commercial insurance, began defining probabilities from statistical data collected on births and deaths. Today, this approach is called The relative frequency of Occurrence. It defines probability as either:
1. The observed relative frequency of an event in a very large number of trials, or
2. The proportion of times that an event occurs in the long run when conditions are stable.
This method used the relative frequencies of past occurrences as probabilities. We determine how often something has happened in the past and use that figure to predict the probability that it will happen again in the future.
For example. Suppose an insurance company knows from past actuarial data that of all males 40 years old, about 60 out or every 100,000 will die within a 1 – year period. Using this method, the company estimates the probability of death of that age group as:
___60____ or 0.0006
1000
A second characteristic of probabilities established by the relative frequency of occurrence method can be shown by tossing one of our fair coins 300 times. Here we can see that although the proportion of heads was far from 0.5 in the first 100 tosses, it seemed to stabilize and approach 0.5 as the number of tosses increased.
In statistical language, we would say that the relative frequency becomes stable as the number of tosses becomes large (if we are tossing the coin under uniform conditions). Thus, when we use the relative frequency approach to establish probabilities, our probability figure will gain accuracy as we increase the number of observations. Of course, this improved accuracy is not free; although more tosses of our coin will produce a more accurate probability of head occurring, we must bear the time and the cost of additional observations.
One difficulty with the relative frequency approach is that people often use it without evaluating a sufficient number of outcomes. If you heard someone say, “My aunt and uncle got the flu this year, and they are both over 65, so everyone in that age bracket will probably get the flu, “you would know that your friend did not base his assumptions on enough evidence. His observations were insufficient data for establishing a relative frequency of occurrence probability.
Explain the limitations of statistics in your own words
Limitations of Statistics:
1. Statistics does not deal with qualitative data. It deals only with quantitative data: Statistical methods can be applied only to numerically expressed data. Qualitative characteristics can be studied only if an alternative method of numerical measurement is introduced.
2. Statistics does not deal with individual fact: Statistical methods can be applied only to aggregate of facts. Single fact cannot be statistically studied.
3. Statistical inferences (conclusions) are not exact: Statistical inferences are true only on an average. They are probabilistic statements.
4. Statistics can be misused: Increasing misuse of Statistics has led to increasing distrust in statistics.
5. Common men cannot handle statistics properly: Only statisticians can handle statistics properly. An illogical analysis of statistical data leads to statistical fallacies.
Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities. Statistics are also used for making informed decisions.
Statistical methods can be used to summarize or describe a collection of data; this is called descriptive statistics. In addition, patterns in the data may be modeled in a way that accounts for randomness and uncertainty in the observations, and then used to draw inferences about the process or population being studied; this is called inferential statistics. Both descriptive and inferential statistics comprise applied statistics. There is also a discipline called mathematical statistics, which is concerned with the theoretical basis of the subject.
1. Statistics does not deal with qualitative data. It deals only with quantitative data: Statistical methods can be applied only to numerically expressed data. Qualitative characteristics can be studied only if an alternative method of numerical measurement is introduced.
2. Statistics does not deal with individual fact: Statistical methods can be applied only to aggregate of facts. Single fact cannot be statistically studied.
3. Statistical inferences (conclusions) are not exact: Statistical inferences are true only on an average. They are probabilistic statements.
4. Statistics can be misused: Increasing misuse of Statistics has led to increasing distrust in statistics.
5. Common men cannot handle statistics properly: Only statisticians can handle statistics properly. An illogical analysis of statistical data leads to statistical fallacies.
Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities. Statistics are also used for making informed decisions.
Statistical methods can be used to summarize or describe a collection of data; this is called descriptive statistics. In addition, patterns in the data may be modeled in a way that accounts for randomness and uncertainty in the observations, and then used to draw inferences about the process or population being studied; this is called inferential statistics. Both descriptive and inferential statistics comprise applied statistics. There is also a discipline called mathematical statistics, which is concerned with the theoretical basis of the subject.
Subscribe to:
Posts (Atom)