The Weighted Means: Essential Bibliography

In the next weeks I will treat the last and more complicated method to analyze the data: the weighted mean. The general hypothesis is that a relatively high number of experiment will be performed (my suggestion: n>5, although it is not so restrictive), and so the method is complimentary to that treated as simple approach – experiment as a random factor.

I have partially treated the argument in:

Goldoni M, Tagliaferri S. Dose-response or dose-effect curves in in vitro experiments and their use to study combined effects of neurotoxicants. Methods Mol Biol. 2011;758:415-34. doi: 10.1007/978-1-61779-170-3_28. Review. PubMed PMID: 21815082  

But I will re-treat the argument step by step to completely explain all the passages. Most of them are available in this essential bibliography:

1)      JR Taylor. An introduction to error analysis:  the study of uncertainties in physical measurements. University Science Books, 1997.

I will adapt the passages proposed for physical measures to toxicology (Chapter in the book entitled: Weighted Means, chapter 7 in the 2^nd edition).

2)      Bland, J. M., and Kerry, S. M. (1998) Weighted comparison of means, BMJ. 316, 129.

This is a very brief but  interesting lecture taken from clinics. http://www.bmj.com/content/316/7125/129

Good Luck!!

An Approximated Method

This method is quite simple to perform, but it does not take into account the number of experiments and it is statistically discussable. It may be used in all the conditions and presents some similarities with the semi-clinical approach (see one of my previous posts).

If we have k experiments with n replicates (n is the same for all the experiments), we may think to calculate the best possible experiment giving to all the experiments the same weight:

-the mean is the mean of k experiments;

-the SD is the mean of SD of k experiments.

In this way, we extrapolate a sort of “mean experiment”, where the number of degree of freedoms (df) remains n-1 for each condition. This method is “dangerous” if we have a low number of replicates and several experiments, and statistically is discussable because it does not take into account all the possible df of our system and gives to all the experiments the same weight.

Let’s look at the data (crude values):

 

 

 The last line represents the values we have to compare, considering n=5 replicates for each conditions. In the statistical softwares, there aren’t general options to compare mean (SD) with ANOVA only knowing that values and n, but we can use some sites that perform such a calculation. See for example:

 

http://www.danielsoper.com/statcalc3/calc.aspx?id=43

In alternative, some softwares perform t-Student tests. We can assess the differences between pairs of conditions using t-Student tests, but we have to consider that we have to apply the Bonferroni’s correction to the p value (true significance is 0.05/m, where m is the number of performed multiple comparisons). The results:

 

To perform the comparisons between exposed conditions and control (software OPENSTAT available at http://www.statprograms4u.com/ or the previous site with pairs of column):

0 vs 1 – p=0.0253

0 vs 2 – p=0.0003

0 vs 3 – p<0.0001

0 vs 4 – p<0.0001

Considering a significant p=0.05/4=0.0125, the condition 1 is the unique not significant condition, and therefore the results are intermediate between A SIMPLE APPROACH with and without random factors (all significant) and the SEMI-CLINICAL approach (no significances). However, this method, although very simplified and discussable, controls fairly the beta error.

 

 

 

THE EXPERIMENT AS RANDOM FACTOR: OUTPUT with SPSS

Let’s go with the analysis, by using the variable “experiment” as random factor.

 

Note that, as for a classical two-way ANOVA, we will have the significant effects of group alone,

experiment alone and the INTERACTION between factors. Interaction means that it evaluates if the trend among experiments is parallel or not. I will show you the meaning with a graph.

Let’s go to the output, looking at the most important tables. The reader may repeat the analysis with normalized data.

 

It is interesting to note that: (1) the significance of the factor group is confirmed (p<0.001); (2) the factor experiment is significant (p=0.002). It means that experiments are not properly homogeneous looking at crude values; (3) interaction is not significant (p=0.185). It means that the trend in the three experiments is substantially parallel; (4) the use of the random factor influences the results of Dunnett’s test, slightly increasing the significance of the difference (as evident in the 1 vs 0 group comparison). Graphically, the trend is the following:

 

It is quite evident that experiment 3 is always the highest, and that experiment 1 is that with major deviations from parallelism, although not significant (e.g. a highest relative effect, to be tested with variable norm1).

CONCLUSION: All the concentrations of the toxicants are significantly effective as compared to controls with p<0.001, but experiments are not perfectly homogeneous (experiment 3 has always higher values as compared to the others), although the trend is overall parallel.

This analysis takes into account both the number of experiment and replicates, and it is particularly efficient with a low number of experiments (3-5). With a higher number of experiments, we may consider other possibilities, as the random factor has too many experiments and therefore a great influence on the statistical analysis, with the risk to create artifacts. I will show you other methods in the next posts.

Note: there is not a best “gold standard” number of experiments to make this approach efficient. It is only my opinion. The conclusions may vary depending on the number of replicates and exposure conditions.

 

 

 

 

  

FIXED AND RANDOM FACTORS

When you perform ANOVA with SPSS, you may have noticed that in Generalized Linear Model  => Univariate…  we have put the variable “group”, which represents the dose of exposure, under FIXED FACTOR(s), but there is another option: RANDOM FACTOR(s).

What about  the difference, using a very simplified terminology? FIXED FACTOR represents a factor which is modified following the design of the researcher. In other words, the researcher FIXES a difference between each category of the factor, as for example the concentration of exposure.  On the other hand, RANDOM FACTOR is a grouping variable on which the researcher cannot act, but can randomly influence the results. To do an example: if we perform a multicentric clinical trial, a category which represents each center is a random factor. The same protocol is used in all the centers, but we cannot know a priori whether  there are uncontrollable differences that may influence the results… YES, when we perform N times the same experiment under the same nominal conditions we introduce a RANDOM FACTOR, as we cannot exclude that there is some type of uncontrollable confounder which may influence the experimental trend.

Therefore, if we introduce the variable “experiment” in which the number of the experiment is indicated, we may think to use it as random factor and perform ANOVA with a fixed factor and a random factor.

 

 

Note that for random factors post-hoc tests cannot be performed as irrelevant for the analysis. IT MAY INFLUENCE THE ANOVA RESULTS, BUT IT PROPERLY TAKES INTO ACCOUNT THE FACT THAT SEVERAL EXPERIMENTS ARE PERFORMED DESPITE OF ONE. It is particularly efficient with a relatively low number of experiments (e.g. 3-5). For a higher number of experiments, other methods may be used (see the next discussions).

In the next post, I will show you the Output of such an analysis.