To SUM UP

First of all, we cannot choose our best test exclusively on the basis of significance (e.g. best=more significant). The reason is that we can do two types of errors (called type I and type II): we see a significance when the observed effect is actually not significant, we don’t see a significance, when the observed effect is actually significant. Therefore, we should choice our test based on its reasonability and statistical correctness:

1)      Semi-clinical approach (12-APR): its use makes sense if each experiment represents a different subject, and replicates are only a measure of intra-subject variability. The test does not consider the replicates, but only mean values. For this reason, it tend to underestimate the significance of the differences and makes the replicates a secondary point;

2)      Simple approach (22-APR): it may be acceptable with few experiments, as it considers all the replicates together, without taking into account the experiments. It may be influenced by outlier experiments, that can increase the sample variance;

3)      The experiment as random factor (9-may): it is a very elegant method to take into account the inter-experiment variability and with few experiments involving the same cells with a couple of replicates it appears a GOOD CHOICE; with several experiments, its interpretation may be difficult and may induce statistical artifacts;

4)      An approximated method (13-may): it is not the best choice, as it tends to underestimate the significance, gives no weight to experiments, and it is statistically discussable. However, it is very simple to perform with few simple calculations;

5)      Weighted means PART 1 (5-jun): it is a good choice particularly with a number of experiments (e.g. >3-5). Its generalization brings to Weighted means PART 2 (6-jun) based on the method proposed by Bland and Kerry. Attention should be given to the degrees of freedom to be used for statistical analysis. The use of n-2 df for each experiment appears a sound choice. The method functions with few and several experiments, but it appears a GOOD CHOICE with k>3-5 experiments.

Although in the peer-review process all these aspects are often not considered by the reviewers and a study is often accepted without these details, I RECOMMEND to indicate the method of considering both experiments and replicates in statistical analysis. It is an appreciable benefit.

Finally, another important consideration: without multiple comparison corrections, the use of t-student/Mann-Whitney test to compare pairs of data with >2 groups IS INCORRECT! TAKE CARE! In some cases, it implies the article rejection!!