I once again find myself personally spending way too much

time both reading and commenting. But so what, it was still worthwhile! ]]>

Oh come on, if just the means are not the same among groups, but variances (and other parameters) are (and there is enough data to estimate the within group variance elsewhere - if all populations are of size one, you are doomed, of course), you do not need to simulate and sample at all - this is why we call these "parametric tests" in the frequentist world. 🙂

With good command of calculus, you can profile the probability of overlaps yourself, others (like me) just "look into the tables" (for the simple case). That was a low hanging fruit.

Challenge: with more than one point you can aim for estimating differences in, let us say, within population kurtosis (e.g. stabilizing vs. disruptive selection) - for suchlike problems, the tools in the frequentists' drawer are not that sharp (or that well known in general), while (I suppose) it may be relatively straightforward to do it under Bayesian framework.

Bonus/malus point for the frequentist approach in the case from the blog: you do not get the shrinkage because of the (wannabe uninformative, yet still present) prior.

summary(lm(snout.vent~as.factor(population),data=snakes))

```
```Call:

lm(formula = snout.vent ~ as.factor(population), data = snakes)

Residuals:

Min 1Q Median 3Q Max

-6.334 -1.466 0.176 1.784 6.906

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 57.100 3.114 18.338 < 2e-16 ***

as.factor(population)2 -15.814 3.266 -4.842 2.43e-05 ***

as.factor(population)3 -11.180 3.266 -3.423 0.00156 **

as.factor(population)4 -2.636 3.266 -0.807 0.42488

as.factor(population)5 1.949 3.266 0.597 0.55438

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

`Residual standard error: 3.114 on 36 degrees of freedom`

Multiple R-squared: 0.8507, Adjusted R-squared: 0.8341

F-statistic: 51.29 on 4 and 36 DF, p-value: 2.208e-14

Basically, I need to generate a pattern from a ppp file or csv file.

]]>To clarify that upfront: effect size and statistical significance (whatever you understand by that) are different things, and should ideally both be reported -- I don't think anyone sane reports just the significance, or just the difference.

Concerning large sample sizes in general: Neither in frequentist nor in Bayesian approach there is a problem with having small but statistically significant difference between groups (detectable thanks to your large sample size) -- it is just not very useful (or interesting), as you mention. Actually, I don't really see any disadvantages of having large datasets, not in Bayesian nor in frequentist approaches either (unless there is a systematic measurement bias that amplifies with more data).

Finally: Yes, posterior distributions of the means will get narrower with increasing sample size. I'd argue that this is a good thing, the more credibility the better. In this respect, it is good to be aware that having narrow posteriors of the means is a completely different thing from the width of your prediction intervals (given sample size and by the variance parameter sigma^2). In other words, you can have highly significant (but small) difference between groups, but when you simulate (predict) from you model, the predictions are still widely spread and all over the place.

Petr

]]>Perhaps you can help me as a Bayesian novice: how reliable is Bayesian inference for very large datasets?

For traditional anova, groups with large sample sizes tend to always be significantly different even when these differences are very small (because the degrees of freedom are so high). In these instances, inference is not very helpful and we have to fall back on subjective comparisons of whether an effect size is practically, as opposed to statistically, different. Does Bayesian anova tend to be like this?

In other words, how sensitive is the posterior distribution of the difference between means to sample size? Does it also become more narrowly distributed when sample sizes are very large?

]]>roaches.data <- list(t.to.death = t.to.death,

t.cen = t.cen,

N = nrow(roaches),

weight = roaches$weight,

is.censored = is.censored) # added