To clarify that upfront: effect size and statistical significance (whatever you understand by that) are different things, and should ideally both be reported -- I don't think anyone sane reports just the significance, or just the difference.

Concerning large sample sizes in general: Neither in frequentist nor in Bayesian approach there is a problem with having small but statistically significant difference between groups (detectable thanks to your large sample size) -- it is just not very useful (or interesting), as you mention. Actually, I don't really see any disadvantages of having large datasets, not in Bayesian nor in frequentist approaches either (unless there is a systematic measurement bias that amplifies with more data).

Finally: Yes, posterior distributions of the means will get narrower with increasing sample size. I'd argue that this is a good thing, the more credibility the better. In this respect, it is good to be aware that having narrow posteriors of the means is a completely different thing from the width of your prediction intervals (given sample size and by the variance parameter sigma^2). In other words, you can have highly significant (but small) difference between groups, but when you simulate (predict) from you model, the predictions are still widely spread and all over the place.

Petr

]]>Perhaps you can help me as a Bayesian novice: how reliable is Bayesian inference for very large datasets?

For traditional anova, groups with large sample sizes tend to always be significantly different even when these differences are very small (because the degrees of freedom are so high). In these instances, inference is not very helpful and we have to fall back on subjective comparisons of whether an effect size is practically, as opposed to statistically, different. Does Bayesian anova tend to be like this?

In other words, how sensitive is the posterior distribution of the difference between means to sample size? Does it also become more narrowly distributed when sample sizes are very large?

]]>roaches.data <- list(t.to.death = t.to.death,

t.cen = t.cen,

N = nrow(roaches),

weight = roaches$weight,

is.censored = is.censored) # added