Estimation of logistic regression models in small samples. A simulation study using a weakly informative default prior distribution

Abstract

In this paper, we used simulations to compare the performance of classical and Bayesian estimations in logistic regression models using small samples. In the performed simulations, conditions were varied, including the type of relationship between independent and dependent variable values (i.e., unrelated and related values), the type of variable (i.e., binary and continuous), and different Binomial distribution values and symmetry (i.e., symmetry and positive asymmetry). Iteratively re-weighted least squares was used as the estimate method to fit the models in both the classical and Bayesian estimations. A weakly informative default distribution was chosen as the prior distribution for Bayesian estimation. The simulation results demonstrate that Bayesian estimations provide more stable distributions but are not able to solve problems generated by asymmetric distributions based on small samples. Additional research using different kinds of priors that is addressed at solving problems caused by asymmetry is needed.