AbstractAim: Joint species distribution models (JSDMs) are an important tool for predicting ecosys-tem diversity and function under global change. The growing complexity of modern JSDMs necessitates careful model selection tailored to the challenges of community prediction under novel conditions (i.e., transferable models). Common approaches to evaluate the per-formance of JSDMs for community- level prediction are based on individual species predic-tions that do not account for the species correlation structures inherent in JSDMs. Here, we formalize a Bayesian model selection approach that accounts for species correlation structures and apply it to compare the community- level predictive performance of alterna-tive JSDMs across broad environmental gradients emulating transferable applications.Innovation: We connect the evaluation of JSDM predictions to Bayesian model selec-tion theory under which the log score is the preferred performance measure for proba-bilistic prediction. We define the joint log score for community- level prediction and distinguish it from more commonly applied JSDM evaluation metrics. We then apply the joint community log score to evaluate predictions of 1918 out- of- sample boreal for-est understory communities spanning 39 species generated using a novel multinomial JSDM framework that supports alternative species correlation structures: independent, compositional dependence and residual dependence.Main conclusions: The best performing JSDM included all observed environmental vari-ables and compositional dependence modelled using a multinomial likelihood. The ad-dition of flexible residual species correlations improved model predictions only within JSDMs applying a reduced set of environmental variables highlighting potential con-founding between unobserved environmental conditions and residual species depend-ence. The best performing JSDM was consistent across successional and bioclimatic gradients regardless of whether interest was in species- or community- level prediction. Our study demonstrates the utility of the joint community log score to compare the pre-dictive performance of JSDMs and highlights the importance of accounting for species dependence when interest is in community composition under novel conditions.