Design and analysis of best-worst scaling studies in agricultural research

Abstract

The research study entitled “Design and analysis of Best-Worst Scaling studies in agricultural research” was undertaken at College of Agriculture, Vellayani, during 2023-2025. The primary objective of the study was to develop a comprehensive web application for designing suitable questionnaires and analysing data for Best-Worst Scaling (BWS) experiments in agriculture, guided by insights obtained from a bibliometric analysis of agricultural studies employing the BWS approach. In agriculture, effective decision-making depends on understanding the preferences and priorities of various stakeholders, including farmers, consumers, researchers, and policymakers. Traditional preference elicitation techniques often fall short in capturing subtle distinctions among choices. BWS offers a robust alternative for quantifying stakeholder preferences across domains such as technology adoption, agricultural policies, consumer behaviour, and resource prioritisation. A bibliometric analysis of agricultural BWS literature from 2011 to 2025 was conducted to identify how BWS has been applied, the experimental situations where the three cases of BWS are adopted, commonly used design methodologies, and the analytical approaches used. These findings provided clarity on current practices and guided what features and analytical capabilities needed to be prioritised in the system development for this research. Despite its growing use, researchers often face challenges in designing BWS questionnaires and analysing the resulting data, particularly when dealing with multiple attributes, complex profiles, or multi-profile choice sets. Manual generation of choice sets can be time-consuming and prone to errors, while advanced analytical models require considerable statistical expertise. These challenges underscore the need for an accessible and efficient tool to streamline both stages of BWS research. To address this gap, the web application, named PEAR-BWS (Preference Evaluation in Agricultural Research using Best-Worst Scaling), was developed using the R Shiny framework. It consists of two core modules, Questionnaire Generation and Statistical Analysis, offering an integrated environment for generating BWS

questionnaires and analysing response data, without the need for programming skills. The Questionnaire Generation module enables users to build BWS-based choice sets for all three BWS cases (Object, Profile, and Multi-profile), ensuring balanced representation of items and profiles. The Statistical Analysis module integrates multiple analytical approaches, including Count Analysis, Multinomial Logit, Paired, Marginal, Marginal Sequential, Hierarchical Bayesian estimation, and Latent Class Analysis models. All questionnaire structures and analysis results generated from both modules can be downloaded as Word documents, facilitating direct use in research reporting, thesis writing, publication work, and field data collection. To demonstrate its analytical capabilities, three hypothetical model datasets were constructed for the three BWS cases, reflecting realistic response structures and consistent scoring (1 for best, -1 for worst, and 0 for others). An online survey conducted among students from different agricultural universities evaluated the usability and performance of the application. The feedback indicated a high level of user satisfaction, highlighting its efficiency and practical relevance. Overall, the study presents PEAR-BWS as a comprehensive and user-friendly tool that simplifies the design and analysis of BWS experiments, thereby enhancing accessibility and promoting evidence-based decision-making in agricultural research. The work provides a foundation that can be further expanded in the future by integrating more analytical methods with enhanced visualisation and direct data collection functionality within the web application.