Chinese Journal of Sociology

• Articles • Previous Articles     Next Articles


Research on Survey Quality: Evaluation of the Representativeness of Survey Responses

  

  1. Author 1:REN Liying,Institute of Social Science Survey,Peking University; Author 2: QIU Zeqi, Department of Sociology, Peking University; Author 3:DING Hua,Institute of Social Science Survey,Peking University; Author 4:YAN Jie,School of Government,Peking University
  • Online:2014-01-20 Published:2014-01-20
  • Contact: REN Liying,Institute of Social Science Survey,Peking University)Email:isssrenly@pku.edu.cn E-mail:isssrenly@pku.edu.cn
  • Supported by:

    This research was supported by the project “Paradata and the Quality Management of Survey Data”(71171004),which was sponsored by National Natural Science Foundation of China.

Abstract: As nonresponses in surveys have increasingly become more common in recent years, the representativeness of survey responses has called attention from survey researchers. Response rate is commonly used as an indicator of survey quality. However, theoretically and empirically there is not necessarily a direct link between response rates and nonresponse biases. So how to get alternative indicators of response representativeness has become a focus in research.After reviewing multiple measures of assessing response representativeness, we consider that Rindicator is the most promising compared with others. Regarding its construction, Rindicator is guided by sound theories, based on rich sampleframe data and paradata, and can be obtained by relatively simple algorithm. Regarding its application, Rindicator can be used for comparing different surveys with the same target population, different waves of a panel survey, or times of measurement in different stages of the same survey.This article introduces the definition of the concept of Rindicator, its computation, composition and limitations. Rindicator was applied to the evaluation of the representativeness of the survey responses in the China Family Panel Studies (CFPS), 2010. We divided the whole fieldwork process into three stages and computed the Rindicator, Maximal Absolute Bias, and partial Rindicators for each stage. The analysis of these indicators led to a discovery that the samples were over or underrepresented in the areas with variations in community attributes, economic development, population density, nonagricultural population ratio, and support from community commissions. Moreover, the seriousness of these problems changed as the survey went on.

Key words: social survey , survey quality , response representativeness , Rindicator