In-depth-pilot study: its conception, execution, and elaboration of pilot materials

Sociological literature distinguishes a number of techniques whose purpose is to validate research tools before proceeding to the research proper. Mentioned are: pilot study, pilot project, pre-test, trial survey, test tube survey, and pilot survey. These terms have various meanings relating to various purposes and requirements. Till now, the pilot practice in Poland, as well as the uses of its results, have been very diverse and unsystematic depending on various theoretical orientations, preferences of particular researchers and research centers, various types of field research organization, various standards of methodological precision adopted by sociologists. Since 1972 the Institute of. Philosophy and Sociology has adopted a uniform system of pilot studies based on a new conception The adoption of this new conception has resulted from: a) a theoretical reflection about a questionnaire interview accompanied by special empirical investigations conducted, since 1966, under the direction of Z. Gostkowski and J. Lutyński, and whose results are being published in the successive volumes of the work Analyses and Tests of Research Techniques in Sociology, b) the introduction of a new uniform organizational structure of field researches conducted by the Institute, of Philosophy and Sociology accompanied by the tendency to raise the methodological level of sociological investigations, especially in respect of questionnaire construction and the quality of basic data. (Every questionnaire must be evaluated by a methodological consultant.) Several phases preceding the preparation of an interview questionnaire may be distinguished. In the first phase the researcher on the basis of literature, own experience and the so-called field reconnaissance, prepares the first version of the questionnaire. Next, this version is submitted to the methodological consultant for evaluation. After the consultations comes the pilot study consisting in a relatively small number of field interviews (30-100); the materials thus obtained are elaborated in a special manner. As a matter of fact the pilot study as a continuation of prior consultations. The methodological programme for a pilot study as well as the ensuing report, is prepared by the consultant. The aim of consultations, and of the pilot study, as to evaluate the validity of the interview questionnaire as a tool for the collection, by the researcher, of the information required. On the basis of results obtained in the pilot study, and the consultations, the researcher is expected to make final corrections and improvements in the questionnaire and interviewing instructions. The methodology of a pilot study is as follows: the evaluation of the questionnaire is done through an analysis of respondents ways of overt behaviour and their answers to particular questions. The analysis should show whether these questions evoke, in respondents, the psychological processes necessary for the obtention of the information desired. Therefore, in the course of a pilot interview, an interviewer should perform the observation, of verbal and non-verbal behaviour of a respondent, self-observation of own tacts and reactions, and observation of the whole interview situation; he should also ask additional pilot questions which frequently make a pilot interview transform itself into an in-depth-interview, free, interview, or an interview about an interview. The conception of the elaboration of pilot materials consists in the following principles: a) each question is elaborated individually, b) the elaboration is done by means of the method, called „relevance analysis of answers”. Each question is evaluated according to whether the answers obtained are relevant or irrelevant, relevant answers being those which bring the kind of information sought by the researcher. Answers to those questions to which correspond two kinds of information sought by the researcher are evaluated twice, i.e. separately from the view-point of one, and separately from the view-point of the other kind of information desired; Next, relevant answers are scrutinized for eventual reservations, and doubts they may evoke in the researcher. Reservations and doubts may be of different kinds — such as, for example, those resulting from a misunderstanding of the question of the alternatives presented, and the task, to be performed by the respondent; from certain undesirable emotional reactions (anxiety, shame, nervousness, intellectual inability, fatigue, etc.). Application of the method of relevance analysis is demonstrated, the exemplification being the analysis of 120 pilot interviews and the en-suing pilot report which inform about the number of cases each question brought irrelevant answers and how many answers to each question seemed doubtful with respect to their validity. In developing further the present conception of a pilot study efforts will be made to work out the methods permitting to evaluate the validity of answers through the external verification, as well as through the use of psychological tests, dummy questions, free interviews, interviews about the prior interviews, probing questions, observation, and a comparison between respondents’ answers obtained in personal roles and those obtained in a situation of a typical formal interview