What to do with incomplete surveys

I had an undergraduate student ask recently what to do with incomplete surveys: should they include all their data, or not?

This is an interesting question. There is no right or wrong answer: only the answer we choose for each project, and how systematic and careful (to quote Bennet, 1991) the justification is in thinking our way through to our particular decision.

The choices - as I see them - are for us to use:

• Only completed surveys; or
• Anything less than complete - usually surveys competed to a certain point onwards. This will be to wherever most makes sense for us to say "This info is essential and without this, the collected data adds nothing to my study"; or
• Any survey which has been started.

We simply (a) use evidence and expert method to come to our decision, (b) write up the justification in our method, underpinned with cited evidence, and (s) apply whatever we have decided consistently across our entire data set.

When we discuss our data profile - which may fit in our methods chapter or at the beginning of our results chapter - we talk about how many surveys were incomplete, how many were complete, and whether there was, or was not, a pattern to be found in where people exited the survey. Thus the shape of the data collection also forms part of our data.

Further, if we allow partially complete survey data, when we speak about this in the body of our text, we also make it explicit - transparent - just how many people have responded to each question. For each graphic (graph) we use, we also include the number of respondents on the graphic (i.e. n=91), so readers can understand exactly how many people responded to each question.

If there were a 'significant' number of incomplete surveys, we might also write this up in the limitations in the methodology. We also need to justify where 'significance' arises; for example, we might use materiality, where we assume anything greater than 5% is significant.

If there were a large number incomplete (such as 25%), which we would probably feel then undermined the quality of the research to a point where the reliability of the research question's answer was jeopardised, we would mention that in our substantive limitations within the conclusions chapter.

It is quite interesting to think about these things ahead of time, but there is a lot to data collection which we don't realise we haven't thought about until we strike it!

Sam

• Reference: Bennett, R. (1991). Chapter 5: What is Management Research? in N. Smith & P. Dainty (Eds) The Management Research Handbook (pp. 67-78). UK: Routledge.