« Previous | Contents | Next »
Listen
Annex 3: Approach to analysis of the consultation - technical note
The main proportion of the consultation findings are based upon the analysis of the written consultation, however findings from the consultation events also feed into the overall analysis.
A mixed method, combining both quantitative and qualitative elements was employed.
Interpretation of findings - notes of caution
- These findings relate only to those consultees and event delegates who answered the consultation document, or attended events and can NOT be generalised to be seen as representative of the views of that sector as a whole, or general opinion overall. Due to the self-selecting nature of consultation exercises these findings are not representative of a wider population.
- Percentages for the written consultation responses have been based on the figure on 187, the total number of analysable responses, and not based on the number who actually answered a particular question. Eg 30% of respondents overall felt… rather than 30% of those who answered this question felt…
- When comparisons have been made between the different background groups, who responded to the written consultation it should be noted that these groups are very different in size, with the voluntary organisations being the most represented on 78, whilst there are only 4 responses from the police.
The analysis process
Analysis of the consultation events:
- Observation of the events
- Analysis of discussion scenarios
- Analysis of Responses to electronic polling questions
Observation of the events
The same researcher was present as an observer at all 7 Protection of Vulnerable Groups, taking notes on delegates' participation in discussion scenarios, their attentiveness during speakers' presentations and the questions that they asked the panel. Research notes on each event were written up, and elements of the observation appear in the analysis of the events.
Analysis of discussion scenarios
There were 3 discussion scenarios at each event (see Annex 4). Delegates were assigned to tables (groups) rather than being allowed to choose where to sit, so that as far as possible, each group would contain individuals from a range of backgrounds. Each group was asked to provide written feedback on the discussion sessions in the form of a feedback sheet. Not all groups at all events returned feedback sheets, nor did all groups answer all questions within a particular scenario. The responses on the feedback sheets were coded to allow for quantitative analysis and the presentation of statistics for each scenario.
Analysis of Responses to electronic polling questions
The same set of 17 electronic polling questions were asked at each of the consultation events, along with questions on delegates' sector and area of work/delivery. (See Annex 5 for list of polling questions). By necessity, the polling questions were closed questions, which asked delegates to select from a list of options. The responses for these were stored in Excel spreadsheets which allowed for statistical analysis, broken down by sector and area of work/service delivery.
Analysis of the Written Consultation
Online responses to the written consultation went automatically into a database. Responses received by hard copy were input into an identical mirror database and the two databases were merged. The file was converted into a spreadsheet for analysis. The spreadsheet was set up to make it possible to read along the questions to see one consultees response in its entirety, and also possible to read down the columns to see all of the responses to one question. The spreadsheet contained all the free text comments from consultees as well as the answers to the multiple choice questions.
The responses to the closed (multiple choice) questions were coded and put into the Statistical Package for Social Sciences ( SPSS), to produce top line statistical findings; percentages for each of the questions, and responses broken down by each of the background categories identified. At the request of the PVG policy team, the percentages provided were based on the total number of analysable responses to the consultation (187), as opposed to being based on the number of consultees who responded to each question. The 'not stated' category in the analysis identifies those who did not respond to the question.
A Qualitative Approach
Consultees free text comments were dealt with in a qualitative manner. In the spreadsheet, it was possible to sort by background group and by those who had answered the closed elements of the question in a certain way, allowing the free text answers to be themed. Qualitative Theme Grids were set up in separate spreadsheets for each question in the consultation, and had separate headings for each of the background groups. Similar types of comments were themed together in the theme grids. It was possible to see differences and similarities within and between different groups.
Using a qualitative approach to the free text responses meant that researchers included the broad spectrum of views that were expressed for each of the issues consulted on, without attempting to add a weighting to them. Quotes were used to illustrate points made by consultees and demonstrate the thought process and complexity of issues that lay behind the answers to the multiple choice questions.
Factual Accuracy
The views presented in this report are the opinions and perceptions of consultees, based on their understanding and could include misconceptions and misunderstanding of what has been presented in the consultation document. Therefore, points that are not factually accurate may appear in the report, as views have not been screened for factual accuracy. Even if a belief held by a consultee is factually incorrect, it is still something that they felt strongly enough about to comment on in the consultation and as such should not be ignored, as it is important to include the complete spectrum of views in the analysis.
Quality Control
The qualitative analysis was carried out by a small team of 3 researchers, with one researcher, taking overall responsibility for the initial sorting and coding of data and the setting up of the theme grids to ensure consistency. The researchers discussed the methods they were using and compiled and followed a set of guidance notes for conducting the qualitative analysis, once again to ensure consistency. The event analysis and the quantitative elements of the written consultation were undertaken by one researcher, who had attended all the events and had gained background knowledge on the policy issues being consulted on.
The Report
The purpose of the report is to bring together in a structured way the key findings from the consultation and to represent the range of opinions expressed by consultees. It focuses primarily on the responses to the written consultation, but also draws on information from the consultation events.
The structure of the report follows the structure of the consultation document. Within each of the chapters, for each question that had a closed, multiple choice element, an overview of the statistics was presented. This was followed by explanations why consultees had answered the way they had, using quotes to illustrate this. For questions that did not have a closed element, the comments were themed and presented, and once again quotes were used. Responses were also looked at by background type, and if there were differences between groups, then this was highlighted, differences within groups were also noted.
« Previous | Contents | Next »