Scalable Cyber Interventions Accelerating Productivity Practice for SMEs' programme (Cyber Well) explores whether the use of an online game can enhance the productivity of Dorset SMEs through encouraging the adoption of better data management and cyber security practices.
How to cite
Robson, Matthew; Pleva, Megan. 2020. "Scalable Cyber Interventions Accelerating Productivity Practice for SMEs' programme (Cyber Well) explores whether the use of an online game can enhance the productivity of Dorset SMEs through encouraging the adoption of better data management and cyber security practices.." AEA RCT Registry. AEARCTR-0006595. https://doi.org/10.1257/rct.6595-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Scalable Cyber Interventions Accelerating Productivity Practice for SMEs' programme (Cyber Well) is a Randomised Control Trial to test whether the use of online gamification can enhance the productivity of Dorset SMEs through encouraging the adoption of better data management and cyber security practices. Cyber Well is an integrated training and self-delivered programme building on four Proof of Concept studies in Dorset. The studies showed evidence that SMEs' growth and productivity can suffer directly from poor digital data management and careless cyber security behaviour. Furthermore, evidence shows that SMEs are unlikely to win new business contracts that increasingly require national security and data management accreditation. Conversely, those that have good practices through a combination of new contracts, compliance certification and organisational behaviour change increase productivity.
The primary outcomes being evaluated are whether the deployment of a cyber game with nudge learning increase the cyber and data knowledge resilience in SMEs, and whether this style of teaching encourages attitudinal changes and increases productivity in SMEs related to cyber behaviours and certification in a more effective way than typical cyber ‘push-learning’ training.
The trial design is a parallel group, two-arm superiority trial with 1:1 allocation ratio of Control: Intervention (aim: 150 SMEs in each). The project team ran three batches of delivery. The first batch started in March 2020 and the final batch in August 2020. Participant eligability requirements included that they were: South England based; an SME; and had enough de Minimis allocation.
The main comparison will be between the Intervention and the Control groups. This between groups comparison will take place following each quantitative data collection phase, i.e. at baseline, at initial follow-up and at two-month follow-up. Due to the use of block randomisation the team will also be able to compare performance across different sized businesses and across different job roles (IT or non-IT role) within each arm. In addition, within group comparisons across the three time points of baseline, initial follow-up and two-month follow-up and for each arm will be conducted. This may be done as a factorial analysis, depending on if the data meets the requirements for such a test (see below).
The analysis to be done will be determined by the normality and distribution of the data. In the requirements for ANOVA are met then the survey items with continuous outcomes such as most of the survey attitudes questions will be analysed using a 2 x 3 mixed ANOVA (intervention vs control/ baseline to initial follow-up to two—month follow-up). Post-hoc tests main effects and interactions will be conducted as appropriate.If the requirements for ANOVA are not met, then the Friedman test will be used to determine changes on continuous responses within each arm over the three quantitative data collection time points. Mann-Whitney will be used to make comparisons between each arm at each data collection point, with Bonferroni corrections applied as appropriate. Chi-square analysis will be used for any outcomes with binary choices, such as some of the true/ false items within the baseline and follow-up surveys. An appropriate variation on a chi-square such as McNemar’s test will be used to establish the statistical significance in any changes from baseline to follow-up.
Descriptive analysis will be used to report outcomes from any response items that use ranking, such as the attitude item on perception of barriers to implementing good cyber security practice. Due to COVID restrictions, the numbers of participants has been much lower than anticipated. As such, the project team decided to interview 20 of the participants who completed the training. This will provide insight into the training and rich evidence to substantiate the quantitative data. The team are currently in the final phase of data collection and will begin writing the final report in the next month.
Intervention
The Intervention has been designed to maximise participation (and learning) compared to more standard ways of learning. Drama, storylines, character immersion and nudges will all be utilised.
The Intervention comprises of:
• Baseline Questionnaire
An online tool/game split into 4 sections
• Module 1: the cyber game (including general cyber practices) which will take approximately 50 minutes to complete
• 15 Minute Nudge following Module 1
• Module 2: the cyber essentials game will take approximately 40 mins to complete (including GDPR and cyber information related to the cyber essentials process).
• 25-minute Nudge following Module 2
• Post Trial Baseline Questionnaire
• Final 2 Month Follow up Baseline Questionnaire
The overall course will take approximately two hours and ten minutes in total, over a two-month period with a week-long gap between each module and nudge. The weeklong gap between each interaction with the participant is to ensure that in case participants do not have time or forget to complete that week’s task, there is time in the ‘mop up’ week to account for this.
Experimental design
The trial design will be a parallel group, two-arm superiority trial with 1:1 allocation ratio of Control: Intervention. The project team are aiming to run two batches of delivery. The first batch will start running at the beginning of March 2020. The second batch (recruitment dependant) will aim to be delivered one month after that.
Once recruited onto the programme, the eligibility of SMEs to participate in the programme will be ascertained by checking that they are: Dorset based; an SME; and have enough de Minimis allocation. If the SME does have Cyber Essentials, they will not be removed from the project, but they will be randomised evenly across the two groups.
Once the sample is finalised baseline data (first questionnaire/test) will be taken. Following the baseline data collection process, the randomisation process will be delivered. This will be ‘blocked randomisation’ and will ensure that the number of participants in the trail arms are as equal as possible. For example, stratification by organisation size (micro, small medium) followed by randomisation within strata will be used to ensure that the two study arms are comparable.
Design details
Cited by
No citation data available for this registration yet.