Self-management open online trials in health [ SMOOTH ] an analysis of existing online trials

BACKGROUND: The use of public engagement and self-management in online clinical trials is growing with benefits, boundaries and minimal methodological guidance. This analysis explores whether running self-recruited online trials can provide trustworthy and useful answers to research questions. AIM: To systematically explore existing selfrecruited online randomized controlled trials of self-management interventions and analyze the trials to assess their strengths and weaknesses, the quality of trials reporting and to report how participants were involved in the research process. METHODS: The Online Randomized Controlled Trials of Health Information Database (ORCHID) will be used as a sampling framework to identify a subset of self-management self-recruited interventions. The trials will be used to explore the qualities of self-recruited online randomized controlled trials and to evaluate how useful they are for obtaining trustworthy answers to questions about health self-management and citizen research involvement. This research employs participatory action research where researchers and participants work as collaborators. SUMMARY: This analysis can provide an overall view of effective methods for online trials and to provide insights into integration for online trials development as early as the protocol planning stage.


FEEDBACK
We would appreciate, insights, practical ideas and suggestions to make this research better.Comments or email welcome.Amy Healingjia@msn.comand we will look for ways to integrate your feedback into our research plan.

Why look at self-recruited online randomized controlled trials?
ThinkWell 1 is an organisation that was founded following a meeting in Birmingham of health researchers, health service users, and clinicians who thought that (a) there was a mismatch between the questions and outcomes of most importance to the public and those that were researched 2 , (b) there was insufficient involvement of the public in health research 3 and (c) the internet and mobile communication were potentially useful and underused tools for re-dressing the balance.ThinkWell's stated mission was "to improve the health and wellbeing of citizens across the world by enabling them to make informed decisions about lifestyle, diet and health interventions through public-led health discussions, education and research, using the internet and the mass media as fundamental tools to reach the masses" 4 .In 2014 ThinkWell was converted into a charity.
Two key principles underlie ThinkWell's workone is evidence-informed decision making and the other is public and patient involvement in all aspects of research process to promote, and generate the evidence required for, informed decisions (see Figure 1)   ThinkWell commits to an evidence-based approach to investigate decisions that affect health and health care in all aspects of its work, such as, how to involve the public

Why engage the public in health research?
The public is the end recipient of healthcare interventions and it is important that research evidence guiding the use of healthcare interventions is relevant and useful to them 10,11 .
The public can become participants in research trials without knowing whether a trial is well run, ethical or even if it will ever be published 12 .Patients can be confused, vulnerable and unsure of how to switch roles between patients and participants 13 .Few resources are available for patients themselves to distinguish between safe and ethical research.Through interactive collaboration the public can share real-life knowledge with practical life problem solving to increase adoption of best practice in research and health literacy.Mining the ORCHID database provides the opportunity to discover what works in terms of engagement and methodology in online trials.This will provide the foundation for building a network of participatory research in online trials where citizens can take part in every aspect of planning a trial and are not limited to being only participants within the trial.

What are the benefits of participatory research?
In other research, even novice human input can be more sophisticated than that of computer automated data mining to discriminate what is of value in research. 14What has proved useful to bridge this gap in other sciences such as neuroscience and astronomy is to crowd source the public as citizen scientists 15 .They are given minimal training to classify scientific data and are able to volunteer and work online at their own pace and intensity 14 .This method may be useful for health research as information and demand are outstripping the ability of healthcare providers, academics and scientists to keep up with the present rate of health care research which doubles every 19 years.For example there are 75 new trials and 11 systematic reviews produced daily 16 .At the current rate health research outstrips medical school learning and becomes outdated during the lifetime of clinical practice 17 .This is where the public and patients can help, as they can become potential experts on their own conditions which is a narrow but important window where

Why this Research is important
ThinkWell plans to train the public and motivated patients who can mentor others, influence existing patient advocacy groups and partner with ThinkWell and other researchers to develop research protocols 1 .This inspiration for an informed health public will spread across cultures and be an agent for change to facilitate communication between research, science and health care.The use of public engagement and selfmanagement in online clinical trials is an emergent framework that will bring unique methodological challenges and benefits 3 .Working with a reliable established database for procuring primary studies can serve researchers and the public in terms of reducing start up research costs.This will be the first research of its kind using the ORCHID database and it is the only existing database that exclusively collates online trials.
This research is important to establish that running self-recruited online trials can be a trustworthy and useful methodology.This is relevant for both ethical and practical reasons, for example, to establish for funders that this is indeed a valid research approach.
Currently online trials may not be perceived by funders to meet the threshold standards of validation or credibility 18 .This reduces their influence as a priority for funding bodies 19 .
This project will employ participatory action research where researchers and participants clinical application.Evaluation will be conducted through systematic review of a subset of the qualifying trials, critical appraisal, surveys and interviews.

Identification of Trials
The ORCHID database represents the most comprehensive collation of online health trials available and was last updated July 2016.It will be used to identify the sub-set of trials that were self-recruited and investigated interventions involving the selfmanagement of health.If relevant online trials are identified that were published before this publication cut-off date, they will be added to ORCHID and included in this study.

Self-management
For the purposes of this research, self-management or self-monitoring of health is defined as use of a medical device, intervention or process that, while it may be recommended by a physician or other clinician, can be used or undertaken by the participant without assistance of a health care professional.For example, self-help, wellness, diet, activity, therapy online or anti coagulation medication that is monitored and titrated by the patient could be included, likewise asthma medications with peak flow measurement recorded by the patients are included, however interventions that are fully physician dependent for interpretation such as radiological films or lab work would not.

Self-Enrolment
Self-enrolment (self-recruitment) online is defined as when participants themselves sign up to, or enroll in, a trial online, via smart phone, tablet or computer.

Screening Title and Abstracts
Two researchers will screen the title and abstract of all citations in ORCHID that match health, self-management, self-help, intervention, self-recruit, self-enroll, and community for citations that match inclusion/exclusion criteria.The citations will be categorized as include, unsure (references to be checked), exclude.Full papers will be retrieved for

Full Paper Retrieval
Two researchers will screen the retrieved full papers to match them against the inclusion and exclusion criteria and categorize them as include, exclude or unsure.There will be discussion between the two researchers to resolve the status of an unsure paper to reach consensus on exclusion or inclusion.A third researcher will be consulted if there is still uncertainty.The rationale for exclusions will be documented in text and with a PRISMA 21 diagram outlining the fate of full papers.The degree of agreement and reasons for discrepancies will be reported in the results.

Obtaining a Representative Sample
The study will use a proportionate stratified sampling technique to include a percentage of subgroups from eligible citations.This allows representation of the online trials population avoiding artificially constraining the size to accommodate resource costs.This method makes it possible to include all subgroups or strata equally and allows for the observation of existing relationships between subgroups.The random selection of all the trials was not used as it could result in groups not having equal representation.We will group the studies into the following strata or sub groups:  Feasibility or Pilot studies  Full trials We will randomly select 50% percent of the studies from each stratum.

Sampling Rationale
There may be fewer feasibility and pilot trials than full trials or other groups however a scoping of the literature and consultation with content experts of trials methodology informed us that important choices about methodology and engagement may be detailed in the feasibility or pilot trial but not included in the final trials report.Sampling increases possibilities for representative inclusion.

Quality Assessment and Reporting
The included trials will be quality assessed for methodology strengths and weaknesses by two review authors.Discrepancies will be discussed and resolved by consensus, or by consultation with a third review author.

Reporting Tools
Critical Appraisal Skills Programme, (CASP) 11 questions to help you make sense of a trial 22 will be used and scored as unacceptable 0-2, Low 3-4, Medium 5-7 Medium High 8-9, High 10-11.The CASP and may be supplemented by Equator 23 guidelines as they pertain to qualities commonly used in reporting randomized controlled online trials for example: Items may be used from CHERRIES 24 for online surveys, CONSORT 25 , CONSORT PRO 26 for reporting patient reported outcomes.The quality tools will be piloted and adapted for best use of time and resources to capture the methodology.Skip technology (branching) will be used to make the critical appraisal process more efficient so that only questions that are relevant to the paper are evaluated and every piece of evidence needs to be handled only once per reviewer.

Cochrane Risk of Bias assessment
The risk of bias will be assessed using the Cochrane 'Risk of Bias' tool categorizing the risks as 'low', 'high', or 'unclear' risk.Individual bias items will be evaluated as described in the Cochrane Handbook for Systematic Reviews of Interventions 27 where any of the specified criteria for a judgment on 'low', unclear' or 'high' risk of bias justifies the associated categorization.

Public and Patient Involvement
There will be a narrative summary of how, when and at what stage(s) public and patient involvement occurred in each included study and whether the impact or value of the public involvement within the study was recorded.

Public and Patient Collaboration
Citizens are invited to comment on this protocol which will be registered and available on a publicly available repository, and shared via social media.There may be important outcomes that citizens can suggest to make online trials useful, engaging or a safer vehicle of research.We will ask how could quality and participation better be improved for online trials.In a recent review citizen input was credited with contributing to clinical safety standards 28 .Citizens will be invited to review, add to, clarity check and prioritize our list of quality assessment criteria and they may be included in the formation of a panel for a future Delphi concerning what to include for a protocol in an online trial.

SMOOTH Survey
Useful details about public and participant involvement in online trials might not be reported in the research publication.Further questions about the use of PPI in the trials will be investigated by inviting trial investigators or corresponding authors of the full sample of included studies to participate in a survey (protocol in appendix-2).The survey will be conducted using a validated survey program and will use guidelines available from Equator "Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES) 24 ".

Analysis
The analysis will explore interactions and correlates for areas of interest across or within the studies.It is not known at the protocol stage the exact areas that contain sufficient data to make this exploration feasible.For example, one question might be "How does participant resource cost throughout the trial and length of the trial influence the compliance, completion rates and effect size of the trial?" is one question that could be asked of the data.It is not known whether there will be sufficient data or homogeneity to perform meta-analysis.If regression modeling is indicated statistical software will be used.The numerical analysis will be supported using graphs and tables for ease of understanding and visual comparisons.

SMOOTH: Self-Management Open Online Trials in Health
What can we learn from existing trials?

Appendix-1 ORCHID search strategy and inclusion/exclusion criteria
The inclusion and Exclusion criteria plus the literature strategy is included below.: Inclusion criteria All studies that meet the following criteria were included:  randomised controlled trials using internet-based technologies in the trial process  studies using mobile technologies where there is also internet-based activity  studies investigating health research, public health research topics and settings. Studies that include educational or behavioral of health and well being topics  Studies involving patients or members of the public Exclusion criteria: The following studies were excluded:  Studies using mobile telecommunications technologies exclusively, with no internetbased content  Studies investigating interventions in social care or educational settings, where the main topic of investigation is not health related, or where clinical interventions are not included in ICD10. Studies where participants were health professionals or students

The ORCHID Search Rationale
A comprehensive literature search was completed to retrieve and import relevant studies into a bibliographic software package where they were screened against inclusion and exclusion criteria.
 Included studies were indexed and coded for the core stages of the research process, topic of study, interventions used, and location of the trial setting.The results were then included in the database to create a known set of internet-based randomised clinical trials in health and well-being  The studies were identified with a broad and comprehensive search strategy to accommodate the wide variations in terminology, definitions and applications of internet-based technologies within the research environment and limited indexing. Terms were mined from the search sources below, along with exploring core articles identified from scoping searches; and by experts in the field.A process of text word analysis was undertaken, and the identified relevant thesaurus and freetext terms, and combinations of terms, tested where possible.

Figure 1
Figure 1 Making informed decision

Figure 2 The
Figure 2 The ThinkWell research process adapted from Price and Burls 6 contributing to the chain to keep best research knowledge current is vital.This is a win/win scenario as the training and practice can enable the citizen scientists to become informed members of a health research team.Citizens can contribute to reducing the PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.2671v1| CC BY 4.0 Open Access | rec: 23 Dec 2016, publ: 23 Dec 2016 SMOOTH: Self-Management Open Online Trials in Health What can we learn from existing trials?workload and become competent to engage in the complex informed shared decision making required for health science research particularly in the design and running of online clinical trials.

Objectives 1 . 1 .
will work as collaborators to answer the research questions20  .This analysis can provide an overall view into what works for online trials and how methodology and public engagement might be best utilized and integrated into the development of online trials as early as the draft protocol stage.SMOOTH: Self-Management Open Online Trials in HealthWhat can we learn from existing trials?Research Question(s)What can be learned about self-recruited online randomized controlled trials for selfmanagement of health interventions from existing trials?1.How useful were these trials for obtaining trustworthy answers to questions about health self-management?2. To what extent did they engage the public in health research 3. What is the quality of these trials in their reporting?Aim To systematically explore existing self-recruited online randomized controlled trials of self-management interventions and analyze the trials to assess their strengths and weaknesses and to find whether, and if so how, participants were involved in the research process and the usefulness, if any, of public involvement.To critically appraise and extract a subset of self-recruited online trials of selfmanagement interventions to:  Identify their strengths and weaknesses  Assess the quality of their reporting 2. Identify how public or patients were involved in the research process 3. To use information obtained during the critical appraisal to see to what extent these factors influence the success or failure of trials.4. To inform the development of guidance for the design, conduct and reporting of online RCTs of self-management interventions Methods The ThinkWell Online Randomized Controlled Trials of Health Information Database (ORCHID) will be used as a sampling framework to identify a subset of self-management self-recruited interventions.The trials will be used to explore the qualities of self-recruited online randomized controlled trials and to evaluate how useful they are for obtaining trustworthy answers to questions about health self-management and public engagement/involvement in research.Usefulness will be identified as ways the trial can contribute to evidence based practice and PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.2671v1| CC BY 4.0 Open Access | rec: 23 Dec 2016, publ: 23 Dec 2016 SMOOTH: Self-Management Open Online Trials in Health What can we learn from existing trials?

Inclusions 11 
Randomized controlled trial  Self-enrolled online  Use of internet-based technologies, computer, tablet, smartphone, in the trial process Interventions of health and well-being including educational or behavioral components Outcomes recorded or reported by the participants themselves Exclusions Studies investigating interventions in social care or educational settings, where the outcome of investigation is not health related.Data not collected online or by mobile technology  Studies where the population was exclusively health professionals, educators, or students for training and not a specific health intervention PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.2671v1| CC BY 4.0 Open Access | rec: 23 Dec 2016, publ: 23 Dec 2016 SMOOTH: Self-Management Open Online Trials in Health What can we learn from existing trials?Studies where the population are enrolled as patients for trial purposes and require physician intervention for reporting outcomes  Studies that are not randomized controlled trials or are secondary analysis, cost effectiveness research No abstract available  Conference presentation or poster only  Aborted/Withdrawn Trials

SMOOTH: Self-Management Open Online Trials in Health What can we learn from existing trials? meaningfully
in research and how to conduct research well.Before ThinkWell launches any online trial, we need to know about best practice for methods.A starting point was to look at existing research that used online trials for health research to see if there were any lessons that could be drawn.A first step was to construct a database of the trials that had used the Internet in a significant way for health research.PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.2671v1| CC BY 4.0 Open Access | rec: 23 Dec 2016, publ: 23 Dec

Self-Management Open Online Trials in Health What can we learn from existing trials? categories
PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.2671v1| CC BY 4.0 Open Access | rec: 23 Dec 2016, publ: 23 Dec 2016 SMOOTH: ; include and unsure based on screening of the title and abstract.Exclusions will not be documented at this stage.