RoB 2 Excel Marco Form Manual (Beta Version 7)

Date: 15 Oct 2020

Table of Contents
1. General notes ..................................................................................................................................... 2
2. Getting started with your first RoB 2 assessment ................................................................................ 2
3. Editing existing assessments ............................................................................................................... 3
4. Deleting an existing assessment .......................................................................................................... 4
5. Creating a summary table and risk of bias graph ................................................................................. 4
6. Risk of bias figures .............................................................................................................................. 5
7. Populating results for printing ............................................................................................................. 5
8. Discrepancy check............................................................................................................................... 6
9. System and software requirements ..................................................................................................... 6
10. Contact ............................................................................................................................................. 6
11. Update ............................................................................................................................................. 7

1
1. General notes
• Enable Macros before you start
• Each workbook can take a maximum of 300 assessments
• Although this tool can manage different outcomes and different effects of interest (i.e. ‘intention-to-treat’
and ‘per-protocol’ effects) in the same review, we recommend that different files are created for different
outcomes and effects of interest, so that separate summary tables and figures are produced for these.
• A YouTube channel contains short clips to illustrate each of the steps described below (see ‘Demo clip’ sign)

<Back to Table of Contents>

2. Getting started with your first RoB 2 assessment Demo clip

❶ Click the 'RoB 2 Assessment Form' button to initialise the user form (note: the form will look
slightly different in Microsoft Excel for Mac, but functionality is not affected).

❷ Give each assessment a ‘Assessment ID’, and enter this in the relevant space on the form.
The ID can be letters, numbers, or a combination of both, and is used to identify a specific risk of bias
assessment. As RoB 2 is an outcome-based assessment, we recommend you to have a naming system
incorporating the outcome. This will be used to match assessments done by another reviewer in the
‘Discrepancy check’ function.

❸ Complete the assessment by filling-in all boxes on the form. Each bias domain of RoB 2 is on a separate tab in
the form. Answers to the signalling questions should be selected from the drop-down menu, and there is
space for justification for your answers.

❹ Give a final judgement and justification for each domain after reviewing your answers to the signalling
questions and the suggested algorithm results. Click the 'Save' button.

❺ Enter a different ‘Assessment ID’ to create a new assessment.

Note
• Double-clicking on the signalling question causes guidance on answering the signalling question to appear.

• Double-clicking the justification column in each domain will populate all the notes from the signalling
questions. This is made to facilitate the Revman outputs.

• RoB 2 incorporates an algorithm to produce risk of bias judgements for each domain, based on the answers
to the signalling questions. Clicking the ‘Algorithm’ button will cause such suggested judgements to appear.
These can be used to inform the ‘Assessor’s judgement’ in the adjacent box.

• There is a box that enables you to input a weight for each result. This is set to 1.00 by default (each result
has equal weight), but can be edited to reflect the weight given to the result has in the meta-analysis. This is
only relevant for the plots created by clicking the ‘Summary’ button on the ‘Intro’ sheet (explained below).
Note that weights should be entered in the same format for all studies (e.g. decimal or percentage)

2
<Back to Table of Contents>

3. Editing existing assessments Demo clip

There are two ways to edit an existing entry: a) by using the interactive form; or b) by directly
editing the excel sheet containing the results:

a. Using the interactive form

❶ Find or type the ‘Assessment ID’ on the user form. The saved version of the assessment will be loaded.

❷ Make the necessary edits.

❸ Click the 'Save' button.

Note
• The suggested risk of bias judgement for the
b. Edit the Results sheet directly
domain will not update automatically. If you have
❶ Find the sheet 'Results'. made edits to your answers to your signalling
questions you may wish to click on the ‘Algorithm’
❷ Make the necessary edits. button again.
❸ Save the excel file.

<Back to Table of Contents>

3
4. Deleting an existing assessment Demo clip
You will need to delete an existing assessment in the 'Results' sheet:

❶ Find the row corresponding to the assessment you wish to delete in the 'Results' sheet.

❷ Delete the whole row.

❸ Save the excel file.

<Back to Table of Contents>

5. Creating a summary table and risk of bias graph Demo clip

This outputs details of each assessment (Assessment ID, Study ID, Reference, Outcome, Result,
Weight) plus domain level and overall risk of bias judgements. It also outputs risk of bias graphs that
illustrate the proportions of results or information at low risk, some concerns or high risk of bias for
each domain (example risk of bias graph shown below).

❶ Click the 'Summary' button on the ‘Intro’ sheet.

❷ All assessments should appear in the 'Summary' sheet.

Note
❸ A plot of the percentage of risk of bias assessments at each
• Change the order of studies by
level of risk of bias per domain will be displayed on the right-hand
sorting results in the ‘Results’
side of the sheet. The default setting results in equal weighting for
sheet.
each study (and the plot can be interpreted as showing the
proportion of results at each level of risk of bias). If weights have
been entered, the plot is weighted by this (and can be interpreted as the proportion of information at each
level of risk of bias).

❹ Cross-check the results if necessary.

<Back to Table of Contents>

4
6. Risk of bias figures Demo clip
This outputs Cochrane-style risk of bias figures (“traffic light plots”), which display the domain and
overall judgements study-by-study, as shown below. Figures are displayed separately according to
effect of interest ('assignment to intervention ' or 'adhering to intervention')

❶ Click the 'Figures' button on the ‘Intro’ sheet.

Note
❷ Figures will be produced in the ‘Figures (ITT)’ or ‘Figure (PP)’
sheet depending on results. • Change the order of studies by
sorting results in the ‘Results’
❸ Figures can be further edited, for example by adding borders
sheet.
and lines using normal excel functionality, before being copied and
pasted into manuscripts (we suggest pasting as 'picture' to avoid distortion in format).

❹ A separate button in each Figure sheet can print figures and saved as PDFs.

Intention-to-
treat Unique ID Study ID Experimental Comparator Outcome Weight D1 D2 D3 D4 D5 Overall
ID1 Study1 Experimental Comparator Effectiveness 1 + ! ! ! ! - + Low risk

ID2 Study2 Experimental Comparator Effectiveness 1 + ! ! ! ! - ! Some concerns

ID3 Study3 Experimental Comparator Effectiveness 1 - ! ! ! ! - - High risk

ID6 Study6 Experimental Comparator Effectiveness 1 ! ! ! ! ! -

ID7 Study7 Experimental Comparator Effectiveness 1 + ! ! ! ! - D1 Randomisation process

ID8 Study8 Experimental Comparator Effectiveness 1 + ! ! ! ! - D2 Deviations from the intended interventions

ID9 Study9 Experimental Comparator Effectiveness 1 + ! ! ! ! - D3 Missing outcome data

ID10 Study10 Experimental Comparator Effectiveness 1 + ! ! ! ! - D4 Measurement of the outcome

D5 Selection of the reported result

<Back to Table of Contents>

7. Populating results for printing Demo clip

This outputs the full results for each study assessment, including the answers to signalling questions
and the justifications given for answers, into a template. This may facilitate double-checking or
allow the creation of supplementary materials for publications.

❶ Click 'Populate printing' on the ‘Intro’ sheet.

Note
❷ Results will be copied and pasted in the 'Print (ITT)' or
‘Print (PP)’ sheets. • Change the order of studies by sorting
results in the ‘Results’ sheet.
❸ You may need to adjust the height and width of
columns and rows to accommodate text used for • To ensure that the width of the
justification of answers. template prints on one A4 sheet, we
suggest selecting the option of scaling
❹ Change page breaks for printing if necessary. so that all columns fit on one page
when printing.

<Back to Table of Contents>

5
8. Discrepancy check Demo clip
This facilitates the discrepancy check between two reviewers' answers and allows the output of
agreed (final) judgements. The two reviewers should enter their assessments into different Excel
files. IT IS IMPORTANT BACK UP YOUR RESULTS (first reviewer’s results) BEFORE USING THIS
FUNCTION as the first reviewer’s results will be replaced with the agreed judgements!

[Setup]

❶ Copy and paste ALL Reviewer 2's results (from their sheet called ‘Results’) into the 'Check' sheet.

❷ Ensure that the same ‘Assessment ID’s have been used for the assessments of the same results in BOTH
reviewers' results.

[Use] Note
❶ Click the 'Discrepancy Check' button on the ‘Intro’ • Reviewer 2's results cannot be edited in
sheet. this form.
❷ A user form should appear.

❸ Both reviewers' answers should be displayed side-by-side. Judgements will be coloured in red if the
judgements reached by the two reviewers are different.

❹ Edit Consensus with final decisions and save. Results can then be produced using the ‘Summary’,
‘Figures’ and ‘Print’ functions as described previously.

<Back to Table of Contents>

9. System and software requirements

Microsoft Excel 2016 in Windows 10/7. The tool appearance looks different and might function differently in the
Macintosh operating system. Please make sure you update your Excel to date.

The compatibility with older versions of Microsoft Excel has not yet been tested.

<Back to Table of Contents>

10. Contact
This is a beta version based on the RoB 2 version dated 22 Aug 2019 If you have any problems while using this form
or any suggestions, please contact:

Vincent Cheng via [email protected]

The development of the RoB 2 tool was supported by the MRC Network of Hubs for Trials Methodology Research
(MR/L004933/2- N61), with the support of the host MRC ConDuCT-II Hub (Collaboration and innovation for Difficult
and Complex randomised controlled Trials In Invasive procedures - MR/K025643/1), by MRC research grant
MR/M025209/1, and by a grant from The Cochrane Collaboration.

6
 The RoB 2 tool is originally licensed under a Creative Commons Attribution-Non Commercial-
NoDerivatives 4.0 International License. This Excel Macro Form is built based on the RoB 2 tool as a
copy and endorsed by the RoB team. This form is licensed under a Creative Commons Attribution-NonCommercial-
ShareAlike 4.0 International License, which gives reviewers freedom to modify and tailor the form for their own use
as long as you credited the original work without making money or trying to make money from the use. However,
any changes, transform or development upon the RoB 2 tool itself (as known as ‘derivative work’) will be required to
contact the RoB 2 team ([email protected]).

<Back to Table of Contents>

11. Update
Key updates from V6:

• Updated the instructions with latest changes in the form and added YouTube videos to illustrate the use of
the form
• Adjusted form size to resolve display problems in devices with low screen resolutions
• Added double-click to populate comments from signalling questions
• De-bugged ‘Discrepancy Check’ function for matching studies from the second reviewer
• De-bugged ‘Summary’ function for ‘pre-protocol’ assessments
• Separated ‘Figures’ into two tabs to simplify the figure production
• Added a new function to print Figures
• Revised the wording and aligned the signalling questions and elaboration with RoB 2 cribsheet dated 22 Aug
2019
• Set ‘intention-to-treat’ assessments as default after creating a new study
• Removed ‘Double-click’ for editing lists of resources and ‘pre-protocol’ questions to simplify the editing
process
• Highlighted certain essential columns in the form

<Back to Table of Contents>