This week, the UK’s National Health Service (NHS) and National Institute for Health and Care Excellence (NICE) released a document titled Evidence standards framework for digital health technologies. An accompanying piece in The Lancet by Greaves et al. (2018) aims to describe the goals of the document at a high level.
The authors state that key principal used to inform these guidelines is that the level of evidence that should be required is proportional to the risk to the patient.
For example, digital tools that provide diagnosis or treatment are in the highest level for which the minimum evidence requirement will be a high-quality experimental or quasi-experimental study with comparative data on patient outcomes. Support services and technologies designed to simply communicate information are lower in the hierarchy and the requirements are therefore designed around the credibility and accuracy of the content, including whether they meet established standards for the quality of the information provided.
The authors note that because digital health technologies tend to be iteratively developed, new techniques may need to be used such as Multiphase Optimisation Strategy (MOST) and sequential, multiple assignment, randomized trials (SMART) [see Collins et al. 2017]
Yet in general, the guidelines impose a similarly high standard of evidence on digital as compared to conventional technologies.
However, there should be no lowering of the quality bar and experimental or quasi-experimental studies using traditional or innovative methodologies need to conform to high standards, including transparency of methods, a-priori analysis plans, and full publication of all results.