Real world data: can we rely on it to shape cancer policies and practice?

Marc Beishon

The digital era has opened up huge possibilities for collecting, storing and analysing data. How wonderful if it could be harnessed to answer questions that would allow us to get the best for every patient and for health systems overall. Marc Beishon takes a look at what current initiatives can offer, and at hopes for the future.

In November 2018, the US Food and Drug Administration granted accelerated approval to larotrectinib (Vitrakvi), a drug that targets a neurotrophic receptor tyrosine kinase (NTRK) gene fusion in a range of adult and paediatric tumours. European approval followed last autumn. It was the second approval for a ‘tumour agnostic’ drug and the first for one designed to be so, but was given the green light on trials with just 55 patients, and has a dizzying price tag, like many of these new agents.

The increasing number of hugely costly drugs – including those such as larotrectinib that target small populations of patients with rare mutations – is placing even more pressure on healthcare systems to determine effectiveness in clinical practice. Treatments are not just single oral agents, but can be complex procedures such as CAR-T immunotherapy; they can be given at various lines in disease progression, and in sequences and combinations with other drugs; and treatments can include not just drugs but other often complex and costly interventions such as radiosurgery.

Innovative treatments should not face unnecessary barriers to reaching the clinic, but at present there is considerable uncertainty about their effectiveness and whether they will find their way into standards of care that can be confidently reimbursed by healthcare systems. The extent of this uncertainty was highlighted in a much cited study in the British Medical Journal which found that, of 48 cancer drugs for 68 indications approved by the European Medicines Agency between 2009 and 2013, only 24 of the indications had a significant survival benefit at approval, with a modest median time of just 2.7 months. Three years following approval most still did not have conclusive evidence of either extended or improved life.

There is recognition that fundamental changes may be needed to the current regulatory model, the way treatments are introduced into clinical practice, and the interface with health technology assessment (HTA) agencies.

One solution being increasingly explored is making greater use of data gathered in the ‘real world’. A number of specialist tumour registries, research platforms, health record integration systems and value scales are now live.

Real word data could help resolve uncertainty about the effectiveness of new treatments

What counts as ‘real world data’?

The term ‘real world data’ (other related terms are ‘real world evidence’, and ‘evidence from clinical experience’) generally means anything collected outside of a classical randomised controlled trial (RCT). Data elements typically include the characteristics of patients, the treatments they receive, and the outcomes achieved.

A review of definitions has found that in the spectrum of data – derived from the highly selected RCT at one end, to electronic health records of routine clinical practice at the other ‒ there is a variety of opinion on what qualifies.
Definitions have been developed by various groups such as the Innovative Medicines Initiative (IMI) in its getReal programme, and the Rand Corporation, but some stakeholders have not adopted them and have not even developed their own definition. A common understanding of what ‘real world data’ means would certainly help to inform decision making when stakeholders discuss its use, say the review authors.
Various types of study can contribute to real world data, though not all definitions include them all. They are:

  • Observational studies
  • Studies that may have a protocol but investigators do not have control over aspects such as treatment assignment or selection of a study population
  • ‘Pragmatic clinical trials’, which measure the benefit a treatment produces in routine clinical practice, and may be randomised.

A broad definition admits all these types of studies.

Clinical data sources are the other main contributor to real world data. Registries and electronic health records are most cited as resources, followed by insurance and administration databases (i.e. healthcare usage), and patient reported outcomes. In cancer, there has been much activity in registry and health records, and many researchers and organisations see great promise in large-scale data collection and analysis.

CancerLinQ (US)

One of the most ambitious attempts to gather, curate and analyse clinical data on inputs and outcomes is the CancerLinQ project, which was set up by the American Society of Clinical Oncology (ASCO). In the US, where only 3% of cancer patients are enrolled in clinical trials, CancerLinQ aims to gather and utilise information from the broader patient population by capturing data from the records of participating oncology practices.

The ultimate goal is to uncover patterns and trends, and allow centres to measure their care against that of their peers and recommended guidelines. But it is has faced major data challenges, as previously reported by Cancer World (‘Harnessing big data to drive up quality of care’, September 2016). While some types of clinical data are coded and easily accessible, such as tumour stage and patient performance status, other parts of a record typically comprise unstructured data from clinics ‒ pathology and imaging reports, for example.
Robert Miller, CancerLinQ’s medical director, has said this “greatly impacts” the system, because someone has to read these reports to glean insights from clinical events, and it is hard to convince doctors to record such data in a structured way. ASCO has been using “human curators” to abstract the data, a process that can be complemented with natural language technology, but there is still a lot of local variation and lack of quality control that renders some documents unusable.

Oncology Data Network

Europe now has a similar project in the Oncology Data Network, which has reached a milestone in signing up 100 hospitals in six countries. This network ‒ said to be the first multi-country European initiative of its kind ‒ aims to aggregate data on how drugs are being used in practice “in near real time”. Backed by a consortium of pharmaceutical companies, and led by data sciences company IQVIA, through the Collaboration for Oncology Data in Europe (CODE), it is an ambitious project and in its early stages, where the key is the technology and data platform.

The network automatically collates a standardised dataset from existing hospital patient records, says IQVIA, thereby minimising administrative burden for the treatment centre. Initially it is collecting structured data on patient attributes ‒ disease diagnosis, histology, stage, death etc. ‒ as well as drug regimen, including line and dosing; and timeframe and geography.

Questions it could answer include: What are the chemotherapy completion and discontinuation rates for different regimens in different settings of care and patient cohorts? How does the rate of adoption of a new anti-cancer medicine at a centre compare? Are there patients that could meet the eligibility criteria for upcoming clinical trials?

Compared with ASCO’s CancerLinQ, IQVIA says the level of automation is likely to be an advantage, and it will also provide input to comparative drug access and financial analysis around Europe.


Completeness and robustness of data are among the major concerns about using such sources as inputs to decision making about treatment effectiveness. In the UK, Public Health England’s National Cancer Registration and Analysis Service (NCRAS) has invested heavily in data quality as it builds on one of the world’s largest cancer patient databases. In addition to analysing the comparative performance of cancer centres and the trends traditionally addressed by population registries, NCRAS is also working towards the possibility of clinicians using the datasets for personalised medicine.

This includes the Systemic Anti-Cancer Treatment (SACT) dataset, which aims to help oncologists compare their patients with others, and can also help evaluate the effectiveness of new therapies. The advantage that the UK’s National Health Service has is that it can mandate the collection of such data to an agreed and published standard.

NCRAS emphasises a drive to obtain complete data and to eliminate as far as possible anomalies that can dilute the robustness of even a huge database. Some of the population-level statistics are already available freely at, including treatment breakdown by tumour type for chemotherapy, radiotherapy and surgery. Certain datasets are also available for download by researchers.

ESME (France)

In France, which also has a universal healthcare system, one of the largest real world research studies in Europe is Unicancer’s Epidemiological Strategy and Medical Economics (ESME) programme, which is industry funded and uses a methodology similar to a clinical trial to collect data from patients in comprehensive cancer centres. It started with metastatic breast cancer (MBC) patients, and has about 23,000 MBC cases on the platform; its primary aims are to identify and quantify unmet medical needs and evaluate management. It is a retrospective project with no influence on treatment practice or interaction with oncologists.

As Suzette Delaloge, a medical oncologist and head of breast cancer at Gustave Roussy Institute, Paris, explains, “A secondary aim is to identify what we do well and less well to predict better treatments with the outcome measures of overall and progression free survival.”

ESME would seem to be an ideal platform on which to investigate recently introduced agents now being applied in MBC. But Delaloge says there are major limitations. “For now, this type of real world data must be used with caution, as strong methodology is needed to evaluate drug effectiveness.” The outcome most important to her as an oncologist is overall survival, which means waiting for around five years to see effects in MBC and, while trends in subtypes such as HER2 are clear, investigating certain regimens within the database over a long period across many centres has too many biases to make recommendations about effectiveness.

“We try to collect data objectively using external people and not by oncologists and their teams, but it is costly and just not perfect at present,” Delaloge says. “We also don’t have other data, including on safety and adverse events, and we do not yet link to the national insurance system, which would help measure economic costs and value.”

ESME has also been developing a database on ovarian cancer and is embarking on a lung cancer database. A lot of projects about medical use will come from it, no doubt similar to England’s NCRAS system, as it is also has the advantage of reporting from a healthcare system that admits all regardless of socioeconomic background.

Dutch Cancer Treatment Registries

The need for information about costs and value was a key driver behind the decision in the Netherlands to set up the Dutch Melanoma Treatment Registry (DMTR) in 2013. The treatment landscape for advanced melanoma had undergone rapid change, with the advent of new immunotherapies and BRAF inhibitors, which have a high price tag together with considerable uncertainty how to use the treatments to best effect. The Dutch Ministry of Health has made reimbursement conditional on treatment taking place only in centres taking part in the DMTR.

Wim Goettsch, an HTA specialist at the National Health Care Institute in the Netherlands, and associate professor in HTA at Utrecht University, says the registry is part of a “framework of disease-specific registries” that is planned to gain insight into how drugs work in practice, by “collecting more data at or around, and even before, marketing authorisation, and it should be an ongoing process so we can follow how drugs work over time.”

In addition to clinical data, the registry records data on quality of life, healthcare utilisation, informal care and productivity losses. It is a step up from ESME in these aspects, and is put forward as a blueprint for such registries. But it also comes at probably even higher cost. “If we wanted to do this for every disease it may be very expensive in the time and training of data managers and physicians, as well as building systems that are standard ways to collect such information,” says Goettsch. “It’s certainly the future in the Netherlands, but we should look for more efficient ways to do it, for instance by building on existing data sources that use electronic patient records.”

A paper on the DMTR notes the financial and administrative burden and its sustainability, estimating the cost per patient at about €500. But this amounts to less than 1% of the cost of these expensive drugs, and as the authors comment, “It will be important to decision-makers whether securing a small percentage of the total treatment budget for obtaining quality information is acceptable” (Eur J Cancer 2017, 72:156‒65).

Funding is crucial, Goettsch adds, and pharmaceutical companies are an obvious source. “Manufacturers should be responsible for some additional real-life work on drugs – why should society pay for all of it? But we need to be thoughtful about the balance between public and private funding so we secure independent monitoring and don’t create registries dependent on private sources.”

Flatiron – a Roche subsidiary

Flatiron built up an independent business in the US by curating and harmonising clinical data from hundreds of thousands of electronic health records from patients in US academic medical centres and community clinics. In a transaction valued at US$1.9 billion, Roche recently acquired the company, and is now using it to generate real world data on the effectiveness of its new drugs, such as alectinib (Alecensa) for non-small-cell lung cancer.

Disease-specific registries and studies

Responding to increasing pressure to demonstrate real world effectiveness, some pharmaceutical companies have also launched increasingly ambitious registries and registry studies in cancers where they have a particular interest.

Takeda’s Insight MM on multiple myeloma is one. This global, prospective, non-interventional observational study is gathering data on the demographic and clinical characteristics of patients treated in real world settings, together with details of their therapeutic regimen, and outcomes, including patient-reported outcomes. This is a disease where a rapid expansion of treatment options over the past 20 years has greatly improved survival rates, but with very little clarity about which options work best for which patients in which disease settings, or what the optimal dose, timing, duration or combinations and sequences are. The study seeks to answer some of those questions.

Lung cancer is another field that has seen a significant expansion of treatment options together with improvements in survival over the recent decades. BMS, a developer of immuno-oncology drugs, which are increasingly important in this disease, has set up I-O Optimise, which draws data from a number of real world data sources across Europe relating to characteristics, treatment patterns and clinical outcomes of more than 45,000 patients a year with non-small-cell lung cancer, small-cell lung cancer and mesothelioma.

These are some of the larger examples among a vast array of disease-specific registries set up by companies, academia and even patient advocacy groups. While each of them generate data that throw light on a variety of aspects of the management of different indications, their proliferation raises questions about fragmentation and the potential to harmonise and standardise the data collected to maximise its power.

No substitute for randomised trials

High-quality databases and registries are clearly needed, and if there is one thing everyone agrees on, it is that the data need to be robust. In a white paper led by the EORTC (European Organisation for Research and Treatment of Cancer) together with members of the BioMed Alliance, the authors say that it is critical to have Europe-wide population-based registries with clinical data, biological and imaging data, biomarker test results, and data on all therapies received and outcomes, but “controlling the quality and accuracy of the data is essential”, and that “minimal quality control requirements for building these databases and registries will need to be developed and implemented” (Eur Respir J 2019, 53:1801870).

But as part of a ‘treatment optimisation’ manifesto, the EORTC and others are also calling for a new model of treatment development ADD LINK that better addresses the needs of patients and society (Mol Oncol 2019, 13:558–566). The way forward, the authors say, is to ramp up not just data collection but also applied clinical research, from the time treatments are authorised, which would still ensure patients receive them, with regulators and HTA agencies committed to a post-authorisation strategy. Funding of research could come jointly from healthcare systems and industry.

Denis Lacombe, EORTC director general, points to EORTC’s own work, for example on the use of radiotherapy and drugs in treating glioblastoma, as the kind of practice-changing research that is needed. In their paper, the DANTE trial in the UK and STOP-GAP in Canada are mentioned – both are RCTs supported by independent funders and are looking at optimal treatment duration of immunotherapies in melanoma, which was not adequately determined during development.

Lacombe expresses concerns about the limitations of initiatives like ESME and CancerLinQ, commenting that it would take an enormous effort to structure such data to iron out treatment-related uncertainties. “It depends on the purpose for assembling real world data. These projects can certainly be helpful in diagnostic, monitoring and uptake of treatment strategies. But when it comes to measuring, assessing and making decisions about treatment effect, things are much more difficult, so it is advisable that they are seen as complements to classical clinical research methodologies.”

“When it comes to… making decision about treatment effect, it is advisable that real world data are seen as complements to classical clinical research”

His remarks would seem to be borne out by the findings from two studies using data from the ESME database of metastatic breast cancer (MBC). On the positive side is a paper that identified improvements in survival, but only in HER2+ cases. It highlighted the unmet need in the luminal and triple negative subtypes, which may be addressed as new targeted and immunotherapies enter clinical practice (Eur J Cancer 2018, 96:17–24).

However, that database was found to be insufficiently robust to answer questions about the impact of adding bevacizumab (Avastin) to chemotherapy in treating patients with HER2-negative metastatic breast cancer (Ann Oncol 2016, 27:1725‒32). ESME provided a large patient sample – about 3,400 receiving bevacizumab and paclitaxel, and 1,300 paclitaxel alone, and the bevacizumab group had significantly better overall and progression free survival. But the authors said: “Our data cannot support extension of current use of bevacizumab in MBC,” citing limitations in the data, including patient selection differences, lack of inclusion of performance status, potential bias of only being done in top centres, and possible data collection differences.

As oncologist Christopher Booth at Queen’s University Cancer Research Institute, Canada, notes with colleagues in a paper, ‘Real-world data: towards achieving the achievable in cancer care,’ “…the comparison of outcomes between nonrandomised groups of patients who have received different treatments in routine practice remains problematic and is not a substitute for RCTs,” (Nat Rev Clin Oncol 2019, 16:312–25). Real world data can offer important insights, they argue, but investigators “need to move beyond simply describing gaps in care and towards designing intervention studies to improve patient care and outcomes”.

The EORTC’s treatment optimisation manifesto sets out the principles. Taking this forward will be a political as well as a scientific task, and Cancer World will shortly report on progress for this optimisation agenda.


1 Trackbacks & Pingbacks

  1. Does it work for my patient? A pragmatic approach to building evidence on clinical effectiveness - Cancerworld

Leave a Reply

Your email address will not be published.