Conducting clinical trials in people with rare cancers is not easy when the number in a small trial do not add up to convincing evidence. Now some researchers are pressing for a new approach–using Bayesian trial designs to make the most of available knowledge.
When you are diagnosed with cancer, the last thing you want to hear is that medical experts have few treatment options to choose from and not much evidence on which to make that choice. Yet this remains the reality for many of the four million cancer patients in the EU-27 who are living with a ‘rare cancer’.
There are 186 of these rare cancers (using the recently proposed definition of a cancer diagnosed in fewer than 6 in every 100,000 people per year), with the seemingly contradictory result that almost one in four cancer patients has a rare cancer.
The consequences can be seen in their markedly poorer prognosis. Five-year survival figures – which broadly speaking reflect the efficacy of treatment – show patients with rare cancers do significantly worse, with fewer than half surviving for five years (47%), compared with almost two-thirds for patients with more common cancers (65%). While this may in part reflect the inherent nature of these particular cancers, it is also probably a result of the comparative difficulty of learning about how to treat rare cancers.
How to advance the cause of this disparate group of patients, and unblock progress in improving treatment strategies and developing new therapies, is a question that is commanding the attention of an increasingly coherent community specialising in rare cancers. In early February, around 50 of them – clinicians, researchers, patient advocates, statisticians, epidemiologists, pathologists and representatives from cooperative trials groups and pharmaceutical companies – spent two days trying to find common ground on how to conduct clinical research where patient populations are small.
Organised by ESMO and Rare Cancers Europe, the conference had two aims: to bring everyone involved in rare cancers behind a research strategy that could significantly speed up the development of an evidence base, and to build a united front that can be used to seek agreement on how regulators and payers can better meet the needs of this group of patients, for whom traditional standards of evidence are difficult to achieve.
A question of numbers
Paolo Bruzzi, clinical epidemiologist at the National Cancer Institute in Genova, Italy, explained the nub of the problem. According to the traditional rules of medical evidence nothing counts but the numbers – which is exactly what rare cancers don’t have.
To demonstrate the value of a new treatment or treatment strategy, you have to show beyond doubt that it benefits the intended group of patients – better survival, better quality of life – more than a comparator, which is usually the standard of care. Using the traditional ‘frequentist’ approach, this involves treating enough patients to show that the difference in outcome between the trial arms is big enough for it not to have come about by chance – the all-important P-value.
The standard P-value required by regulators – and payers – is P ≤ 0.05, which means the odds of the demonstrated difference having come about by mere chance is less than 1 in 20. As chance will always play a larger role in the outcome where numbers are small (throwing a six two times out of four is much more likely than throwing a six 20 times out of 40), proving a difference is not just chance requires large numbers of patients: the smaller the difference, the larger the number of patients required.
Where there are too few patients to prove that an observed difference could not have come about by chance, the study is said to be ‘underpowered’. As a general rule of thumb, said Bruzzi, trials of therapies for patients with early-stage disease require 500–5000 patients, while in advanced disease the numbers are a bit lower, at 300 –1000.
As a consequence, groups of patients who cannot muster this level of participation in a clinical trial risk being excluded from the world of evidence-based medicine: no research, no ‘standard of care’, no guidelines, no access to approved therapies.
“As a rule of thumb trials of therapies for patients with early-stage disease require 500–5000 patients”
The Bayesian alternative
Prompted by the need to develop an evidence base for treating these smaller groups of patients, doctors, researchers and drug developers have begun to turn to an alternative methodology. Originating from a theorem developed by an English priest and mathematician, the Reverend Thomas Bayes, and first published in 1763, the Bayesian approach to modelling probability has one great advantage over frequentist approaches: it enables all types of relevant information to be fed into the probability model.
Using a frequentist approach, you may have a well-designed and rigorously executed randomised controlled trial that comes up with impressive results, yet fails the test of significance because too few patients were spread across the trial arms to demonstrate that the result could not have come about by chance. End of story.
Using a Bayesian approach, however, the results of that same trial could be looked at taking into account the strength of ‘prior’ evidence – relevant information that could have been gathered from any number of sources: biological and preclinical studies, case reports, uncontrolled studies, studies with surrogate endpoints, studies on other similar cancers, or studies on the same cancer in different stages.
The advantages of this approach when working with small groups of patients is clear. The disadvantage is that the process of defining the strength of prior evidence is open to subjectivity and therefore to potential bias, which is why it has been regarded with scepticism by the scientific and medical establishment, and has never yet formed the basis for approval of a new therapy by the regulators or indeed reimbursement. One of the tasks this conference set itself, therefore, was to start building agreement around rules for defining prior probabilities that can command confidence.
The proposal put to the conference suggested a scoring system for rating studies for their validity and pertinence, “so that the assumptions of all calculations are explicit and can be criticised”. A study in an identical patient population would score higher on pertinence than one with the same cancer but at an earlier stage; a well-designed trial with a control arm would score better on validity than a study that had used historic controls (results from an earlier sequence of patients) or none at all.
The proposal suggests using a transparent and open consensus process to generate the scores, so as to minimise the risk of bias. The credibility of the result can be further tested by subjecting the model to a sensitivity analysis: controversial values can be changed, or part of the evidence can even be erased to see what impact more (or indeed less) sceptical assumptions would have on the final probability distribution.
Pathological pathways and prior probabilities
Building a consensus around the use of Bayesian approaches to clinical studies could be key to giving patients with rarer cancers access to a whole range of new biological therapies, the conference was told.
Building a consensus around Bayesian approaches could be key to giving patients access to new biological therapies
This is because biologicals that target mutational pathways, or combinations of pathways, are rarely specific to a single cancer, which means that there is a probability that a therapy developed and approved to treat patients with one of these cancers could also be of benefit to patients with cancers that share the same mutated pathways. Bayesian methodologies allow drug developers to take into account knowledge gained in trials in one indication when investigating the same drug used against the same pathways but in a different indication. If regulators, and indeed payers, are prepared to accept rigorous well-designed Bayesian studies as a basis for approving access to the market and reimbursement, this could substantially reduce the number of patients needed to provide the necessary evidence. This in turn would make it commercially more feasible to develop the drug even where the potential market is small, and would also cut the time taken for patients to get access to the drug.
This new paradigm for developing drugs across tumour types that share a mutational pathway has a number of advantages, comments Andras Fehervary, head of Market Access for Novartis Europe. “By predicting response, it reduces the number of patients needed in clinical trials; by determining response as early as possible, it means trials can be concluded faster; and by predicting not only activity but also adverse events it provides the basis for ‘companion diagnostics’ that can be used in routine clinical practice to see which patients would benefit most, and which might suffer the greatest toxic effects. It would also accelerate development of new drugs and reduce attrition.”
It’s a win–win scenario, says Fehervary, which aims at getting the right therapy at the right dose for the right patient at the right time – a goal shared by the industry, health authorities, physicians and patients and their associations alike.
“For this to happen in a sustainable way, we do need to cooperate to create a more efficient system, which must be a patient centric, and patient outcomes centric, system,” he adds. “And this calls for new models of collaboration between industry and its partners on important steps.”
Fehervary would like to see incremental changes to the current system that would make it easy to conduct trials of targeted drugs in patients with cancers where the targeted pathway is known to play a role, particularly when that drug has already been approved in one indication, and where the disease is serious and there are few or no therapeutic options. Such a system, he suggests, could be based on numerous, fairly small, investigator-initiated trials; an effective way of identifying eligible patients; and agreement from both regulators and payers on allowing information gathered both before and after the trials to form part of the overall evaluation of the efficacy and value of the drug in that setting.
He paints this scenario:
“Assume we are working within INCa [the French cancer research network], and assume a patient has not been accurately diagnosed, but is clearly suffering cancer-related symptoms. The patient is sent to the Institut Gustave Roussy, and is comprehensively screened against a range of biomarkers. That patient is identified as potentially suffering from a rare cancer, and there is a clinical trial running in one of the 21 centres linked to INCa specifically for that form of cancer. The patient is very quickly moved into that trial, and is matched [confirmed to have the relevant diagnosis for the trial]. The trial is set up to run on Bayesian principles.
“Assume the patient responds well to the treatment, and that these results contribute to the overall clinical trial results that show the drug is an active molecule and effective in that setting. In principle, assume there are 30 patients in that group, we should be able to go to EMA [the European regulators] and say: ‘We have an active molecule that should be made available to patients, but hasn’t gone through the full safety tests that come through the larger trials. However, the risk profile of a patient suffering from a rare cancer is different, because of the poorer outlook for rare cancers, and patients want access.’ Hypothetically this could lead to approval.”
Fehervary also mentions the need for greater involvement of patients groups in clinical trial design and execution, a stronger focus on patient adherence to their prescribed treatment, better management of side-effects, and a more equitable access to drugs and to optimal standards of treatment as important areas for improvement.
Bigger is ‘not always better’
Paolo Casali is a medical oncologist who has spent most of his career trying to improve outcomes for patients with sarcomas – rare cancers now thought to consist of more than 50 (even more rare) subtypes. One of the key organisers of the conference, he is an avowed Bayesian enthusiast, and insists that just because a trial is small this does not mean it has to be either methodologically unsound or inconclusive. He does accept, however, that the smaller the trial the more important is a rigorous methodology: transparent, pre-agreed, open to sensitivity testing. And that is exactly why it is so important to build a consensus around how this can be achieved.
Casali points out that, over the years, more than a few drugs have been approved for small populations on the back of a fairly poor evidence base – for instance trials that had no control arm, or that provide data only on tumour shrinkage. In these cases, where the regulators accept that it would not be possible to run a fully powered phase III randomised controlled trial, pharmaceutical companies (or other sponsors of new drugs) are always uncertain about how much evidence regulators will demand to back up the application for approval, and this may deter them from developing drugs for rare indications.
How much better would it be to have trials designed according to agreed Bayesian principles from the outset, argues Casali. Instead of bringing in additional information at the end of the trial, to be assessed, evaluated and applied by the team of regulators as they see fit, that same information would have to be submitted in advance, with a consensus over its validity and pertinence, transparency about how that consensus was reached, and a sensitivity analysis on more uncertain assumptions.
Casali also questions the received wisdom that bigger trials are always necessarily better. “In rare cancers in particular, large collaborations inevitably lead to involving a large number of institutions whose skills in the disease are limited, which has implications for the quality of care within these studies,” he says.
“Large collaborations inevitably lead to involving institutions whose skills in the disease are limited”
Standard quality control measures used in large trials, such as central pathology review and central review of scans, are useful but introduce their own problems, and will never cover all aspects of good-quality care says Casali. “Every clinician knows that tumour response assessment involves complex clinical reasoning, as does every clinical decision. Clinically speaking, by definition, ‘blind’ central reviews, which skip clinical data, will lead to a worse tumour response assessment, not a better one.”
“There has to be some trade-off between the methodological requirements and clinical quality, otherwise, clinicians will not believe in their own studies,” argues Casali, adding that this is precisely what has happened with several adjuvant randomised trials and “basically all” randomised trials comparing adriamycin against adriamycin and ifosfamide in soft tissue sarcomas. “In fact, many sarcoma experts currently rely on small uncontrolled studies for their everyday decisions on medical therapy more than on large randomised trials, even though these are in fact available.”
Bayesian is ‘second best’
Denis Lacombe, director of the headquarters of Europe’s largest clinical trials organiser, the EORTC, is distinctly cautious about the use of Bayesian methodologies for getting new drugs approved or extending their use to new indications, on the basis of results from small trials.
“While alternative designs should be investigated to allow therapeutic progress for these patients [with rare indications], academia has first a role to work together to assess the feasibility of conclusive trials using the most robust methodology… Research groups should avoid applying whenever possible what can possibly be more debatable methodology,” he told the conference.
Where patient numbers are small, says Lacombe, the answer is to seek international collaboration, if necessary between collaborative groups. One such collaboration, which answered several questions about the best use of temozolomide in patients with the aggressive brain tumour glioblastoma, involved three major North American groups – RTOG, the NCCTG and the Canadian NCI – in addition to the EORTC. Lacombe does not deny the challenges posed by these sorts of collaborations – a lot can go wrong without meticulous planning and unrelenting efforts to keep everyone in step every step along the way. But this should not be an excuse for not trying, he insisted, and he cautioned against any recommendations that could be interpreted as a green light to do “local, small and inconclusive initiatives.”
Matt Seymour, director of the UK’s National Cancer Research Network (NCRN), and a specialist in gastro-intestinal cancers, took a similar line, but added that there would be a lot less need for huge international collaborations if more countries made more consistent and concerted efforts to increase the proportion of cancer patients treated within trials.
The UK tried it, he said, and as a result “over the last 10 years the number of cancer patients enrolled in trials has increased five fold, to one in every six cancer patients – matching the trial population of the whole of North America.” One consequence of this, he says, is that the NCRN was able to recruit sufficient patients from the UK alone to run large randomised controlled trials in some very rare cancers, including one demonstrating efficacy of palliative chemotherapy in glandular carcinoma and another in anal cancer demonstrating the efficacy of chemoradiation.
International collaboration will still be essential particularly for very rare cancers, added Seymour, and indeed he was instrumental in launching the International Rare Cancer Initiative last year as a partnership between Cancer Research UK, the EORTC, the UK’s NCRN and the US National Cancer Institute. This group focuses principally on trials in cancer indications with no more than 3 new cases annually per 100,000. Examples include trials in the adjuvant and advanced setting comparing treatment strategies for patients with small bowel adenocarcimona – a cancer with only around 6 new cases diagnosed per 1,000,000 each year.
“We need to be ambitious; we can achieve randomised controlled trials in an international setting. Where protocols are well designed, well written and very clear, you can achieve high quality even for rare cancers where centres are putting smaller numbers of patients.”
Seymour accepts, however, that some cancers are so rare that, even with international cooperation, it is simply not possible to recruit enough patients to answer a trial question within a reasonable length of time using the traditional frequentist methods and significance levels.
“We need to be ambitious; we can achieve randomised controlled trials in an international setting”
It is only in these situations that he would consider turning to Bayesian methodologies, and even then, only where some “genuinely credible” prior evidence is available, and only where all prior evidence is agreed by everyone before the trial starts. Like Lacombe, he cautions against allowing Bayesian designs and prior probabilities to be used “as an excuse for underpowering”.
He cautions against allowing Bayesian designs to be used “as an excuse for underpowering”
Where next?
The challenge in the coming months will be to amend the draft recommendation to reach a consensus on the way forward for methods for clinical research into rare cancers that satisfies the needs expressed by Casali, while addressing the concerns expressed by Lacombe and Seymour. The hope is to publish a consensus document in the Autumn.
Roger Wilson, who is currently in treatment for recurrent myxofibrosarcoma, and is a former chair of the UK’s National Cancer Research Institute Consumer Liaison Group (and former NCRI board member), says it will be a question of striking the right balance.
“What I as a rare cancer patient need is for my scientists to recognise that there are benefits in both approaches and that they need to find the balance which delivers patient benefit. The scientist who drives through a 10-year study in a rare cancer to deliver a result that has been overtaken by a clinical development which he knew nothing about when he designed the study is not a bad scientist, just a brave and unlucky one. The scientist who used a Bayesian approach and was able to adapt his study to account for the new development and then delivered a result which is more relevant clinically at the time it is completed is not just a lucky scientist – he made his luck.”
Doing nothing is not an option, he insists. “We need our researchers and scientists to start to use Bayesian techniques today in rare tumours so we can assess what issues it raises, if any.” Casali agrees, and he told the conference that during the course of this year, together with colleagues from the worldwide sarcoma community, he will be starting some Bayesian designed studies on new agents in sarcomas.
“We need to start to use Bayesian techniques today in rare tumours so we can assess what issues it raises”
Leave a Reply