"When a task is exceedingly difficult, it's best to create a new tool." - Charles C. Schwartz, my father
Acknowledgments:
I thank our impromptu internet family for providing near continuous help, inspiration, and education.
My background is in computers. In the past, programmers for every DOS application had to write code for printer drivers for each printer on the market. Windows solved this problem by assigning this task to the operating system. Install a printer once in Windows, and all programs can use it. Nobody has to write code for printer drivers, except the printer manufacturer, and just once. Randomized controlled studies are like the old DOS system. Each study must create its own control arm.
|
Randomized trials represent the most rigorous method of evaluating medical interventions. However, some interventions are extremely difficult to conduct using this study design. For example, individuals with cancers that are incurable with standard treatments logically seek new treatment by participating in clinical trials. It can be immoral to randomly assign these individuals to one of two interventions when one, and not the other, has no possibility of improving survival, or when one treatment has more toxicity--especially when the toxicity undermines the patient's ability to benefit optimally from subsequent treatments. Randomization also accounts for significant delays in the evaluation process.
Despite the significant ethical challenges and delays that often result from conducting randomized trials, only cancer drugs proven to have clinical benefit in this way can win F.D.A. approval. Historical controls--a type of control that compares outcomes to results from case histories (retrospective data) are considered unreliable.
In this text we describe the limitations of historical and randomized controls. We then propose an alternative that we believe can make well-designed nonrandomized trials possible.
Background:
Weaknesses of historical controls
Historical controls are considered unreliable and inferior to randomized controls because they cannot account well for the following:
-
How the drugs were administered, such as dosing and timing.
-
How the outcomes were assessed, such as types of tests and the testing intervals.
-
The selection of patients, such as age, health, immune status, tumor burden, treatment history, etc.
-
The treatment setting: initial treatment, refractory, relapsed, etc.
It is believed that the failure of retrospective data to account for these variables can lead to misleading conclusions about clinical benefit, and result in the approval of drugs that have little or no value.
Weaknesses of randomized controls
The controls in randomized trials, while superior to historical controls, do not yield perfect statistical results for many reasons. Randomized controls also raise troubling ethical concerns in some circumstances:
-
Delays are indescribably costly to patients with incurable cancers. The delays caused by randomized trials are considered necessary because of the inferiority of historical controls described above.
In general, it is more difficult to recruit patients into randomized studies, especially in blinded studies. This problem will be compounded as additional subtypes of cancers are identified by advanced molecular diagnostics techniques, which will cause the patient pool to shrink, making adequate accrual impossible in some cases.
Delays in accrual will also occur when the drugs being compared do not have equivalent toxicity, or one has a perceived survival advantage over the other.
Delays in accrual can:
-
Make study questions obsolete before the completion of the trial.
-
Postpone evaluation of potentially better drugs, urgently needed by patients with incurable cancers.
-
Increase the cost and financial risks of drug evaluation to the point where some useful agents will not be developed.
-
Randomized controls can give inaccurate conclusions, unless the test groups are much larger than are practical to achieve in many cases.
Here are some of the variables that can contribute to false statistical conclusions in randomized trials:
-
No two cancers are alike. Cancers given the same name, are not the same at a molecular level.
-
No two patients are alike. Age and diagnosis do not account for differences in immune status, and genetics. These and other variables can sometimes influence outcomes.
-
Date of diagnosis does not determine how long a patient has lived with their cancer. This variable can only be estimated.
Study arms may not be large enough to overcome these and other variables in randomized trials.
-
Ethical problems can arise when randomized studies do the following:
-
Assigns patients to toxic versus non-toxic interventions;
-
Exposes patients in the control arm to an unnecessary medical procedure with known side effects or risks. For example, surgery to remove a lymph node--when no vaccine is created from the tissue removed;
-
Exposes patients to a toxic pretreatment that's likely to preclude the patient's optimal use of subsequent therapies, without also providing the potential benefit of the new agent;
-
Asks patients to take a chance on a therapy when there is no genuine clinical or community uncertainty about which drug is superior. This problem can arise when the new treatment has shown evidence of improvement over other treatment options, or equivalent outcomes with less toxicity, or a chance for improved survival over a treatment that has demonstrated no survival benefit that has known toxicity.
Patients with incurable cancers consider trials as treatment. Having one life to experiment with, they cannot afford to receive less than optimal treatment, especially during initial treatment. Typically, the patient has only so many chances to succeed. We know that subsequent cancer treatments are likely to have fewer responses and shorter response durations.
All agree that each type of control has defects. Patients who suffer from the disease and the consequences of the treatment will often avoid randomized trials. Presently, the F.D.A., the final arbiter of methods of evaluation, believes that randomized trials are superior - that it's better to be right than to be fast. Given the two methods of comparing drugs described, it is not difficult to respect F.D.A. judgment in the majority of past clinical trial designs.
But today is not like yesterday, or last year. New targeted therapies may sometimes be less toxic than available treatments and are therefore not comparable. Furthermore, targeted agents may have subtle benefits that will not show their full potential as single agents. Therefore, we have reason to fear that the high costs of conducting randomized trials, and the time it takes to accrue patients, will cause useful agents to fall between the cracks unless policy shifts and methods of testing new agents advance along with treatment technologies.
Randomization adds substantial time and cost to the drug evaluation process. We can only guess how many therapies are not examined adequately as a result. Importantly, we can dramatically improve outcomes for thousands of individuals if we can find a responsible alternative to randomized trials that accelerates drug evaluation and reduces cost. The alternative need not be perfect to replace an imperfect system. It need only to be better and faster; or equivalent and faster.
An Alternative: Set Standards to Create Reliable Historical Data
While the F.D.A. has not approved cancer drugs using historical controls, the ICH guidance document used by the agency clearly contains provisions for the use of historical controls in special circumstances. We may take "special" to mean when ethical randomized trials are not possible to conduct and patients face life-threatening disease.
You may recall that a strong argument for randomization is that it ensures that monitoring of efficacy is done the same for each arm: when scans are done, at what interval, with what kinds of equipment, in which labs, at what interval, etc. Perhaps we can remedy many of these weaknesses by creating new, more reliable historical data as we conduct each trial, and in the day-to-day clinical practice of oncology.
To accomplish this goal we propose the Standard Outcomes and Administration Protocol (SOAP). The goal of SOAP is to create outcome data that can be used responsibly in comparisons with new agents for cancers that are not curable with standard treatments.
The goals of this protocol are:
-
Create standards for administration of approved therapeutics in all settings, including all clinical trials and general practice.
-
Create standards for monitoring outcomes of approved treatments by setting disease-specific standards for tests used, test intervals, equipment used, etc.
-
Use electronic data collection to standardize how data is collected, and to make both the entry of data, and access to records as efficient as possible.
-
Record patient baseline status to fit common inclusion/exclusion for clinical trials and known prognostic indicators, such as tumor burden, LDH, bone marrow involvement, renal function, etc.
-
Record treatment settings: Front-line, Relapse, Refractory and treatment history information.
-
Collect and record potential surrogate biomarker data. (In new studies, surrogate biomarkers might be compared to responders and non-responders who where given the same treatment.
-
From the above, gather primary endpoint data: *reliable* tumor response & time to progression records.
These methods (SOAP) can be superimposed on any study, or used in general practice by participating physicians. The data and the methods can then be used to carry out well-designed nonrandomized comparison studies with this new and more valid historical data in circumstances where ethical or timely trials cannot be conducted otherwise.
Our proposal would not require all to adopt SOAP standards; only those who choose to participate. Those that participate will add to the pool of reliable retrospective data. For this reason, we might provide incentives to encourage oncologists to use the SOAP protocol.
Data from the control arms of randomized trials that append the SOAP protocol can also be used to build reliable outcome data. These data will be especially relevant and useful for a variety of reasons.
Ultimately, perhaps in the time it takes to conduct any single trial, nonrandomized studies can be designed that can use the new data as a control.
Benefits of SOAP
Adopting standards for outcome measurements and the administration of cancer therapies can create retrospective data sets to meet an urgent need for alternatives to randomized trials that will become increasingly difficult to conduct.
The adoption of SOAP standards has the potential to create more reliable outcome data, as over time the data collected will be far larger than for the control group of any single randomized study.
The adoption of SOAP standards can help to reduce the redundant task of creating a control group for each new phase III study.
The adoption of the SOAP protocol will let patients make treatment *decisions* instead of taking treatment *chances.*
SOAP will provide a needed answer to the problem of the shrinking patient pool that is sure to emerge as additional molecular subtypes for cancers are identified.
The protocol will help to accelerate the development of new drugs that are urgently needed for patients with incurable cancers by increasing accrual rates while simultaneously reducing the number of patients needed to conduct studies. This will dramatically reduce the costs of conducting individual trials. Removing delays will result in the timely evaluation of new agents that can improve survival and the quality of life for cancer patients. Drugs that don't make the grade will fail quickly. Reducing the costs of testing agents will give incentives to drug sponsors to develop new agents. Taken together, these are powerful advantages that we ought not ignore.
Importantly, the adoption of outcome and administration standards can also help to ensure that quality cancer care is provided to all Americans, as it will help us to better determine which treatments are effective in regular practice, and which are substandard.
Perhaps there are weaknesses in the SOAP proposal that escape this lay person. But we know that every system has weaknesses, so we must not dismiss this or other new proposals out of hand for the weaknesses they are sure to contain. We must, instead, objectively compare the merit/weakness ratio of this system to existing methods. We must give proper weight to the costs of delays to the patient with cancer who has no satisfactory treatment to turn to; and learn to identify with the ethical concerns of patients who face the disease and the consequences of treatments.
The evaluation system must evolve to better serve us. That, after all, is the reason it exists. Therefore, the patient perspective must be articulated, understood, and valued; and it must also help to shape and properly balance the methods we use to evaluate new drugs.
Karl Schwartz, Patient Advocate
Send comments by
|
