An assumption of the two-sample test is that your samples are i.i.d., or independent and identically distributed. The independent part is easy. The identically distributed part (i.e., there's a good match between the populations) is much more important and more difficult. The point of randomizing treatments to make sure that your samples are drawn from the same populations (i.e., identically distributed).
Your 88% is a number, not a sample from an identically distributed population---a one-sample test. That's fine but not very realistic. The biggest issue is that it is a number that may or may not have any connection to the Montefiore cohort. We don't know. After that number was put out there, there has been a boatload of objections that it is way too high as an estimate of "mortality rate among ventilator patients." Other cohorts saw 75%. Michigan says 20-50%. Which is right for patients like those at Montefiore? What was the mortality rate at Montefiore? I've heard 7/11 ("we saved 4!"
, 5/10, 6/10, 4/10. It depends on how you count.
I've tried hard to be optimistic based on the anecdotal data, but it's really hard to make an even slightly compelling statistical case for how these forecast the results of the trials. One thing that helps make the case in an impressionistic sense (not quantitative) is that the Montefiore cohort was in terrible shape pre-leronlimab. Immediately after leronlimab, the IL-6 levels began a dramatic and sustained decline in every single patient. Tantalizing results from France and China suggest that lowering IL-6 leads to increased survival. The Montefiore study may provide further evidence, although without a control population, it's hard to say.
The Montefiore cohort was in very rough shape. All had serious co-morbidities. Their IL-6 levels were off the charts high, well into the high risk zone for mortality. I would expect their mortality rate to be higher than average for intubated patients. But, then again, 3 were not intubated and the ages were quite a bit younger than average for fatal cases. Would the cohort really be expected to have an average mortality rate for intubated patients? Some would be much more likely to die and some less? No way to tell.
We don't know what the expected mortality rate is for generic ventilator patients (30-88%?), we don't know how this cohort matches up, we don't even know what the mortality rate is for Montefiore, and we don't know if the measure of mortality rate for Montefiore matches that used in a reference population, e.g., the 88% estimate is based on number of fatalities [at what point in time?] and the number of discharged patients...but what about the patients that were alive but had not been discharged? Chances are that most of those would end up being discharged eventually because this disease kills fairly quickly when it kills. That means the 88% is probably too high of an estimate. A little too high? Way too high?
Small convenience sample, no control group. When you don't have good samples, statistical formulas are more misleading than informative because they give an unrealistic semblance of certainty when there is nothing there. Quantitative assessments of mortality and how it compares to what would be expected for non-treated patients in this case are not much more than pulling numbers out of thin air. That's why we need clinical trials with carefully selected samples.
Your simple little calculations are fine, and I do that sort of thing all the time. But I recognize that the numbers that come out the other side are little more than speculation and are guaranteed to be wrong. The question is "how wrong" and "in what direction". Don't put too much stock in any of the guesses.
If you are making plays in a whole bunch of areas, then, by all means, make a quick little calculation or two on dubious data and run with it. Some will work out right. Some won't. On average, you should do pretty well.
Or if you don't make a lot of plays (like me...I have a wholly different life) but like to gamble, then run with the simple little calculations. You might win big. Or you might lose big. I like CYDY right now, but there are still some major, lingering questions. I've put enough in so that it would make a significant difference in my life if it works out but wouldn't kill me if it doesn't. A few months ago, I thought the chances of SP going to 0 were about the same as the chances of going to $10. I think $10 is more likely now; and, if $10, then $20, $30 are clear possibilities. If NASH works out and a few cancers, sky's the limit.
Your 88% is a number, not a sample from an identically distributed population---a one-sample test. That's fine but not very realistic. The biggest issue is that it is a number that may or may not have any connection to the Montefiore cohort. We don't know. After that number was put out there, there has been a boatload of objections that it is way too high as an estimate of "mortality rate among ventilator patients." Other cohorts saw 75%. Michigan says 20-50%. Which is right for patients like those at Montefiore? What was the mortality rate at Montefiore? I've heard 7/11 ("we saved 4!"
, 5/10, 6/10, 4/10. It depends on how you count. I've tried hard to be optimistic based on the anecdotal data, but it's really hard to make an even slightly compelling statistical case for how these forecast the results of the trials. One thing that helps make the case in an impressionistic sense (not quantitative) is that the Montefiore cohort was in terrible shape pre-leronlimab. Immediately after leronlimab, the IL-6 levels began a dramatic and sustained decline in every single patient. Tantalizing results from France and China suggest that lowering IL-6 leads to increased survival. The Montefiore study may provide further evidence, although without a control population, it's hard to say.
The Montefiore cohort was in very rough shape. All had serious co-morbidities. Their IL-6 levels were off the charts high, well into the high risk zone for mortality. I would expect their mortality rate to be higher than average for intubated patients. But, then again, 3 were not intubated and the ages were quite a bit younger than average for fatal cases. Would the cohort really be expected to have an average mortality rate for intubated patients? Some would be much more likely to die and some less? No way to tell.
We don't know what the expected mortality rate is for generic ventilator patients (30-88%?), we don't know how this cohort matches up, we don't even know what the mortality rate is for Montefiore, and we don't know if the measure of mortality rate for Montefiore matches that used in a reference population, e.g., the 88% estimate is based on number of fatalities [at what point in time?] and the number of discharged patients...but what about the patients that were alive but had not been discharged? Chances are that most of those would end up being discharged eventually because this disease kills fairly quickly when it kills. That means the 88% is probably too high of an estimate. A little too high? Way too high?
Small convenience sample, no control group. When you don't have good samples, statistical formulas are more misleading than informative because they give an unrealistic semblance of certainty when there is nothing there. Quantitative assessments of mortality and how it compares to what would be expected for non-treated patients in this case are not much more than pulling numbers out of thin air. That's why we need clinical trials with carefully selected samples.
Your simple little calculations are fine, and I do that sort of thing all the time. But I recognize that the numbers that come out the other side are little more than speculation and are guaranteed to be wrong. The question is "how wrong" and "in what direction". Don't put too much stock in any of the guesses.
If you are making plays in a whole bunch of areas, then, by all means, make a quick little calculation or two on dubious data and run with it. Some will work out right. Some won't. On average, you should do pretty well.
Or if you don't make a lot of plays (like me...I have a wholly different life) but like to gamble, then run with the simple little calculations. You might win big. Or you might lose big. I like CYDY right now, but there are still some major, lingering questions. I've put enough in so that it would make a significant difference in my life if it works out but wouldn't kill me if it doesn't. A few months ago, I thought the chances of SP going to 0 were about the same as the chances of going to $10. I think $10 is more likely now; and, if $10, then $20, $30 are clear possibilities. If NASH works out and a few cancers, sky's the limit.
3
