The thing about immuno-compromised patients is that they are among the least effective vectors of transmission, either because they have been debilitated by the illness or have isolated themselves as a defensive measure. So even if they generate a variant with greater fitness, it is less likely to propagate. This is why viruses are frequently seen to become less virulent: relatively healthy patients are more capable of exposing the virus to a broad spectrum of new targets, which can be considered an indirect aspect of viral fitness.