Are COVID-19 Vaccines Saving Lives in Mumbai? Here’s the Math

Here is what mathematician Murad Banaji has to say about the COVID trends in Mumbai so far.

By now, we know that COVID-19 vaccines save lives. There is plenty of evidence for this from vaccine trials, and from countries such as the UK and Israel which have seen rapid vaccine roll-outs.

In India, vaccination has been slow – a modest 8-9 percent of the population has received at least one vaccine dose. But the drive has, rightly, been focussed on the elderly and others most vulnerable to severe disease. This should have a clear effect when it comes to reducing hospitalisations and deaths. Hard data to confirm this is limited, so far.

One concern is that we don’t know how the vaccines will perform against some variants of SARS-CoV-2, the virus which causes COVID-19. Limited genome sequencing data indicates that in different parts of the country different variants are dominant. Could it be that some of these variants can cause severe disease even in the vaccinated population?

When we examine Mumbai’s data we’ll find that we are seeing fewer deaths than expected in the elderly. Crucially, the scale and timing of this effect suggests it is a consequence of vaccination.

How Can We Show that the Vaccines Are Effective?

To measure vaccine effectiveness outside of a trial, we would, ideally, track disease and fatalities in the vaccinated and unvaccinated populations, and after matching for age and other risk factors show that vaccinated populations are seeing less disease and fewer deaths. But it is not clear to what extent this kind of monitoring is being done.

Recently, the Indian Council of Medical Research presented claims that “breakthrough infections” amongst vaccinated people were very very rare. But the analysis turned out to be weak and misleading. Very limited data had been spun to tell an unreliable story – a pattern which risks undermining confidence in vaccines.

Even without direct evidence, we can still look for circumstantial evidence to support a vaccine effect. If, say, we see a sharp drop in fatalities in a population where vaccination has been high, this would be a hopeful sign. But we run into two problems:

1. The current surge: A huge wave of disease is sweeping the country. With rapidly rising fatalities, it becomes much harder to spot if, indeed, vaccines are saving lives.
2. The integrity of fatality data: We know that many, many COVID-19 deaths are currently going unrecorded, especially in some parts of the country.

Despite these problems, we can check, locally, if there are fewer deaths than “expected” in populations with high vaccine coverage. Let’s try this in Mumbai.

A Shift in the Ages of Those Dying of COVID in Mumbai

During March, daily cases in Mumbai grew fast, shattering all previous records by mid-March 2021 before peaking at roughly 10,000 around 7 April. Although cases stabilised and even fell a little after this, recorded deaths have risen through April so far, averaging around 61 per day in the week ending 24 April.

We expect such a time-lag between cases and deaths.

Hidden in Mumbai’s data is a very striking trend. During April, the over-60s – let’s say the “elderly”, for short – have formed a declining fraction of the city’s total COVID-19 fatalities.

To effect is clear and the scale is now huge. From the start of the year to the end of March, the elderly made up 80 percent of the city’s fatalities.

If this had continued, then in the week up to 24 April, there would have been 300 more COVID deaths in the elderly than observed. We see a strong divergence between total observed deaths, and expected deaths based on trends in the under-60s. Fatalities in the city today are around 44 percent lower than expected.

Could the Age-Shift be Driven by a Surge in Deaths in Younger People?

The huge slum surge during April-June 2020 coincided with a sharp increase in younger people dying.

Could something like this explain the increasingly young profile of those dying of COVID in Mumbai today? The answer is, broadly, no.

We can confirm this by looking at the case fatality rate (CFR). We find that a rise in cases at a given time generally leads to a rise in recorded deaths around 18 days later. By taking the ratio of deaths at a given moment to cases 18 days earlier, we can calculate Mumbai’s delayed CFR, and find a drop of around 33 percent during April.

The drop, alongside the age-shift, tells us that the dominant effect is fewer deaths in the elderly, and not more in younger groups.

In fact, the 33 percent drop in CFR during April could underestimate the true fall in the COVID-19 fatality rate: the rapid rise in cases and test positivity during late March and early April should lead to a rise, not a fall, in the delayed CFR during April.

Is the Age-Shift Consistent With a Vaccine Effect?

The answer is a clear yes.

After an initial phase for healthcare and frontline workers, vaccination opened out to the elderly and over-45s with comorbidities on 1 March. The bulk of those vaccinated during March were the elderly, and by the end of March, around 44 percent of Mumbai’s elderly had had one vaccine dose. The great majority of these doses were of Covishield, the Oxford-AstraZeneca vaccine.

Randomised trial data indicated that Covishield should be highly effective at preventing severe disease 22 days after a single vaccine dose. If this holds in Mumbai, then by 22 April, around 44 percent of the city’s elderly should be largely protected against severe disease.

A 44 percent drop in fatalities in the elderly would correspond to a 34 percent drop across all groups.

Summary

Let’s summarise what we’ve found so far.

1. Recorded deaths in the city today are around 33 percent below what we expect from case data. The true fall in fatality rate in the city could be even higher.
2. Recorded deaths in the city today are around 44 percent below what we would expect if trends in the age-structure of fatalities had continued from March.
3. Given vaccination of the elderly during March, deaths in the city today should be about 34% below baseline. Possibly more if coverage has been highest in the most vulnerable groups, or if some protection kicks in earlier than 22 days after a first dose.

The rough match between these three, entirely different, calculations is remarkable.

Some Questions Remain

Could it be that as hospitals filled up more and more COVID-19 deaths of the elderly at home went unrecorded? Have the elderly managed to ‘shield’ more successfully than younger people?

We can’t completely rule out such possibilities — but vaccine protection is the simplest explanation based on the scale and timing of the trends. Up to 580 lives could already have been saved by the vaccination drive focussed on the elderly.

There were fears that the vaccines might be less effective against the variants in circulation in Mumbai. But the evidence so far should allay these fears. A practical conclusion is that the city authorities must urgently reach the significant number of elderly people who have so far been missed in the vaccination drive.

What could we see next? From early April, vaccination opened up to all those over 45. Since then 45-60 year olds have received the bulk of new vaccine doses. With increasing protection in younger age groups, we could even see a slowing or reversal of the trend for an increasing fraction of fatalities in the under-60s.

This should not be alarming provided it comes alongside a reduction in COVID-19 deaths in the city as a whole. We can hope for some positive news in these grim times.

[A version of this piece with more data and details of the calculations is available here.]

(Murad Banaji is a mathematician with an interest in disease modelling. He tweets @muradbanaji. This is an opinion piece, and the views expressed are the author’s own. The Quint neither endorses nor is responsible for the same.)