June 14, 2022

Active syndromic surveillance of COVID-19 in Israel


We advertised to people in Israel who made COVID-19 related symptoms queries through the Google Ads platform. People who clicked on the ads were referred to the Microsoft Healthcare Bot. Interaction with the bot helped people understand the severity of their symptoms relative to their demographics and underlying medical conditions. The Bot’s COVID19 instance was created and operated by Israel’s second-largest hospital.

The advertising system has been put in place to maximize the number of conversions. People who “converted” were defined as people whose responses to the Healthcare Bot indicated that they needed urgent medical attention.

Note that the algorithm used by the Healthcare Bot was independent of the advertising system. Therefore, the only way to increase conversion rates by the advertising system is to identify the people most likely to be hospitalized based on the information the advertising system had about its users.

Analysis of user interactions with advertisements and the Healthcare Bot at country and city level predicted the number of COVID-19 cases and hospitalizations.

Advertising campaign on the Search Network

Search Network advertising campaigns consist of three main elements: keywords which, if used by a searcher, will trigger an ad to appear; the text of the announcements; and the landing page, which is the page a user will be taken to by clicking on an ad.

Ads were shown on the Google search engine to people in Israel. Google Adwords specifies that the physical location of a user is determined by the IP address or by the location of the device (see https://support.google.com/google-ads/answer/2453995). The ads were shown in response to queries containing any of the following terms (in Hebrew): dry cough, fever, stuffy nose, difficulty breathing, diarrhea, fatigue, headache, nausea, sore throat or vomiting. The list of symptoms was based on the UK’s first National Health Service (FF100) investigation into COVID-1919.

Ads consisted of a short title (one sentence) and a body of 1 to 3 sentences. The advertisements are shown in Table 1. In practice, Google has banned advertisements mentioning COVID-19, and therefore, two advertisements were not served by the advertising system.

The campaign was run in Hebrew.

Once the campaign elements are decided (for example, the ad copy is prepared), it takes 1 to 2 hours to set up the campaign.

Google AdWords provides aggregated information on the results of the advertising campaign, including campaign metrics (number of impressions, clicks and conversions) over time, stratified by city (e.g. Haifa, Jerusalem, etc. .), gender and age group (18-24, 25-34, 35-44, 45-54, 55-64 and 65 years and over). We refer to the number of clicks on all impressions as the click-through rate (CTR). The conversion rate on all clicks is referred to as the Conversion Rate (ConvR). Aggregated information can be downloaded at any time, and its processing, to predict, for example, future hospitalization rates, can occur in near real time.

Table 1 Advertising text.

Microsoft health robot

People who clicked on campaign ads were referred to an instance of the Microsoft Healthcare Bot (https://www.microsoft.com/en-us/research/project/health-bot/) which was installed by Tel Aviv Sourasky Medical Center, Israel’s second largest hospital. The bot was set up by medical professionals to ask people about their demographics, underlying health issues, and symptoms they were experiencing. At the end of the interaction, the bot was offering people one of three outcomes: staying home, calling the family doctor, or going to the emergency room. The latter was used as a feedback signal to the conversion optimization mechanism.

Ground truth case data

We retrospectively compared the data from the ad campaign to the COVID-19 data repository of the Israeli Ministry of Health (https://data.gov.il/dataset/covid-19). Specifically, our data was compared to the COVID-19 dataset by area, which provides information on the number of new COVID-19 cases per 100,000 population in each city each day and the number of hospitalizations in these temporal and spatial resolutions. We call them the case rate and the hospitalization rate, respectively.

We note that the number of cases and hospitalizations are known to be loud indicators of disease incidence, affected by policy, resources and test accuracy. However, at the time of writing, this is the most comprehensive municipal data available to the public on COVID-19 in Israel.

Demographic data

We collected city-level demographic data from two official sources: the total population and the percentage of Arab citizens in a city from the Israel Bureau of Statistics.20 and the median monthly income (2011), the percentage of people earning less than the minimum wage and the Gini index of the National Insurance Institute of Israel21. These were used to assess differences in the demographics of people who used the campaign.

Data analysis

The correlation between ad system metrics and pandemic metrics, taking into account the share of keywords for which the campaign ads were served, was estimated using a linear model operating at a daily granularity. The independent parameters of the model were either CTR or ConvR, spend per day, and share of search impressions that day. The model-dependent parameters were case rates and hospitalization rates. Since it is assumed that the campaign data can predict future pandemic indicators, the model is estimated at different lags between the advertising system data and that of the pandemic indicators, where a negative lag of k days means ad system data correlates with pandemic data taken k days later. The models were estimated for lags ranging from – 14 to + 14 days.