Covid-19 has been challenging for society as a whole, but one of the most impacted groups (outside of those who are sick) has been the healthcare facilities and workers. Not only have they risked their own health to help Covid-19 patients, but they have also had to deal with PPE shortages, long working hours, and the burden of additional safety measures to keep patients and staff safe. Thanks to amazing work by the pharmaceutical industry, several vaccines are on their way to distribution, but experts predict that it will be almost a year before this translates into significant herd immunity. Meanwhile, many patients have been under-treated for other ailments, either because non-emergency facilities were closed or severely scaled down, or because the patients themselves delayed their visits out of fear of being exposed to the virus. We are slowly getting data on the adverse health effects of this phenomenon. Hospitals have started to accept regular patients and they have developed workflows to ensure the health and safety of patients and staff. With the recent spike in cases and hospitalizations, however, the strain on the healthcare system remains.

Pre-screening, consisting of measuring body temperature, checking mask and hand sanitization compliance and having the patients answer a brief questionnaire, is critical in reducing the risk of an encounter between a patient with potential exposure to Covid-19 and other patients and staff. Nurses at the entrance of clinics perform pre-screening, making them unavailable for other patient care duties and also potentially exposing them to infected patients.

A ceiling-mounted thermal camera, paired with a regular security camera and powered by real- time video analytics software, can report the body temperature of the patient within FDA guidelines by identifying the inner canthus of the patient’s eye and measuring the temperature at that point. This is already a useful capability: the temperature measured this way is more accurate than the temperature taken by a hand thermometer aimed at the forehead, because skin temperature can deviate significantly from body temperature, leading to false alarms when the outside temperature is high (e.g., in summer). Worse yet, skin temperature readings may miss a febrile patient when it is cold outside and the forehead temperature is lowered enough to hide the patient’s fever. Furthermore, a nurse does not have to take the temperature in person. In the system described above, the patient does not have to stand in a precise location and can walk naturally, thus removing friction.

Attractive as this capability is, it alone does not go far enough to solve the entire problem. Temperature is a static attribute. If it is high, reporting is sufficient because there is nothing the patient can do to address it. We can however help patients comply with mask and hand sanitization compliance by checking compliance, reminding them to put a mask on or sanitize their hands. This seemingly simple task for a human is quite a feat for AI.

Detecting proper hand sanitization requires the interpretation of human motion: patient gets hand sanitizer from a sanitizer dispenser and rubs her hands together. This must happen in real-time so that the system can give feedback to the patient. Checking compliance for proper mask usage is a challenge as well with the very large variety of masks now in use and the various ways a mask can be improperly worn (e.g., under the nose or even under the chin).

Delivering the questionnaire poses some challenges since we do not want patients to use a touch screen (creating a potential contamination hazard). Furthermore, the needs of patients with challenges must be accommodated (e.g., a patient on a wheelchair, a patient with poor hearing or sight, etc.). A multi-modal presentation (text and audio) can be paired with gesture recognition to interact effectively with most patients.

Finally, the system must gracefully pass challenging cases on to the nursing staff before the patient is frustrated. Keeping such cases to a minimum is critical in meeting productivity targets.

The complete status for each patient is presented to the staff member who checks in the patient. But wait! There is the waiting room. Patients must follow social distancing guidelines. The system checks whether two or more patients violate social distancing requirements for longer than a period of time and creates an alert for the staff in such cases. The challenge is to reduce false alarms due to people passing by or people talking to the staff, etc.

This use case illustrates why we do not interact with AI systems often in our daily lives yet. Some of the technological challenges above have only recently been solved. The good news:  the solution is ready in time to help our healthcare institutions and workers open safely back up to cure non-Covid-19 patients.