Not that intelligent? AI-App tricked by its users

Thu, 2018 / 02 / 15

About the author

Ihre Ansprechpartnerin Dipl.-Kauffrau Heike Kielhorn-Schönermark

Dipl.-Kauffrau Heike Kielhorn-Schönermark
Founder and Managing Director

Fon: +49 511 64 68 14 – 0
Fax: +49 511 64 68 14 18

E-Mail More Contact Info

Currently, there are more than 318,000 digital health apps available on the global app market, of which 40% directly affect the health conditions of its users. Current estimates suggest a potential saving of roughly USD 7 billion of the U.S. healthcare spending per year. One of these apps is the British health app „Babylon Health“. The launch of this app was anticipated with great and visionary expectations, such as the reduction of unnecessary physician visits and patient waiting time, simultaneously increasing the time for vital medical consultation. The company which developed the app received financial support worth millions, including support from the national health service of the United Kingdom (NHS).

The technical solution behind the app is based on artificial intelligence: A programmed Chatbot asks the user about his/her symptoms and then provides the “correct” diagnosis – for simple health complaints, the user receives a home remedy; users with more complicated symptoms are directly referred to a specialist. So far, so good. However, the first test attempt led to a huge disillusionment:

During the test, the users grasped the way the app was functioned and then outsmarted the chatbot by faking severe symptoms in order to receive a faster access to a physician. Hence, the app fails in practice. From the point of view of the Clinical Commission Group in North-West London, it is “unlikely” that the “symptom checker” will actually lower the demand for a physician’s consultation. How Babylon Health will deal with these outcomes and the criticism that comes with it will become apparent in the future.

Working closely together with its clients, the SKC Beratungsgesellschaft mbH supports the development of sustainable, digital health care solutions. According to our experience, the subject of market access in Germany, market exploration and strategic positioning in a digitized healthcare market is very important in this context.

BY Heike Kielhorn-Schönermark, MBA, Founder and Managing Director, SKC Beratungsgesellschaft mbH and Michèl Schikowski, M.Sc. Health Economics, SKC Beratungsgesellschaft mbH

Sources:
t3n: Diese App sollte den Arztbesuch überflüssig machen – doch die Nutzer bewirkten das Gegenteil (German only)
Mobilbranche.de: KI-App Babylon Health wird mit Scheinkrankheiten ausgetrickst (German only)
Healthcaredive: Digital health could be saving $7B today