NHS-backed GP chatbot is branded a 'public health danger'


An NHS-backed medical app has been branded a ‘public health danger’ after failing to suggest a 66-year-old woman’s breast lump could be cancer.

After watching a video of the interaction, Twitter users flocked to criticise Bablyon Health’s ‘flawed’ symptom-checking system.

Someone claiming to be an NHS consultant told the artificial intelligence a painless breast lump – a hallmark of cancer – was their only symptom.

Despite the made-up patient being over 66 years old, so being 10 years older than when the average menopause ends, the ‘utterly absurd’ bot still asked whether she was pregnant or breastfeeding.

And, after asking numerous questions, the app’s best guess was that the patient had osteoporosis, a condition which weakens people’s bones as they get older.

Babylon Health, which runs the app, told MailOnline it chooses to not suggest breast cancer and instead says ‘something potentially serious’, which was the third possible cause.

Twitter user Dr Murphy, who claims to be an NHS consultant, tested Babylon Health's symptom-checker by pretending to be a 66-year-old woman with a breast lump, and was outraged when the NHS-backed app did not list cancer as a potential cause

Twitter user Dr Murphy, who claims to be an NHS consultant, tested Babylon Health's symptom-checker by pretending to be a 66-year-old woman with a breast lump, and was outraged when the NHS-backed app did not list cancer as a potential cause

Twitter user Dr Murphy, who claims to be an NHS consultant, tested Babylon Health’s symptom-checker by pretending to be a 66-year-old woman with a breast lump, and was outraged when the NHS-backed app did not list cancer as a potential cause

Babylon claimed the most likely cause of the made-up woman's lump was osteoporosis, a condition which causes the bones to weaken as people get old

Babylon claimed the most likely cause of the made-up woman's lump was osteoporosis, a condition which causes the bones to weaken as people get old

Babylon claimed the most likely cause of the made-up woman’s lump was osteoporosis, a condition which causes the bones to weaken as people get old

Twitter user @DrMurphy11’s video has been watched more than 13,000 times and commented on by senior doctors expressing their disbelief.

The NHS website says in bold at the top of a page on breast lumps: ‘If you feel a lump in your breast, you should always get it checked by a GP.’

After telling the app their hypothetical patient had a breast lump, Dr Murphy said they were a smoker, but had no pain, discharge, change in the appearance of the breast, swelling in the neck or recent breast injuries.

And it ruled out a family history of cancer in the breast or thyroid.

At the end, the app said osteoporosis was ‘moderately likely’, and listed it as the top potential cause of the patient’s symptoms.

Other ‘less likely’ causes were an underactive thyroid, a breast abscess, fibrocystic disease, myxedema coma, duct ectasia, and hypercorticolism.

READ  Dr Miriam Stoppard: New blood test that can spot Alzheimer’s offers diagnosis hope

One option was named simply ‘something potentially serious’ – although this was only one out of four on the likelihood scale.

Breast surgeon at Imperial College Healthcare in London, Paul Thiruchelvam, tweeted: ‘WTF. This needs to be shut down, it’s a public health danger. My 4 year old daughter knows more about breast masses than this app #artificialincompetence’.

A cardiologist for Frimley Health Foundation Trust in Surrey, Navin Chandra, added: ‘This is so bad it’s almost funny… until you realise it’s bloody dangerous…’

A public health nurse, Julia Smith, tweeted: ‘Multiple questions are a waste of time. Only outcome this need[s] to be given would be you need further assessments.

‘Book a GP appointment after giving the information of breast lump, not pregnant, not breastfeeding.’

Twitter user Thomas Sutton added: ‘That a system this dangerous can not only be allowed on the market, but actively promoted by the NHS, is an absolute travesty.’ 

Paul Thiruchelvam, a breast surgeon at Imperial College Healthcare in London, called Babylon's checker 'a public health danger' and said it 'needs to be shut down'

Paul Thiruchelvam, a breast surgeon at Imperial College Healthcare in London, called Babylon's checker 'a public health danger' and said it 'needs to be shut down'

Paul Thiruchelvam, a breast surgeon at Imperial College Healthcare in London, called Babylon’s checker ‘a public health danger’ and said it ‘needs to be shut down’

Navin Chandra, a cardiologist for Frimley Health Foundation Trust in Surrey, called the oversight 'b****y dangerous'

Navin Chandra, a cardiologist for Frimley Health Foundation Trust in Surrey, called the oversight 'b****y dangerous'

Navin Chandra, a cardiologist for Frimley Health Foundation Trust in Surrey, called the oversight ‘b****y dangerous’

Julia Smith, a public health nurse, said the app didn't need to ask any more questions after being told about a breast lump because they should all be checked by a doctor

Julia Smith, a public health nurse, said the app didn't need to ask any more questions after being told about a breast lump because they should all be checked by a doctor

Julia Smith, a public health nurse, said the app didn’t need to ask any more questions after being told about a breast lump because they should all be checked by a doctor

Twitter user Thomas Sutton said the app being actively promoted by the NHS is 'an absolute travesty' and the mistrust it could cause may damage the reputation of artificial intelligence

Twitter user Thomas Sutton said the app being actively promoted by the NHS is 'an absolute travesty' and the mistrust it could cause may damage the reputation of artificial intelligence

Twitter user Thomas Sutton said the app being actively promoted by the NHS is ‘an absolute travesty’ and the mistrust it could cause may damage the reputation of artificial intelligence

The outrage comes after Dr Murphy criticised the app in July last year for diagnosing a nosebleed as potential erectile dysfunction.

Babylon Health runs the NHS’s GP at Hand service through its app – a service for patients in London to phone or video-call GPs instead of going to a surgery.

The company is paid for each NHS patient it treats in the same way as GP practices get money for people on their books. 

The controversial service – which critics warn will destabilise traditional practices as they lose money from patients who deregister to join Babylon – is expected to expand to Birmingham and further across England later this year.

It has been praised by Health and Social Care Secretary, Matt Hancock, who last year claimed he has ‘become known for using the app’.

But the Labour Party has accused him of breaking ethics rules by publicly and privately endorsing an app which also makes money from private paying patients.

Dr Murphy tweeted alongside his video: ‘The resulting list of “possible causes” – is the product of a poorly developed, inadequately tested, unvalidated, flawed decision tree.’

In comments, people revealed they had tried to diagnose the same symptoms in other similar apps – namely Ada Health and Isabel Healthcare – and both had suggested breast cancer as a possible cause.  

A spokesperson for Babylon Health told MailOnline the company does not suggest people might have cancer.   

They said: ‘Not all lumps in the breast are breast cancer – most of the time lumps are non-cancerous.

‘In this particular scenario, the lump might be cancerous, and smoking is a risk factor, but there are no other symptoms to suggest this apart from the lump. 

‘In order to represent what would happen in face to face clinical practice, where a doctor would highlight that there might be something more serious (such as cancer), we recently made some design changes to our app. 

‘We now say “something potentially serious”, rather than cancer, based on user feedback which showed people were more motivated to actually go and see a doctor with this approach. 

‘In this scenario, the Babylon app behaves in the same way as a GP, in that it spots risk factors for osteoporosis, and suggests this as an outcome, as well as highlighting that the breast lump may be “something potentially serious”.’

The company added that it appreciates feedback from users but accused Dr Murphy – who has criticised the app on multiple occasions – of ‘trolling’.

READ  Blood supplies inadequate in many countries

‘We appreciate feedback from all our users,’ the spokesperson said. ‘Including those such as Dr Murphy who run hundreds of tests on Babylon’s AI output, and while the vast majority of tests prove correct, they only report sensationally on the few outlier problems they find. 

‘Clearly these individuals spends many hours testing Babylon’s system, trying to identify any possible area for complaint. 

‘Unfortunately this specific individual has repeatedly chosen to rebuff Babylon’s attempts to constructively engage. 

‘They remain anonymous, though claim to be a doctor (such behaviour is against General Medical Council’s code of conduct and would lead to punitive action) and repeatedly troll Babylon on Twitter.’

HOW BABYLON HEALTH WORKS

Patients download the Babylon Health app and provide their name and a password.

They are first asked their date of birth – only over 16s can use the app as it has not been rigorously tested on children.

Patients are then asked to ‘briefly describe the symptom that’s worrying you most’.

If they reply a headache, for example, they will be asked up to 30 other questions.

They will be asked where in the head they feel the pain, how long they have had it and whether it came on suddenly.

Other questions include whether patients have any other symptoms such as dizziness, flashing lights in their eyes or sickness.

They will also be asked if they have lost or gained weight (weight loss can be a sign of cancer) and if they are more stressed.

At the end of the questions, patients will be told the most likely cause of their symptoms and the action they should take.

An example message reads: ‘People with symptoms similar to yours usually have the following conditions: tension headache.

‘This can usually be treated by a pharmacist’.

They will also be given a second possible cause, which in this example would be a cluster headache.

The message reads: ‘Another possible cause of these symptoms is Cluster headache (attacks of severe, one-sided headaches which occur in clusters).

‘This usually requires seeing a GP’.



READ SOURCE

LEAVE A REPLY

Please enter your comment!
Please enter your name here