My buddy the AI skeptic flipped my view on ethics in AI last week

I was talking with my friend Mike, who works in social services, and he told me he hates how AI chatbots are being used in mental health apps. He said they give generic advice to people in crisis and it can actually make things worse. At first I argued they're just tools, but then he showed me a screenshot where an app told someone to just breathe through a panic attack when they needed real medical help. Now I'm wondering if these companies are moving too fast without thinking about the harm they could do. Has anyone else had a conversation that made them question something they thought was obvious?

2 comments

Log in to join the discussion

Log In

2 Comments

gavinp442mo ago

My buddy showed me a screenshot last week from one of those Woebot apps where it told someone with suicidal thoughts to try a gratitude journal. That one stuck with me because it's not just unhelpful, its dangerous. How do these companies not have a real person reviewing conversations when someone says they want to hurt themselves? Every single chat in a mental health app should have safety checks built in, not just some algorithm guessing. What would it take for them to actually hire people to watch for the serious stuff?

1

terryallen2mo ago

Gavinp44, Woebot actually does use both humans and AI for crisis detection, not just an algorithm.

6