FWIW my wife works for Kaiser and finds a lot of value in the the medical LLM tools available to her. She tells me being able to do live translation, summarize notes, and quickly get comprehensive answers save her time and help her give better care. Her older patients also frequently come in bringing AI-powered alerts from their apple watches that detected cardiac events.

It's annoying that we use broad terms to describe a set of technologies that in some ways can be problematic and in another ways are very beneficial. We gotta evaluate each of these as they come rather than talk about blanket bans.

"You wouldn't believe how much of a relief it has been. In your last visit, you saw me typing everything you were saying, right? I don't have to. I can listen to you and take very specific notes as necessary as opposed to focusing on both typing and listening to you at the same time. It has bought my stress levels down to here." (Indicated by his hand lowering)

That being said, more than one (female at that) doctor has told me in confidence and based upon their observations during residencies, etc. that if I'm ever admitted, be very careful how I modulate my interactions with nurses. They're not all Florence Nightingale and Mother Theresa and there exist those who will @#$% you up on various pretexts or are just plain sloppy and negligent.

Despite all the moralising fluff, it's just a job, not some saintly vocation. Some safety oversight is needed, just as it is for any other work function. Still, can bet that anything 'Corporate' has mandated will be Goodharted up the wazoo.

I wondered why they zip in and out of the rooms, when just a few years ago they would spend fifteen to twenty minutes in each room. The patient load hadn't grown. The number of nurses has gone up, not down.

So I'm blaming the stupid metric on their evaluations for a worse standard of care.

Some of the tech is pretty scary. One big vendor's solution [0] can provide not just AI agents but also use AI to snoop on calls in progress, evaluating sentiment from both sides [1], verifying phrases are said - pretty dystopian in theory. From experience, these things tend to go downhill based on the attitude at the top - is the mission to slash costs or take care of customers? A 1000 decisions follow from this one, and like Jira, it can be a useful tool or a prison-like hell.

[0] https://www.cisco.com/c/dam/en/us/products/collateral/contac...

[1] https://www.cisco.com/c/en/us/products/contact-center/webex-...

Do these idiots not know how intelligent assholes work? If you make this a policy, everyone who has a gripe will eventually learn to delay a call to 14 minutes and then make an absurd ask.

The loser, as always, is the patient's quality of care.

The sad thing is that there are many issues, but the study didn't really address them. One would hope for more effective critics.

The AI application I'd like to see would be to identify the elective hyper-utilizers (so we could try to demotivate them by addressing what's really bothering them). That would improve quality of care for them and quality of life for providers, and create time for outreach to the under-utilizers who need preventative care.

https://marshallbrain.com/manna

> A company spokesperson said, "Kaiser Permanente does not use Average Handle Time to assess agent performance"

So uh, average time wasn't raised as a concern, calls beyond a certain threshold was. I wish this semantic discrepancy was better highlighted in the article.

https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....

This is a well understood phenomenon.

what significant improvements to society or humanity have come about as direct result of AI, that wouldn't have been achieved without it? faster protein folding is the only one I can think of and that more a matter of "faster" than "impossible without AI"

I think at some point there's going to be some version of the Butlerian Jihad

[edit] preemptively, if you're going to claim "but this isn't art so irrelevant" then I claim bull fucking shit. It's the same problem no matter how you slice it whether it be engineering, support, art, or medicine. Get real. Look inward. Touch grass. If you think AI is a good thing for your profession--whatever that profession might be--you're probably a delusional psychotic. It's OK, you'll thank me later.

About 14% of Americans think AI is moving us towards a better world. About 17% are creationists. About 26% believe in Telekinesis.

It’s going to be very improbable that these statements are true.

They will vocally rationalize it.

I did it. "I'm more productive work from home." But then I do dishes, take an hour break, paid.

Foucault says that when people are observing them, power is placed over them.

If you are a worker you should hate this.

If you are a customer or owner, you should like this.

But I certainly won't be automatically believing people under surveillance who make claims it makes their quality worse.

> Another nurse speaking on condition of anonymity said “AI did not understand our job and would grade us wrong all the time.”

It's always worth remembering Goodhart's law https://en.wikipedia.org/wiki/Goodhart%27s_law - "When a measure becomes a target, it ceases to be a good measure."

In theory AI could usher in the first time in history where one can escape from this trap - because qualitative judgments can be made at scale, from an unbiased and universal baseline. In this situation, for instance, rather than collapsing call transcripts and reports into metrics, it could evaluate whether red flags are encountered in the context of a call, and allow for qualitative guidance on improvement, across a comparative corpus of situations that are themselves chosen qualitatively.

But very few managers are empowered to take this kind of approach; they're evaluated by their ability to report quantitative metrics, and thus they must implement regimes of quantitative metrics. And leadership instructs them to use AI to build that regime more quickly.

If you want to see an "AI native" organization, it's one where leadership actively fights this tendency, and sees managers as product designers who make the end-user experience a beloved and empathy-driven one, as opposed to a gear that turns accountability into a single number on a screen.

People started hating tech right around the time metrics became popular. I don't think it's a coincidence. AI just accelerates the trend.

The problem is the misidentification of AI as the issue. As long as we don't understand the real issue, we won't solve it. AI is just a tool. It's being used in a way that denies human agency.

Our cultural values need to shift away from safetism that demands centralization. And shift toward valuing human agency. That starts with talking about the core issue.

Given how healthcare is one of these sectors that seems to relentlessly resist efficiency increases and is the prime example of Baumol's cost disease, I think any developed country with a costly healthcare system needs to do these AI experiments. The current versions will be shit, but the only way out is through if you still want to provide affordable care.

I honestly have no doubt that AI going forward will be able to do a good job at triaging via calls and also being empathetic about it. But of course it needs careful experimentation.