Colin, I hear you. I made similar points here:

https://becominghuman.ai/why-large-language-models-like-chatgpt-are-bullshit-artists-c4d5bb850852

But then I did a poll on LinkedIn:

https://www.linkedin.com/posts/valliappalakshmanan_what-would-you-do-call-this-the-birthday-activity-7029289117171519488-CVg6

And found that 1/4 th of people will choose conveniently available fluent bullshit over accuracy.

So, I'm resigned to this being the new Bitcoin. A lot of people are going to get hurt.