AI has a strong bias towards what it was trained on. Think about the amount of garbage that's on the Internet, the content that has been distributed with no peer or editorial review. You want to run your business or make life decisions on that?
AI is indeed a great tool. I'm a software engineer and have seen much of software engineering's history. In the beginning, we had to know how the chips and machine code worked. Then we had assembly language and that was a big step forward. The along came higher level languages and LALR parsers that could compile the languages to machine code for us. But we still had to know the assembly language because the compliers were't that good and some code had to be hand optimized to run on the computers available at the time. But then compilers got good enough and machines got fast enough that we didn't worry them anymore and kind of forgot the machine architecture and assembly language at the foundation.
Then along cam Model Base System Engineering were would could "code" in models and run them or compile them to specific platforms.
Now we have AI and software engineering is disappearing, being replaced by a series of prompts.
In every case, the level of abstraction was raised, hiding low level details and variability, and letting us spend more time thinking about the problems we are trying to solve and the outcomes we are trying to achieve.
And there is the root problem. If we have become disconnected from what it means to be human, to engage with each other, to love, create, enjoy, grow. Then how will we know what problems to solve or what outcomes to achieve?
I use AI every day at work. But I also feel fortunate to have had an education and career opportunities that taught me to think (my background is mathematics and computer science), to use evidence and reasoned argument to inform action, and to be able to think critically about what problems I'm trying to solve. AI is a very useful tool in that context. What I'm afraid of is that AI will be used to replace all that so a few people can make a lot of money at the expense of everyone else - just like what happened with social media.