By Bob Glaser

I realized that most of my posts about AI have focused on the more cautionary aspects. I stand by them, but I also believe that AI can be a great service. I have found enormous productivity gains and useful organization of ideas, alternative perspectives, and concepts I had not previously thought of.

Here’s where the line between being productive and being problematic. If you don’t have knowledge and experience in the area that you’re asking AI about (even rag based by LLMs), then you don’t have any way of assessing the accuracy of the results. The important bias here is that the results are geared toward what you want to hear and not necessarily a balanced or even potentially negative assessment of what you requested. Even having the knowledge doesn’t guarantee that you’re asking the right questions based on the token functionality of the LLM.

This is like the magic mirror in sleeping beauty. Wanting to hear what you hope and expect is very different than wanting to hear the truth. The bias here is that we often believe that what we want to hear is truth.

This is like playing with statistics. You can present them in a way that supports an argument in a specious if not deceptive way. There are many scenarios where you can adjust what you present in a way that sports your argument. If the audience doesn’t recognize this, is their ignorance really to blame? In usability this is referred to as “dark UX.”

I recently used AI (in this case Copilot) to provide me with legal advice. It was useful and fairly accurate considering that locale has a significant effect on the laws you have to deal with.

They also worked recently as I have in the past with vibe coding. This has been very useful but at the same time is a nightmare of usability. UX for engineers makes perfect sense to them because they are engineers. Their focus is on a logical process that is efficient, brief, and effective in solving a problem. They also know the less obvious requirements, data assets and structures that are necessary This is definitely not the average user. It’s like saying that anyone can be a surgeon. All you is to know what you want to do and have a Doctorate in medicine and have completed a surgical internship.

Vibe coding can be amazing but, if you don’t understand the logical structure necessary to create an app, website, or agent, then you may be spending a lot of time and money trying to accomplish the task you wish to achieve. Like many advanced disciplines, this means you don’t just start creating, you do a lot of research and planning.