It's tough to answer because you want to hedge for both an AI enthused employer and an AI hesitant employer with limited information about who they are and how they personally use these products. I've been responding with a sort of long winded answer about how 'there is clearly a learning curve for how this technology fits into any process and how I always always always double double double check yadayadayada'

I'm probably using the chat/ask functionality on a daily basis for quick debugging / new technology learning questions but I have yet to really use the fully agent or computer-use products because I've had more bad results than good the few times I've tried them (re-factoring a big repo of decades old fortran+C code for modern compiler/OS some things started to work but ultimately I abandoned that effort).

As of 2023, 27% of American working-age adults were at a PIAAC Literacy Level of 1 or below, out of a total of 5 levels. This has gotten drastically worse in the past 10 years as, in 2013, Level 1 and below was only 17%.

Full scores for 2023 are: % Level 1 or below: 27% Level 2: 29% Level 3: 31% Level 4/5: 13%

For reference, Level 1 means someone can't really handle a full page of text, and can sort of handle simple 1-page web pages. Level 2 is the point where someone can start to handle a few pages of straightforward text, but still nothing particularly complicated.

(Both of those descriptions undersell just how bad it really is, but I'll leave it at that, for the sake of brevity.)

People that aren't using AI at all often aren't using it because they effectively can't. On a fundamental level.

Source: https://nces.ed.gov/surveys/piaac/2023/national_results.asp

In my experience, it's a mixed bag. I wrote this comment[0], yesterday. It reflects my current work, and how I am integrating an LLM.

I have used it for two parts of my project:

1) The backend (PHP), and

2) The frontend (Swift)

It has been a huge help, in both, but #2 is a cautionary tale. It really needs adult supervision, in developing native UIKit Swift apps. I'm realizing how truly bad the code it wrote was. I mean, terrible.

That's jarring, because it did a great job with #1. It made sound, reasonable design decisions, and provided code that is better than what I would write.

With #2, it behaved exactly like an inexperienced engineer, panicking, when confronted with real-world problems. My rewrite is going to feature a much simpler, sound approach.

All that said, it has been a net positive, and has increased my productivity by a large margin.

I guess the lesson I needed to get from this, is that it is good at helping me to find problems, but maybe not so good at fixing them.

[0] https://news.ycombinator.com/item?id=48515217

But 2. For most other things, LLMs are fairly underwhelming. Research is usually mediocre. Try being rigorous and repeat your research prompt many times - then make a confusion matrix to tally up how many false positives and false negatives occur. And for the rest, be honest and ask yourself if the LLM is doing much more than a basic search engine query or trip to Wikipedia would have told you. For “normie” use cases, it’s handy-ish but far from revolutionary

That's an interesting analogy as, despite the real ecological issues with it and principled arguments against meat eating, in general meat consumption has trended upward globally in country after country for decades.

How does this connect to everyone's high level ideas/thoughts about "tech", "AI" and "morals and feels" etc. These lines can start to seem a little blurry, at least for me.

For example, would we say my partner is "using AI" (for all intents and purposes), if she's frequently using Google.com throughout the day, and then ends up picking and believing the AI generated answer overview at the top of the SERPs almost every time?

Or do we feel "uses AI", is more along the lines of the vampire kids running 1000 sub-agents on a mattress floor in SF?

I kind of find the whole spectrum really interesting because even basic phone use is now stuffed with AI, whether we choose to label it or not.

"low effort and convenient" seems to consistently win over "best quality" and this is going to be a downgrade in everything, for everyone

For example; ChatGPT is replacing my Google searching. Not necessarily because it's better, or because it's summaries are better than Google (I find them subjectively better but it's not clear cut).

But because the app has a nice history; can ask a relatively complicated question and go do something else and then come back to it, ask a follow up. Etc.

None of that is specifically an AI benefit, but it's a workflow that really helps, well, flow.

This makes me less bearish on the AI investments that are being made, if 70% of the working age population isn't using AI then there still is a lot of growth. The future is here, it's just not evenly distributed (yet)

"No, everyone is not using the internet for everything."

Which would have been entirely true when written, and entirely false a relatively short time later.

Everyone does use the internet for everything today, and everyone will use AI for everything soon.

I don't get these comments.

If you consider things like the machine learning filters in your smartphone camera and Google's AI Overviews for searches it's entirely plausible that the US is currently at 75%+ of AI usage.

If I worked in marketing/growth for an AI company I would try to consider some ways of breaking through this gap.

Looking things up and asking questions was always something for a minority of the population so the language model usage being relatively low isn't a surprise.

Problem arises if the non-AI segment is leveraged to create regulations that impact the AI using segment negatively.

i am not saying it's really powerful or great. but the lure is undeniable. because of how low friction it has become.

They are great on exploring, understanding and finding bugs in existing codebase.

They are great for simple or one time scripts/programs.

They are terrible, really terrible coders. The overengineering is so deep in their training that no matter what is your prompt, your skills or agents.md/claude.md, if you don't babysit them continuously, at some point they will just fuck up your codebase.

Software engineers are definitely in a bit of a bubble here. Are we just early adopters who see the value sooner, or does it uniquely benefit software engineering, or do we just like cool automation and we're deluding ourselves that this adds value beyond the cost?

- I'm getting my roof replaced due to hail damage. Insurance originally covered only $5k due to depreciation. I fed the insurance policy to AI. I learned about the appraisal clause and invoked it. At the end, I got another $6,500 back.

- I was having issues with plumbing. Four different plumbers came, they all said the cast iron pipes under the house need to change. Quotes ranged from $35k to $55k. I had AI walk me through the process. It taught me about the yard line vs. under-slab distinction, and suggested getting just the yard line replaced first because it's much cheaper and can fix the issue. I did that and spent $6k. The issue was fixed. I "saved" $30k for now by deferring that massive month-long project. (For brevity, I'm omitting a ton of boring technical stuff I learned about plumbing that helped me make the optimal decision - none of the contractors bothered explaining any of it.)

- My 2010 Hyundai Santa Fe is starting to show its age. I've taken it to multiple different repair shops, then fed their diagnoses and recommendations to AI and figured out which ones are trying to fleece me and which ones are being more careful and conservative with their repair recommendations. Probably saved several thousand dollars there. Learned a lot about cars too!

- My partner and I are converting the backyard to a wildlife sanctuary. The AI helped us plan what to plant where (depending on lots of factors like sunlight location, irrigation access, etc.) and it has been going really well. Also planned out a dragonfly pond to deal with mosquitoes. AI created a project plan, including schematics, material purchase list and step-by-step instructions.

- I've been wanting to do various other home improvement projects, but only ones that make financial sense. I took photos of my house, both inside and outside, and fed them to AI, and said "give me a list of projects I can do that will have high ROI for when I decide to sell this house". It spent 15 mins doing deep research, then came back with a long, prioritized list. If I do all the projects, I'd be spending about $40k and it would improve the house valuation by about $90k.

I can go on. There's probably dozens of stuff that I've used it for over the past year that led to massive time and money savings, and I've learned a ton as well about topics I normally would not have been exposed to or bothered to research myself. And I'm not even including all the work-related usage, both for my employer and my side business. That would be its own very long list.

Nor should they! It's such a shit thing to be emotionally invested in. Imagine people would have been upset about databases. It's really fantastic software and we should be happy to have it, and now go and make the most of it, for all of us.

I also just bought a completely mechanical film camera to learn a new old skill with no tech to fall back on.

Actually anything that is about 90% great and 10% disastrously wrong is utter crap given the way people want and do use AI models.

They are great tools in the right hands and awful in the wrong.

My wife uses it for a (non-computer related) business though and it's great for all sorts of normally tedious marketing/social media type jobs though. Stuff that doesn't really require accuracy just needs text on pictures that looks good quickly.

I think everyone just has FOMO and doesn't want to lose to competitors. Eventually it'll die down.

That aside, this piece is interesting and ties together some useful numbers and studies.

I hadn't seen the recent Microsoft paper showing:

> 30 percent of the US working-age population is using AI [...] with at least 90 minutes of usage time in a given month.

I'm honestly impressed at how high that number is! That's a lot of adoption for a technology (LLM chatbots) that didn't exist four years ago.