FRESH

Hacker News

Home

We’re years away from anyone creating artificial intelligence, says Martha Wells

45 points by sohkamyung

by clauderoux

0 subcomment

Some people still think that LLM are just word predictors. Technically, it is not. First, transformer architectures don't process words, they process semantic representations stored as vectors or embeddings in a continuous space. What a lot of people don't understand is that in an LLM, we go from discrete values (the tokens) to continuous values (embeddings) that the transformer takes as input. A transformer is a polynomial function that will project into the latent embedding space. It doesn't generate word per se, but a vector that is then compared against the latent embedding space to find the closest matches. The decoding part is usually not deterministic. This huge polynomial function is the reason we can't understand what is going on in a transformer. It doesn't mimick human speech, it builds a huge representation of the world, which it uses to respond to a query. This is not a conceptual graph, it is not a mere semantic representation. It is a distillation of all the data it ingested. And each model is unique as the process itself is split over hundreds of gpu, with no control over which GPU is going to churn out which part of the dataset and in which order.

by zamalek

1 subcomments

This has been my opinion for some time too. I don't think I'll see AGI in my lifetime. I think the current widespread belief comes from the massive leap that transformers provided, but transformers have their limits. We would need another radically new idea in order to create AGI - which, just like all discoveries that aren't evolutionary, boils down to random chance[1]. What transformers have given us is substantially more infrastructure for trying new ideas out, so the probability of AGI being discovered has increased.
[1]: https://en.wikipedia.org/wiki/Eureka_effect

by cgriswald

0 subcomment

Well, Wells actually says "...years and years and years away from anyone creating an actual artificial intelligence."
You know, in case you correctly interpreted the headline to mean Wells is saying aliens developed AI out there.

by vouaobrasil

2 subcomments

Hyping up AGI is a good way for tech companies to distract people into thinking AI is actually not that big a deal, when it is. It may not be in terms of its pure reasoning or in the goal of reaching AGI, but it is very disruptive, and it's a guaranteed way to heavily reinforce the requirements of using big tech in daily life, without actually improving it.
Yes, it may not be AGI and AGI may not come any time soon, but by focusing on that question, people become distracted and don't have as much time to think about how parasitic big tech really is. If it's not a strategy used consciously, it's rather seredipitous for them that the question has come about.

by tartoran

2 subcomments

What we're currently on with LLMs is some kind of scripted artficial intelligence. I would say that is not necessarily a bad thing considering that true artificial intelligence that had autonomy and goals for preserving itself could easily escape our control and wreak real havoc unless we approach it with tiny steps and clear goals.

by baal80spam

2 subcomments

You cannot be "light-years away" from a specific point in time. Who is this person and why is what they say important?

by noiv

0 subcomment

Well, considering the impact current models already have now, these are good news.

by lostmsu

0 subcomment

I know the phrase is a metaphor, but it is ironic that light-year away in time is, well, a year.

0 subcomment

by ninetyninenine

2 subcomments

I can’t read the site it requires subscription. But I and many other researchers disagree. George Hinton for example, massive disagreement.
It’s not just LLMs that were a leap and bound. For the past decade and more ML has been rising at a breakneck velocity. We see models for scene recognition, models that can read your mind, models that recognize human movement. We were seeing the pieces and components and amazing results constantly for over 10 years and this is independent of LLMs.
And then everyone thinks AI is thousands of years away because we hit a small blip with LLMs in 2 years.
And here’s the thing. The blip isn’t even solid. Like LLMs sometimes gets shit wrong and sometimes gets shit right we just can’t control it. Like we can’t definitively say LLMs can’t answer a specific question. Maybe another LLM can get it right, maybe if prompted a different way it will get it right.
The other strange thing is that the LLM shows signs of lying. Like it’s not truthful. It has knowledge of the truth but the things purpose is not really to tell us the truth.
I guess the best way to put it that current AI sometimes behaves like AGI and sometimes doesn’t. It is not consistently AGI. The fact that we built a machine that inconsistently acts like agi shows how freaking close we are.
But the reality is no one understands how LLMs work. This fact is definitive. Like if you think we know how LLMs work then you are out of touch with reality. Nobody knows how LLMs work so this article and my write up are really speculation. We really dont know.
But the 10 year trendline of AI in general is the one that has a more accurate trendline into future progress. Basing the future off a 2 year trendline of a specific problem with a specific model of ML of LLMs hallucinating is not predictive.

by metalman

0 subcomment

While "true AI" is likely impossible, that discussion detracts from the fact that a whole new and very powerfull ability to process information is here , which will/is bieng used to automate routine managerial tasks and run certain types of robotic equipment. I am slowly preparing myself to use these new tools, but will never consider them "clean", and will impliment there use in certain hermeticaly compartmented areas of my professional and financial undertakings.

by jmclnx

1 subcomments

>We’re Light-Years Away
Needs to be pointed out :) If I move billions of Light-Years from here I will be able to create AI :) A Light-Year is distance, the title should say maybe "decades away".
But I fully believe her argument, I think kids being born today will not see any real AI implementation in their lifetime.