Say we discover a new architecture breakthrough like Yann LeCun's proposed JEPA. Won't scaling laws apply to it anyway?
Suppose training is so efficient that you can train state of the art AGI on a few GPUs. If it's better than current LLMs, there will be more demand/inference, which will require more GPUs and we are back at the same "add more gpus".
I find it hard to believe that we, as a humanity, will hit the wall of "we don't need more compute", no matter what the algorithms are.
If it was AI-generated I had no difficulty with it, certainly on par with typical surface level journalist summaries, and vastly better than losing 2 hours of my life to watching some video interviews.. :) AI as we know it may not be real intelligence but it certainly has valid uses
I get a lot from seeing the person talk vs reading a summary. I have gone back and watched a lot of interviews and talks with Ilya. In hindsight, it is easy to hear the future ideas in his words at the time.
That said, I use AI summaries for a lot of stuff that I don't really care about. For me, this topic is important enough to spend two hours of my life on, soaking up every detail.
This may not be AGI, but I think LLMs as is, with no other innovation, are capably enough for gigantic labor replacement with the right scaffolding. Even humans need a lot of scaffolding at scale (e.g. sales reps use CRMs even though they are generally intelligent). LLMs solve a “fuzzy input” problem that traditional software struggles with. I’m guessing something like 80% of current white collar jobs can be automated with LLMs plus scaffolding.
My first job out of uni was in creating automated tests to validate some set top box. It involved using library of "blocks" to operate a remote control. Some of the people I have been working with spent their whole career in this narrow area, building those libraries of block and using them for customer and I have no doubts a LLM can today produce the same tests without any human intervention
Replacing labor doesn't require replacing whole jobs, it's enough to only replace specific tasks within those jobs which will reduce the number of workers needed for the same amount of work.
To pick a rather extreme example, the fraction of the population involved in farming is rather lower than in the past. Due to productivity improvements.
Most QA, most analyst positions, a good chunk of the kludge in intellectually challenging jobs, like medical diagnostics or software engineering, most administrative work, including in education and in healthcare, about 80% of customer success, about 80% of sales, are all within striking distance of automation with current-generation LLMs. And taht's entirely ignoring the 2nd-order effects in robotics and manufacturing.
Such a decision merely tips the scale into a brittle structure territory. It introduces critical points of failure (funneling responsibility through fewer "nodes", stronger reliance on compute, electricity, internet, and more) and reduces adaptability (e.g. data bias, data cutoff dates, unaware of minute evolving human needs, etc).
I agree, AI image recognition is so good already that it can tell what someone is doing or what is happening in a picture. Just have that run at 30 fps and make the robot's movements align with that understanding and bam, you effectively have "AGI" in some sense no? I mean sure, maybe it doesn't really remember anything like a human would and it doesn't learn on the fly yet but it's definitely some kind of intelligent, autonomous thing that will be able to respond to just about anything in the world. Making it able to learn on demand is something people are working on. Chatgpt remembers some stuff already too after all. It's very small and very spotty, weird memory but hey, it's something. As soon as that becomes a tiny bit better you'll already beat humans at memory.
This opinion is not based in reality. The only way to understand that is to go outside and talk to real people who are neither techies nor managers, and, better yet, try to do their jobs better than they do.
All the frontier houses know this too. They also know it will be extremely difficult to raise more capital if their pitch is "we need to go back to research, which might return nothing at all."
Ilya did also acknowledge that these houses will still generate gobs of revenue, despite being at a dead end, so I'm not sure what the criticism is, exactly.
Everyone knows another breakthrough is required for agi to arrive; sama explicitly said this. Do you wait and sit on your hands until that breakthrough arrives? Or make a lot of money while skating to where the puck will be?
I would say the issue is that most of the big AI players are burning a lot more cash than they earn, and the main thesis is that they are doing so because their product will be so huge that they will need 10x-100x infrastructure to support it.
But what we're seeing at the moment, is a deceleration, not an acceleration.
There is no example of a leading company that ships a world-changing product, yet somehow runs out of cash.
Maybe they lose relevance. Maybe they miss the breakthrough. That becomes the reason. So perplexity? Sure. Anthropic, even? Yep. Google? OpenAI? Nah.
Regardless, viewing the unit economics, there are very clear sight lines to profitability if they want it. Just like with Amazon, Tesla, Apple, etc., when you want to grow, hoarding cash is a bad play.
OpenAI is very unlikely to go bankrupt, but they could be in such a difficult financial position that they would have to make painful compromises with Microsoft and/or Nvidia and lose most of their leverage.
Microsoft, largest software company in the world, has (publicly and privately) admitted that it is in their best interest to ensure that OpenAI remains the leading ai company for at least the next seven years.
As for nvidia, if OpenAI has less leverage, that necessitates a different ai company having more. Who would it be?
No matter the unit economics, commoditization and advancement of open models and small startups means that there is at almost a year or two to exploit competitive advantage. If scaling stops, the window to make a profit is extremely narrow.
People at Anthropic have a vested interest in getting you to believe that they are creating new and exciting things. Of course they say that a breakthrough is imminent. Doesn't make it in the least true, though.
> Models look god-tier on paper:
> they pass exams
> solve benchmark coding tasks
> reach crazy scores on reasoning evals
Models don't look "god-tier" from benchmarks. Surely an 80% is not godlike. I would really like more human comparisons for these benchmarks to get a good idea of what an 80% means though.
I would not say that any model shows a "crazy" score on ARC-AGI.
I broadly have seen incremental improvements in benchmarks since 2020, mostly at a level I would believe to be below average human reasoning, but above average human knowledge. No one would call GPT-3 godlike and it is quite similar to modern models in benchmarks; it is not a difference of like 1% vs 90%. I think most people would consider gpt-3 to be closer to opus 4.5 than opus 4.5 is to a human.
Roughly I'd agree, although I don't have hard numbers, and I'd say GPT-4 in 2023 vs GPT-3 as the last major "wow" release from a purely-model perspective. But they've also gotten a lot faster, which has its own value. And the tooling around them has gotten MASSIVELY better - remember the "prompt engineering" craze? Now there are a lot of tools out there that will take your two-sentence prompt and figure out - even asking you questions sometimes - how to best execute that based on local context like in a code repository, and iterate by "re-prompting" itself over and over. In a fraction of the time you could've done that by manual "prompt engineering."
Though I do not fully know where the boundary between "a model prompted to iterate and use tools" and "a model trained to be more iterative by design" is. How meaningful is that distinction?
But the people who don't get this are the less-technical/less-hands-on VPs, CEOs, etc, who are deciding on layoffs, upcoming headcount, "replace our customer service or engineering staffs with AI" things. A lot of those moves are going to look either really silly or really genius depending on exactly how "AGI-like" the plateau turns out to be. And that affects a LOT of people's jobs/livelihood, so it's good to see the hype machine start to slow down and get more realistic about the near-term future.
How do you avoid overfitting with the automated prompts? It seems to add lots of exceptions from what I've seen in the past versus generalize as much as a human would.
I dunno, some of the questions on things like Humanity's Last Exam sure strike me as "godlike." Yes, I'm happy that I can still crush LLMs on ARC-AGI-2 but I see the writing on the wall there, too. Barely over a year ago LLMs were what, single digit percentages on ARC-AGI-1?
Didn’t we just see big pretraining gains from Google and likely Anthropic?
I like Dario’s view on this, we’ve seen this story before with deep learning. Then we progressively got better regularization, initialization, and activations.
I’m sure this will follow the same suit, the graph of improvement is still linear up and to the right
While I think there's obvious merit to their skepticism over the race towards agi, Sutskever's goal doesn't seem practical to me. As Dwarkesh also said, we reach to a safe and eventually perfect system by deploying it in public and iterating over it until optimal convergence dictated by users in a free market. Hence, I trust that Google, OpenAI or Anthropic will reach there, not SSI.
> we reach to a safe and eventually perfect system by deploying it in public and iterating over it until optimal convergence dictated by users in a free market
Possibly... but also a lot of the foundational AI advancements were actually done in skunkworks-like environments and with pure research rather than iterating in front of the public.
It's not 100% clear to me if the ultimate path to the end is iteration or something completely new.
Some have been saying this for years now, but the consensus in the AI community and SV has been visibly shifting in the recent months.
Social contagion is astonishingly potent around ideas like this one, and this probably explains why the zeitgeist seems to be immovable for a time and then suddenly change.
People have been saying this before chatgpt and ever since. And they're right.
Its charlatans like sama that muddy the waters by promising the sky to get money for their empire building.
LLMs can make and are great great products. But its sneaky salesmen that are the ones saying scaling is the path to AGI. The reality is that they're just aiming for economies of scale to make their business viable
>What high quality data sources are not already tapped
Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic). Maybe the "AGI needs to be embodied" people will be right, because that's the only way to get enough coherent multimodal data to do things like long-range planning, navigation, game-playing, and visual tasks.
> Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic).
Be careful with mistaking data for information.
You are getting a digital (maybe lossy compressed) samples of photons and sound waves. It is not unlimited, a camera pointed at a building at night is going to have very little new information from second to second. A microphone outside is going to have very little new information second to second unless something audible is happening close by.
You can max out your storage capacity by adding twenty ML high megapixel cameras recording frames as tiff file but gain little new useful information for every camera you add.
This is where it is a bit confusing for people not familiar with the state of the art.
Some people don't seem to realize how critical the "eval" function is for machine learning.
Raw data is not much more useful than noise for the current recipes of model training.
Human produced data on the internet (text, images, etc.) is highly structured and the eval function can easily be built.
Chess or Go has rules and the eval function is more or less derived or discovered from them.
But the real world?
For driving you can more or less build a computer vision system able to follow a road in a week, because the eval function is so simple. But for all the complex parts, the eval function is basically one bit (you crashed/not crashed) that you have to sip very slowly, and it very inefficient to train such a complex system with such a minimal reward even in simulations.
The real world is governed by physics, so isn't "next state prediction" a sufficient eval function that forces it to internalize a world model? And increasing the timespan over which you predict requires an increasing amount of "intelligence" because it requires modeling the real-world behavior of constituent subsystems that are often black-boxes (e.g. if there is a crow on a road with a car approaching, you can't just treat it as a physics simulation, you need to know that crows are capable of flying to understand what is going to happen).
I don't see how this is any less structured than the CLM objective of LLMs, there's a bunch of rich information there.
I thought about the huge pile of hard drives in Utah this morning. The TLAs in the USA have a metric shit ton of data that _should_ not be used but _could_ be used.
Even still, we need evolutions in model architecture to get to the next level. Data is not enough.
I thought Ilya said we have more companies than we have ideas. He also noted that our current are resulting in models which are very good at benchmarks but have some problems with generalization (and gave a theory as to why).
But I don't recall him actually saying that the current ideas won't lead to AGI.
Then, he starts to talk about the other ideas but his lawyers / investors prevent him from going into detail: https://youtu.be/aR20FWCCjAs?t=1939
The worrisome thing is that he openly talks about whether to release AGI to the public. So, there could be a world in which some superpower has access to wildly different tech than the public.
To take Hinton's analogy of AGI to extraterrestrial intelligence, this would be akin to a government having made contact but withholding the discovery and the technology from the public: https://youtu.be/e1Hf-o1SzL4?t=30
A problem is that the bulk of the people behind these labs are people that were conditioned from an early age to achieve high scores in standardized tests and conflate that with intelligence. Then apply that mentality to their models resulting in these leaderboards that nobody cares about.
Absolutely this, models acheive very highly on kind problems; ones that you can master with sufficient practice. Which is just remarkable, but the world is a wicked learning environment, and repetition is not enough.
The frenzy around AI is to do with growth fueled cocaine capitalism seeking 'more' where rational minds can see that we don't have that much more runway left with our current mode of operation.
While the tech is useful, the mass amounts of money being shoveled into AI has more to do with the ever escaping mirage of a promised land where there will be an infinite amount of 'more'. For some people that means post scarcity, for others it means a world dominating AGI that achieves escape velocity against the current gridlock of geopolitics, for still others it means ejecting the pesky labour class and replacing all labour needs with AI and robots. Varied needs, but all perceived as urgent and inescapable by their vested interests.
I am somewhat relieved that we're not headed into the singularity just yet, I see it as way too risky given the current balance of power and stability across the planet. The outcome of ever accelerating tech progress at the expense of all other dimensions of wellbeing is not good for the majority of life here.
> The frenzy around AI is to do with growth fueled cocaine capitalism seeking 'more' where rational minds can see that we don't have that much more runway left with our current mode of operation.
When talking with non-tech people around me, it’s really not about “rational minds”, it’s that people really don’t understand how all this works and as such don’t see the limitations of it.
Combine that with a whole lot of FOMO which happens often with investors and you have a whole pile of money being invested.
From what I hear, most companies like Google and Meta have a lot of money to burn, and their official position towards investors is “chances of reaching ASI/AGI are very low, but if we do and we miss out on it, it will mean a huge opportunity loss so it’s worth the investment right now”.
Say we discover a new architecture breakthrough like Yann LeCun's proposed JEPA. Won't scaling laws apply to it anyway?
Suppose training is so efficient that you can train state of the art AGI on a few GPUs. If it's better than current LLMs, there will be more demand/inference, which will require more GPUs and we are back at the same "add more gpus".
I find it hard to believe that we, as a humanity, will hit the wall of "we don't need more compute", no matter what the algorithms are.
This is an AI generated article based on real interviews.
Watch the original Sutskever interview: https://www.youtube.com/watch?v=aR20FWCCjAs
And LeCun: https://www.youtube.com/watch?v=4__gg83s_Do
If it was AI-generated I had no difficulty with it, certainly on par with typical surface level journalist summaries, and vastly better than losing 2 hours of my life to watching some video interviews.. :) AI as we know it may not be real intelligence but it certainly has valid uses
I get a lot from seeing the person talk vs reading a summary. I have gone back and watched a lot of interviews and talks with Ilya. In hindsight, it is easy to hear the future ideas in his words at the time.
That said, I use AI summaries for a lot of stuff that I don't really care about. For me, this topic is important enough to spend two hours of my life on, soaking up every detail.
As for being on par with typical surface level journalism. I think we might be further into the dead internet than most people realize: https://en.wikipedia.org/wiki/Dead_Internet_theory
This may not be AGI, but I think LLMs as is, with no other innovation, are capably enough for gigantic labor replacement with the right scaffolding. Even humans need a lot of scaffolding at scale (e.g. sales reps use CRMs even though they are generally intelligent). LLMs solve a “fuzzy input” problem that traditional software struggles with. I’m guessing something like 80% of current white collar jobs can be automated with LLMs plus scaffolding.
What's a white collar job than can be automated with LLMs plus scaffolding?
My first job out of uni was in creating automated tests to validate some set top box. It involved using library of "blocks" to operate a remote control. Some of the people I have been working with spent their whole career in this narrow area, building those libraries of block and using them for customer and I have no doubts a LLM can today produce the same tests without any human intervention
Replacing labor doesn't require replacing whole jobs, it's enough to only replace specific tasks within those jobs which will reduce the number of workers needed for the same amount of work.
But then it becomes a competitive advantage for another firm to use the same employees to do more work, leading to the jobs not being replaced.
To pick a rather extreme example, the fraction of the population involved in farming is rather lower than in the past. Due to productivity improvements.
Most QA, most analyst positions, a good chunk of the kludge in intellectually challenging jobs, like medical diagnostics or software engineering, most administrative work, including in education and in healthcare, about 80% of customer success, about 80% of sales, are all within striking distance of automation with current-generation LLMs. And taht's entirely ignoring the 2nd-order effects in robotics and manufacturing.
Business Intelligence Engineers
> LLMs solve a “fuzzy input” problem that traditional software struggles with
This is basically all of it.
Kind of how word processors solved the writing is tedious struggle and search solved the "can't curate the internet" struggle.
Such a decision merely tips the scale into a brittle structure territory. It introduces critical points of failure (funneling responsibility through fewer "nodes", stronger reliance on compute, electricity, internet, and more) and reduces adaptability (e.g. data bias, data cutoff dates, unaware of minute evolving human needs, etc).
I agree, AI image recognition is so good already that it can tell what someone is doing or what is happening in a picture. Just have that run at 30 fps and make the robot's movements align with that understanding and bam, you effectively have "AGI" in some sense no? I mean sure, maybe it doesn't really remember anything like a human would and it doesn't learn on the fly yet but it's definitely some kind of intelligent, autonomous thing that will be able to respond to just about anything in the world. Making it able to learn on demand is something people are working on. Chatgpt remembers some stuff already too after all. It's very small and very spotty, weird memory but hey, it's something. As soon as that becomes a tiny bit better you'll already beat humans at memory.
Same here. Scale existing chips 100-1000x - there's plenty left to do. With that - we'll likely need 100x more power production too.
You sound like a manager that doesn't understand what your employees are doing.
Theyre not deterministic and real wirld work is risk adverse. People aew confusing sinusidal growth with singularity.
This opinion is not based in reality. The only way to understand that is to go outside and talk to real people who are neither techies nor managers, and, better yet, try to do their jobs better than they do.
Try talking to white collar workers outside your bubble. Better yet get a job.
All the frontier houses know this too. They also know it will be extremely difficult to raise more capital if their pitch is "we need to go back to research, which might return nothing at all."
Ilya did also acknowledge that these houses will still generate gobs of revenue, despite being at a dead end, so I'm not sure what the criticism is, exactly.
Everyone knows another breakthrough is required for agi to arrive; sama explicitly said this. Do you wait and sit on your hands until that breakthrough arrives? Or make a lot of money while skating to where the puck will be?
I would say the issue is that most of the big AI players are burning a lot more cash than they earn, and the main thesis is that they are doing so because their product will be so huge that they will need 10x-100x infrastructure to support it.
But what we're seeing at the moment, is a deceleration, not an acceleration.
There is no example of a leading company that ships a world-changing product, yet somehow runs out of cash.
Maybe they lose relevance. Maybe they miss the breakthrough. That becomes the reason. So perplexity? Sure. Anthropic, even? Yep. Google? OpenAI? Nah.
Regardless, viewing the unit economics, there are very clear sight lines to profitability if they want it. Just like with Amazon, Tesla, Apple, etc., when you want to grow, hoarding cash is a bad play.
OpenAI is very unlikely to go bankrupt, but they could be in such a difficult financial position that they would have to make painful compromises with Microsoft and/or Nvidia and lose most of their leverage.
Microsoft, largest software company in the world, has (publicly and privately) admitted that it is in their best interest to ensure that OpenAI remains the leading ai company for at least the next seven years.
As for nvidia, if OpenAI has less leverage, that necessitates a different ai company having more. Who would it be?
Houses?
So we're rehypothecating CDOs like the last bubble?
Revenue isn't profit
Countless examples of a lack of profit not spelling the doom of a company (uber, amazon, tesla).
Countless examples of companies that strive for profit too early, only to die
Amazon was profitable, they just prioritized scaling up so they reinvested all profits.
OpenAI is also very profitable if they deprioritize... scaling up.
A distinction that matters if the unit economics are bad, which nobody has full visibility over.
No matter the unit economics, commoditization and advancement of open models and small startups means that there is at almost a year or two to exploit competitive advantage. If scaling stops, the window to make a profit is extremely narrow.
Open models serve as a safeguard, not a competitor, to closed models.
They still cost billions to pre-train
Which is an issue with unit economics.
Google made $35 billion in profit last quarter.
Source?
Everyone at anthropic is saying ASI is imminent…
People at Anthropic have a vested interest in getting you to believe that they are creating new and exciting things. Of course they say that a breakthrough is imminent. Doesn't make it in the least true, though.
Reminds me of this commercial [1]
[1] https://www.youtube.com/watch?v=MzakqMAaHME
>> Everyone at anthropic is saying ASI is imminent…
Who exactly is saying this, other than C-level people?
I disagree with the framing in 2.1 a lot.
Models don't look "god-tier" from benchmarks. Surely an 80% is not godlike. I would really like more human comparisons for these benchmarks to get a good idea of what an 80% means though.I would not say that any model shows a "crazy" score on ARC-AGI.
I broadly have seen incremental improvements in benchmarks since 2020, mostly at a level I would believe to be below average human reasoning, but above average human knowledge. No one would call GPT-3 godlike and it is quite similar to modern models in benchmarks; it is not a difference of like 1% vs 90%. I think most people would consider gpt-3 to be closer to opus 4.5 than opus 4.5 is to a human.
Roughly I'd agree, although I don't have hard numbers, and I'd say GPT-4 in 2023 vs GPT-3 as the last major "wow" release from a purely-model perspective. But they've also gotten a lot faster, which has its own value. And the tooling around them has gotten MASSIVELY better - remember the "prompt engineering" craze? Now there are a lot of tools out there that will take your two-sentence prompt and figure out - even asking you questions sometimes - how to best execute that based on local context like in a code repository, and iterate by "re-prompting" itself over and over. In a fraction of the time you could've done that by manual "prompt engineering."
Though I do not fully know where the boundary between "a model prompted to iterate and use tools" and "a model trained to be more iterative by design" is. How meaningful is that distinction?
But the people who don't get this are the less-technical/less-hands-on VPs, CEOs, etc, who are deciding on layoffs, upcoming headcount, "replace our customer service or engineering staffs with AI" things. A lot of those moves are going to look either really silly or really genius depending on exactly how "AGI-like" the plateau turns out to be. And that affects a LOT of people's jobs/livelihood, so it's good to see the hype machine start to slow down and get more realistic about the near-term future.
How do you avoid overfitting with the automated prompts? It seems to add lots of exceptions from what I've seen in the past versus generalize as much as a human would.
Ask the agent "Is this over-fitting?"
I'm not joking.
I dunno, some of the questions on things like Humanity's Last Exam sure strike me as "godlike." Yes, I'm happy that I can still crush LLMs on ARC-AGI-2 but I see the writing on the wall there, too. Barely over a year ago LLMs were what, single digit percentages on ARC-AGI-1?
Didn’t we just see big pretraining gains from Google and likely Anthropic?
I like Dario’s view on this, we’ve seen this story before with deep learning. Then we progressively got better regularization, initialization, and activations.
I’m sure this will follow the same suit, the graph of improvement is still linear up and to the right
The gains were on benchmarks. Ilya describes why this is a red herring here: https://youtu.be/aR20FWCCjAs?t=286
Gemini 3 is a huge jump. I can't imagine how anyone who uses the models all the time wouldn't feel this.
This caught my eye.
> The industry is already operating at insane scale.
Sounds a lot like "640K ought to be enough for anybody", or "the market can stay irrational longer than you can stay solvent".
I don't doubt this person knows how things should go but I also don't doubt this will get bigger before it gets smaller.
While I think there's obvious merit to their skepticism over the race towards agi, Sutskever's goal doesn't seem practical to me. As Dwarkesh also said, we reach to a safe and eventually perfect system by deploying it in public and iterating over it until optimal convergence dictated by users in a free market. Hence, I trust that Google, OpenAI or Anthropic will reach there, not SSI.
> we reach to a safe and eventually perfect system by deploying it in public and iterating over it until optimal convergence dictated by users in a free market
Possibly... but also a lot of the foundational AI advancements were actually done in skunkworks-like environments and with pure research rather than iterating in front of the public.
It's not 100% clear to me if the ultimate path to the end is iteration or something completely new.
Some have been saying this for years now, but the consensus in the AI community and SV has been visibly shifting in the recent months.
Social contagion is astonishingly potent around ideas like this one, and this probably explains why the zeitgeist seems to be immovable for a time and then suddenly change.
People have been saying this before chatgpt and ever since. And they're right.
Its charlatans like sama that muddy the waters by promising the sky to get money for their empire building.
LLMs can make and are great great products. But its sneaky salesmen that are the ones saying scaling is the path to AGI. The reality is that they're just aiming for economies of scale to make their business viable
Dupe: https://news.ycombinator.com/item?id=46048125
I personally don’t think the scaling hypothesis is wrong, but it is running up against real limits
What high quality data sources are not already tapped?
Where does the next 1000x flops come from?
>What high quality data sources are not already tapped
Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic). Maybe the "AGI needs to be embodied" people will be right, because that's the only way to get enough coherent multimodal data to do things like long-range planning, navigation, game-playing, and visual tasks.
Also self-driving cars which are hoovering this data already. Both alphabet and grok have an unusual advantage with those data sources.
> Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic).
Be careful with mistaking data for information.
You are getting a digital (maybe lossy compressed) samples of photons and sound waves. It is not unlimited, a camera pointed at a building at night is going to have very little new information from second to second. A microphone outside is going to have very little new information second to second unless something audible is happening close by.
You can max out your storage capacity by adding twenty ML high megapixel cameras recording frames as tiff file but gain little new useful information for every camera you add.
This is where it is a bit confusing for people not familiar with the state of the art.
Some people don't seem to realize how critical the "eval" function is for machine learning.
Raw data is not much more useful than noise for the current recipes of model training.
Human produced data on the internet (text, images, etc.) is highly structured and the eval function can easily be built.
Chess or Go has rules and the eval function is more or less derived or discovered from them.
But the real world?
For driving you can more or less build a computer vision system able to follow a road in a week, because the eval function is so simple. But for all the complex parts, the eval function is basically one bit (you crashed/not crashed) that you have to sip very slowly, and it very inefficient to train such a complex system with such a minimal reward even in simulations.
The real world is governed by physics, so isn't "next state prediction" a sufficient eval function that forces it to internalize a world model? And increasing the timespan over which you predict requires an increasing amount of "intelligence" because it requires modeling the real-world behavior of constituent subsystems that are often black-boxes (e.g. if there is a crow on a road with a car approaching, you can't just treat it as a physics simulation, you need to know that crows are capable of flying to understand what is going to happen).
I don't see how this is any less structured than the CLM objective of LLMs, there's a bunch of rich information there.
If Transformers were able to learn in such a trivial fashion, we would already have AGI.
There is at least one missing piece to the puzzle, and some say 5-6 more breakthrough are necessary.
Dropbox, O365, Google workspace, AWS s3.
NSA datasets. Did your eye catch on the “genesis” project?
I thought about the huge pile of hard drives in Utah this morning. The TLAs in the USA have a metric shit ton of data that _should_ not be used but _could_ be used.
Even still, we need evolutions in model architecture to get to the next level. Data is not enough.
Modern ML builds on two pillars: GPUs and autodiff. Given that GPUs are running out of steam, I wonder what we should focus on now.
Not even close, I haven't heard a good argument from either of them. They should read the bitter lesson again.
Maybe you should read the bitter lesson again. Their current rhetoric is a direct extension of that perspective.
How do? Isn’t the bitter lesson about more search and compute? As opposed to clever algorithms
I thought Ilya said we have more companies than we have ideas. He also noted that our current are resulting in models which are very good at benchmarks but have some problems with generalization (and gave a theory as to why).
But I don't recall him actually saying that the current ideas won't lead to AGI.
He says the current models generalize dramatically worse than people: https://youtu.be/aR20FWCCjAs?t=1501
Then, he starts to talk about the other ideas but his lawyers / investors prevent him from going into detail: https://youtu.be/aR20FWCCjAs?t=1939
The worrisome thing is that he openly talks about whether to release AGI to the public. So, there could be a world in which some superpower has access to wildly different tech than the public.
To take Hinton's analogy of AGI to extraterrestrial intelligence, this would be akin to a government having made contact but withholding the discovery and the technology from the public: https://youtu.be/e1Hf-o1SzL4?t=30
It's a wild time to be alive.
The current idea is keep doing more of the same & expect different results.
A problem is that the bulk of the people behind these labs are people that were conditioned from an early age to achieve high scores in standardized tests and conflate that with intelligence. Then apply that mentality to their models resulting in these leaderboards that nobody cares about.
Absolutely this, models acheive very highly on kind problems; ones that you can master with sufficient practice. Which is just remarkable, but the world is a wicked learning environment, and repetition is not enough.
Why is Yann Lecun in same article as Ilya?
I can't tell if you meant this as a slight on Ilya Sutskever or on Yann LeCun. Both are well-known names in AI.
Pretty sure the article is AI slop, so it's kind of connect the dots
Ilya has appeared to shift to closer to Yann's position, though: he's been on the "scaling LLMs will fail to reach AGI" beat for a long time.
,> Pretty sure the article is AI slop
Yeah, the actual video with transcripts (YouTube link in bottom of TFA):
https://www.dwarkesh.com/p/ilya-sutskever-2
Ed: TFA is basically a dupe of
https://news.ycombinator.com/item?id=46048125
well, take a stab at it
so the top dogs state the obvious, again?
every LLM easily misaligned, "deceived to deceive" and whatnot and they want to focus on adding MORE ATTACK SURFACE???
and throw more CPU at it?
This is glorious.
time to invest in the pen & paper industry!
Every computer scientist with a grain of salt knows this…
Then they should speak up more because non-technical people and the general public are still confused.
the market doesnt want to listen, because lines go up...
The frenzy around AI is to do with growth fueled cocaine capitalism seeking 'more' where rational minds can see that we don't have that much more runway left with our current mode of operation.
While the tech is useful, the mass amounts of money being shoveled into AI has more to do with the ever escaping mirage of a promised land where there will be an infinite amount of 'more'. For some people that means post scarcity, for others it means a world dominating AGI that achieves escape velocity against the current gridlock of geopolitics, for still others it means ejecting the pesky labour class and replacing all labour needs with AI and robots. Varied needs, but all perceived as urgent and inescapable by their vested interests.
I am somewhat relieved that we're not headed into the singularity just yet, I see it as way too risky given the current balance of power and stability across the planet. The outcome of ever accelerating tech progress at the expense of all other dimensions of wellbeing is not good for the majority of life here.
> The frenzy around AI is to do with growth fueled cocaine capitalism seeking 'more' where rational minds can see that we don't have that much more runway left with our current mode of operation.
When talking with non-tech people around me, it’s really not about “rational minds”, it’s that people really don’t understand how all this works and as such don’t see the limitations of it.
Combine that with a whole lot of FOMO which happens often with investors and you have a whole pile of money being invested.
From what I hear, most companies like Google and Meta have a lot of money to burn, and their official position towards investors is “chances of reaching ASI/AGI are very low, but if we do and we miss out on it, it will mean a huge opportunity loss so it’s worth the investment right now”.