Multiple years have passed since then. GPT spark got me excited again, but somehow that seems to have faded right back into obscurity.
Can somebody explain why there’s so little apparent progress here despite the theoretically massive advantage? Can I still expect this to happen eventually?
I think wafer scale could improve performance of models and has some applications, but from a manufacturing perspective this approach seems cursed. Defect density is irrelevant when your target is an actual barn door. You can make the system resilient to defects, however the tradeoff is that you have to hedge for defects being anywhere. With chiplets, you accept that some units of space will be completely unusable. The trade off is that others are much higher performance because we don't have to spend any space or time on redundancy.