FRESH

Hacker News

Waypoint-1: Real-Time Interactive Video Diffusion from Overworld

91 points by avaer

by ecmulli

2 subcomments

I dont have a big enough GPU but I was able to play around with the model using this plugin https://github.com/daydreamlive/scope-overworld via Runpod - very cool!

by roskelld

2 subcomments

The context seemed to last a few seconds. I went from a mock up screenshot of a fantasy video game, complete with first person weapon. Then as I moved forward the weapon became part of the scenery and the whole world blurred and blended until it became some sort of sci-fi abstract space. Spinning the camera completely changed look and style.
I ended up with a UI that closely resembled the Cyberpunk 2077 one complete with VO modal popup. I guess it must have featured a lot in the training data.
Really not sure what to make of this, seems to have no constraints on concept despite the prompt (I specifically used the word fantasy), no spatial memory, no collision, or understanding of landscape features in order to maintain a sense of place.

by lcastricato

0 subcomment

BTW, there is a gradio space here:
https://huggingface.co/spaces/Overworld/waypoint-1-small
And our streamed version:
https://overworld.stream

by avaer

0 subcomment

If you think this is cool you might also be interested in https://github.com/MineDojo/NitroGen which is kind of the opposite (and complimentary).

by Plankaluel

0 subcomment

An RTX 5090 for 20-30fps for the small model: That is not as unreasonable as I had feared :D

by dsrtslnd23

1 subcomments

by khimaros

0 subcomment

by lcastricato

3 subcomments

0 subcomment