You're in a room without windows. Everything outside the room is culled. Frame rate is very high. Then you open the door and go outside into a large city. Some buildings have big windows showing the interior, so you can't cull the building interior. You're on a long street and can look off into the distance. Now keep the frame rate from dropping while not losing distant objects.

Games with fixed objects can design the world to avoid these situations. Few games have many windows you can look into. Long sightlines are often avoided in level design. If you don't have those options, you have to accept that distant objects will be displayed, and level of detail handling becomes more important than occlusion. Impostors. Lots of impostors.

Occlusion culling itself has a compute cost. I've seen the cost of culling big scenes exceed the cost of drawing the culled content.

This is one of those hard problems metaverses have, and which, despite the amount of money thrown at the problem, were not solved during the metaverse boom. Meta does not seem to have contributed much to graphics technology.

This is much of why Second Life is slow.

- do we have libraries in c++ that can handle this for us?

It was used extensively in outdoor games like Jak and Daxter.

The BSP tree was also extremely useful for optimizing netcode for games like Quake 3 Arena and games within that family and time period I believe.

https://www.youtube.com/watch?v=CHYxjpYep_M

https://slitherworld.com

This worked very well but of course if the camera turned quickly, you would see pop-in. Not a modern technique, but an oldschool one.