FRESH

Hacker News

Mixing Visual and Textual Code

77 points by doppioandante

by kitd

1 subcomments

A very long time ago I wrote an Eclipse plug in that would read from source comments an ascii diagram describing the state/transition flow of a java class object, and generate/update the necessary state fields and transition methods.
It was pretty cool in principle but nearly unworkable in practice, purely because maintaining an ascii diagram in a text editor for anything more than "Hello world" is a massive PITA.
The simple text editor has a lot to answer for when it comes to how we think about programming, in a Sapir-Whorf kind of way. It's a shame there has been no cast iron standard for mixing live text and visuals. Our industry might look very different.

by Izkata

1 subcomments

SikuliX was a UI automation tool from over a decade ago with an IDE that looked similar to their screenshots: https://avleonov.com/wp-content/uploads/2017/02/script_windo...
The language was just python and the IDE swapped out certain functions' arguments with the image and targets within the image that the code referred to. It also had a bunch of built-in functionality that let you manipulate the images and targets (the little red crosses in that screenshot) directly within the IDE instead of having to edit the underlying python code.
Looks like it has a successor called Oculix but I know nothing about that.

by WillAdams

0 subcomment

Every to often a project will come along which reminds me of the visions which Hermann Hesse's _The Glass Bead Game_ (originally published as _Magister Ludi_) inspired in me --- this looks to be the latest.
Looking forward to reading the paper and hope that it is widely read/discussed, and that it influences projects/developers such as: https://www.inkandswitch.com/
I'd dearly love to see a 3D CAD program which used this methodology.

by Garlef

1 subcomments

Hm...the screenshots don't really sell it;
Maybe the authors should just vibe code a cljs port and put it in a browser?
And showcase some program written in this language that sells it better?

by donaldihunter

1 subcomments

Visual languages often fail when it comes to concurrent development where the state of the art is textual diff and merge. The IBM Rational suite attempted visual model diff/merge but I recall it being unworkable in practice.
Modern collaborative visual tools like Miro sidestep the concurrent development problem by being live multiplayer and by essentially having no version management at all.
Most visual BPM tools that I'm aware of try to mix flowchart style programming with e.g. Javascript on activity nodes, but they fail provide any developer-time syntax checking or completion. They also tend to serialise to XML which is unworkable in practice for diff/merge of the visual logic.
In a previous job we developed an in-house flowchart based language that embedded Javascript. We put the effort into writing a first-class Eclipse plugin that had syntax checking and completion across all graphical and Javascript constructs. It even had interactive debugging that interleaved graphical and textual single-stepping. We never solved the diff/merge problem in a satisfactory way.

by Twey

0 subcomment

by doctorhandshake

0 subcomment

I work daily in TouchDesigner and am constantly wrestling with the integration of its node and Python scripting contexts. In particular order of operations, delegation of responsibility, and store of state are very unsolved areas when expressing intent across both domains.

by gr__or

0 subcomment

https://bsky.app/profile/watwa.re/post/3lizpdz34522g
I'm working on a similar idea, it's been a while since I published about it but open to input and collabs!

by summarybot

0 subcomment

So cool! Love that Clojure(Script) is the language of choice here. Seeing a graphical tile nestled in a function call blew my mind for a hot minute, there.

by michael-ax

1 subcomments

We have the complete AST in almost every IDE. Why are we not learning to render abstracted code and systems from that??

by cml123

0 subcomment

I feel it would be remiss of me not to mention the humble TempleOS support for mixing code and sprites. I can't say I've ever used the system myself, but its DolDoc format [0] offered some pretty nifty features for directly interspersing sprites into source code viewable together as one. You can see some screenshots on the hackmag page [1] under the section that reads: "If that’s not enough to impress you, open any game file and behold the unthinkable: the sprites are embedded right in the code!"
0) https://tinkeros.github.io/WbTempleOS/Doc/DolDocOverview.htm... 1) https://hackmag.com/security/templeos

by conartist6

2 subcomments

Oh dear. I think they may fall over in surprise when they realize that this technology become commercially viable while they were busy writing this academic paper...