FRESH

Hacker News

Home

The unexpected effectiveness of one-shot decompilation with Claude

238 points by knackers

by simonw

6 subcomments

For anyone else who was initially confused by this, useful context is that Snowboard Kids 2 is an N64 game.
I also wasn't familiar with this terminology:
> You hand it a function; it tries to match it, and you move on.
In decompilation "matching" means you found a function block in the machine code, wrote some C, then confirmed that the C produces the exact same binary machine code once it is compiled.
The author's previous post explains this all in a bunch more detail: https://blog.chrislewis.au/using-coding-agents-to-decompile-...

by saagarjha

2 subcomments

It's worth noting here that the author came up with a handful of good heuristics to guide Claude and a very specific goal, and the LLM did a good job given those constraints. Most seasoned reverse engineers I know have found similar wins with those in place.
What LLMs are (still?) not good at is one-shot reverse engineering for understanding by a non-expert. If that's your goal, don't blindly use an LLM. People already know that you getting an LLM to write prose or code is bad, but it's worth remembering that doing this for decompilation is even harder :)

by t_mann

4 subcomments

> The ‘give up after ten attempts’ threshold aims to prevent Claude from wasting tokens when further progress is unlikely. It was only partially successful, as Claude would still sometimes make dozens of attempts.
Not what I would have expected from a 'one-shot'. Maybe self-supervised would be a more suitable term?

by reactordev

1 subcomments

I’ve been having fun sending Claude down the old school MUD route, giving it access to a SMAUG derivative and once it’s mastered the play, give it admin powers to create new play experiences.
I stayed away from decompilation and reverse engineering, for legal reasons.
Claude is amazing. It can sometimes get stuck in a reason loop but will break away, reassess, and continue on until it finds its way.
Claude was murdered in a dark instance dungeon when it managed to defeat the dragon but ran out of lamp oil and torches to find its way out. Because of the light system it kept getting “You can’t seem to see anything in the darkness” and randomly walked into a skeleton lair.
Super fun to watch from an observer. Super terrifying that this will replace us at the office.

by rlili

12 subcomments

Makes me wonder if decompilation could eventually become so trivial that everything would become de-facto open source.

by knackers

6 subcomments

I've been experimenting with running Claude in headless mode + a continuous loop to decompile N64 functions and the results have been pretty incredible. (This is despite already using Claude in my decompilation workflow).
I hope that others find this similarly useful.

by ACCount37

5 subcomments

If you aren't using LLMs for your reverse engineering tasks, you're missing out, big time. Claude kicks ass.
It's good at cleaning up decompiled code, at figuring out what functions do, at uncovering weird assembly tricks and more.

by sehugg

0 subcomment

I ran Node with --print-opt-code and had Opus look at Turbofan's output. It was able to add comments to the JIT'ed code and give suggestions on how to improve the JavaScript for better optimization.

by Nevermark

1 subcomments

There are quite a few comments here on code obfuscation.
The hardest form of code obfuscation is called homomorphic computing, which is code transformed to act on encrypted data isomorphically to regular code on regular data. The homomorphic code is hard obfuscated by this transformation.
Now create a homomorphic virtual machine, that operates on encrypted code over encrypted data. Very hard to understand.
Now add data encryption/decryption algorithms, both homomorphically encrypted to be run by the virtual machine, to prepare and recover inputs, outputs or effects of any data or event information, for the homomorphic application code. Now that all data within the system is encrypted by means which are hard obfuscated, running on code which is hard obfuscated, the entire system becomes hard^2 (not a formal measure) opaque.
This isn't realistic in practice. Homomorphic implementations of even simple functions are extremely inefficient for the time being. But it is possible, and improvements in efficiency have not been exhausted.
Equivalent but different implementations of homomorphic code can obviously be made. However, given the only credible explanations for design decisions of the new code are, to exactly match the original code, this precludes any "clean room" defenses.
--
Implementing software with neural network models wouldn't stop replication, but would decompile as source that was clearly not developed independent from the original implementation.
Even distilling (training a new model on the "decompiled" model) would be dead giveaway that it was derived directly from the source, not a clean room implementation.
--
I have wondered, if quantum computing wouldn't enable an efficient version of homomorphic computing over classical data.
Just some wild thoughts.

by benmccann

0 subcomment

I used Gemini to compare the minimized output of the Rollup vs Rolldown JavaScript bundlers to find locations where the latter was not yet at the same degree of optimization. It was astoundingly good and I'm not sure how I would have been able to accomplish the task without an LLM as an available tool.

by hombre_fatal

0 subcomment

Wow, I haven't thought of this game since I played it as a kid. My friend would bring it over all the time for sleep overs. I'm going to try to emulate it right now for old time's sake. I loved this game.

by viraptor

2 subcomments

Yeah, it works great for porting as well. I tried it on the assembler sources of Prince of Persia for Apple ii and went from nothing to basics being playable (with a few bugs but still) on modern Mac with SDL graphics within a day.

by heavyset_go

1 subcomments

Am I just wrong in thinking doing decompilation of copyrighted code via the cloud is a bad idea?
Like, if it ever leaks, or you were planning on releasing it, literally every step you took in your crime is uploaded to the cloud ready to send you to prison.
It's what's stopped me from using hosted LLMs for DMCA-legal RE. All it takes is for a prosecutor/attorney to spin a narrative based on uploaded evidence and your ass is in court.

by throwaway81523

0 subcomment

Rather than insisting on byte perfect matches, sometimes you can prove code equivalence of machine code sequences using SAT solvers. That might be an interesting extension, maybe giving clearer code output and/or solution to difficult functions in some cases.

by butz

0 subcomment

Are there any similar specialized decompilation LLM models available to be used locally?

0 subcomment

by wiz21c

0 subcomment

Last day I asked Claude to estimate a loop of a dozen 6502 instructions. It failed but his estimate was not bad at all. Amazing!

by grim_io

0 subcomment

I need to try using a frontier LLM for deobfuscation. That's a huge pain in the ass for a noob like me.

by DrNosferatu

0 subcomment

More than an overview, a step by step tutorial on this would be awesome!

by VikingCoder

0 subcomment

I've been waiting for decompilation to show up in this space.

by xnx

0 subcomment

Great use case. Curious to see how Gemini fares when tested.

by jamesbelchamber

0 subcomment

This is a refreshingly practical demonstration of an LLM adding value. More of this please.

by knallfrosch

1 subcomments

We're wasting Energy reverse-engineering code, which, by definition, already exists now. Oh god.
Have you tried asking them to simply open source the code?