You think we are far ahead of where we are... and I also wish we were there, but we are not.

Not saying one day will not be possible but have you tried asking gpt4 to multiply two 3 digits numbers?

Yeah, GPT sucks on tasks which require actual thinking and personally I am kind of skeptical about it's actual usefulness tbh. But my impression is that despite being primarily built to work with natural language, it actually does work better with computer code. Probably because computer code has much simpler structure. This got me thinking that building something more specialized that would be required to only work with computer code would actually be an easier task - more similar to automated translation perhaps, which is already working pretty well using ML.

There's already research on this. For example, see "DIRECT: A Transformer Model for Decompiled Variable Name Recovery" by Nitin et al.

Decompilers already exist though.

GPT4 will do this to an extent out of the box, feed it some assembly and it will hypothesise a corresponding program in the language of your choice. For me it still has that disassembler character of over-specificity, but I didn't try very hard to get idiomatic result.

It can give detailed analysis of assembly too, including assessing what it does at a high level in plain english. Useful!

Edit: Of course it's going to fail hopelessly for large/complex programs.

Yeah, but they're hand-crafted algorithms and produce code that's hard to read.

that said, it might be also a good approach to preprocess the input with a classical algorithm and then train a model on refactoring that decompiled code, so it becomes more readable

Friends working in rev.ng told me that it's very difficult to decompile to the original high level structures actually used in the source code. Maybe C have a few ways to code a loop but c++ has many and figuring out the source code from assembly is very hard to achieve with rule based systems.

I think that if you can have better variable names that is already a big selling point

That depends on the language/compiler used. Java and C# have decompilers that turn out great code.

Exactly, a lot of the tech is already there, it's perhaps more about unobfuscating the code rather than decompiling it.

I've had some luck doing this with chatGPT too. Mainly feeding it bits of 6502 code and then saying, 'Please explain the branch logic in a higher level language". It's also reasonably able to give plain english explanations if you can let it know the context and what various addresses may represent.

Why do you need a generative model for this? Couldn’t this be done with standard supervised learning?

Good question.

The generative model might be able to learn with fewer examples? Because if already knows more about coding in various languages?

Just a guess.

Clever. Should be very possible.

And there is already OpenAI-based plugin for them

decompiler childs play, train a model that reconstructs servers and databases based on api endpoints

You could certainly do that to some extent, but I suspect that wouldn't generalize very well to programs that do things significantly different from anything in the training set. Transforming the syntax alone would probably be straightforward enough, but the parts that need more "interpretation" of what's going on (such as assigning plausible variable/function/class names, nevermind something like writing comments) I just can't see a standard supervised model handling particularly gracefully. Whereas that's one of the areas LLM excel at.

I can't speak for GPT-4, but in my experience with ChatGPT, I would definitely not say it is better with code. It's just absurdly, terribly, unbelievably bad at maths. It's a bit better at dealing with code, but it doesn't mean it's good, you're just comparing it with its weakest area. It's not really capable of generating code that does anything even a little complex without heavy guidance directing it towards mistakes and getting it to make revision after revision (and even that is non-trivial to get it to do, it tends to just start generating completely different programs with completely different problems instead)

That being said, I can definitely believe it could do okay at decompilation. It's an easy enough task in general, comparatively, and the "trickiest" bit (interpreting what the program is supposed to be doing, to have the context to name variables etc) feels like the kind of thing it'd perform surprisingly well at. Getting a general "vibe" and sticking with it, and translating A to B, it tends to do okay. It's when it needs to generate entirely novel outputs that need to fulfill multiple requirements at once that it starts failing miserably.

Yes, there’s already work on this:

https://arxiv.org/abs/2010.00770

Sorry if this is a very basic question. But what are the applications of such a model? What can we do with the decompiled code?

Ultimately, you could force open-sourcing of any software (if you can run it on your device, not SaaS)

Better question can we train a cross-language IDE that allows you to translate between different OOPLs

Possibly! But it also seems like a good sequence-to-sequence translation problem, just line up the two streams of tokens and let the model figure it out.

Those languages don't compile to machine code, they compile to a special bytecode that runs in a VM.

Yes, that is obviously a hugely valuable application of machine intelligence. Want to work on it?

Most of those problems are due to the input/memory limitations for general use. I can imagine locally hosted GPTs that have training access to an organization's source code, development standards, and database data structures. Such a system could be incredibly useful. Human developers would just provide the prompts, supervise, approve, and test new/updated code.

Would have to be locally hosted, because most orgs are NOT going to feed their source code to an outside agency regardless of the promises of efficiency.

In most cases yes, but inherently no. Understand that compilers, as part of their optimization step, might compile high level code into something that you can't really connect with the actual code. Part of the information is lost in the optimization step and so in a general case you will not be able to revert the compilation step. At least not fully, of course you will be able to get something resembling the solution, but it is not guaranteed to be the exact code that compiled into your starting input.

This is, of course, after taking into consideration you will not be able to recover dead source code if it's never compiled into something. Because if you take this into account, even if a language does not optimize the source code otherwise, if it only discards dead code: you are also losing information.

And also, this is disregarding name mangling. Obviously name mangling can be done in a way you have information loss, but this is probably irrelevant since concrete entity names are not that relevant.

That would be breaching copyright. Depending on the company and the product, you'd get anywhere from a pretty nasty debt to straight up ruining your whole life (and potentially the lives of your family and people associated with you).

The same way you wouldn't steal from the mob, you would not steal from a company that makes money on a product FOSS can't compete with. Aside from that, decompilers exist for a very long time yet we have not witnessed such vigilantism.

Also: "Augmenting Decompiler Output with Learned Variable Names and Types" by Chen and Lacomis et al. https://jeremylacomis.com/materials/ChenDIRTY2022.pdf

These generative models also seem better at learning though right?

It could better understand what the user wants too

It’s very useful for malware analysis. In malware it’s all about hiding your tracks. Clearing up the intent of even just some code helps white hats a lot. Example: Perhaps it inserts some magic bytes into a file to exploit an auto run vulnerability. ChatGPT might recognize that context from its training data much more quickly.