Yeah, that's definitely one of the things it offers right now. If you want a particular data shape out of GPT, we handle that, both on the side of crafting the prompt to elicit the type and on the parsing side to get the data out of the raw GPT response.

We're also building more tools to make the development process easier, which depend on the fact that imaginary functions are easy to do static analysis on. The first tool is an IDE plugin that lets you directly run and test imaginary functions in VS Code and to compare different versions of an imaginary function to see how they do on various test inputs. We also plan to add simple annotations to the comment format to let you easily switch to other LLMs for your runtime to manage the cost/quality/privacy tradeoff.

ETA: One thing it also does right now is lets you switch between models (ada, babbage, curie, davinci, gpt-3.5-turbo, gpt-4) with just a configuration switch. If you use OpenAI's APIs you need to change your client code, because the GPT-3 models have a different API than GPT-3.5 and GPT-4.

Thanks! It's definitely a little mind-bending, but we are enjoying exploring how far the idea can go.

Thanks for the response!

I may not be completely understanding the question, but from my perspective, the OpenAI APIs are just as non-deterministic as imaginary functions. If you call OpenAI directly multiple times with the exact same prompt and a temperature above 0, you will get different responses each time. The same is true of imaginary functions. (As an interesting side note, we default temperature in imaginary functions to 0, so unless you modify it in the comment, imaginary functions do by default return the same responses for the same set of arguments.)

Now, I do think that introducing this kind of non-determinism into your web code, whether through OpenAI's APIs or imaginary programming, presents some interesting wrinkles. For a traditional web developer like me, the fuzziness and non-determinism is frankly a bit scary. The thing we're working on now is tools that you can use to consistently test your imaginary functions and make sure that they are returning acceptable answers. Our hope is that this will give frontend devs the ability to use AI in their apps with reasonable confidence that the AI is doing what they want it to.

We have tried out other runtimes, and a lot of them seem to work decently well. If memory serves, Claude was quite good. I'm definitely interested in supporting other hosting and other models as a way to balance quality, cost, and privacy, and we are currently building IDE tools that will let you test your imaginary functions in ways that will hopefully surface those tradeoffs.

Yep, I should have been clearer that (at least for now) it only supports JSON-compatible types and doesn’t support type aliases or classes (although we have a prototype version that does). The limitations are documented at https://imaginary.dev/docs/writing-an-imaginary-function .

So usually, yes, that’s true about TypeScript. Types are removed by the compiler.

But we wrote a typescript & Babel compiler plug-in, which allows us to replace the imaginary function with whatever code we want. So we replace the imaginary function with code that includes a run-time type check for the appropriate return type from the TypeScript definition. Does that make sense?

Great question! We spent a lot of time experimenting with how to cue GPT into returning the right JSON type, and it’s pretty darned compliant. GPT-3 doesn’t always do a good job adhering to strict JSON syntax, but we wrote an extremely lenient parser that understands the weird things that GPT-3 gets wrong. (Sometimes it uses single quotes instead of double, sometimes it puts new lines in strings, sometimes it decides a semicolon is a better choice than a comma!). GPT-4 and GPT-3.5 are significantly better at JSON syntax.

On the question of returning the actual type you asked for, we do a run time type check to make sure it’s right. So if you get a value back, you can be sure it’s the type you wanted.

Ha, as a programmer, I too am doomed to laziness. :)

All great points! I tend to think that it's best suited to problems that you can't solve with traditional programming. If the problem you have is "reverse this array of numbers", then writing the code or having Copilot write the code is a better answer.

But if the problem you want to solve is "come up with good titles for this blog post" or "summarize this user email" or "categorize these customer service complaints by anger level", there really isn't a JavaScript/TypeScript function you can write to do that. In this case, I think the latency is often worth the functionality.

As to the non-determinism, I think that's a real issue. Right now, the state of the art of testing GPT prompts feels very shaky; one person I talked to said that they "change the prompt and then bang the keyboard to try four or five inputs". This clearly isn't ok for serious projects. To help with this, we're currently building some tools to help developers generate test inputs and evaluate how their imaginary functions perform.

ETA: and thanks for the comment about the name! (I can't take credit for it, though; I believe it was first coined by Shane Milligan.)

You can check out the code at https://github.com/imaginary-dev/imaginary-dev , and there's code for demo projects as well:

Blog Writing demo from the screencast: Live site: blog-demo.imaginary.dev Code: https://github.com/imaginary-dev/imaginary-dev/tree/main/example-clients/nextjs-blog

Emojifier Live site: emojifier.imaginary.dev Code: https://github.com/imaginary-dev/imaginary-dev/tree/main/example-clients/nextjs-api

I'm also happy to answer any questions you might have!