At first I was like: "okay, it's just a decorator to add a prompt when you have str as an input and str as an output.
Then I kept on reading, and I have to admit that the object creation with LLMs is really amazing!
The API looks very clean. Today I learned about "..." in Python
I built a similar package for Typescript[0], with the goal of having type-safe LLM responses automagically.
It's pretty fun, but I've found that having the LLM write code is often-times what I actually want most of the time.
Awesome job with the simplicity, gonna play with it. Have you tried using yaml as the format with the models instead of JSOn? Feel like you'll use far fewer tokens to describe the same thing. Perhaps it's a bit more forgiving as well.
EDIT: Just tried using the decorator to output a fairly complex pydantic model and it failed with "magentic.chat_model.openai_chat_model.StructuredOutputError: Failed to parse model output. You may need to update your prompt to encourage the model to return a specific type."
I typically try to give examples in the pydantic Config class, perhaps those could be piped in for some few-shot methods, and also have some iteration if the model output is not perfectly parseable to correct the output syntax..
Very cool! At first the title reminded me of a project me and my colleague are working on called OpenAI-Functools [1], but your concept is quite the opposite, combining LLMs in your code rather seamlessly instead of the other way around. Quite cool, and interesting examples :)
I’ll definitely try to apply it in one of my pet projects. Good stuff
I've personally found frameworks like this to get in the way of quality COT: It's rare for a prompt that takes great advantage of the LLM's reasoning to fit in the format these generators encourage
A friend mentioned how terrible most cold email generators are at actually generating natural feeling emails. It just took asking him questions about how actual people in marketing come up with emails to come up with a chain of thought that produces intentionally uncanny emails for a wide range of inputs: https://rentry.co/54hbz
It's not like you can't technically fit what I described into bunch of comments (or an obnoxiously long multiline comment), but it'd be bulky and not conducive to general happiness of anyone involved.
I much prefer repurposing Nunjucks templates to keep all of that a separate document that's easy to manage with version control
This is great. I hacked a smaller version of this together when I built an LLM app with Elixir. Honestly, the async by default of Ex is so much better suited to this stuff, especially as it’s just api calls.
Tempted to have a go at porting these ideas. Should be v doable with the macro system.
Really like how this is implemented with decorators. Everything just feels really smooth
Looks great! I don't normally like these LLM libraries but this one sparks joy. I'll try it out on my next experiment.
Could you highlight how you're parsing to structured objects and how it can fail? Ever since I discovered guidance's method of pattern guides I've been wanting this more and more (only works for local hugging face models though). Wish OpenAI offered a similar API.
I think the comments on great api design got me thinking of a world in which you can have multiple of these frameworks orchestrate together. I could see use in adding this to a platform I’m currently building to overcome some issues llamaindex eg introduces.
Curious as to why you chose to do it as a decorator instead of just a function call?
See also: `antiscope`, an experiment in subjunctive programming
Looks super cool! A few questions:
1) Can you get the actual code output or will this end up calling OpenAI each function call? 2) What latency does it add? What about token usage? 3) Is the functionality deterministic?
Nice! I’m going to try it out and possibly integrate it into my Python package: https://vanna.ai
Does this do System vs. Assistant vs. User prompting?
I am amazed that `...` is a valid syntax in Python, not a pseudo grammar.
This library is impressive, I appreciate it and I will apply it to my project.
This is neat. It makes it easy to prototype, and then you can just remove the decorator and write a specific implementation if you need to.
Is it really LLMs (plural) when you only have OpenAPI integration?
Super cool! Looks quite intuitive, especially for function calls.
Just wanna say, that's pretty great API design :)
Do you support custom LLMs?
Seems a lot like https://github.com/PrefectHQ/marvin?
The prompting you do seems awfully similar to:
Pretty cool, I made something similar (lambdaprompt[1]), with the same ideal of functions being the best interface for LLMs.
Also, here's some discussion about this style of prompting and ways of working with LLMs from a while ago [2].
[1] https://github.com/approximatelabs/lambdaprompt/ [2] https://news.ycombinator.com/context?id=34422917
Are you familiar with https://github.com/PrefectHQ/marvin? This looks very similar
Write some tests for those functions. It will be worth it. No, I am not kidding, especially for AI we need tests, but we should report accuracy instead of a hard fail/pass.
We need a new language/DSL. Python is a lost cause for strings as first-class.
This is also similar in spirit to LMQL
This looks really useful. Langchain is not my idea of a fun time.
Love the examples too. Low-effort humor is the best:
> create_superhero("Garden Man")
> # Superhero(name='Garden Man', age=30, power='Control over plants', enemies=['Pollution Man', 'Concrete Woman'])