How does it differentiate from using json output with OpenAI?
I think it can be useful for some cases, but you’ll always have the limitation of the model size to compete against large closed source models.
can't speak to the actual service, but can confirm they have excellent bananas (not kidding)
Nice, makes sense to chain models so you don't waste the attention of the big smart model on the grunt work of JSON structure.
I bet there are some good post-processing heuristics you could also apply for hallucination with a flag for "this should be in the text verbatim" & then string matching whether the answer it outputted was a string from the text or not.