VLLM with JSON Guided Generation
Show HN: VLLM with JSON Guided Generation
Operating in show hn launch, VLLM with JSON Guided Generation operates.
Profile
Our project, Outlines, now offers guided/constrained generation (e.g. according to a JSON schema) via the VLLM library.My colleague, Rémi, created some patches that allow one to pass vLLM a JSON schema along with the prompt, which dramatically simplifies deployment of JSON-guided generation. He also added a new `serve` interface that puts it all together and makes serving such models a 2-3 line process.Check it out and tell us what you think!
The company sits in the show hn launch segment, where it competes on its core product or service.
The company is listed at the Show HN launch stage of development.
| Sector | Show HN launch |
|---|---|
| Stage | Show HN launch |
Website: outlines-dev.github.io/outlines/reference/vllm
Browse related
Create a free account