Hacker News

Introducing the UHK 80

Hacker News - Thu, 11/07/2024 - 11:18am
Categories: Hacker News

Interview with the lead author of Let Me Speak Freely?

Hacker News - Thu, 11/07/2024 - 11:11am

JSON Mode has been one of the biggest enablers for working with Large Language Models! JSON mode is even expanding into Multimodal Foundation models! But how exactly is JSON mode achieved?

There are generally 3 paths to JSON mode:

1. Constrained Generation (such as Outlines)

2. Begging the model for a JSON response in the prompt

3. A two stage process of generate-then-format (or generate-then-retry)

Although most of the field has converged on the first method, Let Me Speak Freely? is a new paper challenging the potential tradeoffs in achieving JSON mode with constrained generation.

I am BEYOND EXCITED to publish the 108th Weaviate Podcast with Zhi Rui Tam, the lead author of Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models!

As the title of the paper suggests, although constrained generation is awesome because of its reliability, we may be sacrificing the performance of the LLM by producing our JSON with this method.

The podcast dives into how these experiments identify this and all sorts of details about the potential and implementation details of Structured Outputs. I particularly love the conversation topic of incredibly Complex Structured Outputs, such as generating 10 values in a single inference or say HTML templates.

https://www.youtube.com/watch?v=UsVIX9NJ_a4

Comments URL: https://news.ycombinator.com/item?id=42077920

Points: 1

# Comments: 0

Categories: Hacker News

Game Development Caution [video]

Hacker News - Thu, 11/07/2024 - 11:11am
Categories: Hacker News

Show HN: I built an agent to make open source contributions easier

Hacker News - Thu, 11/07/2024 - 11:05am

Over the last few months I’ve built a tool that breaks down your codebase into a knowledge graph, generates and stores inferences about the code with each node and provides tooling for agents to fetch graph structure, perform similarity search, fetch code etc in order to do engineering tasks.

My favourite thing about this is how easy it makes getting started with a new codebase that I have no context about. As a young engineer when I was trying to get into open source, it always seemed out of reach because the large codebases always scared me and documentation was not fun to read.

Use cases I’ve tried so far:

- Generating “Getting started” documentation for a project: I could not find any steps on how to build and test the AgentOps SDK locally. I’m more of a microservices guy, had never heard of tox or built SDKs. The tool was able to read the configuration files and tell me exactly how to build and run the tests [1]

- Generating a plan for changes to be made for a Github issue description - Tried this with the Portkey AI gateway and Signoz codebase where I picked up an open issue from the repo and tried to get the agent to tell me what needs to be done. [2][3]

- Understanding behaviour of a repo that is not documented - I was debugging a latency issue and noticed an evaluation task prompt in our LLM observability, our agents are built with CrewAI, but I could not find any reference to this on their documentation. Using the tool I was able to pinpoint where it was coming from and its purpose. [4]

I’m experimenting with new use cases everyday and trying to see what else is possible. How would you use these agents?

[1] Video: https://youtu.be/KonguA9PD80

Chat: https://app.potpie.ai/chat/0192dd3f-078e-753f-b5f8-00b724d9085e [2] Video: https://youtu.be/XSZMtIawy5s

Chat: https://app.potpie.ai/chat/0192c3ea-3dfa-7090-b5fb-3f3d50bbc301 [3] Video: https://youtu.be/N9wUI8h0f5w

Chat: https://app.potpie.ai/chat/0193013e-a1bb-723c-805c-7031b25a21c5 [4] Video: https://youtu.be/FeTQhSBguf4

Chat: https://app.potpie.ai/chat/0193074b-5fec-7308-9158-67dbbf4cd516

Comments URL: https://news.ycombinator.com/item?id=42077849

Points: 3

# Comments: 0

Categories: Hacker News

Parental leave at early stage startups

Hacker News - Thu, 11/07/2024 - 10:21am
Categories: Hacker News

Pages