July 2025 Product Updates

Here's what's new in July 2025 (and the last couple of months):

Tool support in the playground:

We released tool usage in the Agenta playground - a key feature for anyone building agents with LLMs.

Agents need tools to access external data, perform calculations, or call APIs. Now you can:

• Define tools directly in the playground using JSON schema

• Test how your prompt generates tool calls in the playground and simulate the response

• Run evaluation and test whether calls are done correctly with custom evaluators

The tool schema is saved with your prompt configuration, making integration easy when you fetch configs through the API.

Image support in the playground:

Agenta now supports images in the playground, test sets, and evaluations. This enables a systematic workflow for developing and testing applications that use vision models. Now you can:

• Add images directly to your prompts when experimenting in the playground.

• Create and manage test sets that include image inputs alongside text.

• Run evaluations on prompts designed to process images, allowing for systematic comparison of different prompt versions or models.

User feedback API:

One of the recurring feature requests we had was the ability to capture user feedback and annotations (e.g. scores) to LLM responses traced in Agenta.

As of now you can use the annotation API to add annotations to LLM responses traced in Agenta. Now you can:

• Collect user feedback on LLM responses

• Run custom evaluation workflow

• Measure application performance in real-time

Check out the how to annotate traces from API for more details. Or try our new tutorial (available as jupyter notebook).

New integrations to Observability

We've added integrations to OpenAI Agents SDK, PydanticAI, LangGraph and LLamaIndex with more to come.

Rewrite to the self-hosting docs

Self-hosting Agenta used to be a hassle. We've restructured the logic, reworked the environment variables and rewritten the docs.

Now you can self-host Agenta very easily in all scenarios, whether you're using a different port, using HTTPS, or running behind nginx - we've got you covered.

As the LLMs say, that's a wrap for today.

We're planning a major launch week culminating with a Product Hunt launch in the beginning of October. Lots of exciting updates soon!