Skip to main content
Version: v1.0

Configuring Applications

Applications are the functions you want to evaluate. They receive inputs from your test data and return outputs that evaluators will check.

Basic Application Structure​

An application is any Python function decorated with @ag.application. The decorator tells Agenta this function should be evaluated.

import agenta as ag

ag.init()

@ag.application(
    slug="my_app",
    name="My Application",
    description="Describes what this application does"
)
async def my_app(input_text: str):
    # Your application logic here
    result = process(input_text)
    return result

The application decorator takes these parameters:

  • slug (required): A unique identifier for your application
  • name (optional): A human-readable name shown in the UI
  • description (optional): Explains what the application does

Understanding Application Inputs​

Applications receive inputs from your test cases. The function parameters must match field names in your test data.

Example:

# Your test case
test_case = {
    "country": "France",
    "language": "French",
    "capital": "Paris"
}

# Your application receives these as parameters
@ag.application(slug="country_info")
async def country_info(country: str, language: str):
    # country = "France"
    # language = "French"
    # Note: capital is not used by this application
    return f"The capital of {country} is well known!"

You only need to declare parameters for the fields your application uses. Extra fields in the test case are ignored.

Application Return Values​

Applications should return the output you want evaluators to check. The return value can be:

  • String: Text responses
  • Dictionary: Structured data
  • List: Multiple items
  • Number: Numeric results
  • Any JSON-serializable value

String Returns​

Most common for text-based applications:

@ag.application(slug="question_answerer")
async def question_answerer(question: str) -> str:
    answer = generate_answer(question)
    return answer  # Simple string

Dictionary Returns​

Useful for structured outputs:

@ag.application(slug="entity_extractor")
async def entity_extractor(text: str) -> dict:
    return {
        "entities": ["Paris", "France"],
        "count": 2,
        "confidence": 0.95
    }

List Returns​

For multiple items:

@ag.application(slug="keyword_extractor")
async def keyword_extractor(text: str) -> list:
    keywords = extract_keywords(text)
    return keywords  # ["keyword1", "keyword2", ...]

Application Examples​

Simple Lookup Application​

@ag.application(
    slug="capital_lookup",
    name="Capital City Lookup",
    description="Returns the capital city for a given country"
)
async def capital_lookup(country: str) -> str:
    """Look up a country's capital city."""
    capitals = {
        "France": "Paris",
        "Germany": "Berlin",
        "Spain": "Madrid",
        "Italy": "Rome",
    }
    return capitals.get(country, "Unknown")

LLM-Based Application​

import openai

@ag.application(
    slug="question_answerer",
    name="Question Answering System",
    description="Answers questions using GPT-4"
)
async def question_answerer(question: str, context: str) -> str:
    """Answer questions based on provided context."""
    client = openai.AsyncOpenAI()

    response = await client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "Answer based on the context provided."},
            {"role": "user", "content": f"Context: {context}\n\nQuestion: {question}"}
        ]
    )

    return response.choices[0].message.content

Synchronous vs Asynchronous​

Applications can be either synchronous or asynchronous:

Synchronous​

Working with Application Parameters​

On this page