LLMs in the Context of Applications

When using Large Language Models (LLM’s) within software applications, structure and consistency matter. 

You can use Python to achieve this, and this post will show you how.

With tools like Pydantic and Magentic, you can generate complex structured outputs from LLMs using Python and OpenAI’s GPT-4o (latest as of 7/1/24). 

Precisely structured outputs from LLMs matter to ensure seamless integration with other components such as APIs and data pipelines.

While you may interact with tools like ChatGPT in a conversational manner, you might not be familiar with programmatic interaction or techniques for generating precise responses.

Whether you are a technical program manager or a software developer, this guide will equip you with practical insights on creating precise, strongly-typed responses from LLMs.

Unstructured LLM Output Example

Let’s use the example of asking an LLM: “What are the country-level population indicators for the United States?” 

This can lead to ambiguous results due to unclear expectations about which indicators are of interest and how the response should be formatted.

By default, LLMs tend to respond in a conversational style. This is suitable for human interactions, but it poses challenges when composing larger systems with AI-enabled components. 

Here is a sample output of that query in ChatGPT:

While informative, this output is not structured in a way that is useful within the context of an application.

Python as a Tool to Manage LLM Outputs

Python empowers you to define custom objects that communicate your desired response structures to LLMs effectively.

Pydantic and Magentic are two powerful libraries that you can use to add LLMs to data pipelines.

A Pattern for Generating Structured Outputs from LLMs

To effectively communicate the desired structure for LLM responses, you can leverage a structured pattern as follows:

  1. Model the Desired Response: Define the response structure as a Python object.
  2. Incorporate Pydantic: Use Pydantic for type information to express types and validate response.
  3. Develop a Function: Create a Python function that returns the defined response object.
  4. Decorate with magentic.prompt: Apply the magentic.prompt decorator to this function, embedding your LLM prompt within it.
  5. Invoke the Function: Call your LLM-enabled function like any standard Python function.

By following this pattern, you ensure that responses from LLMs are consistently structured and validated, facilitating more reliable and predictable outputs.

Let’s see this in action with some example code showing how to get a structured response object for your LLM prompt: “What are the country-level population indicators for the United States?” 

1) Model Your Desired Response

You can start by creating a simple Python class to model your desired response from the LLM. The CountryPopulationIndicators class contains information that you want to capture, such as country name, population, median age, birth rate, death rate, and life expectancy.

class CountryPopulationIndicators:
    def __init__(self, country_name, population, median_age, birth_rate, death_rate, life_expectancy):
        self.country_name = country_name
        self.population = population
        self.median_age = median_age
        self.birth_rate = birth_rate
        self.death_rate = death_rate
        self.life_expectancy = life_expectancy

However, there is a problem: what data types should be used? 

For instance, should median_age be an integer or a decimal value? This ambiguity can lead to inconsistencies and errors in your application. To address this issue more robustly and ensure type safety and validation, you can leverage Pydantic.

2) Sprinkle in Type Information using Pydantic

To enhance the robustness and clarity of your desired response, you can leverage Pydantic’s type validation and metadata capabilities. 

You can start by converting your simple Python class to a stronger-typed class which subclasses the Pydantic library’s BaseModel

Initially, you can define a basic CountryPopulationIndicators class with type annotations for each attribute.

from pydantic import BaseModel

class CountryPopulationIndicators(BaseModel):
    country_name: str
    population: int
    median_age: float
    birth_rate: float
    death_rate: float
    life_expectancy: float

While this approach provides basic type checking, you can further improve it by adding descriptive metadata using Pydantic’s Field function. 

This not only validates the data types, but also adds meaningful descriptions to each attribute, making the model more informative and self-documenting.

from pydantic import BaseModel, Field

class CountryPopulationIndicators(BaseModel):
    country_name: str = Field(description="The name of the country")
    population: int = Field(description="The total population of the country")
    median_age: float = Field(description="The median age of the population in years")
    birth_rate: float = Field(description="The birth rate per 1000 individuals")
    death_rate: float = Field(description="The death rate per 1000 individuals")
    life_expectancy: float = Field(description="The average life expectancy in years")

By incorporating Field descriptions, you add an extra layer of clarity, making it easier for anyone reading or using these models to understand their purpose and constraints. 

This small change significantly enhances the maintainability and usability of your code.

3) Develop a Function

Now that you have a strongly-typed desired response object, let’s create a Python function which will return that desired response object.

def generate_country_population_indicators() -> CountryPopulationIndicators: ...

Notice how simple this function is. To start off, you can just give it an intuitive name, it takes no arguments, and it returns your CountryPopulationIndicators object.

4) Decorate with magentic.prompt

Now that you have defined a strongly-typed desired response object with a corresponding function, it’s time to add in your custom LLM prompt. 

This is the fun part! 

You can add an LLM prompt by using the prompt decorator from the magentic Python library. This decorator helps you create an LLM-enabled function by templating standard Python function parameters into your LLM prompt and casting the response into the proper type.

You can slap the prompt decorator on your existing function to LLM-enable it. The decorator takes a string template for the LLM prompt, which can include placeholders for function parameters.

@prompt("""What are the country-level population indicators for the following country: {country_name}?""")
def get_country_population_indicators(country_name: str) -> CountryPopulationIndicators:
    ...

In this example, {country_name} within the triple-quoted string will be replaced by the actual argument passed to get_country_population_indicators function.  By adding the country_name parameter, your function can now potentially generate population indicators for any country!

Now that your function is properly defined and decorated, calling it is straightforward. 

Here’s an example of how you might call this function and handle its output:

# Example usage of get_country_population_indicators
country_name = "Japan"
population_indicators = get_country_population_indicators(country_name)
print(population_indicators)
>>> country_name='Japan' population=125360000 population_density=347.8 population_growth=-0.3 birth_rate=7.3 death_rate=11.1 migration_rate=0.6

In this example, calling get_country_population_indicators("Japan") will generate a prompt asking for Japan’s population indicators and return an instance of CountryPopulationIndicators, which you then use to print specific information.

By leveraging the power of LLMs through structured prompts and type-safe responses, you can easily integrate advanced capabilities into your Python applications while maintaining clarity and correctness in your codebase.

5) Putting it all together…Invoke the Function

To wrap up your code example, let’s put the whole thing together. 

The goal is to demonstrate how an LLM-enabled function can be utilized just like any other Python function. 

You will call this function in a for loop with a list of countries and display the generated output. The following code showcases this integration using the magentic library, pydantic for data validation, and a custom model CountryPopulationIndicators.

from magentic import prompt
from pydantic import BaseModel, Field
class CountryPopulationIndicators(BaseModel):
    country_name: str = Field(description="The name of the country")
    population: int = Field(description="The total population of the country")
    median_age: float = Field(description="The median age of the population in years")
    birth_rate: float = Field(description="The birth rate per 1000 individuals")
    death_rate: float = Field(description="The death rate per 1000 individuals")
    life_expectancy: float = Field(description="The average life expectancy in years")
@prompt("""What are the country-level population indicators for the following country: {country_name}?""")
def get_country_population_indicators(country_name: str) -> CountryPopulationIndicators:
    ...
COUNTRIES = ("Japan", "Brazil", "USA", "Westeros", "Mordor")
  
for country in COUNTRIES:
    print(get_country_population_indicators(country))

When you run this code, you get an output similar to:

country_name='Japan' population=125960000 median_age=48.4 birth_rate=6.9 death_rate=11.0 life_expectancy=84.6 
country_name='Brazil' population=214000000 median_age=33.5 birth_rate=13.3 death_rate=6.7 life_expectancy=76.2 
country_name='USA' population=331893745 median_age=38.3 birth_rate=11.0 death_rate=8.7 life_expectancy=78.9 
country_name='Westeros' population=19000000 median_age=30.5 birth_rate=14.2 death_rate=8.3 life_expectancy=74.5 
country_name='Mordor' population=1000000 median_age=35.0 birth_rate=11.0 death_rate=9.0 life_expectancy=67.0

In this final composite code sample, you integrated an LLM-enabled function within a typical Python script using magentic and pydantic. You validated that your function can handle multiple input cases by iterating through a list of countries and displaying their respective demographic indicators as structured objects.

Be mindful that while LLMs can generate structured data for real-world entities accurately, they may produce plausible but fictitious outputs when given non-real entities as input (e.g., Westeros and Mordor). Therefore, always validate your data sources and cross-check with reliable information where necessary.

Conclusion

In this blog post, we’ve explored the process of generating structured outputs from Large Language Models (LLMs) using Python, with a focus on ensuring type safety and validation through libraries like Pydantic and Magentic. 

By modeling your desired response, leveraging Pydantic for robust type annotations, developing clear functions, and decorating them with magentic.prompt, you can create reliable and predictable outputs from LLMs. This structured approach not only enhances the integration of AI capabilities into software applications, but also ensures that these integrations are maintainable and understandable.

At Makepath, we specialize in helping clients AI-enable their business workflows. 

Whether you’re looking to enhance your data pipelines or build sophisticated AI-driven applications, Makepath is here as a partner and guide through the AI landscape.

If you have any questions or suggested improvements to this workflow, please drop us a line at contact@makepath.com.

Links