← 课程目录 · 东灵培智
GitHub 原文 ↗ 中文
III · 构建 AI 应用

Building Text Generation Applications

微软《Generative AI for Beginners》 · 第 06 课 · 英文原版 · 本地镜像

Building Text Generation Applications

(Click the image above to view video of this lesson)

You've seen so far through this curriculum that there are core concepts like prompts and even a whole discipline called "prompt engineering". Many tools you can interact with like ChatGPT, Office 365, Microsoft Power Platform and more, support you using prompts to accomplish something.

For you to add such an experience to an app, you need to understand concepts like prompts, completions and choose a library to work with. That's exactly what you'll learn in this chapter.

Introduction

In this chapter, you will:

Learning goals

At the end of this lesson, you'll be able to:

What is a text generation app?

Normally when you build an app it has some kind of interface like the following:

Console and UI apps are limited

Compare it to a command-based app where you type a command:

Benefits of text generation apps

So how is a text generation app different?

In a text generation app, you have more flexibility, you're not limited to a set of commands or a specific input language. Instead, you can use natural language to interact with the app. Another benefit is that you're already interacting with a data source that has been trained on a vast corpus of information, whereas a traditional app might be limited on what's in a database.

What can I build with a text generation app?

There are many things you can build. For example:

How can I get started?

Well, you need to find a way to integrate with an LLM which usually entails the following two approaches:

Libraries/SDKs

There are a few well known libraries for working with LLMs like:

Then there are libraries that operate on a higher level like:

First app using openai

Let's see how we can build our first app, what libraries we need, how much is required and so on.

Install openai

There are many libraries out there for interacting with OpenAI or Azure OpenAI. It's possible to use numerous programming languages as well like C#, Python, JavaScript, Java and more. We've chosen to use the openai Python library, so we'll use pip to install it.

pip install openai

Create a resource

You need to carry out the following steps:

[!NOTE] At the time of writing, you need to apply for access to Azure OpenAI.

Locate API key and endpoint

At this point, you need to tell your openai library what API key to use. To find your API key, go to "Keys and Endpoint" section of your Azure OpenAI resource and copy the "Key 1" value.

Keys and Endpoint resource blade in Azure Portal

Now that you have this information copied, let's instruct the libraries to use it.

[!NOTE] It's worth separating your API key from your code. You can do so by using environment variables.

  • Set the environment variable OPENAI_API_KEY to your API key. export OPENAI_API_KEY='sk-...'

Setup configuration Azure

If you're using Azure OpenAI (now part of Microsoft Foundry), here's how you setup configuration. We use the standard OpenAI client pointed at the Azure OpenAI /openai/v1/ endpoint, which works with the Responses API and needs no api_version:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    base_url=f"{os.environ['AZURE_OPENAI_ENDPOINT'].rstrip('/')}/openai/v1/",
)

Above we're setting the following:

[!NOTE] > os.environ reads environment variables. You can use it to read environment variables like AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT. Set these environment variables in your terminal or by using a library like dotenv.

Generate text

The way to generate text is to use the Responses API via the responses.create method. Here's an example:

prompt = "Complete the following: Once upon a time there was a"

response = client.responses.create(
    model="gpt-5-mini",  # this is your model deployment name
    input=prompt,
    store=False,
)
print(response.output_text)

In the above code, we create a response and pass in the model we want to use and the prompt. Then we print the generated text via response.output_text.

Multi-turn conversations

The Responses API is well suited for both single-turn text generation and multi-turn chatbots - you provide a list of messages in input to build up a conversation:

from openai import OpenAI

client = OpenAI(api_key="sk-...")

response = client.responses.create(model="gpt-5-mini", input="Hello world", store=False)
print(response.output_text)

More on this functionality in an upcoming chapter.

Exercise - your first text generation app

Now that we learned how to set up and configure openai, it's time to build your first text generation app. To build your app, follow these steps:

  1. Create a virtual environment and install openai:

bash python -m venv venv source venv/bin/activate pip install openai

[!NOTE] If you're using Windows type venv\Scripts\activate instead of source venv/bin/activate.

[!NOTE] Locate your Azure OpenAI key by going to https://portal.azure.com/ and search for Open AI and select the Open AI resource and then select Keys and Endpoint and copy the Key 1 value.

  1. Create an app.py file and give it the following code:

```python import os from openai import OpenAI

client = OpenAI( api_key="", base_url="/openai/v1/", ) deployment_name = ""

# add your completion code prompt = "Complete the following: Once upon a time there was a"

# make a request using the Responses API response = client.responses.create(model=deployment_name, input=prompt, store=False)

# print response print(response.output_text) ```

[!NOTE] If you're using plain OpenAI (not Azure), use client = OpenAI(api_key="<replace this value with your OpenAI key>") (no base_url) and pass a model name like gpt-5-mini instead of a deployment name.

You should see an output like the following:

```output very unhappy _____.

Once upon a time there was a very unhappy mermaid. ```

Different types of prompts, for different things

Now you've seen how to generate text using a prompt. You even have a program up and running that you can modify and change to generate different types of text.

Prompts can be used for all sorts of tasks. For example:

A more practical use case: a recipe generator

Imagine you have ingredients at home and you want to cook something. For that, you need a recipe. A way to find recipes is to use a search engine or you could use an LLM to do so.

You could write a prompt like so:

"Show me 5 recipes for a dish with the following ingredients: chicken, potatoes, and carrots. Per recipe, list all the ingredients used"

Given the above prompt, you might get a response similar to:

1. Roasted Chicken and Vegetables:
Ingredients:
- 4 chicken thighs
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 2 tablespoons olive oil
- 2 cloves garlic, minced
- 1 teaspoon dried thyme
- 1 teaspoon dried oregano
- Salt and pepper, to taste

2. Chicken and Potato Stew:
Ingredients:
- 2 tablespoons olive oil
- 1 onion, diced
- 2 cloves garlic, minced
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 teaspoon dried oregano
- 1 teaspoon dried thyme
- 1 cup chicken broth
- Salt and pepper, to taste

3. Chicken and Potato Bake:
Ingredients:
- 2 tablespoons olive oil
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 onion, diced
- 2 cloves garlic, minced
- 1 teaspoon dried oregano
- 1 teaspoon dried thyme
- 1 cup chicken broth
- Salt and pepper, to taste

4. Chicken and Potato Soup:
Ingredients:
- 2 tablespoons olive oil
- 1 onion, diced
- 2 cloves garlic, minced
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 teaspoon dried oregano
- 1 teaspoon dried thyme
- 4 cups chicken broth
- Salt and pepper, to taste

5. Chicken and Potato Hash:
Ingredients:
- 2 tablespoons olive oil
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 onion, diced
- 2 cloves garlic, minced
- 1 teaspoon dried oregano

This outcome is great, I know what to cook. At this point, what could be useful improvements are:

For the above cases, let's add an additional prompt:

"Please remove recipes with garlic as I'm allergic and replace it with something else. Also, please produce a shopping list for the recipes, considering I already have chicken, potatoes and carrots at home."

Now you have a new result, namely:

1. Roasted Chicken and Vegetables:
Ingredients:
- 4 chicken thighs
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 2 tablespoons olive oil
- 1 teaspoon dried thyme
- 1 teaspoon dried oregano
- Salt and pepper, to taste

2. Chicken and Potato Stew:
Ingredients:
- 2 tablespoons olive oil
- 1 onion, diced
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 teaspoon dried oregano
- 1 teaspoon dried thyme
- 1 cup chicken broth
- Salt and pepper, to taste

3. Chicken and Potato Bake:
Ingredients:
- 2 tablespoons olive oil
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 onion, diced
- 1 teaspoon dried oregano
- 1 teaspoon dried thyme
- 1 cup chicken broth
- Salt and pepper, to taste

4. Chicken and Potato Soup:
Ingredients:
- 2 tablespoons olive oil
- 1 onion, diced
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 teaspoon dried oregano
- 1 teaspoon dried thyme
- 4 cups chicken broth
- Salt and pepper, to taste

5. Chicken and Potato Hash:
Ingredients:
- 2 tablespoons olive oil
- 2 chicken breasts, cut into cubes
- 2 potatoes, cut into cubes
- 2 carrots, cut into cubes
- 1 onion, diced
- 1 teaspoon dried oregano

Shopping List:
- Olive oil
- Onion
- Thyme
- Oregano
- Salt
- Pepper

That's your five recipes, with no garlic mentioned and you also have a shopping list considering what you already have at home.

Exercise - build a recipe generator

Now that we have played out a scenario, let's write code to match the demonstrated scenario. To do so, follow these steps:

  1. Use the existing app.py file as a starting point
  2. Locate the prompt variable and change its code to the following:

python prompt = "Show me 5 recipes for a dish with the following ingredients: chicken, potatoes, and carrots. Per recipe, list all the ingredients used"

If you now run the code, you should see an output similar to:

```output -Chicken Stew with Potatoes and Carrots: 3 tablespoons oil, 1 onion, chopped, 2 cloves garlic, minced, 1 carrot, peeled and chopped, 1 potato, peeled and chopped, 1 bay leaf, 1 thyme sprig, 1/2 teaspoon salt, 1/4 teaspoon black pepper, 1 1/2 cups chicken broth, 1/2 cup dry white wine, 2 tablespoons chopped fresh parsley, 2 tablespoons unsalted butter, 1 1/2 pounds boneless, skinless chicken thighs, cut into 1-inch pieces -Oven-Roasted Chicken with Potatoes and Carrots: 3 tablespoons extra-virgin olive oil, 1 tablespoon Dijon mustard, 1 tablespoon chopped fresh rosemary, 1 tablespoon chopped fresh thyme, 4 cloves garlic, minced, 1 1/2 pounds small red potatoes, quartered, 1 1/2 pounds carrots, quartered lengthwise, 1/2 teaspoon salt, 1/4 teaspoon black pepper, 1 (4-pound) whole chicken -Chicken, Potato, and Carrot Casserole: cooking spray, 1 large onion, chopped, 2 cloves garlic, minced, 1 carrot, peeled and shredded, 1 potato, peeled and shredded, 1/2 teaspoon dried thyme leaves, 1/4 teaspoon salt, 1/4 teaspoon black pepper, 2 cups fat-free, low-sodium chicken broth, 1 cup frozen peas, 1/4 cup all-purpose flour, 1 cup 2% reduced-fat milk, 1/4 cup grated Parmesan cheese

-One Pot Chicken and Potato Dinner: 2 tablespoons olive oil, 1 pound boneless, skinless chicken thighs, cut into 1-inch pieces, 1 large onion, chopped, 3 cloves garlic, minced, 1 carrot, peeled and chopped, 1 potato, peeled and chopped, 1 bay leaf, 1 thyme sprig, 1/2 teaspoon salt, 1/4 teaspoon black pepper, 2 cups chicken broth, 1/2 cup dry white wine

-Chicken, Potato, and Carrot Curry: 1 tablespoon vegetable oil, 1 large onion, chopped, 2 cloves garlic, minced, 1 carrot, peeled and chopped, 1 potato, peeled and chopped, 1 teaspoon ground coriander, 1 teaspoon ground cumin, 1/2 teaspoon ground turmeric, 1/2 teaspoon ground ginger, 1/4 teaspoon cayenne pepper, 2 cups chicken broth, 1/2 cup dry white wine, 1 (15-ounce) can chickpeas, drained and rinsed, 1/2 cup raisins, 1/2 cup chopped fresh cilantro ```

NOTE, your LLM is nondeterministic, so you might get different results every time you run the program.

Great, let's see how we can improve things. To improve things, we want to make sure the code is flexible, so ingredients and number of recipes can be improved and changed.

  1. Let's change the code in the following way:

```python no_recipes = input("No of recipes (for example, 5): ")

ingredients = input("List of ingredients (for example, chicken, potatoes, and carrots): ")

# interpolate the number of recipes into the prompt an ingredients prompt = f"Show me {no_recipes} recipes for a dish with the following ingredients: {ingredients}. Per recipe, list all the ingredients used" ```

Taking the code for a test run, could look like this:

```output No of recipes (for example, 5): 3 List of ingredients (for example, chicken, potatoes, and carrots): milk,strawberries

-Strawberry milk shake: milk, strawberries, sugar, vanilla extract, ice cubes -Strawberry shortcake: milk, flour, baking powder, sugar, salt, unsalted butter, strawberries, whipped cream -Strawberry milk: milk, strawberries, sugar, vanilla extract ```

Improve by adding filter and shopping list

We now have a working app capable of producing recipes and it's flexible as it relies on inputs from the user, both on the number of recipes but also the ingredients used.

To further improve it, we want to add the following:

```python filter = input("Filter (for example, vegetarian, vegan, or gluten-free): ")

prompt = f"Show me {no_recipes} recipes for a dish with the following ingredients: {ingredients}. Per recipe, list all the ingredients used, no {filter}" ```

Above, we add {filter} to the end of the prompt and we also capture the filter value from the user.

An example input of running the program can now look like so:

```output No of recipes (for example, 5): 3 List of ingredients (for example, chicken, potatoes, and carrots): onion,milk Filter (for example, vegetarian, vegan, or gluten-free): no milk

  1. French Onion Soup

Ingredients:

-1 large onion, sliced -3 cups beef broth -1 cup milk -6 slices french bread -1/4 cup shredded Parmesan cheese -1 tablespoon butter -1 teaspoon dried thyme -1/4 teaspoon salt -1/4 teaspoon black pepper

Instructions:

  1. In a large pot, sauté onions in butter until golden brown.
  2. Add beef broth, milk, thyme, salt, and pepper. Bring to a boil.
  3. Reduce heat and simmer for 10 minutes.
  4. Place french bread slices on soup bowls.
  5. Ladle soup over bread.

  6. Sprinkle with Parmesan cheese.

  7. Onion and Potato Soup

Ingredients:

-1 large onion, chopped -2 cups potatoes, diced -3 cups vegetable broth -1 cup milk -1/4 teaspoon black pepper

Instructions:

  1. In a large pot, sauté onions in butter until golden brown.
  2. Add potatoes, vegetable broth, milk, and pepper. Bring to a boil.
  3. Reduce heat and simmer for 10 minutes.

  4. Serve hot.

  5. Creamy Onion Soup

Ingredients:

-1 large onion, chopped -3 cups vegetable broth -1 cup milk -1/4 teaspoon black pepper -1/4 cup all-purpose flour -1/2 cup shredded Parmesan cheese

Instructions:

  1. In a large pot, sauté onions in butter until golden brown.
  2. Add vegetable broth, milk, and pepper. Bring to a boil.
  3. Reduce heat and simmer for 10 minutes.
  4. In a small bowl, whisk together flour and Parmesan cheese until smooth.
  5. Add to soup and simmer for an additional 5 minutes, or until soup has thickened. ```

As you can see, any recipes with milk in it has been filtered out. But, if you're lactose intolerant, you might want to filter out recipes with cheese in them as well, so there's a need to be clear.

For this functionality, we could either try to solve everything in one prompt or we could split it up into two prompts. Let's try the latter approach. Here we're suggesting adding an additional prompt, but for that to work, we need to add the result of the former prompt as context to the latter prompt.

Locate the part in the code that prints out the result from the first prompt and add the following code below:

```python old_prompt_result = response.output_text prompt = "Produce a shopping list for the generated recipes and please don't include ingredients that I already have."

new_prompt = f"{old_prompt_result} {prompt}" response = client.responses.create(model=deployment_name, input=new_prompt, max_output_tokens=1200, store=False)

# print response print("Shopping list:") print(response.output_text) ```

Note the following:

  1. We're constructing a new prompt by adding the result from the first prompt to the new prompt:

    python new_prompt = f"{old_prompt_result} {prompt}"

  2. We make a new request, but also considering the number of tokens we asked for in the first prompt, so this time we say max_output_tokens is 1200.

    python response = client.responses.create(model=deployment_name, input=new_prompt, max_output_tokens=1200, store=False)

    Taking this code for a spin, we now arrive at the following output:

    ```output No of recipes (for example, 5): 2 List of ingredients (for example, chicken, potatoes, and carrots): apple,flour Filter (for example, vegetarian, vegan, or gluten-free): sugar

    -Apple and flour pancakes: 1 cup flour, 1/2 tsp baking powder, 1/2 tsp baking soda, 1/4 tsp salt, 1 tbsp sugar, 1 egg, 1 cup buttermilk or sour milk, 1/4 cup melted butter, 1 Granny Smith apple, peeled and grated -Apple fritters: 1-1/2 cups flour, 1 tsp baking powder, 1/4 tsp salt, 1/4 tsp baking soda, 1/4 tsp nutmeg, 1/4 tsp cinnamon, 1/4 tsp allspice, 1/4 cup sugar, 1/4 cup vegetable shortening, 1/4 cup milk, 1 egg, 2 cups shredded, peeled apples Shopping list: -Flour, baking powder, baking soda, salt, sugar, egg, buttermilk, butter, apple, nutmeg, cinnamon, allspice ```

Improve your setup

What we have so far is code that works, but there are some tweaks we should be doing to improve things further. Some things we should do are:

  1. Create a .env file with the following content:

    bash OPENAI_API_KEY=sk-...

    Note, for Azure OpenAI in Microsoft Foundry, you need to set the following environment variables instead:

    bash AZURE_OPENAI_API_KEY=<replace> AZURE_OPENAI_ENDPOINT=<replace> AZURE_OPENAI_API_VERSION=2024-10-21

    In code, you would load the environment variables like so:

    ```python import os from dotenv import load_dotenv from openai import OpenAI

    load_dotenv()

    client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) ```

To change the tokens used, you can use the max_output_tokens parameter. For example, if you want to use 100 tokens, you would do:

python response = client.responses.create(model=deployment, input=prompt, max_output_tokens=100, store=False)

To alter the temperature, you can use the temperature parameter. For example, if you want to use a temperature of 0.5, you would do:

python response = client.responses.create(model=deployment, input=prompt, temperature=0.5, store=False)

Note, the closer to 1.0, the more varied the output.

In short: temperature/top_p are still valid on many models (Llama, Mistral, Phi, and the GPT-4.x family - though GPT-4.x is deprecating), but the direction of travel is prompt engineering + reasoning controls on reasoning models like GPT-5.

Assignment

For this assignment, you can choose what to build.

Here are some suggestions:

Solution

Study buddy

Below is a starter prompt, see how you can use it and tweak it to your liking.

- "You're an expert on the Python language

    Suggest a beginner lesson for Python in the following format:

    Format:
    - concepts:
    - brief explanation of the lesson:
    - exercise in code with solutions"

History bot

Here are some prompts you could be using:

- "You are Abe Lincoln, tell me about yourself in 3 sentences, and respond using grammar and words like Abe would have used"
- "You are Abe Lincoln, respond using grammar and words like Abe would have used:

   Tell me about your greatest accomplishments, in 300 words"

Knowledge check

What does the concept temperature do?

  1. It controls how random the output is.
  2. It controls how big the response is.
  3. It controls how many tokens are used.

🚀 Challenge

When working on the assignment, try to vary the temperature, try setting it to 0, 0.5, and 1. Remember that 0 is the least varied and 1 is the most. What value works best for your app?

Great Work! Continue Your Learning

After completing this lesson, check out our Generative AI Learning collection to continue leveling up your Generative AI knowledge!

Head over to Lesson 7 where we will look at how to build chat applications!