PDL Language Tutorial

The following sections give a step-by-step overview of PDL language features. All the examples in this tutorial can be found in examples/tutorial.

Simple text

The simplest PDL program is one that generates a small text (file):

description: Hello world!
text:
    Hello, world!

This program has a description field, which contains a title. The description field is optional. It also has a text field, which can be either a string, a block, or a list of strings and blocks. A block is a recipe for how to obtain data (e.g., model call, code call, etc...). In this case, there are no calls to an LLM or other tools, and text consists of a simple string.

To render the program into an actual text, we have a PDL interpreter that can be invoked as follows:

pdl examples/tutorial/simple_program.pdl

This results in the following output:

Hello, world!

Calling an LLM

description: Calling a model on the implicit background context
text:
- "Hello\n"
- model: ollama_chat/granite3.2:2b
  parameters:
    stop: ['!']

In this program (file), the text starts with the word "Hello\n", and we call a model (ollama/granite3.2:2b) with this as input prompt. The model is passed a parameter stop to indicate the stop sequences.

A PDL program computes two data structures. The first is a JSON corresponding to the result of the overall program, obtained by aggregating the results of each block. This is what is printed by default when we run the interpreter. The second is a conversational background context, which is a list of role/content pairs (list of messages), where we implicitly keep track of roles and content for the purpose of communicating with models that support chat APIs. The contents in the latter correspond to the results of each block. The conversational background context is what is used to make calls to LLMs via LiteLLM.

In this example, the input of the model is [{"role": "user", "content": "Hello\n"}] which corresponds to the entire text up to that point using the default role user.

When we execute this program using the interpreter, we obtain the following result where the second Hello has been generated by Granite:

Hello
Hello

The input of to the model can also be provided explicitly using the input field. Here is an example of model call using this feature (file):

description: Calling a model with an input text
text:
- "Hello\n"
- model: ollama_chat/granite3.2:2b
  input:
    Translate the word 'Hello' to French

In this case, the input passed to the model is [{"role": "user", "content": "Translate the word 'Hello' to French"}] and nothing else from the surrounding document. When we execute this program, we obtain the following result where the second line is generated by the model.:

Hello
Bonjour (pronounced bon-zhoor) is the translation for "Hello" in French. It's an informal greeting used during the day, similar to how we use "Hi" or "Hello." For a more formal context, you might say "Bonjour," which means "Good day."

Using the input field, we can also give a directly an array of messages (role/content) to the model (file):

description: Calling a model with an explicit list of messages
text:
- "Hello\n"
- model: ollama_chat/granite3.2:2b
  input:
    array:
    - role: system
      content: You are a helpful assistant that is fluent in French.
    - role: user
      content: Translate the word 'Hello' to French

This has a similar output as the previous program. An alternative way of writing this program using a variable to store the prompt is this program.

Parameter defaults for watsonx Granite models

When using Granite models, we use the following defaults for model parameters:

temperature: 0
max_new_tokens: 1024
min_new_tokens: 1
repetition_penalty: 1.05

Also if the decoding_method is sample (`watsonx_text text completion endpoint), then the following defaults are used:

temperature: 0.7
top_p: 0.85
top_k: 50

The user can override these defaults by explicitly including them in the model call.

Building the background context with `lastOf`

The pervious example explicitly provides a list of messages with different roles to the LLM call. This can also be done implicitly using the background context.

Each block can be annotated with a role field indicating the role that is used when a message is added to the background context by the block or any of the sub-block that does not redefine it. In this example, we add a system message asking the model to provide answer formally (file):

description: Explicit use of role
role: user
text:
- "Hello\n"
- role: system
  text: "You are a polite assistant that likes to answer very formally."
- model: ollama_chat/granite3.2:2b

In this program, we explicitly indicated the top-level user role which is added automatically by the interpreter otherwise. This role is inherited by the "Hello\n" block and masked in the next block to define a system prompt. So the context provided as input to the LLM is [{"role": "user", "content": "Hello\n"}, {"role": "system", "content": "You are a polite assistant that likes to answer very formally."}]. The answer produced by the model block has the assistant role.

The execution of this program produces:

Hello
You are a polite assistant that likes to answer very formally.
Greetings! I trust this message finds you in good health and high spirits. How may I be of assistance today? Please feel free to pose your query or request, knowing that I am here to serve with diligence and precision.

If we want to add the system message to the background context without having it present in the result, we can use a lastOf block. The lastOf is associated to a list of blocks that are executed in sequence. Each sub-block contributes messages to the background context but the result of the block is the result of the last block. The following program provides the same input to the LLM, but the system prompt is not part of the result (file):

description: Explicit use of role
role: user
text:
- "Hello\n"
- lastOf:
  - role: system
    text: "You are a polite assistant that likes to answer very formally."
  - model: ollama_chat/granite3.2:2b

Therefore, the result of the program is:

Hello

Greetings! I trust this message finds you in good health and high spirits. How may I be of assistance today? Please feel free to pose your query or request, knowing that I am here to serve with diligence and precision.

Model Chaining

In PDL, we can declaratively chain models together as in the following example (file):

description: Model chaining
text:
- "Hello\n"
- model: ollama_chat/granite3.3:8b
  parameters:
    stop: ["!"]
- "\nTranslate the above to French\n"
- model: ollama_chat/granite3.3:8b
  parameters:
    stop: ["!"]

In this program, the first block result is Hello\n and adds a message with this value to the background context. The second block calls Granite on the background context containing the Hello message and adds the response of the model to the result and context. The following block contributes the sentence: \nTranslate the above to French\n. The final line of the program takes the entire context produced so far and passes it as input to the Granite model. Notice that the input passed to this model is the context up to that point, represented as a conversation. This makes it easy to chain models together and continue building on previous interactions. Notice how the conversational context is accumulated implicitly without requiring the user to explicitly manage messages.

When we execute this program, we obtain:

Hello
Hello
Translate the above to French
Bonjour

Variable Definition and Use

Any block can define a variable using a def: <var> field. This means that the result of that block is assigned to the variable <var>, which may be reused at a later point in the document.

Consider the following example (file):

description: Variable def and use
text:
- "Hello\n"
- model: ollama_chat/granite3.2:2b
  parameters:
    stop: ['!']
  def: GEN
- "\nThe variable GEN is equal to: ${ GEN }"

Here we assign the response of the model to variable GEN using the def field. The last line of the program prints out the value of GEN. Notice the notation ${ } for accessing the value of a variable. Any Jinja expression is allowed to be used inside these braces. These expressions are also used to specify conditions for loops and conditionals. See for example this file.

When we execute this program, we obtain:

Hello
Hello
The variable GEN is equal to: Hello

Local Computation Using `defs`

In the previous example, the value of the variable GEN computed by the model block is part of the result and is added to the background context. To define the variable GEN without contributing to the result and context, the model block can be moved into a defs (file):

description: Local computations using defs
text:
- "Hello\n"
- defs:
    GEN:
      model: ollama_chat/granite3.2:2b
      parameters:
        stop: ['!']
- "The variable GEN is equal to: ${ GEN }"

The execution of this program produces:

Hello
The variable GEN is equal to: Hello

The defs field can be added on any block and can introduce multiple variables. The following program defines two variables fr and es associated to a text block that uses them (file):

text:
- "Hello\n"
- defs:
    fr:
      lastOf:
      - "\nTranslate to French\n"
      - model: ollama_chat/granite3.2:2b
    es:
      lastOf:
      - "\nTranslate to Spanish\n"
      - model: ollama_chat/granite3.2:2b
  text: |

    In Fench: ${ fr }

    In Spanish: ${ es }

This program first output Hello and add it in the context. Then, the blocks defining the fr and es variables are both executed in a context containing only the Hello message. These blocks are using a lastOf that adds the value to each sub-blocs to the context and output the value of the last block. Finally, the value of the variables are used in the text block.

The output of this program is:

Hello

In Fench: Bonjour!

Translation of "Hello" in French is "Bonjour".

In Spanish: Hola!

La traducción de "Hello" al español es "Hola".

Control Block Outputs with `contribute`

By default, when a PDL block is executed, it produces a result that is contributed to the overall result, and it also contributes to the background context. We saw that defs and lastOf gives some control over the contribution to the result of context. defs executes a block without contributing to the context and name the result that can be used later. lastOf contributes only to the context for all of its sun-blocks except the last one. It is also possible to control the contribution of each block using the contribute field.

Consider an example similar as above, but that uses contribute instead of defs and lastOf (file):

description: Control block outputs with `contribute`
text:
- "Hello\n"
- text: "\nTranslate to French\n"
  contribute: [context]
- model: ollama_chat/granite3.2:2b
  contribute: []
  def: fr
- |

  In Fench: ${ fr }

Instead of a lastOf, we set contribute to [context] for the block that produces "\nTranslate to French\n". That way, we only contribute to the context and not to the result. We set contribute to [] for the call to the LLM such that it does not produce an output but only save the result in the fr variable that is used in the last block of the program. When we execute this program, we obtain:

Hello

In Fench: Bonjour!

Translation of "Hello" in French is "Bonjour".

In general, contribute can be used to set how the result of the block contribute to the final result and the background context. Here are its possible values:

[]: no contribution to either the final result or the background context
[result]: contribute to the final result but not the background context
[context]: contribute to the background context but not the final result
[result, context]: contribute to both, which is also the default setting.

Function Definition

PDL supports function definitions to make it easier to reuse code. Suppose we want to define a translation function that takes a string and calls a Granite model for the translation. This would be written in PDL as follows (file):

description: Function definition and call
defs:
  translate:
    function:
      sentence: string
      language: string
    return:
      lastOf:
      - |
        Translate the sentence '${ sentence }' to ${ language }.
        Only give the result of the translation.
      - model: ollama_chat/granite3.2:2b
text:
- call: ${ translate }
  args:
    sentence: I love Paris!
    language: French
- "\n"
- call: ${ translate }
  args:
    sentence: I love Madrid!
    language: Spanish

In this program, the defs field defines a function translate that takes as parameters sentence and language, both of which are of type string. The body of the function is defined by its return field. In this case, we formulate a translation prompt using the parameters and send it to a Granite model.

The body of the program is a text block that calls this function twice, as indicated by call: ${ translate }. The call block specifies the arguments to be passed. When we execute this program, we obtain:

J'aime Paris !
Me encanta Madrid.

When we call a function, we implicitly pass the current background context, and this is used as input to model calls inside the function body. In the above example, since the input field is omitted, the entire document produced at that point is passed as input to the Granite model.

To reset the context when calling a function, we can pass the special argument: pdl_context: [] (see example).

Functions can be declared with optional parameters (see example).

PDL is a language with higher order functions meaning that functions are values. So for example, a function can be aliased (see example).

PDL functions can also be called from Jinja expressions as in the following example (file):

description: Calling a PDL function from Jinja
defs:
  translate:
    function:
      sentence: string
      language: string
    return:
      lastOf:
      - |
        Translate the sentence '${ sentence }' to ${ language }.
        Only give the result of the translation.
      - model: ollama_chat/granite3.2:2b
text: |
  The way to say hello in French is ${ translate("Hello", language="French") }.

Notice that arguments can be positional or named.

Building Data Structures

In PDL, the user specifies step by step the shape of data they wish to generate. A text block takes a list of blocks, stringifies the result of each block, and concatenates them.

An array takes a list of blocks and creates an array of the results of each block:

array:
  - apple
  - orange
  - banana

This results in the following output:

["apple", "orange", "banana"]

Each list item can contain any PDL block (strings are shown here), and the overall result is presented as an array.

An object constructs an object:

object:
  name: Bob
  job: manager

This results in the following output:

{"name": "Bob", "job": "manager"}

Each value in the object can be any PDL block, and the result is presented as an object.

A lastOf is a sequence, where each block in the sequence is executed and the overall result is that of the last block.

lastOf:
  - 1
  - 2
  - 3

This results in the following output:

Each list item can contain any PDL block (strings are shown here), and the result of the whole list is that of the last block.

Notice that block types that require lists (repeat, for, if-then-else) need to specify the shape of data in their bodies, for example text or array. It is a syntax error to omit it. For more detailed discussion on this see this section.

Input from File or Stdin

PDL can accept textual input from a file or stdin. In the following example (file), the contents of this file are read by PDL and incorporated in the document. The result is also assigned to a variable HELLO.

description: PDL code with input block
text:
- read: ./data.txt
  def: HELLO

In the next example, prompts are obtained from stdin (file). This is indicated by assigning the value null to the read field.

description: PDL code with input block
text:
- "The following will prompt the user on stdin.\n"
- read:
  message: "Please provide an input: "
  def: STDIN

If the message field is omitted then one is provided for you.

The following example shows a multiline stdin input (file). When executing this code and to exit from the multiline input simply press control D (on macOS).

description: PDL code with input block
text:
- "A multiline stdin input.\n"
- read:
  multiline: true

Finally, the following example shows reading content in JSON format.

Consider the JSON content in this file:

{
    "name": "Bob",
    "address": {
        "number": 87,
        "street": "Smith Road",
        "town": "Armonk", 
        "state": "NY",
        "zip": 10504
    }
}

The following PDL program reads this content and assigns it to variable PERSON in JSON format using the parser (file). The reference PERSON.address.street then refers to that field inside the JSON object. Note that the PDL interpreter performs automatic repair of JSON objects generated by LLMs.

description: Input block example with json input
defs:
  PERSON:
    read: ./input.json
    parser: json
text:
- "${ PERSON.name } lives at the following address:\n"
- "${ PERSON.address.number } ${ PERSON.address.street } in the town of ${ PERSON.address.town }, ${ PERSON.address.state }"

When we execute this program, we obtain:

Bob lives at the following address:
87 Smith Road in the town of Armonk, NY

Parsing the output of a block

As we saw in the previous section, it is possible to use the parser: json setting to parse the result of a block as a JSON. Other possible values for parser are yaml, jsonl, or regex.

The following example extracts using a regular expression parser the code between triple backtick generated by a model:

description: Parse a block output using a regex
defs:
  output:
    model: ollama_chat/granite3.2:2b
    parameters:
      temperature: 0
    input: Write a Python function that perform the addition of two numbers.
    parser:
      spec:
        code: string
      regex: (.|\n)*```python\n(?P<code>(.|\n)*)```(.|\n)*
text: ${ output.code }

Here is another example using a regular expression:

We support the following operations with theregex parser (indicated with the mode field):

fullmatch (default)
search
match
split
findall

Here is an example using the findall mode that returns the list ['1', '2', '3', '4']:

data: "1 -- 2 -- 3 -- 4"
parser:
  regex: '[0-9]+'
  mode: findall

See here for more information on how to write regular expressions.

Calling code

The following script shows how to execute python code (file). The python code is executed locally (or in a containerized way if using pdl --sandbox). In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable result internally to be propagated to the result of the block. A variable def on the code block will then be set to this result.

In order to define variables that are carried over to the next Python code block, a special variable PDL_SESSION can be used, and variables assigned to it as fields. See for example: (file).

description: Hello world showing call to Python code
text:
- "Hello, "
- lang: python
  code: 
    |
    import random
    import string
    result = random.choice(string.ascii_lowercase)

This results in the following output (for example):

Hello, r!

PDL also supports Jinja code blocks, shell commands, as well as PDL code blocks for meta-cycle programming. For more examples, see (Jinja code), (shell command), (PDL code).

Calling REST APIs

PDL programs can contain calls to REST APIs with Python code. Consider a simple weather app (file):

description: Using a weather API and LLM to make a small weather app
text:
- def: QUERY
  text: "What is the weather in Madrid?\n"
- model: ollama_chat/granite3.2:2b
  input: |
      Extract the location from the question.
      Question: What is the weather in London?
      Answer: London
      Question: What's the weather in Paris?
      Answer: Paris
      Question: Tell me the weather in Lagos?
      Answer: Lagos
      Question: ${ QUERY }
  parameters:
    stop: ["Question", "What", "!", "\n"]
  def: LOCATION
  contribute: []
- lang: python
  code: |
    import requests
    #result = requests.get('https://api.weatherapi.com/v1/current.json?key==XYZ=${ LOCATION }').text
    #Mock result:
    result = '{"location": {"name": "Madrid", "region": "Madrid", "country": "Spain", "lat": 40.4, "lon": -3.6833, "tz_id": "Europe/Madrid", "localtime_epoch": 1732543839, "localtime": "2024-11-25 15:10"}, "current": {"last_updated_epoch": 1732543200, "last_updated": "2024-11-25 15:00", "temp_c": 14.4, "temp_f": 57.9, "is_day": 1, "condition": {"text": "Partly cloudy", "icon": "//cdn.weatherapi.com/weather/64x64/day/116.png", "code": 1003}, "wind_mph": 13.2, "wind_kph": 21.2, "wind_degree": 265, "wind_dir": "W", "pressure_mb": 1017.0, "pressure_in": 30.03, "precip_mm": 0.01, "precip_in": 0.0, "humidity": 77, "cloud": 75, "feelslike_c": 12.8, "feelslike_f": 55.1, "windchill_c": 13.0, "windchill_f": 55.4, "heatindex_c": 14.5, "heatindex_f": 58.2, "dewpoint_c": 7.3, "dewpoint_f": 45.2, "vis_km": 10.0, "vis_miles": 6.0, "uv": 1.4, "gust_mph": 15.2, "gust_kph": 24.4}}'
  def: WEATHER
  parser: json
  contribute: []
- model: ollama_chat/granite3.2:2b
  input: |
      Explain the weather from the following JSON:
      ${ WEATHER }

In this program, we first define a query about the weather in some location (assigned to variable QUERY). The next block is a call to a Granite model with few-shot examples to extract the location, which we assign to variable LOCATION. The next block makes an API call with Python (mocked in this example). Here the LOCATION is appended to the url. The result is a JSON object, which may be hard to interpret for a human user. So we make a final call to an LLM to interpret the JSON in terms of weather. Notice that many blocks have contribute set to [] to hide intermediate results.

Data Block

PDL offers the ability to create JSON data as illustrated by the following example (described in detail in the Overview section). The data block can gather previously defined variables into a JSON structure. This feature is useful for data generation. Programs such as this one can be generalized to read jsonl files to generate data en masse by piping into another jsonl file (file).

description: Code explanation example
defs:
  CODE:
    read: ./data.yaml
    parser: yaml
  TRUTH:
    read: ./ground_truth.txt
  EXPLANATION:
    model: ollama_chat/granite3.2:2b
    input:
      |
        Here is some info about the location of the function in the repo.
        repo:
        ${ CODE.repo_info.repo }
        path: ${ CODE.repo_info.path }
        Function_name: ${ CODE.repo_info.function_name }


        Explain the following code:
        ```
        ${ CODE.source_code }```
  EVAL:
    lang: python
    code:
      |
      import textdistance
      expl = """
      ${ EXPLANATION }
      """
      truth = """
      ${ TRUTH }
      """
      result = textdistance.levenshtein.normalized_similarity(expl, truth)
data:
  input: ${ CODE }
  output: ${ EXPLANATION }
  metric: ${ EVAL }

Notice that in the data block the values are interpreted as Jinja expressions. If values need to be PDL programs to be interpreted, then you need to use the object block instead (see this section).

In the example above, the expressions inside the data block are interpreted, but in some cases it may be useful not to interpret the values in a data block. The raw field can be used to turn off the interpreter inside a data block. For example, consider the (file):

description: Raw data block
data:
  name: ${ name }
  phone: ${ phone }
raw: True

The result of this program is the JSON object:

{
  "name": "${ name }",
  "phone": "${ phone }"
}

where the values of name and phone have been left uninterpreted.

Import Block

PDL allows programs to be defined over multiple files. The import block allows one file to incorporate another, as shown in the following example:

defs:
  lib: 
    import: import_lib
text:
- call: ${ lib.a }
  args: 
    arg: Bye!

which imports the following file:

The import block means that the PDL code at that file is executed and its scope is assigned to the variable defined in that block. So all the defs in the imported file are made available via that variable. This feature allows reuse of common templates and patterns and to build libraries. Notice that relative paths are relative to the containing file.

Prompt Library

A prompt library is included in the contrib/ directory. These modules define some common patterns such Chain-of-Thought, ReAct, and ReWOO. Example usage can be found in examples/prompt_library. Note that import blocks resolve file paths relative to the PDL file being executed.

Chain-of-Thought

In the GSM8K example below, we import CoT (Wei et al., 2022) and define our model, demonstrations, and the question. Demonstrations are a list of dictionaries with question, reasoning, and answer keys. The cot.chain_of_thought function returns a dictionary with an answer key.

examples/prompt_library/gsm8k_cot.pdl

description: Demo of CoT pattern
defs:
  cot:
    import: ../../contrib/prompt_library/CoT
  model: ollama/granite3.2:8b
  demonstrations:
    data:
      - question: |-
          Noah charges $60 for a large painting and $30 for a small painting.
          Last month he sold eight large paintings and four small paintings.
          If he sold twice as much this month, how much is his sales for this month?
        reasoning: |-
          He sold 8 large paintings and 4 small paintings last month.
          He sold twice as many this month.
          8 large paintings x $60 = << 8*60= 480 >> 480
          4 small paintings x $30 = << 4*30= 120 >> 120
          So he sold << 480+120= 600 >> 600 paintings last month.
          Therefore he sold << 600*2= 1200 >> this month.
        answer: $1200
      - question: |-
          Noah charges $30 for a large vases and $10 for a small vases.
          Last month he sold five large vases and three small vases.
          If he sold three times as much this month, how much is his sales for this month?
        reasoning: |-
          He sold 5 large vases and 3 small vases last month.
          He sold three times as many this month.
          5 large vases x $30 = << 5*30= 150 >> 150
          3 small vases x $10 = << 3*10= 30 >> 30
          So he sold << 150+30= 180 >> 180 vases last month.
          Therefore he sold << 180*3= 540 >> this month.
        answer: $540
  question: "Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?"

text:
  # CoT
  - "Answer the questions to the best of your abilities.\n\n"
  - call: ${ cot.chain_of_thought }
    def: cot_result
    contribute: []
    args:
      examples: "${ demonstrations }"
      question: "${ question }"
      model: "${ model }"
  - "Result: ${ cot_result }"

Executing this example produces the following background context (model response highlighted):

Answer the questions to the best of your abilities.

Question: Noah charges $60 for a large painting and $30 for a small painting.
Last month he sold eight large paintings and four small paintings.
If he sold twice as much this month, how much is his sales for this month?
Answer: Let's think step by step. He sold 8 large paintings and 4 small paintings last month.
He sold twice as many this month.
8 large paintings x $60 = << 8*60= 480 >> 480
4 small paintings x $30 = << 4*30= 120 >> 120
So he sold << 480+120= 600 >> 600 paintings last month.
Therefore he sold << 600*2= 1200 >> this month.
The answer is $1200

Question: Noah charges $30 for a large vases and $10 for a small vases.
Last month he sold five large vases and three small vases.
If he sold three times as much this month, how much is his sales for this month?
Answer: Let's think step by step. He sold 5 large vases and 3 small vases last month.
He sold three times as many this month.
5 large vases x $30 = << 5*30= 150 >> 150
3 small vases x $10 = << 3*10= 30 >> 30
So he sold << 150+30= 180 >> 180 vases last month.
Therefore he sold << 180*3= 540 >> this month.
The answer is $540

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
Answer: Let's think step by step.
1. Jacob earns $6 per hour.
2. Jake earns thrice what Jacob does, so Jake earns 3 * $6 = $18 per hour.
3. Jake works 8 hours a day, so he earns $18 * 8 = $144 per day.
4. Jake works 5 days, so he earns $144 * 5 = $720 in 5 days.
The answer is $720.

The final result is:

Answer the questions to the best of your abilities.

Result: {'answer': '1. Jacob earns $6 per hour.\n2. Jake earns thrice what Jacob does, so Jake earns 3 * $6 = $18 per hour.\n3. Jake works 8 hours a day, so he earns $18 * 8 = $144 per day.\n4. Jake works 5 days, so he earns $144 * 5 = $720 in 5 days.\nThe answer is $720.'}

ReAct

In the ReAct (Yao et al., 2022) example below, we import the ReAct library and provide tools from tools.pdl. Demonstrations consist of agent trajectories represented as a list of lists with key names question or task, thought, action, and observation. Actions follow this pattern: {"name": "tool_name", "arguments": {"arg1": "..."}}. The function returns a dictionary with a answer key, containing the answer.

examples/prompt_library/gsm8k_react.pdl

description: Demo of ReAct pattern
defs:
  react:
    import: ../../contrib/prompt_library/ReAct
  tools:
    import: ../../contrib/prompt_library/tools
  model: ollama/granite3.2:8b
  demonstrations:
    data:
      - - question: Noah charges $60 for a large painting and $30 for a small painting.  Last month he sold eight large paintings and four small paintings.  If he sold twice as much this month, how much is his sales for this month?
        - thought: He sold 8 large paintings and 4 small paintings last month.  He sold twice as many this month. I need to calculate (8 large paintings x $60 + 4 small paintings x $30)
        - action: '{"name": "calculator", "arguments": {"expr": "8*60+4*30"}}'
        - observation: 600
        - thought: He sold twice as many paintings this month, therefore I need to calculate 600*2.
        - action: '{"name": "calculator", "arguments": {"expr": "600*2"}}'
        - observation: 1200
        - thought: He sold $1200 this month.
        - action: '{"name": "finish", "arguments": {"answer": "$1200"}}'
      - - question: Teresa is 59 and her husband Morio is 71 years old. Their daughter, Michiko was born when Morio was 38. How old was Teresa when she gave birth to Michiko?
        - thought: I need to calculate the difference in age between Teresa and Morio.
        - action: '{"name": "calculator", "arguments": {"expr": "71-59"}}'
        - observation: 12
        - thought: I need to calculate how old Teresa is when their daughter is born.
        - action: '{"name": "calculator", "arguments": {"expr": "38-12"}}'
        - observation: 26
        - thought: Teresa was 26 when she gave birth to Michiko.
        - action: '{"name": "finish", "arguments": {"answer": "26"}}'
  question: "Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?"

text:
  # ReAct
  - call: ${ react.react }
    def: react_result
    contribute: []
    args:
        task: ${ question }
        model: ${ model }
        tool_schema: ${ tools.tool_schema }
        tools: ${ tools.tools }
        trajectories: ${ demonstrations }
  - "Result: ${ react_result }"

Produces background context (model responses highlighted):

Question: Noah charges $60 for a large painting and $30 for a small painting.  Last month he sold eight large paintings and four small paintings.  If he sold twice as much this month, how much is his sales for this month?
Tho: He sold 8 large paintings and 4 small paintings last month.  He sold twice as many this month. I need to calculate (8 large paintings x $60 + 4 small paintings x $30)
Act: {"name": "calculator", "arguments": {"expr": "8*60+4*30"}}
Obs: 600
Tho: He sold twice as many paintings this month, therefore I need to calculate 600*2.
Act: {"name": "calculator", "arguments": {"expr": "600*2"}}
Obs: 1200
Tho: He sold $1200 this month.
Act: {"name": "finish", "arguments": {"answer": "$1200"}}

Question: Teresa is 59 and her husband Morio is 71 years old. Their daughter, Michiko was born when Morio was 38. How old was Teresa when she gave birth to Michiko?
Tho: I need to calculate the difference in age between Teresa and Morio.
Act: {"name": "calculator", "arguments": {"expr": "71-59"}}
Obs: 12
Tho: I need to calculate how old Teresa is when their daughter is born.
Act: {"name": "calculator", "arguments": {"expr": "38-12"}}
Obs: 26
Tho: Teresa was 26 when she gave birth to Michiko.
Act: {"name": "finish", "arguments": {"answer": "26"}}

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
Tho: Jacob earns $6 per hour. Jake earns thrice as much.
Act: {"name": "calculator", "arguments": {"expr": "6*3"}}
Obs: 18
Tho: Jake earns $18 per hour.
Act: {"name": "calculator", "arguments": {"expr": "18*8"}}
Obs: 144
Tho: Jake earns $144 in a day.
Act: {"name": "calculator", "arguments": {"expr": "144*5"}}
Obs: 720
Tho: Jake earns $720 in 5 days.
Act: {"name": "finish", "arguments": {"answer": "$720"}}

The final result is:

Result: {'answer': '$720'}

Tools

Tools allow the agentic patterns to call PDL functions. The tools are defined in two parts. The tools object is a dictionary of tool names to PDL functions. The tool_schema is a JSON schema, represented as a data block consisting of list of dictionaries, that describes the tools to the LLM. The tools library contains a calculator and a Wikipedia search tool by default. There are a few requirements to consider when using tools.

You must have a finish tool in the schema. You do not need a PDL function for this, but the LLM needs to be aware of the finish tool so the ReAct loop can end.
A PDL tool function must accept only arguments: obj as parameters.

Here we define our tools in PDL, with an example using a python block. Note the use of arguments: obj.

tools:
  object:
    hello:
      function:
        arguments: object
      return:
        lang: python
        code: |
          result = None
          def main(name: str, *args, **kwargs) -> str:
            return f"hello {name}"
          result = main(**arguments)

    another_function: ...

Once we have defined our PDL functions, we describe them in the tool_schema.

tool_schema:
  data:
    - name: hello
      description: Hello function, returns "hello <name>"
      parameters:
        type: object
        properties:
          name:
            type: string
            description: Name to greet
        required:
          - name

    - name: finish
      description: Respond with the answer
      parameters:
        type: object
        properties:
          answer:
            type: string
            description: The answer
          required:
            - answer

Another useful thing to remember is that data blocks are templated in PDL. For example, this is valid:

finish_action:
  data:
    name: finish
    description: Respond with the answer
    parameters:
      type: object
      properties:
        answer:
          type: string
          description: The answer
        required:
          - answer

tool_schema:
  data:
    - name: calculator
      description: Calculator function
      parameters:
        type: object
        properties:
          expr:
            type: string
            description: Arithmetic expression to calculate
        required:
          - expr
    - ${ finish_action }

In fact, you can reuse the finish_action from tools in your own tool schemas:

defs:
  tools:
    import: ../../contrib/prompt_library/tools
  my_tool_schema:
    data:
      - ${ tools.finish_action }
      - ...

Additionally, there is a helper function filter_tools_by_name in tools, that given a JSON tools schema and a list of tool names, returns a schema with only those tools in it. This is useful if you have defined many tools, but don't need all of them for a particular task.

call: ${ tools.filter_tools_by_name }
args:
  tools: ${ tools.tool_schema }
  tool_names: ['calculator', 'finish']

ReWOO

The tools can be reused for ReWOO (Xu et al., 2023). Demonstrations follow a similar pattern as for ReAct, except that ReWOO does not use a finish tool/action, nor does it include observation, and thus should be omitted from the demonstrations. The argument show_plans is used for displaying the extracted ReWOO plans in the background context for debugging purposes i.e. --stream context. The function returns a dictionary with a answer key, containing the answer.

examples/prompt_library/gsm8k_rewoo.pdl

description: Demo of ReWOO pattern
defs:
  rewoo:
    import: ../../contrib/prompt_library/ReWoo
  tools:
    import: ../../contrib/prompt_library/tools
  model: ollama/granite3.2:8b
  demonstrations:
    data:
      - - question: Noah charges $60 for a large painting and $30 for a small painting. Last month he sold eight large paintings and four small paintings. If he sold twice as much this month, how much is his sales for this month?
        - thought: He sold 8 large paintings and 4 small paintings last month. He sold twice as many this month. I need to calculate (8 large paintings x $60 + 4 small paintings x $30)
        - action: '{"name": "calculator", "arguments": {"expr": "8*60+4*30"}}'
        - thought: He sold twice as many paintings this month, therefore I need to calculate 600*2.
        - action: '{"name": "calculator", "arguments": {"expr": "600*2"}}'
      - - question: Teresa is 59 and her husband Morio is 71 years old. Their daughter, Michiko was born when Morio was 38. How old was Teresa when she gave birth to Michiko?
        - thought: I need to calculate the difference in age between Teresa and Morio.
        - action: '{"name": "calculator", "arguments": {"expr": "71-59"}}'
        - thought: I need to calculate how old Teresa is when their daughter is born.
        - action: '{"name": "calculator", "arguments": {"expr": "38-12"}}'
  question: "Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?"

text:
  # ReWoo
  - call: ${ rewoo.rewoo }
    def: rewoo_result
    contribute: []
    args:
        task: ${ question }
        model: ${ model }
        tool_schema: ${ tools.tool_schema }
        tools: ${ tools.tools }
        trajectories: ${ demonstrations }
        show_plans: false
  - "Result: ${ rewoo_result }"

This results in the following background context:

For the following task, make plans that can solve the problem step by step. For each plan, indicate which external tool together with tool input to retrieve evidence. You can store the evidence into a variable #E that can be called by later tools. (Plan, #E1, Plan, #E2, Plan, ...)

Tools can be one of the following:
[{'type': 'function', 'function': {'name': 'calculator', 'description': 'Calculator function', 'parameters': {'type': 'object', 'properties': {'expr': {'type': 'string', 'description': 'Arithmetic expression to calculate'}}, 'required': ['expr']}}}]

Task: Noah charges $60 for a large painting and $30 for a small painting. Last month he sold eight large paintings and four small paintings. If he sold twice as much this month, how much is his sales for this month?
Plan: He sold 8 large paintings and 4 small paintings last month. He sold twice as many this month. I need to calculate (8 large paintings x $60 + 4 small paintings x $30) #E1 = {"name": "calculator", "arguments": {"expr": "8*60+4*30"}}
Plan: He sold twice as many paintings this month, therefore I need to calculate 600*2. #E2 = {"name": "calculator", "arguments": {"expr": "600*2"}}

Task: Teresa is 59 and her husband Morio is 71 years old. Their daughter, Michiko was born when Morio was 38. How old was Teresa when she gave birth to Michiko?
Plan: I need to calculate the difference in age between Teresa and Morio. #E1 = {"name": "calculator", "arguments": {"expr": "71-59"}}
Plan: I need to calculate how old Teresa is when their daughter is born. #E2 = {"name": "calculator", "arguments": {"expr": "38-12"}}

Begin!
Describe your plans with rich details. Each Plan should be followed by only one #E.

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
[["First, I need to calculate Jake's hourly wage. Since Jake earns thrice what Jacob does, and Jacob earns $6 per hour, Jake earns 3 * $6 = $18 per hour.", "#E1", "{\"name\": \"calculator\", \"arguments\": {\"expr\": \"6*3\"}}"]]

Solve the following task or problem. To solve the problem, we have made step-by-step Plan and retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might contain irrelevant information.

Plan: First, I need to calculate Jake's hourly wage. Since Jake earns thrice what Jacob does, and Jacob earns $6 per hour, Jake earns 3 * $6 = $18 per hour.
Evidence: 18

Now solve the question or task according to provided Evidence above. Respond with the answer directly with no extra words.

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
Response: Jake earns $18 per hour. In 5 days, working 8 hours a day, Jake works 5 * 8 = 40 hours. Therefore, Jake earns 40 * $18 = $720 in 5 days.

Answer: $720

The final result of this example is

Result: {"answer": "Jake earns $18 per hour. In 5 days, working 8 hours a day, Jake works 5 * 8 = 40 hours. Therefore, Jake earns 40 * $18 = $720 in 5 days.\n\nAnswer: $720"}

Conditionals and Loops

PDL supports conditionals and loops as illustrated in the following example (file), which implements a chatbot.

description: Chatbot
text:
# Allow the user to type any question, implicitly adding the question to the context.
- read:
  message: "What is your query?\n"
- repeat:
    text:
    # Send context to Granite model hosted at ollama
    - model: ollama_chat/granite3.2:2b
    # Allow the user to type 'yes', 'no', or anything else, storing
    # the input into a variable named `eval`.  The input is also implicitly
    # added to the context.
    - read:
      def: eval
      message: "\nIs this a good answer[yes/no]?\n"
    - "\n"
    # If the user only typed "no", prompt the user for input to add to the context.
    - if: ${ eval == 'no' }
      then:
        text:
        - read:
          message: "Why not?\n"
  # If the user typed only "yes", finish the `repeat` and end the program
  until: ${ eval == 'yes'}

The first block prompts the user for a query, and this is contributed to the background context. The next block is a repeat-until, which repeats the contained text block until the condition in the until becomes true. The field repeat can contain a string, or a block, or a list. If it contains a list, then the list must be a text, array or lastOf (which means that all the blocks in the list are executed and the result of the body is that of the last block).

The example also shows the use of an if-then-else block. The if field contains a condition, the then field can also contain either a string, or a block, or a list (and similarly for else).

The chatbot keeps looping by making a call to a model, asking the user if the generated text is a good answer, and asking why not? if the answer (stored in variable eval) is no. The loop ends when eval becomes yes. This is specified with a Jinja expression on line 18.

Notice that the repeat and then blocks are followed by text. This is because of the semantics of lists in PDL. If we want to aggregate the result by stringifying every element in the list and collating them together, then we need the keyword text to precede a list. The number of iterations of a loop can be bounded by adding a maxIterations field.

The way that the result of each iteration is collated with other iterations can be customized in PDL using the join feature (see the following section).

Another simple example of using an if statement is this.

For Loops

PDL also offers for loops over lists. The following example stringifies and outputs each number.

description: for loop creating a string
for:
  i: [1, 2, 3, 4]
repeat: 
  ${ i }

This program outputs:

To output a number of each line, we can specify which string to use to join the results.

description: for loop with new lines between iterations
for:
  i: [1, 2, 3, 4]
repeat: 
  ${ i }
join:
  with: "\n"

To create an array as a result of iteration, we would write:

description: Array comprehension
for:
  i: [1, 2, 3, 4]
repeat:
  ${ i }
join:
  as: array

which outputs the following list:

[1, 2, 3, 4]

To retain only the result of the last iteration of the loop, we would write:

description: Loop where the result is the result of the last iteration
for:
  i: [1, 2, 3, 4]
repeat:
  ${ i }
join:
  as: lastOf

which outputs:

When join is not specified, the collation defaults to

join:
  as: text
  with: ""

meaning that result of each iteration is stringified and concatenated with that of other iterations. When using with, as: text can be elided.

Note that join can be added to any looping construct (repeat) not just for loops.

The for loop construct also allows iterating over 2 or more lists of the same length simultaneously:

description: for loop over multiple lists
defs:
  numbers:
    data: [1, 2, 3, 4]
  names:
    data: ["Bob", "Carol", "David", "Ernest"]
for:
  number: ${ numbers }
  name: ${ names }
repeat:
  "${ name }'s number is ${ number }\n"

This results in the following output:

Bob's number is 1
Carol's number is 2
David's number is 3
Ernest's number is 4

The loop constructs also allow to build an object:

description: for loop creating an object
defs:
  numbers:
    data: [1, 2, 3, 4]
  names:
    data: ["Bob", "Carol", "David", "Ernest"]
for:
  number: ${ numbers }
  name: ${ names }
repeat:
  data:
    ${ name }: ${ number }
join:
  as: object

This results in the following output:

{"Bob": 1, "Carol": 2, "David": 3, "Ernest": 4}

Other loops

The following example shows a while loop in PDL:

defs:
  i: 0
while: ${ i < 3 }
repeat:
  defs:
    i: ${i + 1}
  text: ${i}

The while field indicates the looping condition and repeat contains the body of the loop.

This loop can be rewritten using maxIterations to bound the number of iterations and index to name the iteration number.

index: i
repeat:
  text: ${i + 1}
maxIterations: 3

Notice that for, while, until, and maxIterations can all be combined in the same repeat block. The loop exits as soon as one of the exit conditions is satisfied:

description: repeat loop with multiple conditions
defs:
  numbers:
    data: [42, 2, 4012, 27]
  names:
    data: ["Bob", "Carol", "David", "Ernest"]
for:
  number: ${ numbers }
  name: ${ names }
index: i
repeat:
  "${i}: ${ name }'s number is ${ number }\n"
until: ${ name == "Carol"}
maxIterations: 3

Match block

PDL provides a match block for convenience. Consider the example. This shows retrieved RAG documents that are then submitted with a query to a RAG Granite model. The output contains an answer to the query together with hallucination score and possibly a citation.

To obtain and install the Granite model locally follow these instructions.

The end of this program contains a match block:

...
  The answer is: ${ out[0].sentence }
- match: ${out[0].meta.hallucination_level}
  with:
  - case: "high"
    then: Totally hallucinating, sorry!
  - case: "low"
    if: ${ out[0].meta.citation }
    then: |
      I am not hallucinating, promise!
      The citation is: ${ out[0].meta.citation.snippet }
  - then: Not sure if I am hallucinating...

The match field indicates an expression to match on. The cases follow the with field. Additional conditions can be indicated as shown in the second case.

Roles and Chat Templates

Consider again the chatbot example (file). By default blocks have role user, except for model call blocks, which have role assistant. If we write roles explicitly for the chatbot, we obtain:

description: chatbot
text:
- read:
  message: "What is your query?\n"
  contribute: [context]
- repeat:
    text:
    - model: replicate/ibm-granite/granite-3.1-8b-instruct
      role: assistant
    - read:
      def: eval
      message: "\nIs this a good answer[yes/no]?\n"
      contribute: []
    - if: ${ eval == 'no' }
      then:
        text:
        - read:
          message: "Why not?\n"
  until: ${ eval == 'yes'}
role: user

In PDL, any block can be adorned with a role field indicating the role for that block. These are high-level annotations that help to make programs more portable across different models. If the role of a block is not specified (except for model blocks that have assistant role), then the role is inherited from the surrounding block. So in the above example, we only need to specify role: user at the top level (this is the default, so it doesn't need to be specified explicitly).

PDL takes care of applying appropriate chat templates (done either in LiteLLM or at the server side).

The prompt that is actually submitted to the first model call (with query What is APR?) is the following:

<|start_of_role|>user<|end_of_role|>What is APR?<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>

To change the template that is applied, you can specify it as a parameter of the model call:

model: replicate/ibm-granite/granite-3.1-8b-instruct
parameters:
  roles:
    system:
       pre_message: <insert text here>
       post_message: <insert text here>
    user:
       pre_message: <insert text here>
       post_message: <insert text here>
    assistant:
       pre_message: <insert text here>
       post_message: <insert text here>

Type Checking

Consider the following PDL program (file). It first reads the data found here to form few-shot examples. These demonstrations show how to create some JSON data.

# Expected not to type check
description: Creating JSON Data
defs:
  data:
    read: type_checking_data.yaml
    parser: yaml
    spec: { questions: [string], answers: [object] }
text:
  - model: ollama_chat/granite3.2:2b
    def: model_output
    spec: {name: string, age: integer}
    input:
      array:
      - role: user
        content:
          text:
          - for:
              question: ${ data.questions }
              answer: ${ data.answers }
            repeat: |
              ${ question }
              ${ answer }
          - >
            Question: Generate only a JSON object with fields 'name' and 'age' and set them appropriately. Write the age all in letters. Only generate a single JSON object and nothing else.
    parser: yaml
    parameters:
      stop: ["Question"]
      temperature: 0

Upon reading the data we use a parser to parse it into a YAML. The spec field indicates the expected type for the data, which is an object with 2 fields: questions and answers that are an array of string and an array of objects, respectively. When the interpreter is executed, it checks this type dynamically and throws errors if necessary.

Similarly, the output of the model call is parsed as YAML, and the spec indicates that we expect an object with two fields: name of type string, and age of type integer.

When we run this program, we obtain the output:

{'name': 'John', 'age': '30'}
type_checking.pdl:9 - Type errors during spec checking:
type_checking.pdl:9 - twentyfive should be of type <class 'int'>

Notice that since we asked the age to be produced in letters, we got a string back and this causes a type error indicated above.

In general, spec definitions can be a subset of JSON schema, or use a shorthand notation as illustrated by the examples below:

boolean or {type: boolean}: boolean
string or {type: string}: string
integer or {type: integer}: integer
number or {type: number}: floating point numbers
"null" or {type: "null"}: type of the null value
array or {type: array}: array with elements of any type
object or {type: object}: object with any fields
{type: string, pattern: '^[A-Za-z][A-Za-z0-9_]*$'}: a string satisfying the indicated pattern
{type: number, minimum: 0, exclusiveMaximum: 1}: a float satisfying the indicated constraints
[integer]: an array of integers
{type: array, items: { type: integer }}: same as above
[{ type: integer, minimum: 0}]: a list of integers satisfying the indicated constraints
{type: array, items: { type: integer, minimum: 0}}: same as above
{type: array, minItems: 1}, a list with at least one element
{latitude: number, longitude: number}: an object with fields latitude and longitude
{object: {latitude: number, longitude: number}}: same as above
{question: string, answer: string, context: {optional: string}}: an object with an optional field context
{object: {question: string, answer: string, context: {optional: string}}}: same as above
[{question: string, answer: string}]: a list of objects
{enum: [red, green, blue]}: an enumeration

Another example of type checking a list can be found here.

Structured Decoding

When a type is specified in a PDL block, it is used for structured decoding with models that support it. The fields guided_json and response_format are added automatically by the interpreter with a JSON Schema value obtained from the type. Models on platforms that support structured decoding will then use this to generate JSON of the correct format.

The following program:

text:
- role: system
  text: You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
  contribute: [context]
- "\nWhat is the color of the sky? Write it as JSON\n"
- model: watsonx/ibm/granite-34b-code-instruct
  parser: json
  spec: { color: string }

produces the output:

What is the color of the sky?
{'color': 'blue'}

Python SDK

PDL programs can be defined and called programmatically directly in Python. In the following example, the PDL program is defined as a string and then parsed and executed using the exec_str function (file).

from pdl.pdl import exec_str

HELLO = """
text:
- >+
  Hello
- model: ollama_chat/granite3.2:2b
  parameters:
    stop: ['!']
"""


def main():
    result = exec_str(HELLO)
    print(result)


if __name__ == "__main__":
    main()

The SDK also provides functions to execute programs defined in a file (see example), as a Python dictionary (see example), or PDL abstract syntax tree defined by a Pydantic data structure (see example). The documentation of the API is available here.

A way to handle the processing of large datasets using PDL is to use Python multiprocessing capabilities to launch multiple instances of the PDL interpreter. The example below, w are using the Python's concurrent.futures.ProcessPoolExecutor to execute in parallel multiple instances of the PDL program HELLO where the free variable name is instantiated with a different value for each instance (file).

import concurrent.futures

from pdl.pdl import exec_str

HELLO = """
text:
- >+
  Hello, my name is ${name}
- model: ollama_chat/granite3.2:2b
"""


def _run_agent(name):
    pdl_output = exec_str(
        HELLO,
        scope={"name": name},
        config={
            "yield_result": False,
            "yield_background": False,
            "batch": 1,  # disable streaming
        },
    )
    return pdl_output


if __name__ == "__main__":
    data = ["Alice", "Nicolas", "Rosa", "Remi"]
    with concurrent.futures.ProcessPoolExecutor() as executor:
        futures = {executor.submit(_run_agent, name) for name in data}
        executor.map(_run_agent, data)
        for future in concurrent.futures.as_completed(futures):
            try:
                result = future.result()
            except Exception as e:
                print(f"Task raised an exception: {e}")
            else:
                print(result)

Finally, it possible to interleave the use of Python and PDL. You can find an example here of a Python application which is using a function defined in PDL which itself depend on the Python application.

Debugging PDL Programs

We highly recommend to edit PDL programs using an editor that support YAML with JSON Schema validation. For example, you can use VSCode with the YAML extension and configure it to use the PDL schema. The PDL repository has been configured so that every *.pdl file is associated with the PDL grammar JSONSchema (see settings). This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion. The PDL interpreter also provides similar error messages. To make sure that the schema is associated with your PDL files, be sure that PDL Schemas appear at the bottom right of your VSCode window, or on top of the editor window.

PDL provides a graphical user experience to help with debugging, program understanding and live programming. You may install this via brew install pdl on MacOS. For other platforms, downloads are available here. You may also kick the tires with a web version of the GUI here.

To generate a trace for use in the GUI:

pdl --trace <file.json> <my-example.pdl>

This is similar to a spreadsheet for tabular data, where data is in the forefront and the user can inspect the formula that generates the data in each cell. In the Live Document, cells are not uniform but can take arbitrary extents. Clicking on them similarly reveals the part of the code that produced them.

Finally, PDL includes experimental support for gathering trace telemetry. This can be used for debugging or performance analysis, and to see the shape of prompts sent by LiteLLM to models.

For more information see here.

Using Ollama models

Install Ollama e.g., brew install --cask ollama
Run a model e.g., ollama run granite-code:8b. See the Ollama library for more models
An OpenAI style server is running locally at http://localhost:11434/, see the Ollama blog for more details.

Example:

text:
- Hello,
- model: ollama_chat/granite-code:8b
  parameters:
    stop:
    - '!'
    decoding_method: greedy

If you want to use an external Ollama instance, the env variable OLLAMA_API_BASE should be defined, by default is http://localhost:11434.

Alternatively, one could also use Ollama's OpenAI-style endpoint using the openai/ prefix instead of ollama_chat/. In this case, set the OPENAI_API_BASE, OPENAI_API_KEY, and OPENAI_ORGANIZATION (if necessary) environment variables. If you were using the official OpenAI API, you would only have to set the api key and possibly the organization. For local use e.g., using Ollama, this could look like so:

export OPENAI_API_BASE=http://localhost:11434/v1
export OPENAI_API_KEY=ollama # required, but unused
export OPENAI_ORGANIZATION=ollama # not required

pdl <...>

Strings in Yaml

Multiline strings are commonly used when writing PDL programs. There are two types of formats that YAML supports for strings: block scalar and flow scalar formats. Scalars are what YAML calls basic values like numbers or strings, as opposed to complex types like arrays or objects. Block scalars have more control over how they are interpreted, whereas flow scalars have more limited escaping support. (Explanation in this section are based on yaml-multiline.info by Wolfgang Faust.)

Block Scalars

Block Style Indicator: The block style indicates how newlines inside the block should behave. If you would like them to be kept as newlines, use the literal style, indicated by a pipe |. Note that without a chomping indicator, described next, only the last newline is kept.

PDL:

text:
  - |
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."

Output:

Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.
End.

If instead you want them to be replaced by spaces, use the folded style, indicated by a right angle bracket >. To get a newline using the folded style, leave a blank line by putting two newlines in. Lines with extra indentation are also not folded.

PDL:

text:
  - >
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."

Output:

Several lines of text, with some "quotes" of various 'types', and also a blank line:
and some text with
    extra indentation
on the next line, plus another line at the end.
End.

Block Chomping Indicator: The chomping indicator controls what should happen with newlines at the end of the string. The default, clip, puts a single newline at the end of the string. To remove all newlines, strip them by putting a minus sign - after the style indicator. Both clip and strip ignore how many newlines are actually at the end of the block; to keep them all put a plus sign + after the style indicator.

PDL:

text:
  - |-
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."

Output:

Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.End.

PDL:

text:
  - |+
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."

Output:

Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.


End.

If you don't have enough newline characters using the above methods, you can always add more like so:

text:
  - |-
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "\n\n\n\n"
  - "End."

Output:

Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.



End.

Indentation Indicator: Ordinarily, the number of spaces you're using to indent a block will be automatically guessed from its first line. You may need a block indentation indicator if the first line of the block starts with extra spaces. In this case, simply put the number of spaces used for indentation (between 1 and 9) at the end of the header.

PDL:

text:
  - |1
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line.

Output:

 Several lines of text,
 with some "quotes" of various 'types',
 and also a blank line:

 and some text with
     extra indentation
 on the next line.

Flow Scalars

Single-quoted:

PDL:

text: 'Several lines of text,
  containing ''single quotes''. Escapes (like \n) don''t do anything.

  Newlines can be added by leaving a blank line.
    Leading whitespace on lines is ignored.'

Output:

Several lines of text, containing 'single quotes'. Escapes (like \n) don't do anything.
Newlines can be added by leaving a blank line. Leading whitespace on lines is ignored.

Double-quoted:

PDL:

text: "Several lines of text,
  containing \"double quotes\". Escapes (like \\n) work.\nIn addition,
  newlines can be esc\
  aped to prevent them from being converted to a space.

  Newlines can also be added by leaving a blank line.
    Leading whitespace on lines is ignored."

Output:

Several lines of text, containing "double quotes". Escapes (like \n) work.
In addition, newlines can be escaped to prevent them from being converted to a space.
Newlines can also be added by leaving a blank line. Leading whitespace on lines is ignored.

Plain:

PDL:

text: Several lines of text,
  with some "quotes" of various 'types'.
  Escapes (like \n) don't do anything.

  Newlines can be added by leaving a blank line.
    Additional leading whitespace is ignored.

Output:

Several lines of text, with some "quotes" of various 'types'. Escapes (like \n) don't do anything.
Newlines can be added by leaving a blank line. Additional leading whitespace is ignored.

PDL Language Tutorial

Simple text

Calling an LLM

Parameter defaults for watsonx Granite models

Building the background context with lastOf

Model Chaining

Variable Definition and Use

Local Computation Using defs

Control Block Outputs with contribute

Function Definition

Building Data Structures

Input from File or Stdin

Parsing the output of a block

Calling code

Calling REST APIs

Data Block

Import Block

Prompt Library

Chain-of-Thought

ReAct

Tools

ReWOO

Conditionals and Loops

For Loops

Other loops

Match block

Roles and Chat Templates

Type Checking

Structured Decoding

Python SDK

Debugging PDL Programs

Using Ollama models

Strings in Yaml

Block Scalars

Flow Scalars

Building the background context with `lastOf`

Local Computation Using `defs`

Control Block Outputs with `contribute`