Examples#

Kork allows providing examples in the prompt.

  1. Examples help the LLM understand the syntax of the language

  2. Examples can be dynamically generated based on the user query, to surface the most relevant programs.

The format of examples is a sequence of 2-tuples of the form: (str, Program).

Let’s see how to specify them.

The Simplest Way#

from kork.parser import parse

examples_as_strings = [
    (
        "declare a variable called `y` and assign to it the value 8",
        "var y = 8",
    )
]

examples = [(query, parse(code)) for query, code in examples_as_strings]

examples

[('declare a variable called `y` and assign to it the value 8',
  Program(stmts=(VarDecl(name='y', value=Literal(value=8)),)))]

The Hardest Way#

Let’s import AST elements and write an example query and a corresponding short program.

from kork.ast import Program, VarDecl, Literal

examples = [
    (
        "declare a variable called `y` and assign to it the value 8",
        Program(stmts=[VarDecl("y", Literal(value=8))]),
    )
]

Formatted Example#

The chain formats the example using an AstPrinter to format the examples as a sequence of the 2-tuple of (str, str).

Here’s what happens:

from kork.examples import format_examples
from kork import AstPrinter

print(format_examples("SmirkingCat", examples, AstPrinter()))

[('declare a variable called `y` and assign to it the value 8', '```SmirkingCat\nvar y = 8\n```')]

print(
    format_examples(
        "SmirkingCat", examples, AstPrinter(), input_formatter="text_prefix"
    )
)

[('Text: """\ndeclare a variable called `y` and assign to it the value 8\n"""', '```SmirkingCat\nvar y = 8\n```')]

The Hardest Way Made Easier#

There are utility functions that make it easier to generate programs that invoke foreign functions – which is expected to be a common pattern.

from kork import c_, r_

?c_

Signature: c_(name: Callable, *args: Any) -> kork.ast.FunctionCall
Docstring: Create a kork function call.
File:      ~/src/kork/kork/examples.py
Type:      function

?r_

Signature: r_(expr: kork.ast.Expr) -> kork.ast.Program
Docstring: Assign last program expression to a result variable.
File:      ~/src/kork/kork/examples.py
Type:      function

import math
import operator

Here’s a small program that invokes the foreign function log10 with arg 2

c_(math.log10, 2)

FunctionCall(name='log10', args=[Literal(value=2)])

A program that invokes log10 with arg of 2 and stores the result in a result variable in the global namespace.

r_(c_(math.log10, 2))

Program(stmts=[VarDecl(name='result', value=FunctionCall(name='log10', args=[Literal(value=2)]))])

program = r_(c_(operator.add, 2, c_(math.log10, 3)))

print(AstPrinter().visit(program))

var result = add(2, log10(3))

examples = [("take the log base 10 of 3 and add 2 to the result.", program)]