Tutorial — rellm

Exhibit I

Build an agent.

An agent is created with AgentBuilder. Required: a provider (LM Studio, OpenRouter, OpenAI, or your own implementation) and a conversation to store history. Everything else has a sensible default.

main.go · a minimal agent on LM Studio

package main

import (
	"context"
	"fmt"

	"github.com/dbedla/rellm/pkg/rellm"
)

func main() {
	provider, err := rellm.NewLMStudioProvider(
		"google/gemma-4-26b-a4b", "http://127.0.0.1", "1234",
	)
	if err != nil {
		panic(err)
	}

	agent, err := rellm.NewAgentBuilder().
		WithProvider(provider).
		WithAgentName("MyAgent").
		WithMaxAgentSteps(20).
		WithConversation(rellm.NewInMemoryConversation()).
		WithImageGenerationKeepInTheLoop().
		WithUnknownConversationElementKeepInTheLoop().
		WithSystemMessage("You are a helpful assistant.").
		Build()
	if err != nil {
		panic(err)
	}

	finalReport, err := agent.Ask(context.Background(), "Hello!")
	if err != nil {
		panic(err)
	}
	fmt.Println(finalReport.Message)
}

Ask is the minimal entry point for a plain-text question. For finer control — temperature, reasoning effort, output caps — build a prompt with PromptBuilder and call Execute:

prompt.go · inference parameters per prompt

prompt, err := rellm.NewPromptBuilder().
	WithMessage("Write a haiku about tide pools.").
	WithTemperature(0.7).
	WithMaxOutputTokens(100).
	Build()
if err != nil {
	log.Fatal(err)
}
finalReport, err := agent.Execute(ctx, prompt)
answer := finalReport.Message

The full runnable version lives in examples/endpoint_agent, which shows how to switch providers with flags. A Report carries the final message, every generated image, and one usage stat per provider call — fresh data for each Ask or Execute, never mixed with earlier runs.

Exhibit II

Give the agent hands.

Tools define how the agent can interact with your system. Implement the Toolset interface and register it with WithToolset on the builder:

pkg/rellm/agent.go · the Toolset interface

type Toolset interface {
    // Definitions returns the tool definitions advertised to the model.
    Definitions() []rellm.ToolDefinition

    // Dispatch executes a tool call requested by the model.
    Dispatch(ctx context.Context, name string, arguments json.RawMessage) (rellm.ToolCallResult, error)
}

Keep the functionality separate from the toolset implementation. Write your logic as a plain struct with methods, then wrap it in a thin Toolset that only translates between the model and your code. The Calculator sample in pkg/toolsets shows the split:

calculator.go · your functionality: no rellm types involved

type Calculator struct{}

func (c *Calculator) Add(a, b float64) float64 { return a + b }
func (c *Calculator) Sub(a, b float64) float64 { return a - b }
func (c *Calculator) Mul(a, b float64) float64 { return a * b }

calculatortoolset.go · a thin adapter that exposes Calculator to the agent

type CalculatorToolset struct {
    impl *Calculator
}

func NewCalculatorToolset(c *Calculator) *CalculatorToolset {
    return &CalculatorToolset{impl: c}
}

var _ rellm.Toolset = (*CalculatorToolset)(nil)

func (t *CalculatorToolset) Definitions() []rellm.ToolDefinition {
    return []rellm.ToolDefinition{
        {
            Type:        "function",
            Name:        "Calculator_Add",
            Description: "Add two numbers.",
            Parameters: map[string]any{
                "type": "object",
                "properties": map[string]any{
                    "a": map[string]any{"type": "number"},
                    "b": map[string]any{"type": "number"},
                },
                "required": []string{"a", "b"},
            },
        },
        // ...Calculator_Sub and Calculator_Mul follow the same pattern
    }
}

func (t *CalculatorToolset) Dispatch(_ context.Context, name string, args json.RawMessage) (rellm.ToolCallResult, error) {
    var a struct {
        A, B float64
    }
    if err := json.Unmarshal(args, &a); err != nil {
        return rellm.ToolCallResult{Err: err}, nil
    }

    switch name {
    case "Calculator_Add":
        return rellm.ToolCallResult{Value: t.impl.Add(a.A, a.B)}, nil
    case "Calculator_Sub":
        return rellm.ToolCallResult{Value: t.impl.Sub(a.A, a.B)}, nil
    case "Calculator_Mul":
        return rellm.ToolCallResult{Value: t.impl.Mul(a.A, a.B)}, nil
    default:
        return rellm.ToolCallResult{}, fmt.Errorf("unknown tool: %s", name)
    }
}

Why the split pays off Calculator is tested as ordinary Go code — no model, no JSON schema, no agent in the loop. The toolset adapter is the only part that needs a Toolset-level test. A Toolset is simple, repetitive work — one definition and one case per method — and works best when generated by a coding agent: give it your struct, let it produce the adapter, then review and test the result.

Register it on the builder, choosing whether the model may call tools in parallel (ParallelToolCallsDisable forces one call per response for toolsets with ordering-dependent side effects):

registering a toolset

agent, err := rellm.NewAgentBuilder().
	WithProvider(provider).
	WithConversation(rellm.NewInMemoryConversation()).
	WithToolset(toolsets.NewCalculatorToolset(&toolsets.Calculator{}), rellm.ParallelToolCallsEnable).
	Build()

A dispatched tool error does not break the agentic loop — put the error in ToolCallResult.Err and the model gets it back as feedback. See the complete agent in examples/lms_math_agent.

Exhibit III

Pin down the output.

Ask for JSON that conforms to a schema instead of free text. Two lines do most of the work: reflect a JSON schema straight from a Go struct, and describe each field with a jsonschema tag so the contract is readable to the model.

main.go · structured output via OpenRouter

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/invopop/jsonschema"
	"github.com/joho/godotenv"

	"github.com/dbedla/rellm/pkg/rellm"
)

// The struct tags describe the schema.
type Person struct {
	Name string `json:"name" jsonschema:"description=Full name of the person"`
	Age  int    `json:"age"  jsonschema:"description=Age in years"`
	City string `json:"city" jsonschema:"description=City of residence"`
}

func main() {
	// Schema generated from the struct tags above.
	textFormat := rellm.TextFormat{
		Type:   "json_schema",
		Name:   "person",
		Strict: true,
		Schema: (&jsonschema.Reflector{DoNotReference: true}).Reflect(&Person{}),
	}

	_ = godotenv.Load() // reads OPENROUTER_API_KEY from .env
	apiKey := os.Getenv("OPENROUTER_API_KEY")
	if apiKey == "" {
		panic("OPENROUTER_API_KEY is not set")
	}

	provider, err := rellm.NewOpenRouterProvider(apiKey, "openai/gpt-5.6-luna")
	if err != nil {
		panic(err)
	}

	const sysMsg = `You are an assistant that extracts structured information
from user text and returns only valid JSON matching the requested schema.`

	agent, err := rellm.NewAgentBuilder().
		WithProvider(provider).
		WithAgentName("StructuredOutputAgent").
		WithMaxAgentSteps(20).
		WithConversation(rellm.NewInMemoryConversation()).
		WithImageGenerationKeepInTheLoop().
		WithUnknownConversationElementKeepInTheLoop().
		WithSystemMessage(sysMsg).
		WithTextFormat(textFormat).
		Build()
	if err != nil {
		panic(err)
	}

	prompt, err := rellm.NewPromptBuilder().
		WithMessage("I am John Snow from Winterfell, I have 100 years...").
		Build()
	if err != nil {
		panic(err)
	}

	finalReport, err := agent.Execute(context.Background(), prompt)
	if err != nil {
		panic(err)
	}

	var person Person
	err = json.Unmarshal([]byte(finalReport.Message), &person)
	if err != nil {
		panic(err)
	}
	fmt.Printf("after unmarshal to struct: %+v\n", person)
}

Local models (LM Studio) a small model served by LM Studio may ignore the text.format field. If the output drifts, embed a stringified copy of the schema in the system prompt with an extra instruction to return raw JSON only (no markdown) — see e2e_tests/e2e_format_text_agent/agent.go.

Build an agent, step by step.

Build an agent.

Give the agent hands.

Pin down the output.