Partial Variable Substitution

Partial Variable Substitution

Problem

You need to substitute variables in a prompt template incrementally as data becomes available, rather than requiring all variables at once. This is useful for streaming scenarios or when building prompts dynamically.

Solution

Implement a partial substitution system that allows replacing variables one at a time, tracks which variables have been substituted, and can complete the substitution when all variables are available. This works because prompt templates use placeholders that can be replaced independently.

Why This Matters

Standard Go text/template requires all variables to be present at render time. If any variable is missing, the template either errors or produces empty strings (depending on the missingkey option). This is fine for simple cases, but production prompt construction often involves variables that arrive at different times from different sources: the user’s role from authentication middleware, retrieved documents from a RAG pipeline, and the user’s query from the HTTP request.

Partial substitution decouples variable availability from template rendering. You can fill in variables as they arrive, inspect the template’s current state at any point (useful for debugging), and check which variables are still missing before sending the prompt to the LLM. The Complete() method with defaults provides a safety net: if some variables never arrive (e.g., a retrieval step times out), the template can still be rendered with fallback values rather than failing entirely.

The OTel instrumentation in each method is deliberate. Tracking which variables are substituted and when, along with the value length, creates a trace that shows the prompt construction timeline. This is valuable for diagnosing cases where prompts are unexpectedly empty or malformed — you can see exactly which variable was missing or had an unexpected value.

Code Example

package main

import (
    "context"
    "fmt"
    "regexp"
    "strings"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/trace"
)

var tracer = otel.Tracer("beluga.prompts.partial_substitution")

// PartialPromptTemplate supports incremental variable substitution
type PartialPromptTemplate struct {
    template      string
    substituted   map[string]string
    remainingVars []string
    pattern       *regexp.Regexp
}

// NewPartialPromptTemplate creates a new partial template
func NewPartialPromptTemplate(template string) *PartialPromptTemplate {
    // Extract variable names
    varPattern := regexp.MustCompile(`\{\{\.(\w+)\}\}`)
    matches := varPattern.FindAllStringSubmatch(template, -1)

    remainingVars := []string{}
    seen := make(map[string]bool)
    for _, match := range matches {
        varName := match[1]
        if !seen[varName] {
            remainingVars = append(remainingVars, varName)
            seen[varName] = true
        }
    }

    return &PartialPromptTemplate{
        template:      template,
        substituted:   make(map[string]string),
        remainingVars: remainingVars,
        pattern:       varPattern,
    }
}

// Substitute replaces a variable with a value
func (ppt *PartialPromptTemplate) Substitute(ctx context.Context, varName string, value string) error {
    ctx, span := tracer.Start(ctx, "partial_template.substitute")
    defer span.End()

    span.SetAttributes(
        attribute.String("variable", varName),
        attribute.Int("value_length", len(value)),
    )

    // Check if variable exists
    if !ppt.hasVariable(varName) {
        err := fmt.Errorf("variable %s not found in template", varName)
        span.RecordError(err)
        span.SetStatus(trace.StatusError, err.Error())
        return err
    }

    // Store substitution
    ppt.substituted[varName] = value

    // Remove from remaining
    for i, v := range ppt.remainingVars {
        if v == varName {
            ppt.remainingVars = append(ppt.remainingVars[:i], ppt.remainingVars[i+1:]...)
            break
        }
    }

    span.SetStatus(trace.StatusOK, "variable substituted")
    return nil
}

// GetCurrentPrompt returns the prompt with current substitutions
func (ppt *PartialPromptTemplate) GetCurrentPrompt(ctx context.Context) string {
    ctx, span := tracer.Start(ctx, "partial_template.get_current")
    defer span.End()

    result := ppt.template

    // Replace all substituted variables
    for varName, value := range ppt.substituted {
        placeholder := fmt.Sprintf("{{.%s}}", varName)
        result = strings.ReplaceAll(result, placeholder, value)
    }

    span.SetAttributes(
        attribute.Int("substituted_count", len(ppt.substituted)),
        attribute.Int("remaining_count", len(ppt.remainingVars)),
    )

    return result
}

// IsComplete checks if all variables have been substituted
func (ppt *PartialPromptTemplate) IsComplete() bool {
    return len(ppt.remainingVars) == 0
}

// GetRemainingVariables returns variables that still need substitution
func (ppt *PartialPromptTemplate) GetRemainingVariables() []string {
    return ppt.remainingVars
}

// Complete substitutes all remaining variables with defaults or errors
func (ppt *PartialPromptTemplate) Complete(ctx context.Context, defaults map[string]string) (string, error) {
    ctx, span := tracer.Start(ctx, "partial_template.complete")
    defer span.End()

    // Substitute remaining variables
    for _, varName := range ppt.remainingVars {
        value, exists := defaults[varName]
        if !exists {
            err := fmt.Errorf("no value or default for variable %s", varName)
            span.RecordError(err)
            span.SetStatus(trace.StatusError, err.Error())
            return "", err
        }

        if err := ppt.Substitute(ctx, varName, value); err != nil {
            return "", err
        }
    }

    result := ppt.GetCurrentPrompt(ctx)
    span.SetStatus(trace.StatusOK, "template completed")

    return result, nil
}

// hasVariable checks if a variable exists in the template
func (ppt *PartialPromptTemplate) hasVariable(varName string) bool {
    placeholder := fmt.Sprintf("{{.%s}}", varName)
    return strings.Contains(ppt.template, placeholder)
}

func main() {
    ctx := context.Background()

    // Create template
    template := "Hello {{.name}}, your order {{.orderId}} is ready. Status: {{.status}}"
    ppt := NewPartialPromptTemplate(template)

    // Substitute incrementally
    ppt.Substitute(ctx, "name", "Alice")
    fmt.Println(ppt.GetCurrentPrompt(ctx)) // "Hello Alice, your order {{.orderId}} is ready. Status: {{.status}}"

    ppt.Substitute(ctx, "orderId", "12345")
    fmt.Println(ppt.GetCurrentPrompt(ctx)) // "Hello Alice, your order 12345 is ready. Status: {{.status}}"

    // Complete with defaults
    defaults := map[string]string{"status": "pending"}
    final, _ := ppt.Complete(ctx, defaults)
    fmt.Println(final) // "Hello Alice, your order 12345 is ready. Status: pending"
}

Explanation

1. Variable extraction at construction time — The constructor parses the template once using a regex to find all {{.VarName}} placeholders, building the remainingVars list. This upfront parsing means GetRemainingVariables() is a constant-time lookup rather than re-parsing the template on every call. The seen map prevents duplicate entries when a variable appears multiple times in the template.

2. Incremental substitution with validation — Each Substitute() call verifies the variable exists in the template before storing it. This catches typos early (substituting “userName” when the template uses “name”) rather than silently producing a prompt with unresolved placeholders. The substitution is stored in a map and applied lazily when GetCurrentPrompt() is called, which means multiple substitutions don’t trigger redundant string replacements.

3. Completion with defaults as safety net — The Complete() method iterates over remaining variables and fills them from a defaults map. If any variable has neither a substitution nor a default, it returns an error rather than producing an incomplete prompt. This fail-fast behavior prevents sending malformed prompts to the LLM, which would waste tokens and produce confusing responses.

4. OTel instrumentation — Each substitution and completion operation gets its own span with attributes for the variable name, value length, substituted count, and remaining count. This creates a traceable timeline of prompt construction that is valuable for debugging prompts that produce unexpected LLM responses.

Testing

func TestPartialPromptTemplate_SubstitutesIncrementally(t *testing.T) {
    template := "Hello {{.name}}, status: {{.status}}"
    ppt := NewPartialPromptTemplate(template)

    ppt.Substitute(context.Background(), "name", "Alice")

    result := ppt.GetCurrentPrompt(context.Background())
    require.Contains(t, result, "Alice")
    require.Contains(t, result, "{{.status}}")
    require.False(t, ppt.IsComplete())
}

Variations

Validation on Substitution

Validate values when substituting:

func (ppt *PartialPromptTemplate) SubstituteWithValidation(ctx context.Context, varName string, value string, validator func(string) error) error {
    // Validate before substituting
}

Template Merging

Merge multiple partial templates:

func (ppt *PartialPromptTemplate) Merge(other *PartialPromptTemplate) *PartialPromptTemplate {
    // Combine templates
}

Related Recipes

• Dynamic Message Chain Templates — Build message chains dynamically
• Streaming Tool Calls — Streaming patterns