Ollygarden otel go setup

Go SDK Setup Conventions

Setup Checklist — verify every item before you finish

Setup is not done when the SDK boots. Each unchecked item below produces a specific telemetry-quality finding in production; work through all of them.

• Thread the request context into the data layer. Every database, HTTP-client, and queue call on a request path must receive the request's ctx — never context.Background() and never a bare global handle. With GORM this means db.WithContext(ctx) at every query site, with the context passed handler → service → model:

  // handler: take ctx from the framework request
  articles, err := svc.ListArticles(c.Request.Context(), filter)
  
  // service: every method takes ctx as its first parameter
  func (s *Service) ListArticles(ctx context.Context, f Filter) ([]Article, error) {
      return s.repo.List(ctx, f)
  }
  
  // repo/model: the only layer touching *gorm.DB
  func (r *Repo) List(ctx context.Context, f Filter) ([]Article, error) {
      var out []Article
      return out, r.db.WithContext(ctx).Where("tag = ?", f.Tag).Find(&out).Error
  }
  

  Verify by auditing every request-path DB call: each must receive the request context — via db.WithContext(ctx), a per-request gorm.Session{Context: ctx}, or database/sql's QueryContext/ExecContext. A quick spot-check is grep -rn "WithContext" --include='*.go' . (zero hits on a GORM codebase is a strong signal the context is not threaded), but the grep alone is not proof — wrappers and reused *gorm.DB handles hide call sites, so walk the request paths. Without the request context, DB spans become detached CLIENT-kind trace roots instead of children of the HTTP span (Root Client Span finding). Refactoring existing ctx-less signatures across layers is part of setup, not optional follow-up.

• Never record SQL parameter values — on any signal. Bound values must not appear in db.query.text, SQL logs, database/sql instrumentation attributes, or custom spans; only ? placeholders are acceptable. With the GORM OTel plugin specifically, pass tracing.WithoutQueryVariables(). Raw values in any of these leak PII (Critical PII Leakage finding).

• Configure the SDK declaratively, not in code. Exporters, processors, sampling, and signal wiring live in an external YAML file (configs/otel.yaml) parsed with otelconf — use the Setup Pattern below, not hand-constructed exporter/provider code. Operators must be able to change the telemetry setup without recompiling, and the app must fall back to no-op providers when the file is absent.

• Inject service.instance.id (a per-process UUID) alongside service.version, as the setup pattern below does programmatically (Missing service.instance.id finding).

• Keep the resource lean. service.name, service.version, service.instance.id, and deployment.environment.name — that is the full set. Do not add resource.WithOS(), resource.WithProcess(), resource.WithHost(), or equivalent detectors: os.* and process.* resource attributes are discouraged (Discouraged Resource Attribute finding).

• Honor the standard OTEL_* environment variables end-to-end. OTEL_EXPORTER_OTLP_*, OTEL_SERVICE_NAME, and OTEL_RESOURCE_ATTRIBUTES must all take effect at runtime. Do not invent custom environment variables (DEPLOYMENT_ENVIRONMENT, SERVICE_VERSION, ...) for values the standard variables already express, and never overwrite an attribute supplied via OTEL_RESOURCE_ATTRIBUTES with a code-level default — a hardcoded fallback like deployment.environment.name = "development" silently clobbers the deployment's real environment and misfiles every signal the service emits.

Required: Verification Report

Setup is not complete until you produce this report. It is a table with one row per checklist item above. Fill each row with artifacts from THIS run — the marker value you sent, an excerpt of the exported span dump, a trace id, the config value you changed. Never a restatement of the requirement, never a bare "done".

The table below is an illustrative example, not a report you can submit: every value in it is a placeholder showing the expected shape of evidence. Replace every cell with your own run's artifacts. If you did not run a check, write GAP — not run in that row and leave it visible — a missing or hand-waved row is itself a finding.

Example (illustrative values — replace every cell with your own run's evidence):

Item	Check performed	Observed evidence
Context threaded to data layer	traced a request end-to-end and inspected the exported spans	DB span is a CHILD of the HTTP server span (same trace id 4bf9…), no parentless CLIENT-kind roots
No SQL parameter values on any signal	ran a query with marker value MARKER_7f3a, inspected the exported DB span, SQL logs, and DB metrics	db.query.text shows only ? placeholders; MARKER_7f3a appears in no span, log, or metric
SDK configured declaratively	changed a value in configs/otel.yaml (e.g. the sampler ratio) and restarted without recompiling; renamed the file to confirm fallback	new sampling behavior took effect from the file alone; with the file absent the app logged the no-op warning and still ran
service.instance.id injected	dumped the exported resource across two process starts	service.instance.id present as a UUID, and it differs between the two boots
Resource kept lean	dumped the exported resource attributes	exactly service.name, service.version, service.instance.id, deployment.environment.name; no os.* or process.* keys
Standard OTEL_* honored	booted with OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES set to non-defaults, and OTEL_EXPORTER_OTLP_ENDPOINT pointed at a marker collector	service.name/deployment.environment.name carry the supplied values on exported telemetry, with no code-level default overwriting them; the marker collector's log / receipt confirms telemetry arrived at the overridden endpoint (the endpoint is a destination, evidenced there, not on the spans)

A row you cannot fill with observed evidence is a visible gap — that item is not done. Do not delete the row, copy these example values, or write "N/A" to hide it; go run the check and record what you actually saw.

Recommended import path

For new code, use the root otelconf package (go.opentelemetry.io/contrib/otelconf) — it tracks the current schema and includes the propagator-from-YAML fix. The schema-pinned otelconf/v0.3.0 subpackage is for keeping existing configs unchanged.

Project Structure

internal/telemetry/
├── const.go          # Service scope and telemetry constants
├── setup.go          # SDK initialization (code below)
├── providers.go      # Provider management utilities
└── carriers.go       # Custom propagation carriers (if needed)
configs/
└── otel.yaml         # Declarative configuration

Setup Pattern

The core setup reads a YAML config file, injects runtime attributes, and creates an SDK instance that provides all three providers (tracer, meter, logger) plus a propagator.

package telemetry

import (
    "context"
    "errors"
    "fmt"
    "os"

    "github.com/google/uuid"
    "go.opentelemetry.io/contrib/bridges/otelzap"
    otelconf "go.opentelemetry.io/contrib/otelconf"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/log"
    "go.opentelemetry.io/otel/log/global"
    "go.opentelemetry.io/otel/metric"
    "go.opentelemetry.io/otel/propagation"
    semconv "go.opentelemetry.io/otel/semconv/v1.41.0"
    "go.opentelemetry.io/otel/trace"
    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
)

type Providers struct {
    TracerProvider trace.TracerProvider
    MeterProvider  metric.MeterProvider
    LoggerProvider log.LoggerProvider
    Logger         *zap.Logger
    Closer         func(ctx context.Context) error
}

func SetupTelemetry(ctx context.Context, serviceName, version, configFile string) (*Providers, error) {
    providers, sdk, err := providersFromConfig(ctx, serviceName, version, configFile)
    if err != nil {
        return nil, err
    }

    otel.SetTracerProvider(providers.TracerProvider)
    otel.SetMeterProvider(providers.MeterProvider)
    global.SetLoggerProvider(providers.LoggerProvider)

    if sdk != nil {
        otel.SetTextMapPropagator(sdk.Propagator())
    } else {
        otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
            propagation.TraceContext{},
            propagation.Baggage{},
        ))
    }

    return providers, nil
}

func providersFromConfig(ctx context.Context, scope, version, cfgFile string) (*Providers, *otelconf.SDK, error) {
    b, err := os.ReadFile(cfgFile)
    if err != nil {
        if errors.Is(err, os.ErrNotExist) {
            logger := zap.Must(zap.NewProduction())
            logger.Warn("OpenTelemetry config file not found, using no-op providers",
                zap.String("config_file", cfgFile))
            return &Providers{
                TracerProvider: trace.NewNoOpTracerProvider(),
                MeterProvider:  metric.NewNoOpMeterProvider(),
                LoggerProvider: log.NewNoOpLoggerProvider(),
                Logger:         logger,
                Closer:         func(ctx context.Context) error { return nil },
            }, nil, nil
        }
        return nil, nil, fmt.Errorf("failed to read config file %s: %w", cfgFile, err)
    }

    b = []byte(os.ExpandEnv(string(b)))

    conf, err := otelconf.ParseYAML(b)
    if err != nil {
        return nil, nil, err
    }

    if conf.Resource == nil {
        conf.Resource = &otelconf.Resource{}
    }
    if conf.Resource.Attributes == nil {
        conf.Resource.Attributes = []otelconf.AttributeNameValue{}
    }
    conf.Resource.Attributes = insertAttribute(conf.Resource.Attributes,
        string(semconv.ServiceVersionKey), version)
    conf.Resource.Attributes = insertAttribute(conf.Resource.Attributes,
        string(semconv.ServiceInstanceIDKey), uuid.New().String())

    sdk, err := otelconf.NewSDK(
        otelconf.WithContext(ctx),
        otelconf.WithOpenTelemetryConfiguration(*conf),
    )
    if err != nil {
        return nil, nil, err
    }

    core := zapcore.NewTee(
        zapcore.NewCore(
            zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
            zapcore.AddSync(os.Stdout),
            zapcore.InfoLevel,
        ),
        otelzap.NewCore(scope, otelzap.WithLoggerProvider(global.GetLoggerProvider())),
    )

    return &Providers{
        TracerProvider: sdk.TracerProvider(),
        MeterProvider:  sdk.MeterProvider(),
        LoggerProvider: sdk.LoggerProvider(),
        Logger:         zap.New(core),
        Closer:         sdk.Shutdown,
    }, &sdk, nil
}

func insertAttribute(attrs []otelconf.AttributeNameValue, name, value string) []otelconf.AttributeNameValue {
    for _, attr := range attrs {
        if attr.Name == name {
            return attrs
        }
    }
    return append(attrs, otelconf.AttributeNameValue{Name: name, Value: value})
}

Main Integration

func main() {
    ctx := context.Background()

    providers, err := telemetry.SetupTelemetry(ctx,
        telemetry.ServiceName,
        telemetry.ServiceVersion,
        "configs/otel.yaml")
    if err != nil {
        log.Fatalf("Failed to setup telemetry: %v", err)
    }

    defer func() {
        shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
        defer cancel()
        if err := providers.Closer(shutdownCtx); err != nil {
            providers.Logger.Error("Failed to shutdown telemetry", zap.Error(err))
        }
    }()

    tracer := otel.Tracer(telemetry.Scope)
    meter := otel.Meter(telemetry.Scope)

    // Application logic...
}

Key Details

• No-op fallback: If the config file doesn't exist, the setup returns no-op providers instead of failing. The application runs without telemetry.
• Runtime attributes: service.version and service.instance.id are injected programmatically because they vary per deployment, not per environment.
• Zap bridge: The otelzap bridge sends structured logs to the OTel LoggerProvider, enabling log correlation with traces. Stdout JSON output is preserved via a tee.
• 10-second shutdown timeout: Bounds shutdown so a hung exporter cannot block process exit.

Cross-References

• Reference: otel-go skill — references/declarative-setup.md for otelconf fetch table, import path facts, schema version mapping; references/breaking-changes.md for SDK/contrib upgrade audits; references/instrumentation-libraries.md for wiring DB/HTTP/gRPC libraries, threading context.Context into the data layer (avoid detached CLIENT-root DB spans), and keeping PII out of db.query.text.
• General conventions: ollygarden-otel-declarative-config — anti-patterns and common YAML patterns.