json-transformer

Why

Go's encoding/json gives you no control over the JSON output once your struct is defined. Real-world systems frequently need to change the JSON shape without touching the data model:

• A service receives camelCase JSON from a frontend but must forward snake_case to a downstream API.
• An API gateway needs to mask sensitive fields (passwords, tokens) before logging.
• A data pipeline reads NDJSON, renames fields to match a new schema, and streams the result to another sink.

The standard workarounds all have significant costs:

Workaround	Problem
Second struct with different tags	Doubles type definitions; breaks with dynamic schemas
json.Unmarshal → rename → json.Marshal	Two full encode/decode passes; entire document loaded into memory
map[string]any manipulation	Verbose, error-prone, loses type information
Custom MarshalJSON per type	Couples serialisation logic to the domain model

json-transformer solves this in one pass: rename keys and transform values as the JSON flows through, without loading the full document into memory.

Design

One pass, no round-trips. When the input is a Go value (map, struct, slice), the library walks it directly and writes JSON with no intermediate marshal step. When the input is raw JSON ([]byte, string, io.Reader), a token-based decoder drives the output writer in lockstep — the document is never fully decoded into memory.

Zero config, zero cost. A Transformer with no options short-circuits to a direct pass-through. You pay only for what you configure.

Concurrency-safe. All per-call state is local to each invocation. One Transformer can be shared freely across goroutines.

Follows encoding/json conventions. Struct tags (json:"name,omitempty", json:"-", anonymous embedding) are respected. Map key order is non-deterministic, consistent with the standard library. Large numbers are preserved exactly via json.Number with no float64 precision loss.

Installation

go get github.com/longbridge/json-transformer

Quick start

import jsontransform "github.com/longbridge/json-transformer"

t := jsontransform.New(
    jsontransform.WithRenameFunc(jsontransform.SnakeCaseRename()),
)

out, err := t.TransformBytes(map[string]any{"userName": "Alice", "userAge": 30})
// out: {"user_name":"Alice","user_age":30}

API

Creating a transformer

t := jsontransform.New(opts ...Option) *Transformer

Option	Description
WithRenameFunc(fn RenameFunc)	Called for every object key. Return nil to keep the original name, or *string with the new name.
WithValueTransformer(fn func(string) ValueTransformFunc)	Called once per unique field name. Return a ValueTransformFunc to transform that field's value, or nil to leave it unchanged.
WithPathRenameFunc(fn PathRenameFunc) / WithPathValueTransformer(fn func(string) ValueTransformFunc)	Same, but matched by root-anchored dot path instead of bare name. See Path-aware callbacks.
WithKeepRenamedOriginal()	Emit renamed fields twice: the original name keeps the input value, the new name gets the transformed one. See Keeping the original field.

Choosing a method

Method	Use when
TransformBytes(src any) ([]byte, error)	You need the result as []byte. The most common case.
Transform(src any, dst io.Writer) error	You already have an io.Writer to write into (HTTP response, file, network connection). Also the right choice when src is an io.Reader.
TransformStream(r io.Reader, w io.Writer) error	Your input contains multiple JSON values in sequence (NDJSON). Each value is transformed and written on its own line. Use Transform for single-value input.

out, err := t.TransformBytes(src)       // → []byte
err      := t.Transform(src, w)         // → io.Writer
err      := t.TransformStream(r, w)     // NDJSON: multiple values

Input dispatch

Transform and TransformBytes accept any of the following as src:

Type	Behaviour
map[string]any	Fast path — traverses the map directly, no marshal/unmarshal
struct / *struct	Fast path — reflection with cached field metadata
slice / array	Fast path
[]byte / string / io.Reader	Streaming path — parses JSON tokens without loading the full document
Primitives (int, bool, …)	Encoded directly with encoding/json

Built-in rename functions

jsontransform.SnakeCaseRename()                          // "UserName"  → "user_name"
jsontransform.CamelCaseRename()                          // "user_name" → "userName"
jsontransform.PascalCaseRename()                         // "user_name" → "UserName"
jsontransform.KebabCaseRename()                          // "UserName"  → "user-name"
jsontransform.MapRename(map[string]string{"id": "ID"})   // exact lookup; returns nil if not found

MapRename returns nil for unknown keys, so it composes naturally with a fallback:

exact    := jsontransform.MapRename(map[string]string{"id": "ID"})
fallback := jsontransform.SnakeCaseRename()

t := jsontransform.New(
    jsontransform.WithRenameFunc(func(name string) *string {
        if s := exact(name); s != nil {
            return s
        }
        return fallback(name)
    }),
)

Examples

Rename fields on a struct

type User struct {
    UserName string `json:"userName"`
    UserAge  int    `json:"userAge"`
}

t := jsontransform.New(jsontransform.WithRenameFunc(jsontransform.SnakeCaseRename()))
out, _ := t.TransformBytes(User{UserName: "Alice", UserAge: 30})
// {"user_name":"Alice","user_age":30}

Rename a key and mask its value

WithValueTransformer receives the original field name. Return a transform function for fields you want to handle, or nil to leave the value unchanged. Renaming and value transformation are independent and can be combined freely.

t := jsontransform.New(
    jsontransform.WithRenameFunc(func(name string) *string {
        if name == "pwd" {
            s := "password"
            return &s
        }
        return nil
    }),
    jsontransform.WithValueTransformer(func(name string) jsontransform.ValueTransformFunc {
        if name == "pwd" {
            return func(v any) any { return "***" }
        }
        return nil
    }),
)

out, _ := t.TransformBytes(map[string]any{"user": "alice", "pwd": "secret"})
// {"user":"alice","password":"***"}

Transform values by pattern

Because WithValueTransformer gives you the field name, you can match by any pattern rather than a fixed string:

t := jsontransform.New(
    jsontransform.WithValueTransformer(func(name string) jsontransform.ValueTransformFunc {
        if strings.HasSuffix(name, "_at") {
            return func(v any) any {
                // reformat Unix timestamp → RFC3339
                if n, ok := v.(json.Number); ok {
                    ts, _ := n.Int64()
                    return time.Unix(ts, 0).UTC().Format(time.RFC3339)
                }
                return v
            }
        }
        return nil
    }),
)

out, _ := t.TransformBytes(map[string]any{"created_at": 1700000000, "name": "Alice"})
// {"created_at":"2023-11-14T22:13:20Z","name":"Alice"}

Stream a large JSON file

t := jsontransform.New(jsontransform.WithRenameFunc(jsontransform.CamelCaseRename()))

f, _ := os.Open("large.json")
defer f.Close()
t.Transform(f, os.Stdout)

Process NDJSON

t := jsontransform.New(jsontransform.WithRenameFunc(jsontransform.SnakeCaseRename()))

r := strings.NewReader(`{"userId":1}` + "\n" + `{"userId":2}`)
t.TransformStream(r, os.Stdout)
// {"user_id":1}
// {"user_id":2}

Path-aware callbacks

WithPathRenameFunc / WithPathValueTransformer match fields by their root-anchored dot-joined path instead of the bare name. Array levels are transparent (no segment). A path callback returning nil falls back to the name-based callback; returning "" omits the field.

tr := jsontransform.New(
    jsontransform.WithPathRenameFunc(func(path string) *string {
        if path == "data.list" { s := "items"; return &s }
        return nil
    }),
    jsontransform.WithPathValueTransformer(func(path string) jsontransform.ValueTransformFunc {
        if path == "data.list.ts" {
            return func(v any) any { /* convert */ return v }
        }
        return nil
    }),
)
// paths always refer to the ORIGINAL input structure: renaming data.list
// does not stop data.list.ts from matching.

Keeping the original field

WithKeepRenamedOriginal makes renaming additive: every renamed field is emitted twice — first under its original name with the input value untouched, then under the new name with the transform applied. Use it to add a converted field to a response without breaking consumers that still read the old one.

tr := jsontransform.New(
    jsontransform.WithRenameFunc(jsontransform.MapRename(map[string]string{"counter_id": "symbol"})),
    jsontransform.WithValueTransformer(func(name string) jsontransform.ValueTransformFunc {
        if name == "counter_id" {
            return func(v any) any { return toSymbol(v) }
        }
        return nil
    }),
    jsontransform.WithKeepRenamedOriginal(),
)

// {"counter_id":"ST/HK/00700"} → {"counter_id":"ST/HK/00700","symbol":"700.HK"}

The retained field is verbatim: nested rename and value rules apply only to the renamed copy, so it keeps exactly the shape the input had.

Three cases are deliberately left alone:

• A field whose name does not change is emitted once — an in-place value transform cannot keep both values under one key. If you need the old value to survive, rename to a new field instead of transforming in place.
• A field omitted by a rename callback (rename to "") stays omitted. Omission is a deletion, not a rename.
• Renaming two sibling fields onto each other's names still yields duplicate keys, exactly as it does without this option. The library never deduplicates output keys.

Notes

• Field name matching (both WithValueTransformer and RenameFunc) always uses the original field name, before any renaming is applied. The lookup function passed to WithValueTransformer is called at most once per unique field name; results are cached for the lifetime of the Transformer.
• Value transformer input when processing JSON text ([]byte/string/io.Reader): values arrive as string, json.Number, bool, nil, map[string]any, or []any. When processing Go values directly, the original Go type is passed.
• Value transformer output is encoded as-is. It is not subject to further renaming or transformation.
• Map key order is non-deterministic, consistent with encoding/json.
• Large numbers in JSON text are preserved exactly via json.Number; there is no float64 precision loss.

Benchmarks

Measured on Windows amd64, Intel Core i9-12900K, Go 1.24.

BenchmarkTransformMap          ~487 ns/op     1 alloc/op
BenchmarkTransformStruct       ~340 ns/op     2 allocs/op
BenchmarkTransformParallel     ~199 ns/op     1 alloc/op   (24 goroutines)
BenchmarkTransformBytes        ~899 ns/op    13 allocs/op  ([]byte input, fastjson path)
BenchmarkTransformNoOp          ~36 ns/op     1 alloc/op   (pass-through baseline, Writer)
BenchmarkTransformLargeJSON    ~975 ms/op   ~108 MB/s      (100 MB JSON array, []byte input)

The []byte/string/io.Reader path is driven by fastjson's arena parser. The fast path (in-memory Go values) has no token-allocation overhead at all.

Run on your own machine:

go test -bench=. -benchmem ./...

License

MIT — see LICENSE.