Writing Bytes as Base-N Text

You have a byte block and need to place it in a text field. The field’s format determines how the bytes should look as text: perhaps uppercase hexadecimal, perhaps URL-safe Base64, or perhaps Base64 wrapped into lines.

BaseNEncoder handles the conversion. BaseNFormat describes the alphabet and layout it should write. This page starts with a simple encoding, then shows how to choose and adjust that format. For the underlying idea and a full round trip, see Encoding Byte Data as Base-N Text.

Start with a Byte Block

The encoder takes a ByteBlock. It does not need to know whether those bytes came from a file, a digest, or some other source. If you omit the format argument, it uses canonical padded Base64.

Here the input bytes spell kaiku, the Finnish word for an echo. The example writes their Base64 representation and shows the available output string widths.

text/BaseN/BaseNDemos.cpp

void encodeSoundTag() {
    // Encode the UTF-8 bytes of a short Finnish sound label.
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    const auto encoded = el::base_n::BaseNEncoder{bytes}.toString();
    el::io::printLine("Base64: "_el, encoded);

    // The same encoder can produce UTF-16 and UTF-32 text when that is the destination's string width.
    const auto encoder = el::base_n::BaseNEncoder{bytes};
    el::io::printLine("UTF-16 characters: "_el, encoder.toU16String().characterLength().toSizeT());
    el::io::printLine("UTF-32 characters: "_el, encoder.toU32String().characterLength().toSizeT());
}

Output

Base64: a2Fpa3U=
UTF-16 characters: 8
UTF-32 characters: 8

toString() returns the ordinary UTF-8 String. If the receiving API needs a specific width, use toU8String(), toU16String(), or toU32String(). The Base-N characters stay the same; the destination string type changes. You can inspect the encoder’s selected format through format().

Choose the Alphabet

A standard format factory is the clearest starting point when a protocol names a familiar encoding. BaseNFormat::base16() writes uppercase hexadecimal digits. base32() and base32Hex() select the two RFC 4648 Base32 alphabets. base64() and base64Url() select ordinary and URL-safe Base64. The default constructor and defaultFormat() both select base64().

The next example uses two bytes that expose the difference between the Base64 alphabets. Ordinary Base64 produces +/8=; Base64url produces -_8=. The bytes are identical, but the text follows different transport rules.

text/BaseN/BaseNDemos.cpp

void chooseAlphabet() {
    const auto bytes = el::ByteBlock{el::Byte{0xfbU}, el::Byte{0xffU}};
    const auto base64 = el::base_n::BaseNFormat::base64();
    const auto url = el::base_n::BaseNFormat::base64Url();
    el::io::printLine("Base64: "_el, el::base_n::BaseNEncoder{bytes, base64}.toString());
    el::io::printLine("Base64url: "_el, el::base_n::BaseNEncoder{bytes, url}.toString());

    // A custom Base16 alphabet changes the text representation of the same bytes.
    const auto custom = el::base_n::BaseNFormat{el::U32String{U"FEDCBA9876543210"_el}};
    el::io::printLine("Custom Base16: "_el, el::base_n::BaseNEncoder{bytes, custom}.toString());
    el::io::printLine("Bits per character: "_el, custom.bitsPerCharacter());
}

Output

Base64: +/8=
Base64url: -_8=
Custom Base16: 0400
Bits per character: 4

If another protocol defines its own alphabet, pass a U32String to the BaseNFormat constructor. You can also change an existing format with setAlphabet(). The alphabet must contain exactly 16, 32, or 64 distinct Unicode scalar values. The custom-alphabet constructor starts without padding and accepts ASCII whitespace while decoding.

alphabet() returns the chosen characters. bitsPerCharacter() reports whether each one stands for four, five, or six bits. For inspecting an external format, valueFor() finds the digit value assigned to a character, while characterFor() performs the reverse lookup. These answers follow the selected alphabet; they are not universal Base-N values.

Choose the Padding Character

Some encodings fill their last group with padding. For example, the Base64 representation of kaiku ends with =. The standard Base32 and Base64 format factories use that character.

padding() reports the configured character, and setPadding() changes it. The character must be a valid Unicode scalar outside the alphabet and the decoder’s ignored whitespace set. The example changes it to ~ and confirms that the matching decoder recovers the original bytes.

text/BaseN/BaseNDemos.cpp

void choosePadding() {
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    auto format = el::base_n::BaseNFormat::base64();
    format.setPadding(el::Char{U'~'});
    const auto encoded = el::base_n::BaseNEncoder{bytes, format}.toString();
    el::io::printLine("Alternate padding: "_el, encoded);
    el::io::printLine("Round trip: "_el, el::base_n::BaseNDecoder{encoded, format}.toDataOrThrow() == bytes);
}

Output

Alternate padding: a2Fpa3U~
Round trip: true

The padding character and the decision to emit it are separate settings. EmitPadding tells the encoder to write canonical padding. Clearing that flag produces an unpadded value. If you built a format from a custom alphabet, assign a padding character before enabling this flag.

Control the Format Flags

The BaseNFormatFlag set also contains RequirePadding and WrapLines. RequirePadding affects decoding; WrapLines affects encoded layout. The flags can be selected independently as long as the resulting format is valid.

flags() returns the BaseNFormatFlags set. hasFlag() asks about one flag. setFlags() replaces the entire set, whereas addFlags() and clearFlags() modify a format you already have.

The example removes both padding flags from Base64. The encoder then omits =, and the decoder accepts the unpadded value. Adding RequirePadding again makes that same text invalid for decoding.

text/BaseN/BaseNDemos.cpp

void chooseFlags() {
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    auto format = el::base_n::BaseNFormat::base64();
    format.clearFlags(el::base_n::BaseNFormatFlag::EmitPadding | el::base_n::BaseNFormatFlag::RequirePadding);
    const auto unpadded = el::base_n::BaseNEncoder{bytes, format}.toString();
    el::io::printLine("Unpadded: "_el, unpadded);
    el::io::printLine("Unpadded accepted: "_el, el::base_n::BaseNDecoder{unpadded, format}.toData().has_value());
    format.addFlags(el::base_n::BaseNFormatFlag::RequirePadding);
    el::io::printLine("Padding now required: "_el, el::base_n::BaseNDecoder{unpadded, format}.toData().has_value());
}

Output

Unpadded: a2Fpa3U
Unpadded accepted: true
Padding now required: false

This separation can help when reading an older unpadded format while writing a newer padded form. The receiving side still needs an explicit policy, described in Reading Base-N Text into Bytes.

Wrap Long Output

Some text formats store Base64 as lines rather than one continuous field. BaseNFormat::base64Pem() is the ready-made choice for 64-character lines separated by LF. It supplies the payload layout, without PEM header or footer lines.

For a different layout, call setLineLength(). The length counts encoded characters, including padding. Call setLineSeparator() to choose the text between lines, then enable WrapLines. The lineLength() and lineSeparator() accessors report the chosen values. Without that flag, the stored length and separator do not change the output.

text/BaseN/BaseNDemos.cpp

void wrapLines() {
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    auto format = el::base_n::BaseNFormat::base64();
    format.setLineLength(el::CpCount{4U})
        .setLineSeparator(el::U32String{U"\n"_el})
        .addFlags(el::base_n::BaseNFormatFlag::WrapLines);
    const auto wrapped = el::base_n::BaseNEncoder{bytes, format}.toString();
    el::io::printLine("Wrapped:\n"_el, wrapped);
    el::io::printLine("Round trip: "_el, el::base_n::BaseNDecoder{wrapped, format}.toDataOrThrow() == bytes);
}

Output

Wrapped:
a2Fp
a3U=
Round trip: true

The encoder inserts a separator between lines, not after the last line. With wrapping enabled, the length must be finite and greater than zero, and the separator must be nonempty. Every separator character must also be in the decoder’s ignored whitespace set. That rule lets the same format read the output it writes.

Format setters validate each resulting combination and throw ParameterFailure if it is invalid. Build and validate a format before encoding data, especially when its settings came from configuration.

Complete demo sources

text/BaseN/BaseNDemos.cpp

// Copyright (c) 2026 Tobias Erbsland - https://erbsland.dev
// SPDX-License-Identifier: Apache-2.0

#include <DemoCommon.hpp>
#include <erbsland/base_n/BaseNDecoder.hpp>
#include <erbsland/base_n/BaseNEncoder.hpp>
#include <erbsland/base_n/BaseNFormat.hpp>
#include <erbsland/base_n/BaseNFormatFlag.hpp>
#include <erbsland/Byte.hpp>
#include <erbsland/ByteBlock.hpp>
#include <erbsland/ByteCount.hpp>
#include <erbsland/Char.hpp>
#include <erbsland/CharSet.hpp>
#include <erbsland/CpCount.hpp>
#include <erbsland/StandardStreams.hpp>
#include <erbsland/U32String.hpp>

#include <span>

namespace demo {

/// Carry a small byte value through a text field and recover exactly the original bytes.
///
/// Base-N encoding changes the representation of a `ByteBlock`, not its contents. The encoder and decoder use the
/// same format so the text can cross a text-only boundary without treating the bytes as Unicode characters.
void roundTripSoundTag() {
    // Treat the sound label as bytes that need to pass through a text field.
    const auto original = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    const auto format = el::base_n::BaseNFormat::base64();
    const auto encoded = el::base_n::BaseNEncoder{original, format}.toString();

    // Decode with the same format and compare the recovered bytes.
    const auto recovered = el::base_n::BaseNDecoder{encoded, format}.toDataOrThrow();
    el::io::printLine("Text field: "_el, encoded);
    el::io::printLine("Bytes recovered: "_el, recovered == original);
}

/// Encode a small binary sound label as portable Base64 text.
///
/// `BaseNEncoder` consumes a `ByteBlock` and a matching `BaseNFormat`. The default format is canonical, padded
/// Base64. Choose a specific format factory when the receiving protocol requires another alphabet or layout.
void encodeSoundTag() {
    // Encode the UTF-8 bytes of a short Finnish sound label.
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    const auto encoded = el::base_n::BaseNEncoder{bytes}.toString();
    el::io::printLine("Base64: "_el, encoded);

    // The same encoder can produce UTF-16 and UTF-32 text when that is the destination's string width.
    const auto encoder = el::base_n::BaseNEncoder{bytes};
    el::io::printLine("UTF-16 characters: "_el, encoder.toU16String().characterLength().toSizeT());
    el::io::printLine("UTF-32 characters: "_el, encoder.toU32String().characterLength().toSizeT());
}

/// Decode Base64 with an explicit byte limit and choose how to handle invalid input.
///
/// `toData()` returns an empty optional for malformed input or an exceeded size limit. `toDataOrThrow()` preserves
/// the distinction as a parse or range exception. Both methods check the decoded byte count before producing data.
void decodeSoundTag() {
    // Decode a small value with the largest byte count this field permits.
    const auto decoded = el::base_n::BaseNDecoder{"a2Fpa3U="_el}.toData(el::ByteCount{5U});
    el::io::printLine("Decoded bytes: "_el, decoded->length().toSizeT());

    // A malformed payload and an oversized payload both fail in the optional form.
    const auto invalid = el::base_n::BaseNDecoder{"a2Fpa3U!"_el}.toData();
    const auto tooLarge = el::base_n::BaseNDecoder{"a2Fpa3U="_el}.toData(el::ByteCount{4U});
    el::io::printLine("Malformed accepted: "_el, invalid.has_value());
    el::io::printLine("Oversized accepted: "_el, tooLarge.has_value());
}

/// Choose a predefined Base-N alphabet or supply a distinct 16-, 32-, or 64-character alphabet.
///
/// Each alphabet character represents four, five, or six bits. The encoder and decoder must use the same alphabet:
/// Base64 and Base64url, for example, assign different characters to their last two values.
void chooseAlphabet() {
    const auto bytes = el::ByteBlock{el::Byte{0xfbU}, el::Byte{0xffU}};
    const auto base64 = el::base_n::BaseNFormat::base64();
    const auto url = el::base_n::BaseNFormat::base64Url();
    el::io::printLine("Base64: "_el, el::base_n::BaseNEncoder{bytes, base64}.toString());
    el::io::printLine("Base64url: "_el, el::base_n::BaseNEncoder{bytes, url}.toString());

    // A custom Base16 alphabet changes the text representation of the same bytes.
    const auto custom = el::base_n::BaseNFormat{el::U32String{U"FEDCBA9876543210"_el}};
    el::io::printLine("Custom Base16: "_el, el::base_n::BaseNEncoder{bytes, custom}.toString());
    el::io::printLine("Bits per character: "_el, custom.bitsPerCharacter());
}

/// Configure the padding character separately from the policy for emitting and requiring it.
///
/// A padding character must be outside both the alphabet and the ignored whitespace set. A custom alphabet starts
/// without padding; assign a character before enabling either padding flag.
void choosePadding() {
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    auto format = el::base_n::BaseNFormat::base64();
    format.setPadding(el::Char{U'~'});
    const auto encoded = el::base_n::BaseNEncoder{bytes, format}.toString();
    el::io::printLine("Alternate padding: "_el, encoded);
    el::io::printLine("Round trip: "_el, el::base_n::BaseNDecoder{encoded, format}.toDataOrThrow() == bytes);
}

/// Select exactly which characters a decoder may ignore between Base-N digits.
///
/// Standard formats accept ASCII whitespace. Replacing that set can make a protocol-specific separator legal while
/// ordinary spaces become invalid. The alphabet, padding, and whitespace sets must remain disjoint.
void chooseWhitespace() {
    auto format = el::base_n::BaseNFormat::base64();
    format.setWhitespace(el::CharSet{U'~'});
    el::io::printLine("Tilde accepted: "_el, el::base_n::BaseNDecoder{"a2Fp~a3U="_el, format}.toData().has_value());
    el::io::printLine("Space accepted: "_el, el::base_n::BaseNDecoder{"a2Fp a3U="_el, format}.toData().has_value());
}

/// Change the three independent Base-N behavior flags.
///
/// `EmitPadding` changes encoded output, `RequirePadding` changes decoding policy, and `WrapLines` changes output
/// layout. `setFlags()` replaces all flags; `addFlags()` and `clearFlags()` adjust an existing format.
void chooseFlags() {
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    auto format = el::base_n::BaseNFormat::base64();
    format.clearFlags(el::base_n::BaseNFormatFlag::EmitPadding | el::base_n::BaseNFormatFlag::RequirePadding);
    const auto unpadded = el::base_n::BaseNEncoder{bytes, format}.toString();
    el::io::printLine("Unpadded: "_el, unpadded);
    el::io::printLine("Unpadded accepted: "_el, el::base_n::BaseNDecoder{unpadded, format}.toData().has_value());
    format.addFlags(el::base_n::BaseNFormatFlag::RequirePadding);
    el::io::printLine("Padding now required: "_el, el::base_n::BaseNDecoder{unpadded, format}.toData().has_value());
}

/// Accept an unpadded Base64 field only when its protocol allows that spelling.
///
/// Clearing `RequirePadding` permits a missing final padding character. Padding that is present must still be
/// canonical, so adding an incorrect number of padding characters does not become valid.
void decodePaddingPolicy() {
    auto format = el::base_n::BaseNFormat::base64();
    el::io::printLine(
        "Default accepts a2Fpa3U: "_el, el::base_n::BaseNDecoder{"a2Fpa3U"_el, format}.toData().has_value());

    format.clearFlags(el::base_n::BaseNFormatFlag::RequirePadding);
    el::io::printLine(
        "Optional padding accepts a2Fpa3U: "_el, el::base_n::BaseNDecoder{"a2Fpa3U"_el, format}.toData().has_value());
    el::io::printLine(
        "Incorrect padding accepted: "_el, el::base_n::BaseNDecoder{"a2Fpa3U=="_el, format}.toData().has_value());
}

/// Wrap encoded output by setting a line length, separator, and `WrapLines` flag.
///
/// The line length counts encoded characters, including padding. Each separator character must also be accepted
/// whitespace during decoding. Wrapping inserts no separator after the final line.
void wrapLines() {
    const auto bytes = el::ByteBlock::fromSpan(std::span<const char>{"kaiku", 5U});
    auto format = el::base_n::BaseNFormat::base64();
    format.setLineLength(el::CpCount{4U})
        .setLineSeparator(el::U32String{U"\n"_el})
        .addFlags(el::base_n::BaseNFormatFlag::WrapLines);
    const auto wrapped = el::base_n::BaseNEncoder{bytes, format}.toString();
    el::io::printLine("Wrapped:\n"_el, wrapped);
    el::io::printLine("Round trip: "_el, el::base_n::BaseNDecoder{wrapped, format}.toDataOrThrow() == bytes);
}

}