String Attributes

A string has more than one meaningful notion of size and identity. Its native encoded length may differ from its number of decoded code points, and two values with identical text may or may not refer to the same backing storage. Even a question as simple as “is this valid text?” has a different cost from asking whether the value is empty.

This page helps you choose the attribute that answers the question you actually have. It uses the common UTF-8 String in its examples, but the same model applies to the UTF-16 and UTF-32 families, their editors, and literals. The table gives you a quick view of the costs before the following sections explore why they differ.

Question	Method	Cost model
Does this string contain any stored text?	`isEmpty()`	Constant time.
Is the stored UTF data well-formed?	`isValidUtf8()`, `isValidUtf16()`, or `isValidUtf32()`	Scans the stored data.
How large is the native storage range?	`length()`	Constant time.
How many decoded code points does it contain?	`characterLength()`	Scans UTF-8 and UTF-16.
Which hash represents the decoded text?	`toHash()` or `toHashCI()`	Scans decoded code points.
Does an index belong to the same visible storage range?	`storageId()`	Constant time.

Empty Text and Well-formed Encodings

Use isEmpty() when you need to know whether a string contains any stored data. This is clearer than comparing a length to zero, and it keeps the code independent of whether the string is stored as UTF-8 bytes, UTF-16 code units, or UTF-32 code points.

Encoding validation is a different question. Use isValidUtf8() for UTF-8 strings, isValidUtf16() for UTF-16 strings, and isValidUtf32() for UTF-32 strings. Validation has to inspect the complete stored range, so it is proportional to the length of the input.

Most string algorithms in Erbsland Core handle invalid encoding data deterministically. When a malformed sequence is decoded, the string APIs use the Unicode replacement character instead of walking into undefined behavior. This makes ordinary processing robust, but it does not replace validation at trust boundaries.

For security-sensitive input, validate the encoding when text enters your application. Then validate the allowed character set with containsOnly() before treating the text as an identifier, path component, protocol token, or command.

/// Create a string that deliberately contains invalid UTF-8 bytes.
///
/// This is only for demonstrating error handling. Do not construct strings this
/// way in application code.
auto createTextWithInvalidUtf8() -> el::String {
    constexpr auto bytes = std::array<const char, 15>{
        'S', 'u', 'n', 'n', 'y', ' ', static_cast<char>(0x82U), ' ', 'W', 'e', 'a', 't', 'h', 'e', 'r'};

    return el::String{std::string_view{bytes.data(), bytes.size()}};
}

/// This demo shows basic state checks for `String`.
///
/// It demonstrates:
/// - checking whether a string is empty,
/// - checking whether its byte data is valid UTF-8,
/// - safely printing strings even when invalid UTF-8 is present.
///
/// Invalid UTF-8 data is handled safely. Encoding errors are represented with
/// the Unicode replacement character U+FFFD.
void basicTests() {
    const auto station = el::String{"🌦️ Station Süd: Nebel über dem Fjord"_el};
    const auto empty = el::String{};
    const auto invalidUtf8 = createTextWithInvalidUtf8();

    el::io::printLine("Input strings:"_el);
    el::io::printLine("  station     : \""_el, station, "\""_el);
    el::io::printLine("  empty       : \""_el, empty, "\""_el);
    el::io::printLine("  invalidUtf8 : \""_el, invalidUtf8, "\""_el);

    // `isEmpty()` checks whether the string contains no bytes.
    el::io::printLine("\nEmpty checks:"_el);
    el::io::printLine("  station.isEmpty()     : "_el, station.isEmpty());
    el::io::printLine("  empty.isEmpty()       : "_el, empty.isEmpty());
    el::io::printLine("  invalidUtf8.isEmpty() : "_el, invalidUtf8.isEmpty());

    // `isValidUtf8()` checks whether all bytes form valid UTF-8 sequences.
    el::io::printLine("\nUTF-8 validity checks:"_el);
    el::io::printLine("  station.isValidUtf8()     : "_el, station.isValidUtf8());
    el::io::printLine("  empty.isValidUtf8()       : "_el, empty.isValidUtf8());
    el::io::printLine("  invalidUtf8.isValidUtf8() : "_el, invalidUtf8.isValidUtf8());
}

Input strings:
  station     : "🌦️ Station Süd: Nebel über dem Fjord"
  empty       : ""
  invalidUtf8 : "Sunny � Weather"

Empty checks:
  station.isEmpty()     : false
  empty.isEmpty()       : true
  invalidUtf8.isEmpty() : false

UTF-8 validity checks:
  station.isValidUtf8()     : true
  empty.isValidUtf8()       : true
  invalidUtf8.isValidUtf8() : false

Two Lengths That Answer Different Questions

length() returns the native storage length. For UTF-8 strings, this is a ByteLength. For UTF-16 strings, this is a U16DataLength. For UTF-32 strings, this is a CpLength, because one UTF-32 storage element is one decoded code point.

Native length is the right value for slicing, reserving storage, reporting byte sizes, and working with native indexes. It is stored with the string range and is therefore fast to query.

characterLength() returns the number of decoded Unicode code points as CpLength. For UTF-8 and UTF-16 text this requires decoding the string because one character may use multiple storage units. For large strings, avoid calling it repeatedly inside loops. If an algorithm needs to inspect each character anyway, count while iterating or use StringCharReader.

/// `length()` returns the native storage length, while `characterLength()`
/// returns the decoded Unicode code-point length.
///
/// UTF-8 counts bytes, UTF-16 counts 16-bit data units, and UTF-32 counts code
/// points directly. For non-ASCII text these values often differ. Use the native
/// length for storage ranges and indexes, and use the character length only when
/// a user-visible code-point count is the actual question.
void lengthAttributes() {
    const auto u8Reading = el::String{"温度計📡: 21℃"_el};
    const auto u16Reading = el::U16String{u"温度計📡: 21℃"_el};
    const auto u32Reading = el::U32String{U"温度計📡: 21℃"_el};
    const auto asciiLabel = el::String{"sensor-21"_el};

    el::io::printLine("Measurement label: "_el, u8Reading);
    el::io::printLine("UTF-8 native length ....: "_el, u8Reading.length());
    el::io::printLine("UTF-8 code points ......: "_el, u8Reading.characterLength());
    el::io::printLine("UTF-16 native length ...: "_el, u16Reading.length());
    el::io::printLine("UTF-16 code points .....: "_el, u16Reading.characterLength());
    el::io::printLine("UTF-32 code points .....: "_el, u32Reading.length());

    el::io::printLine();
    el::io::printLine("ASCII label: "_el, asciiLabel);
    el::io::printLine("ASCII bytes ............: "_el, asciiLabel.length());
    el::io::printLine("ASCII code points ......: "_el, asciiLabel.characterLength());
}

Measurement label: 温度計📡: 21℃
UTF-8 native length ....: 20
UTF-8 code points ......: 9
UTF-16 native length ...: 10
UTF-16 code points .....: 9
UTF-32 code points .....: 9

ASCII label: sensor-21
ASCII bytes ............: 9
ASCII code points ......: 9

The different length types are intentionally not interchangeable. This prevents accidental comparisons such as “byte length equals character length” from becoming a hidden Unicode bug.

Hash the Text, Not Its Encoding

Use toHash() when a string is used as a key in hash-based containers. The hash is calculated from decoded code points, not from raw storage bytes. UTF-8, UTF-16, and UTF-32 strings that contain the same decoded text therefore produce the same regular hash value.

Use toHashCI() when the matching rule is Unicode simple case-folded text. Pair it with a matching case-folded comparison rule. Do not mix case-insensitive hashes with exact comparisons because equal hash buckets alone do not define equality.

The library also provides std::hash specializations for the string and string-view types. Those specializations call toHash() and therefore use exact decoded text.

/// This demo shows how to make use of `String`s hash functions.
void hashing() {
    // Here we create a set of different read-only strings.
    // For real code use the literals directly, like `const auto x = "abc"_el;`
    const auto titlecase = el::String{"Fichte"_el};
    const auto lowercase = el::String{"fichte"_el};
    const auto greek = el::String{"Σκιά"_el};
    const auto greekLower = el::String{"σκιά"_el};

    el::io::printLine("titlecase ............: "_el, titlecase);
    el::io::printLine("lowercase ............: "_el, lowercase);
    el::io::printLine("greek ................: "_el, greek);
    el::io::printLine("greekLower ...........: "_el, greekLower);
    el::io::printLine();

    // String hashes let you use read-only strings in sets and unordered maps.
    el::io::printLine("\nRegular Hash Values:"_el);
    auto hash = titlecase.toHash();
    el::io::printLine("  titlecase.toHash()     → "_el, hashToString(hash));
    hash = lowercase.toHash();
    el::io::printLine("  lowercase.toHash()     → "_el, hashToString(hash));
    hash = greek.toHash();
    el::io::printLine("  greek.toHash()         → "_el, hashToString(hash));
    hash = greekLower.toHash();
    el::io::printLine("  greekLower.toHash()    → "_el, hashToString(hash));

    el::io::printLine("\nCase-Insensitive Hash Values:"_el);
    hash = titlecase.toHashCI();
    el::io::printLine("  titlecase.toHashCI()   → "_el, hashToString(hash));
    hash = lowercase.toHashCI();
    el::io::printLine("  lowercase.toHashCI()   → "_el, hashToString(hash));
    hash = greek.toHashCI();
    el::io::printLine("  greek.toHashCI()       → "_el, hashToString(hash));
    hash = greekLower.toHashCI();
    el::io::printLine("  greekLower.toHashCI()  → "_el, hashToString(hash));
}

titlecase ............: Fichte
lowercase ............: fichte
greek ................: Σκιά
greekLower ...........: σκιά


Regular Hash Values:
  titlecase.toHash()     → 0x29b5533011ce1d3c
  lowercase.toHash()     → 0x29b554d5e70ddac6
  greek.toHash()         → 0x00028253e6edaba4
  greekLower.toHash()    → 0x00028253df6bdd02

Case-Insensitive Hash Values:
  titlecase.toHashCI()   → 0x29b554d5e70ddac6
  lowercase.toHashCI()   → 0x29b554d5e70ddac6
  greek.toHashCI()       → 0x00028253df6bdd02
  greekLower.toHashCI()  → 0x00028253df6bdd02

Know Whether a Position Belongs to This Storage

Use storageId() only for low-level code that stores native indexes outside the string object. The returned StorageIdentifier identifies the visible storage range of the string or view. It changes when a string detaches, reallocates, or when a view selects a different range.

This matters because a native index is meaningful only for the storage it came from. A ByteIndex into a UTF-8 view must not be reused with another UTF-8 view just because the text looks similar. The same applies to U16DataIndex for UTF-16 strings.

Store the storage identifier together with the native index when an index outlives the immediate operation:

/// `storageId()` lets low-level code verify that a cached native index still
/// belongs to the same visible storage range.
///
/// This is useful when a byte index outlives the immediate operation that
/// produced it. Even when two strings contain the same decoded text, a native
/// index from one storage range must not be applied to another one.
void storageIdentifier() {
    struct CachedRange final {
        el::StorageIdentifier storageId;
        el::ByteIndex index;
        el::ByteLength length;
    };

    const auto report = el::String{"温度計A: 21℃; 気圧計B: 1012hPa; 湿度計C: 45%"_el};
    const auto token = el::String{"気圧計"_el};
    const auto cachedToken = CachedRange{report.storageId(), report.find(token), token.length()};
    const auto booleanFormat = el::BooleanFormat::yesNo();

    const auto tryUseCachedRange = [&](const el::String &label, const el::String &candidate) -> void {
        const auto sameStorage = candidate.storageId() == cachedToken.storageId;
        el::io::printLine(label, ":"_el);
        el::io::printLine("  same visible storage range: "_el, booleanFormat, sameStorage);
        if (sameStorage && !cachedToken.index.isNoIndex()) {
            el::io::printLine(
                "  cached range reads ........: "_el,
                candidate.slice(el::ByteRange{cachedToken.index, cachedToken.length}));
        } else {
            el::io::printLine("  cached range reads ........: <not used>"_el);
        }
    };

    const auto copiedReport = report.copy();
    const auto tail = report.slice(el::ByteRange{report.find(token), el::ByteLength::infinite()});

    el::io::printLine("Report: "_el, report);
    el::io::printLine("Cached token: "_el, token);
    el::io::printLine("Cached byte index: "_el, cachedToken.index);
    el::io::printLine();

    tryUseCachedRange("Original string"_el, report);
    tryUseCachedRange("Copied text"_el, copiedReport);
    tryUseCachedRange("Tail slice"_el, tail);
}

Report: 温度計A: 21℃; 気圧計B: 1012hPa; 湿度計C: 45%
Cached token: 気圧計
Cached byte index: 19

Original string:
  same visible storage range: yes
  cached range reads ........: 気圧計
Copied text:
  same visible storage range: no
  cached range reads ........: <not used>
Tail slice:
  same visible storage range: no
  cached range reads ........: <not used>

For ordinary string processing, prefer keeping indexes local, using StringCharReader, or using slices returned by the string API. Those patterns avoid manual storage identity checks.