Text Matching and Splitting

String Pattern

Introduction

StringPattern is a lightweight pattern matcher for decoded string characters. It is intended for small internal checks such as protocol or prefix/suffix matching. The parsed syntax supports literal text, ? for one decoded character, [a-z] character sets, and one * divider. The divider separates front and back matching parts; it is not a repeated wildcard.

Parsed patterns support backslash escapes for ?, *, [, ] and \. Typed construction uses the erbsland::text::pattern element namespace to build immutable compiled pattern data directly without parsing pattern syntax.

Example

using namespace el::text::literals;

const auto parsed = el::StringPattern{"http?://*"_el};
const auto ok = parsed.matches("https://example.test"_el);

using namespace el::text::pattern;

static const auto staticPattern = el::StringPattern{
    Text{U"http"},
    OneChar{},
    Text{U"://"},
    Divider{}};

Fuzzy Matching

Introduction

The text::fuzzy namespace ranks candidate strings by Damerau-Levenshtein edit distance. Matching operates on decoded Unicode code points, and an adjacent transposition counts as one edit. It is suitable for short suggestion lists, command names, option names, and other user-entered identifiers.

Matcher

Matcher stores the reference pattern, an optional maximum distance, an optional result limit, and a CharCompareFn. Distance and result limits are unbounded by default, and matching is case-sensitive unless a comparison callback is configured.

findMatches() accepts a StringList and returns a MatchList ordered by distance and then original candidate order. Equivalent candidates are deduplicated using the configured comparison callback.

Match Results

Match exposes the original candidate and its CpLength edit distance. MatchList is an util::List of matches.

Example

auto matcher = el::text::fuzzy::Matcher{"verbsoe"_el};
matcher.setMaximumDistance(el::CpLength{2U}).setMaximumResults(el::ItemCount{3U});
const auto matches = matcher.findMatches(el::StringList{"verbose"_el, "version"_el, "quiet"_el});

String Splitters

String splitters read an owning Core string sequentially and return copy-on-write slices without copying their text. StringSplitter is the common UTF-8 alias. The U8StringSplitter, U16StringSplitter and U32StringSplitter variants provide the same interface for each supported width.

Separators

A splitter accepts either one Char or a CharSet. Each call to next() reads through the next matching separator. Malformed encoded data is handled tolerantly like the underlying string type.

StringSplitMode::DiscardSeparator returns only the text between separators. Consecutive separators therefore return empty parts, and a trailing separator produces a final empty part. StringSplitMode::KeepSeparator includes the separator at the end of each part. In this mode a trailing separator completes the preceding part without producing another empty part, which is useful for line-oriented processing.

Sequential State

isAtEnd() distinguishes an empty part from the end of the sequence. Calling next() after the end safely returns an empty string. Calling skip() consumes the same next part as next() without constructing its shared slice; calling it after the end has no effect. remaining() returns the unread suffix as another shared slice, and reset() restarts the splitter at the beginning.

An empty source has one empty part. This preserves ordinary split semantics and lets a caller observe the source once before isAtEnd() becomes true.

The UTF-8 implementation has a direct byte-search path for a single ASCII separator. ASCII bytes cannot occur inside a UTF-8 multibyte sequence, so this optimization preserves tolerant decoding behavior. Other separator sets use decoded character matching. The width-specific backends can use independent search optimizations while keeping the same public contract.

Interface

class Match

One fuzzy text match and its edit distance.

Public Functions

Match() = default: Create an empty match.

inline Match(text::String text, unit::CpLength distance) noexcept

Create a match.

Parameters:

text – The matching candidate text.
distance – The Damerau-Levenshtein edit distance.

inline const text::String &text() const noexcept: Get the matching candidate text.

inline const text::String &candidate() const noexcept: Get the matching candidate text.

inline unit::CpLength distance() const noexcept: Get the edit distance from the pattern.

using erbsland::text::fuzzy::MatchList = util::List<Match>: A ranked list of fuzzy text matches.

class Matcher

Find text candidates using bounded Damerau-Levenshtein distance.

Matching operates on decoded Unicode code points. Adjacent transpositions count as one edit. Results are ordered by distance and retain candidate order for ties.

Public Functions

Matcher() = default: Create a matcher for an empty pattern.

inline explicit Matcher(text::String pattern) noexcept

Create a matcher for a pattern.

Parameters:: pattern – The text to compare with candidates.

inline const text::String &pattern() const noexcept: Get the pattern.

Matcher &setPattern(text::String pattern) noexcept: Set the pattern.

inline unit::CpLength maximumDistance() const noexcept: Get the maximum accepted edit distance.

Matcher &setMaximumDistance(unit::CpLength maximumDistance) noexcept: Set the maximum accepted edit distance.

inline unit::ItemCount maximumResults() const noexcept: Get the maximum number of returned matches.

Matcher &setMaximumResults(unit::ItemCount maximumResults) noexcept: Set the maximum number of returned matches.

inline CharCompareFn comparisonFn() const noexcept: Get the optional decoded-character comparison function.

Matcher &setComparisonFn(CharCompareFn comparisonFn) noexcept: Set the decoded-character comparison function. Empty selects exact comparison.

MatchList findMatches(const text::StringList &candidates) const

Find matching candidates.

Parameters:: candidates – Candidate text in preferred tie order.
Returns:: Accepted, deduplicated matches ordered by distance and candidate order.

struct Divider

Split the pattern into front and back matching parts.

struct OneChar

Match one decoded character.

class Set

A set element made of one or more ranges.

Public Functions

constexpr Set() noexcept = default: Create an empty set.

inline constexpr Set(std::initializer_list<Range> ranges): Create a set from a list of ranges.

inline explicit Set(const CharSet &charSet): Create a set from an existing character set.

inline constexpr std::size_t count() const noexcept: The number of ranges in this set.

inline constexpr CharRange range(const std::size_t index) const noexcept: Access the range at the given index.

class Range

A character range element.

Public Functions

constexpr Range() noexcept = default: Create an empty range.

inline constexpr Range(const Char first, const Char last) noexcept: Create a range from two characters.

inline constexpr Range(const char32_t first, const char32_t last) noexcept: Create a range from two code points.

inline explicit constexpr Range(const CharRange range) noexcept: Create a range from an existing character range.

inline constexpr CharRange charRange() const noexcept: Access the range.

class Text

Non-owning UTF-32 text element for static pattern construction.

Public Functions

constexpr Text() noexcept = default: Create an empty text element.

template<std::size_t N> inline explicit constexpr Text(const char32_t (&data)[N]) noexcept: Create a text element from a UTF-32 literal.

inline explicit constexpr Text(const std::u32string_view text) noexcept: Create a text element from a UTF-32 read-only string.

inline constexpr std::u32string_view view() const noexcept: Access the literal text.

class StringPattern

A lightweight decoded-character string pattern.

See: Text Matching and Splitting

Public Functions

StringPattern() noexcept = default: Create a pattern that never matches.

explicit StringPattern(const U8String &pattern): Parse a pattern from a UTF-8 string.

explicit StringPattern(const U16String &pattern): Parse a pattern from a UTF-16 string.

explicit StringPattern(const U32String &pattern): Parse a pattern from a UTF-32 string.

template<pattern::AnyElement... Args> explicit StringPattern(const Args&... elements): Create a typed pattern from pattern elements.

bool matches(const U8String &text) const noexcept: Test if this pattern matches the front or divided front/back parts of the text.

bool matches(const U16String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool matches(const U32String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool trim(U8String &text) const noexcept: Trim the matching part from the view if the pattern matches.

bool trim(U16String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool trim(U32String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool trim(U8StringEditor &text) const: Trim the matching part from the string if the pattern matches.

bool trim(U16StringEditor &text) const: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool trim(U32StringEditor &text) const: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

U8String trimmed(const U8String &text) const noexcept: Return the trimmed view, or the original view if there is no match.

U16String trimmed(const U16String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

U32String trimmed(const U32String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

std::pair<U8String, U8String> split(const U8String &text) const noexcept: Split the text at the matching pattern boundary.

std::pair<U16String, U16String> split(const U16String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

std::pair<U32String, U32String> split(const U32String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

unit::ByteLength length(const U8String &text) const noexcept: Return the native length of the matching text, or zero if there is no match.

unit::U16DataLength length(const U16String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

unit::CpLength length(const U32String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

unit::ByteIndex index(const U8String &text) const noexcept: Return the native split index, or no-index if there is no match.

unit::U16DataIndex index(const U16String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

unit::CpIndex index(const U32String &text) const noexcept: This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Public Static Attributes

static constexpr auto cMaximumElements = pattern::cMaximumElements: Maximum number of elements in a typed pattern.

static constexpr auto cMaximumRanges = pattern::cMaximumStaticRanges: Maximum number of total ranges in a typed pattern.

static constexpr auto cMaximumSetRanges = pattern::cMaximumSetRanges: Maximum number of ranges in one set element.

enum class erbsland::text::StringSplitMode : uint8_t

Select how a string splitter handles separator characters.

Values:

enumerator DiscardSeparator: Exclude the separator from each returned part.

enumerator KeepSeparator: Include the separator at the end of each returned part.

typedef U8StringSplitter erbsland::text::StringSplitter: The common sequential UTF-8 string splitter.

typedef impl::StringSplitter<U16String> erbsland::text::U16StringSplitter: A sequential UTF-16 string splitter.

typedef impl::StringSplitter<U32String> erbsland::text::U32StringSplitter: A sequential UTF-32 string splitter.

typedef impl::StringSplitter<U8String> erbsland::text::U8StringSplitter: A sequential UTF-8 string splitter.