.. Copyright (c) 2026 Tobias Erbsland - Erbsland DEV. https://erbsland.dev SPDX-License-Identifier: Apache-2.0 .. index:: !single: Finding Text Positions single: StringEditor Search single: Text Search single: Character Search single: find single: findFirstOf single: findFirstNotOf single: findLastOf single: findLastNotOf single: ByteIndex ********************** Finding Text Positions ********************** Before you can extract a field, replace a token, or point to an error, you need a reliable position in the source text. That position should be useful to the next operation as well; otherwise a simple search can quietly lead to another scan of the same Unicode data. Erbsland Core search functions therefore return positions in the string's native storage. For the common UTF-8 :cpp:type:`String `, that means a ``ByteIndex`` already located on a decoded character boundary. This page shows how to carry such positions from searching into slicing, repeated matching, and range extraction without converting coordinates unnecessarily. Begin with the Shape of the Match ================================= Use a character-set search when the boundary is any one of several characters. Use text search when the boundary or marker is a complete string. .. list-table:: :header-rows: 1 :widths: 32 34 34 * - Task - Preferred API - Typical follow-up * - Find the next delimiter, marker character, or line break. - :cpp:func:`findFirstOf() ` - Slice before the delimiter or advance past it. * - Skip padding, separators, or other ignored characters. - :cpp:func:`findFirstNotOf() ` - Start the next token or field at the returned position. * - Find the last delimiter before an end position. - :cpp:func:`findLastOf() ` - Split a path-like or record-like string from the right. * - Ignore trailing padding or separators. - :cpp:func:`findLastNotOf() ` - Trim a search range before slicing. * - Find a complete string. - :cpp:func:`find() ` - Continue with a start position, or pass a comparison function. All search functions return the matching position, or the no-index state of the matching index type when there is no match. Check this state with :cpp:func:`isNoIndex() ` before slicing or reading the character at the result. Keep Search Results in Native Coordinates ========================================= Search functions on UTF-8 strings return :cpp:type:`ByteIndex `. This is deliberate. The returned position can be reused without rescanning the text, and it is already on a decoded character boundary. When you need the character at a found position, call :cpp:func:`charAt(ByteIndex) `. When you need the text before, after, or between positions, construct a :cpp:type:`ByteRange ` and pass it to :cpp:func:`slice(ByteRange) `. Do not add raw byte numbers to move through UTF-8 text unless the value is a byte length that came from the string API itself. Use :cpp:func:`advance() ` and :cpp:func:`retreat() ` to move by decoded characters. Search for a Class of Characters ================================ :cpp:class:`CharSet ` lets one search operation cover a complete group of characters. For example, a delimiter set can contain several punctuation marks, and a padding set can contain all ASCII whitespace. The forward functions start at the given position and include that position in the search. The reverse functions search before the given end position. This makes it easy to ask for the last delimiter before a known marker. .. erbsland-demo:: :source: text/String/FindingCharacterSets.cpp :exec: text/string --demo FindingCharacterSets :source-sha256: eea4b53bf18c7b6fe767d6f493c528d7cd81222248dbe50654f2fc9093aec161 .. code-block:: cpp /// Character-set search functions find positions of decoded characters. /// /// Use findFirstOf and findLastOf to locate delimiters from a set. Use /// findFirstNotOf and findLastNotOf to skip padding or other characters that /// are not part of the useful text. All positions returned by String are /// byte indexes that can be reused for slicing or further searches. void findingCharacterSets() { static const auto padding = el::CharSet::from(el::AsciiCategory::Whitespace); static const auto separators = el::CharSet{U':', U'=', U';'}; const auto logLine = el::String{" Dyklogg: mål=Kosterhavet; djup=240m; status=redo "_el}; const auto contentStart = logLine.findFirstNotOf(padding); const auto contentEnd = logLine.findLastNotOf(padding); const auto firstSeparator = logLine.findFirstOf(separators); const auto lastSeparator = logLine.findLastOf(separators); const auto statusStart = logLine.find("status"_el); const auto separatorBeforeStatus = logLine.findLastOf(separators, statusStart); el::io::printLine("Log line: '"_el, logLine, "'"_el); el::io::printLine("First content byte ........: "_el, contentStart); el::io::printLine("Last content byte .........: "_el, contentEnd); el::io::printLine( "First separator ...........: "_el, firstSeparator, " ('"_el, logLine.charAt(firstSeparator), "')"_el); el::io::printLine( "Last separator ............: "_el, lastSeparator, " ('"_el, logLine.charAt(lastSeparator), "')"_el); el::io::printLine( "Separator before status ...: "_el, separatorBeforeStatus, " ('"_el, logLine.charAt(separatorBeforeStatus), "')"_el); } .. erbsland-ansi:: :escape-char: ␛ Log line: ' Dyklogg: mål=Kosterhavet; djup=240m; status=redo ' First content byte ........: 2 Last content byte .........: 50 First separator ...........: 9 (':') Last separator ............: 46 ('=') Separator before status ...: 38 (';') .. erbsland-demo-end:: Follow a Match Across a Character Run ===================================== A common parser pattern is to define the characters that belong to a token and treat everything else as separation. Use :cpp:func:`findFirstOf() ` to find the next token start, then :cpp:func:`findFirstNotOf() ` to find its end. The end position may be missing when the final token reaches the end of the string. In that case, use :cpp:func:`indexAt(StringSide::Back) ` as the end position. .. erbsland-demo:: :source: text/String/TokenRuns.cpp :exec: text/string --demo TokenRuns :source-sha256: 0cc69bb4c2d51981d55d4c33d5100cbb7aacd863d5f2a9188b7ed0a87424fad7 .. code-block:: cpp /// Combining findFirstOf and findFirstNotOf extracts runs of matching characters. /// /// Define the characters that belong to a token, then search for the next /// matching character and the next non-matching character. Everything outside /// the token set acts as a separator, so punctuation, spaces, and symbols do not /// need separate handling. void tokenRuns() { static const auto tokenCharacters = el::CharSet::from(el::UnicodeCategoryGroup::Letter) | el::CharSet::from(el::UnicodeCategory::DecimalNumber) | el::CharSet{U'-'}; const auto route = el::String{"Rutt: Havsörn-7 går mot djupzon Ålvik; prov=A12; temp=4°C"_el}; el::io::printLine("Route note: "_el, route); el::io::printLine("Tokens:"_el); auto tokenStart = route.findFirstOf(tokenCharacters); while (!tokenStart.isNoIndex()) { auto tokenEnd = route.findFirstNotOf(tokenCharacters, tokenStart); if (tokenEnd.isNoIndex()) { tokenEnd = route.indexAt(el::StringSide::Back); } const auto token = route.slice(el::ByteRange{tokenStart, tokenEnd}); el::io::printLine(" ["_el, tokenStart, ", "_el, tokenEnd, "): "_el, token); tokenStart = route.findFirstOf(tokenCharacters, tokenEnd); } } .. erbsland-ansi:: :escape-char: ␛ Route note: Rutt: Havsörn-7 går mot djupzon Ålvik; prov=A12; temp=4°C Tokens: [0, 4): Rutt [6, 16): Havsörn-7 [17, 21): går [22, 25): mot [26, 33): djupzon [34, 40): Ålvik [42, 46): prov [47, 50): A12 [52, 56): temp [57, 58): 4 [60, 61): C .. erbsland-demo-end:: Continue Searching Without Starting Over ======================================== :cpp:func:`find() ` searches for a complete string. Use the overload with a start position to continue after a previous result. When a loop must find overlapping occurrences, move the start position by one decoded character with :cpp:func:`advance() `. When only non-overlapping occurrences are useful, move the start position by the native length of the text being searched. Guard repeated searches against an empty needle. Searching for empty text succeeds at the start position, so a loop must either reject it or handle it as a special case. Pass a :cpp:type:`CharCompareFn ` when matching should use a different character comparison rule. For example, :cpp:func:`Char::compareCaseFolded() ` performs Unicode simple case folding while the original text stays unchanged. .. erbsland-demo:: :source: text/String/FindingAllOccurrences.cpp :exec: text/string --demo FindingAllOccurrences :source-sha256: 07356ce83e6e56a9cdb1b7bd6a8e1a888067c42417e9a40cf92ce3eb0436940d .. code-block:: cpp /// String::find can be called repeatedly to collect every text position. /// /// Pass a start position to continue the search after a previous match. Move /// the start position with advance when overlapping matches should remain /// possible, or add the needle length when only non-overlapping matches are /// useful. A character comparison function can adapt the matching rule without /// first transforming the source text. void findingAllOccurrences() { const auto missionLog = el::String{"ROV Freja såg ljus; rov freja markerade ljus; ROV Freja sparade karta"_el}; const auto needle = el::String{"rov freja"_el}; el::io::printLine("Mission log: "_el, missionLog); el::io::printLine("Needle: "_el, needle); el::io::printLine("Case-folded matches:"_el); auto searchStart = el::ByteIndex::zero(); auto matchIndex = missionLog.find(needle, searchStart, el::Char::compareCaseFolded); while (!matchIndex.isNoIndex()) { const auto match = missionLog.slice(el::ByteRange{matchIndex, needle.length()}); el::io::printLine(" byte "_el, matchIndex, ": "_el, match); searchStart = matchIndex; if (!missionLog.advance(searchStart)) { break; } matchIndex = missionLog.find(needle, searchStart, el::Char::compareCaseFolded); } } .. erbsland-ansi:: :escape-char: ␛ Mission log: ROV Freja såg ljus; rov freja markerade ljus; ROV Freja sparade karta Needle: rov freja Case-folded matches: byte 0: ROV Freja byte 21: rov freja byte 47: ROV Freja .. erbsland-demo-end:: Turn Positions into Useful Slices ================================= Found positions compose naturally with slicing. The common pattern is: * Search for the start boundary. * Move past the boundary with :cpp:func:`advance() ` if the boundary itself should be excluded. * Search for the end boundary from the adjusted start position. * Slice the :cpp:type:`ByteRange ` between the two positions. When a boundary is optional, handle the no-index state before constructing the range. When the end boundary is optional and the slice should continue to the end of the text, use :cpp:func:`indexAt(StringSide::Back) ` or an infinite :cpp:type:`ByteLength `. This approach avoids conversions to character indexes and keeps parsing code close to the data the string stores.