Add a few more UTF-8 utilities by jviotti · Pull Request #2735 · sourcemeta/core

jviotti · 2026-08-16T22:00:17Z

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

augmentcode · 2026-08-16T22:01:36Z

🤖 Augment PR Summary

Summary: This PR adds UTF-8 validation and repair utilities to the Unicode module.

Changes:

Introduces utf8_sequence_size to measure a valid leading UTF-8 sequence.
Introduces is_utf8_tail to apply lead-byte-specific continuation constraints.
Adds to_valid_utf8 to replace malformed input with U+FFFD.
Preserves valid UTF-8 byte sequences unchanged during normalization.
Implements maximal-subpart handling for truncated multibyte sequences.
Rejects overlong encodings, surrogate encodings, and code points beyond U+10FFFF.
Registers and adds unit tests for valid, malformed, truncated, and boundary sequences.

Technical notes: The implementation relies on RFC 3629 lead-byte ranges and produces output accepted by the existing strict UTF-8 decoder.

_{🤖 Was this summary useful? React with 👍 or 👎}

augmentcode

Review completed. 1 suggestion posted.

Comment augment review to trigger a new review at any time.

augmentcode · 2026-08-16T22:01:37Z

+/// #include <sourcemeta/core/unicode.h>
+/// #include <cassert>
+///
+/// assert(sourcemeta::core::to_valid_utf8("a\xFFb") == "a\xEF\xBF\xBDb");


At src/core/unicode/include/sourcemeta/core/unicode.h:157, C++ hexadecimal escapes greedily consume following hex digits, so both trailing b characters are absorbed by \xFF/\xBD; this assertion does not construct the advertised byte sequences. Consumers copying it can therefore receive an out-of-range escape diagnostic or a failing assertion.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

cubic-dev-ai

All reported issues were addressed across 5 files

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

github-actions

Benchmark (linux/gcc)

Details

Benchmark suite	Current: `303fa9c`	Previous: `3e1d118`	Ratio
`JOSE_VerifySignature_RS256`	`14807.256990846592` ns/iter	`22101.53720930166` ns/iter	`0.67`
`JOSE_VerifySignature_ES512`	`315577.50632907526` ns/iter	`576968.7041905654` ns/iter	`0.55`
`GZIP_Compress_ISO_Language_Set_3_Locations`	`20501406.647059303` ns/iter	`36316672.63158234` ns/iter	`0.56`
`GZIP_Decompress_ISO_Language_Set_3_Locations`	`2180342.886731523` ns/iter	`4161923.886904956` ns/iter	`0.52`
`GZIP_Compress_ISO_Language_Set_3_Schema`	`1363572.9432484878` ns/iter	`2021862.5635836513` ns/iter	`0.67`
`GZIP_Decompress_ISO_Language_Set_3_Schema`	`209467.49789283346` ns/iter	`386395.26269801345` ns/iter	`0.54`
`HTML_Build_Table_100000`	`33933621.5000003` ns/iter	`56435371.54544861` ns/iter	`0.60`
`HTML_Render_Table_100000`	`1603044.2281104128` ns/iter	`1792061.7621483617` ns/iter	`0.89`
`JSONLD_Catalog_Annotation_List_Populate`	`374534.3912346327` ns/iter	`681361.1267056442` ns/iter	`0.55`
`JSONLD_Catalog_Materialize`	`1740147.064837846` ns/iter	`3499838.727272866` ns/iter	`0.50`
`JSONL_Parse_Large`	`5289079.166667052` ns/iter	`9820948.267606005` ns/iter	`0.54`
`JSONL_Parse_Large_GZIP`	`6220307.149123385` ns/iter	`11205630.968252731` ns/iter	`0.56`
`URITemplateRouter_Create`	`9720.388344571265` ns/iter	`21016.46062166891` ns/iter	`0.46`
`URITemplateRouter_Match`	`72.36769192008462` ns/iter	`164.87117392571722` ns/iter	`0.44`
`URITemplateRouter_Match_BasePath`	`90.248939099569` ns/iter	`185.43352057601237` ns/iter	`0.49`
`URITemplateRouterView_Restore`	`7406.454216867632` ns/iter	`8394.31779691524` ns/iter	`0.88`
`URITemplateRouterView_Match`	`62.91209032302987` ns/iter	`127.36728231578856` ns/iter	`0.49`
`URITemplateRouterView_Match_BasePath`	`71.52862767159512` ns/iter	`144.87936165921144` ns/iter	`0.49`
`URITemplateRouterView_Arguments`	`300.45547994405047` ns/iter	`470.935384176182` ns/iter	`0.64`
`JSONPath_Descendant_Filter_Nested`	`930.556525276244` ns/iter	`1612.474267705516` ns/iter	`0.58`
`Pointer_Object_Traverse`	`13.417349382537463` ns/iter	`23.19325641885472` ns/iter	`0.58`
`Pointer_Object_Try_Traverse`	`12.340902605988775` ns/iter	`22.744842791922096` ns/iter	`0.54`
`Pointer_Push_Back_Pointer_To_Weak_Pointer`	`62.74124244733255` ns/iter	`132.26060561480196` ns/iter	`0.47`
`Pointer_Walker_Schema_ISO_Language`	`763319.2788144466` ns/iter	`1408135.31160891` ns/iter	`0.54`
`Pointer_Maybe_Tracked_Deeply_Nested/0`	`721906.453333339` ns/iter	`1208688.61063465` ns/iter	`0.60`
`Pointer_Maybe_Tracked_Deeply_Nested/1`	`936712.2143808452` ns/iter	`1722650.3931205398` ns/iter	`0.54`
`Pointer_Position_Tracker_Get_Deeply_Nested`	`411.91943649161976` ns/iter	`599.277775030408` ns/iter	`0.69`
`JSON_Array_Of_Objects_Unique`	`203.51905888689643` ns/iter	`444.1995594437962` ns/iter	`0.46`
`JSON_Parse_1`	`2707.2933715678378` ns/iter	`5249.098796381818` ns/iter	`0.52`
`JSON_Parse_Real`	`2901.9806224964163` ns/iter	`5639.794278062028` ns/iter	`0.51`
`JSON_Parse_Decimal`	`5502.621012586843` ns/iter	`10213.540378370813` ns/iter	`0.54`
`JSON_Parse_Schema_ISO_Language`	`1991067.0959301025` ns/iter	`3601621.481865847` ns/iter	`0.55`
`JSON_Parse_Integer`	`2160.8253404163515` ns/iter	`3682.149235981534` ns/iter	`0.59`
`JSON_Parse_String_NonSSO_Plain`	`1840.9601812738845` ns/iter	`3268.1126063047323` ns/iter	`0.56`
`JSON_Parse_String_SSO_Plain`	`1602.9315587117026` ns/iter	`3014.7198011116825` ns/iter	`0.53`
`JSON_Parse_String_Escape_Heavy`	`9198.433905026293` ns/iter	`16648.717567599706` ns/iter	`0.55`
`JSON_Parse_Object_Short_Keys`	`5070.916082762112` ns/iter	`8379.323650659393` ns/iter	`0.61`
`JSON_Parse_Object_Scalar_Properties`	`2578.6333468305716` ns/iter	`4253.195059744302` ns/iter	`0.61`
`JSON_Parse_Object_Array_Properties`	`3520.479780266427` ns/iter	`6392.622313362967` ns/iter	`0.55`
`JSON_Parse_Object_Object_Properties`	`3622.503634690205` ns/iter	`6591.551761119188` ns/iter	`0.55`
`JSON_Parse_Nested_Containers`	`22378.15588109043` ns/iter	`38786.33619344216` ns/iter	`0.58`
`JSON_From_String_Copy`	`4.815817958060234` ns/iter	`8.692148675840231` ns/iter	`0.55`
`JSON_From_String_Temporary`	`3.887374404886587` ns/iter	`7.398876177322743` ns/iter	`0.53`
`JSON_Number_To_Double`	`11.031930833737354` ns/iter	`21.187564406867164` ns/iter	`0.52`
`JSON_Object_At_Last_Key/8`	`2.131664725565263` ns/iter	`3.7472406751109237` ns/iter	`0.57`
`JSON_Object_At_Last_Key/32`	`7.7581654439640975` ns/iter	`11.840434535447505` ns/iter	`0.66`
`JSON_Object_At_Last_Key/128`	`30.773078033753414` ns/iter	`48.593308623270836` ns/iter	`0.63`
`JSON_Object_At_Last_Key/512`	`160.80984379797317` ns/iter	`360.66574949198844` ns/iter	`0.45`
`JSON_Fast_Hash_Helm_Chart_Lock`	`33.60422688857353` ns/iter	`61.21684197824709` ns/iter	`0.55`
`JSON_Equality_Helm_Chart_Lock`	`75.18429470384551` ns/iter	`147.6973751437302` ns/iter	`0.51`
`JSON_Divisible_By_Decimal`	`147.84524478036877` ns/iter	`266.82155116814897` ns/iter	`0.55`
`JSON_String_Equal/10`	`2.4300114130857637` ns/iter	`5.383067169617077` ns/iter	`0.45`
`JSON_String_Equal/100`	`2.916449381247116` ns/iter	`6.027594028105579` ns/iter	`0.48`
`JSON_String_Equal_Small_By_Perfect_Hash/10`	`0.48812442845217047` ns/iter	`0.6262743049105542` ns/iter	`0.78`
`JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10`	`11.089606462866627` ns/iter	`25.191046211621053` ns/iter	`0.44`
`JSON_String_Fast_Hash/10`	`1.2729352471008133` ns/iter	`2.2642719033122307` ns/iter	`0.56`
`JSON_String_Fast_Hash/100`	`1.2018342252709726` ns/iter	`2.2571210985156624` ns/iter	`0.53`
`JSON_String_Key_Hash/10`	`0.547630443268468` ns/iter	`1.2463032554881113` ns/iter	`0.44`
`JSON_String_Key_Hash/100`	`7.109785348759875` ns/iter	`12.44066967900533` ns/iter	`0.57`
`JSON_Object_Defines_Miss_Same_Length`	`1.5087641385275832` ns/iter	`3.113368885108315` ns/iter	`0.48`
`JSON_Object_Defines_Miss_Too_Small`	`1.6736626677936066` ns/iter	`3.112489518284147` ns/iter	`0.54`
`JSON_Object_Defines_Miss_Too_Large`	`1.4682790973169473` ns/iter	`3.1225383151483173` ns/iter	`0.47`
`Regex_Lower_S_Or_Upper_S_Asterisk`	`0.4746312558998452` ns/iter	`0.9345701875181324` ns/iter	`0.51`
`Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar`	`0.7023459464238755` ns/iter	`0.6228135543831659` ns/iter	`1.13`
`Regex_Period_Asterisk`	`0.49194875622945894` ns/iter	`0.9336394320079977` ns/iter	`0.53`
`Regex_Group_Period_Asterisk_Group`	`0.614925242133672` ns/iter	`0.6224510929740247` ns/iter	`0.99`
`Regex_Period_Plus`	`0.47691332282605386` ns/iter	`0.9346429238473781` ns/iter	`0.51`
`Regex_Period`	`0.7034059560701584` ns/iter	`0.622906331131836` ns/iter	`1.13`
`Regex_Caret_Period_Plus_Dollar`	`0.939138862021624` ns/iter	`0.9343967308518903` ns/iter	`1.01`
`Regex_Caret_Group_Period_Plus_Group_Dollar`	`0.7048605831564315` ns/iter	`0.6246915147986264` ns/iter	`1.13`
`Regex_Caret_Period_Asterisk_Dollar`	`0.48028418253479555` ns/iter	`0.9337307497682605` ns/iter	`0.51`
`Regex_Caret_Group_Period_Asterisk_Group_Dollar`	`0.7077953048019135` ns/iter	`0.622694739284461` ns/iter	`1.14`
`Regex_Caret_X_Hyphen`	`1.9208326108320193` ns/iter	`4.368118029665147` ns/iter	`0.44`
`Regex_Period_Md_Dollar`	`13.937416125062565` ns/iter	`29.425825261233133` ns/iter	`0.47`
`Regex_Caret_Slash_Period_Asterisk`	`1.895512116632097` ns/iter	`4.673294088474014` ns/iter	`0.41`
`Regex_Caret_Period_Range_Dollar`	`0.5604301460432287` ns/iter	`1.24814614420334` ns/iter	`0.45`
`Regex_Nested_Backtrack`	`19.397769081589168` ns/iter	`38.03832628801116` ns/iter	`0.51`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Benchmark (macos/llvm)

Details

Benchmark suite	Current: `303fa9c`	Previous: `3e1d118`	Ratio
`Regex_Lower_S_Or_Upper_S_Asterisk`	`2.00579915029903` ns/iter	`1.868143424877922` ns/iter	`1.07`
`Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar`	`1.8498027340876797` ns/iter	`2.2580766707778217` ns/iter	`0.82`
`Regex_Period_Asterisk`	`1.815312594974945` ns/iter	`1.787528595540095` ns/iter	`1.02`
`Regex_Group_Period_Asterisk_Group`	`1.8465002957176315` ns/iter	`1.8446062102550262` ns/iter	`1.00`
`Regex_Period_Plus`	`2.373461705819016` ns/iter	`2.4034381630774053` ns/iter	`0.99`
`Regex_Period`	`2.1825401475147035` ns/iter	`2.1626874932896687` ns/iter	`1.01`
`Regex_Caret_Period_Plus_Dollar`	`2.187946744098961` ns/iter	`2.202504473735838` ns/iter	`0.99`
`Regex_Caret_Group_Period_Plus_Group_Dollar`	`2.2379138079523795` ns/iter	`2.055102067784934` ns/iter	`1.09`
`Regex_Caret_Period_Asterisk_Dollar`	`1.8406769018968878` ns/iter	`1.782146651301697` ns/iter	`1.03`
`Regex_Caret_Group_Period_Asterisk_Group_Dollar`	`2.116026372472973` ns/iter	`1.8529442196767445` ns/iter	`1.14`
`Regex_Caret_X_Hyphen`	`7.449531149073145` ns/iter	`6.522971040108916` ns/iter	`1.14`
`Regex_Period_Md_Dollar`	`19.136195853415906` ns/iter	`17.263617775183967` ns/iter	`1.11`
`Regex_Caret_Slash_Period_Asterisk`	`4.870097911475717` ns/iter	`5.065924900811986` ns/iter	`0.96`
`Regex_Caret_Period_Range_Dollar`	`2.178220905618364` ns/iter	`2.9408047933410697` ns/iter	`0.74`
`Regex_Nested_Backtrack`	`28.76740767826082` ns/iter	`28.01045652768125` ns/iter	`1.03`
`JSON_Array_Of_Objects_Unique`	`376.5664010158549` ns/iter	`397.42592498420845` ns/iter	`0.95`
`JSON_Parse_1`	`3190.323849824358` ns/iter	`3144.7779434624217` ns/iter	`1.01`
`JSON_Parse_Real`	`5305.789579650017` ns/iter	`5240.181728212203` ns/iter	`1.01`
`JSON_Parse_Decimal`	`6564.392659227211` ns/iter	`5845.891665062545` ns/iter	`1.12`
`JSON_Parse_Schema_ISO_Language`	`3015296.517123046` ns/iter	`2302835.1151318224` ns/iter	`1.31`
`JSON_Parse_Integer`	`3757.718037630469` ns/iter	`3316.329788609883` ns/iter	`1.13`
`JSON_Parse_String_NonSSO_Plain`	`3223.395270270278` ns/iter	`3169.6869318158874` ns/iter	`1.02`
`JSON_Parse_String_SSO_Plain`	`2029.8919002239754` ns/iter	`1804.6337390163744` ns/iter	`1.12`
`JSON_Parse_String_Escape_Heavy`	`17750.209618281086` ns/iter	`17632.58628607189` ns/iter	`1.01`
`JSON_Parse_Object_Short_Keys`	`6177.022808705099` ns/iter	`5693.908461373438` ns/iter	`1.08`
`JSON_Parse_Object_Scalar_Properties`	`3261.986557434255` ns/iter	`2803.3237044861066` ns/iter	`1.16`
`JSON_Parse_Object_Array_Properties`	`3524.2255294167235` ns/iter	`3732.0947755142115` ns/iter	`0.94`
`JSON_Parse_Object_Object_Properties`	`4072.9065971045598` ns/iter	`3726.63825277851` ns/iter	`1.09`
`JSON_Parse_Nested_Containers`	`25345.282346049295` ns/iter	`25879.68694907584` ns/iter	`0.98`
`JSON_From_String_Copy`	`11.298306316501911` ns/iter	`11.082411438320724` ns/iter	`1.02`
`JSON_From_String_Temporary`	`6.433967486097671` ns/iter	`6.748603100057906` ns/iter	`0.95`
`JSON_Number_To_Double`	`37.28223394316145` ns/iter	`34.83491269748545` ns/iter	`1.07`
`JSON_Object_At_Last_Key/8`	`4.6870820812860865` ns/iter	`4.234703042921326` ns/iter	`1.11`
`JSON_Object_At_Last_Key/32`	`12.6948213162839` ns/iter	`12.776145574655306` ns/iter	`0.99`
`JSON_Object_At_Last_Key/128`	`57.24029983080498` ns/iter	`56.94222220977404` ns/iter	`1.01`
`JSON_Object_At_Last_Key/512`	`209.7972953508597` ns/iter	`200.55775220414196` ns/iter	`1.05`
`JSON_Fast_Hash_Helm_Chart_Lock`	`62.847732035124885` ns/iter	`61.411084520874006` ns/iter	`1.02`
`JSON_Equality_Helm_Chart_Lock`	`154.91685317209405` ns/iter	`154.1364276972294` ns/iter	`1.01`
`JSON_Divisible_By_Decimal`	`229.74458213027904` ns/iter	`195.52060480319645` ns/iter	`1.18`
`JSON_String_Equal/10`	`9.320082875080102` ns/iter	`7.426577140374459` ns/iter	`1.25`
`JSON_String_Equal/100`	`8.40978862525647` ns/iter	`7.2283031582439135` ns/iter	`1.16`
`JSON_String_Equal_Small_By_Perfect_Hash/10`	`0.42858014159501745` ns/iter	`0.3520226711945682` ns/iter	`1.22`
`JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10`	`4.436003591016841` ns/iter	`3.3663789447876478` ns/iter	`1.32`
`JSON_String_Fast_Hash/10`	`3.815614660937612` ns/iter	`2.4638133943692107` ns/iter	`1.55`
`JSON_String_Fast_Hash/100`	`2.559253640674747` ns/iter	`2.1718386097666604` ns/iter	`1.18`
`JSON_String_Key_Hash/10`	`2.0679332542237643` ns/iter	`1.7401142870940456` ns/iter	`1.19`
`JSON_String_Key_Hash/100`	`2.4085898720992702` ns/iter	`2.2083027902362624` ns/iter	`1.09`
`JSON_Object_Defines_Miss_Same_Length`	`3.112434053790461` ns/iter	`2.81049696825084` ns/iter	`1.11`
`JSON_Object_Defines_Miss_Too_Small`	`2.824485354051045` ns/iter	`2.7622945159002956` ns/iter	`1.02`
`JSON_Object_Defines_Miss_Too_Large`	`2.7926418261824777` ns/iter	`3.6088322761155145` ns/iter	`0.77`
`Pointer_Object_Traverse`	`20.46309278430362` ns/iter	`19.965211144923373` ns/iter	`1.02`
`Pointer_Object_Try_Traverse`	`22.49797324960558` ns/iter	`24.333243628441544` ns/iter	`0.92`
`Pointer_Push_Back_Pointer_To_Weak_Pointer`	`148.27259129912758` ns/iter	`148.28377026630568` ns/iter	`1.00`
`Pointer_Walker_Schema_ISO_Language`	`1238205.7899460273` ns/iter	`1376223.3727811151` ns/iter	`0.90`
`Pointer_Maybe_Tracked_Deeply_Nested/0`	`824600.7594787054` ns/iter	`902076.290652975` ns/iter	`0.91`
`Pointer_Maybe_Tracked_Deeply_Nested/1`	`1038305.9336349103` ns/iter	`1187770.698052021` ns/iter	`0.87`
`Pointer_Position_Tracker_Get_Deeply_Nested`	`341.3699980703837` ns/iter	`403.2724670981848` ns/iter	`0.85`
`JSONPath_Descendant_Filter_Nested`	`1152.2472084818414` ns/iter	`1230.6775588878556` ns/iter	`0.94`
`URITemplateRouter_Create`	`18269.731911016464` ns/iter	`18855.950062689033` ns/iter	`0.97`
`URITemplateRouter_Match`	`174.33788181902983` ns/iter	`183.44147147054684` ns/iter	`0.95`
`URITemplateRouter_Match_BasePath`	`200.2432583772904` ns/iter	`239.81510733933243` ns/iter	`0.83`
`URITemplateRouterView_Restore`	`10373.365211406208` ns/iter	`11083.809938453245` ns/iter	`0.94`
`URITemplateRouterView_Match`	`141.73294210242386` ns/iter	`146.35830408193226` ns/iter	`0.97`
`URITemplateRouterView_Match_BasePath`	`159.6840587178323` ns/iter	`169.43063824178702` ns/iter	`0.94`
`URITemplateRouterView_Arguments`	`583.1368657096548` ns/iter	`579.8310923353538` ns/iter	`1.01`
`JSONL_Parse_Large`	`7984713.174418767` ns/iter	`6598446.315315249` ns/iter	`1.21`
`JSONL_Parse_Large_GZIP`	`7827508.425531931` ns/iter	`7704164.835164133` ns/iter	`1.02`
`JSONLD_Catalog_Annotation_List_Populate`	`544333.4573990243` ns/iter	`556112.1670000375` ns/iter	`0.98`
`JSONLD_Catalog_Materialize`	`2452664.9865771295` ns/iter	`2506085.793358019` ns/iter	`0.98`
`HTML_Build_Table_100000`	`41383372.00000056` ns/iter	`41020147.05882298` ns/iter	`1.01`
`HTML_Render_Table_100000`	`1674235.7594935875` ns/iter	`1966947.6960486234` ns/iter	`0.85`
`GZIP_Compress_ISO_Language_Set_3_Locations`	`27539309.307695266` ns/iter	`28538121.500001047` ns/iter	`0.97`
`GZIP_Decompress_ISO_Language_Set_3_Locations`	`3385386.418268756` ns/iter	`3975062.9516124865` ns/iter	`0.85`
`GZIP_Compress_ISO_Language_Set_3_Schema`	`1521764.8621444493` ns/iter	`1640259.3333335365` ns/iter	`0.93`
`GZIP_Decompress_ISO_Language_Set_3_Schema`	`276652.666799826` ns/iter	`292111.0996745661` ns/iter	`0.95`
`JOSE_VerifySignature_RS256`	`21485.615525312303` ns/iter	`22259.124734231915` ns/iter	`0.97`
`JOSE_VerifySignature_ES512`	`1023751.6413502583` ns/iter	`1024923.0905232576` ns/iter	`1.00`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Benchmark (linux/llvm)

Details

Benchmark suite	Current: `303fa9c`	Previous: `3e1d118`	Ratio
`Regex_Lower_S_Or_Upper_S_Asterisk`	`1.9239305081619105` ns/iter	`2.0943796831822947` ns/iter	`0.92`
`Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar`	`1.9148384947148969` ns/iter	`2.1506233254065417` ns/iter	`0.89`
`Regex_Period_Asterisk`	`1.909294867617315` ns/iter	`2.1214273287737693` ns/iter	`0.90`
`Regex_Group_Period_Asterisk_Group`	`1.9097761147593644` ns/iter	`2.0973794152843` ns/iter	`0.91`
`Regex_Period_Plus`	`3.003449198813585` ns/iter	`2.079886787199638` ns/iter	`1.44`
`Regex_Period`	`3.0013503747756247` ns/iter	`2.0980421718820783` ns/iter	`1.43`
`Regex_Caret_Period_Plus_Dollar`	`2.9991327840548543` ns/iter	`2.078929371440862` ns/iter	`1.44`
`Regex_Caret_Group_Period_Plus_Group_Dollar`	`3.000080927234838` ns/iter	`2.068902280065019` ns/iter	`1.45`
`Regex_Caret_Period_Asterisk_Dollar`	`2.1853876011646505` ns/iter	`2.1428149690009` ns/iter	`1.02`
`Regex_Caret_Group_Period_Asterisk_Group_Dollar`	`2.1849987652245546` ns/iter	`2.121360686008557` ns/iter	`1.03`
`Regex_Caret_X_Hyphen`	`5.182395904175728` ns/iter	`4.048235373277673` ns/iter	`1.28`
`Regex_Period_Md_Dollar`	`20.24365018207106` ns/iter	`24.710893702540524` ns/iter	`0.82`
`Regex_Caret_Slash_Period_Asterisk`	`5.291860731318862` ns/iter	`3.747774884146645` ns/iter	`1.41`
`Regex_Caret_Period_Range_Dollar`	`3.272313583392864` ns/iter	`2.595284946652597` ns/iter	`1.26`
`Regex_Nested_Backtrack`	`30.294744899586227` ns/iter	`35.844638943767116` ns/iter	`0.85`
`JSON_Array_Of_Objects_Unique`	`342.6113459163901` ns/iter	`376.1713014490205` ns/iter	`0.91`
`JSON_Parse_1`	`3459.2339882473243` ns/iter	`4228.441446352357` ns/iter	`0.82`
`JSON_Parse_Real`	`4048.2646958476043` ns/iter	`4757.448872824584` ns/iter	`0.85`
`JSON_Parse_Decimal`	`6008.8475007705565` ns/iter	`7356.491819814974` ns/iter	`0.82`
`JSON_Parse_Schema_ISO_Language`	`2724383.1607843367` ns/iter	`3357323.88151628` ns/iter	`0.81`
`JSON_Parse_Integer`	`3112.6487400550104` ns/iter	`3628.4130746973924` ns/iter	`0.86`
`JSON_Parse_String_NonSSO_Plain`	`2403.385094652094` ns/iter	`2782.9030677286096` ns/iter	`0.86`
`JSON_Parse_String_SSO_Plain`	`2260.8203333057413` ns/iter	`2307.8706693805552` ns/iter	`0.98`
`JSON_Parse_String_Escape_Heavy`	`9620.838646758677` ns/iter	`14087.211857596665` ns/iter	`0.68`
`JSON_Parse_Object_Short_Keys`	`6275.08477044889` ns/iter	`6817.855256109218` ns/iter	`0.92`
`JSON_Parse_Object_Scalar_Properties`	`3200.161518971467` ns/iter	`3557.7856842762967` ns/iter	`0.90`
`JSON_Parse_Object_Array_Properties`	`4423.614418700758` ns/iter	`4917.800273915879` ns/iter	`0.90`
`JSON_Parse_Object_Object_Properties`	`4452.283837073822` ns/iter	`4882.791186790714` ns/iter	`0.91`
`JSON_Parse_Nested_Containers`	`27821.767953820694` ns/iter	`31822.39754378404` ns/iter	`0.87`
`JSON_From_String_Copy`	`9.990114410357494` ns/iter	`8.828710410613812` ns/iter	`1.13`
`JSON_From_String_Temporary`	`8.063747025650663` ns/iter	`8.41426284142872` ns/iter	`0.96`
`JSON_Number_To_Double`	`17.47808715991824` ns/iter	`21.73328996984484` ns/iter	`0.80`
`JSON_Object_At_Last_Key/8`	`4.912197205015386` ns/iter	`5.813973809296038` ns/iter	`0.84`
`JSON_Object_At_Last_Key/32`	`18.282110595303354` ns/iter	`19.62079718174821` ns/iter	`0.93`
`JSON_Object_At_Last_Key/128`	`70.69533454819945` ns/iter	`75.4027239484618` ns/iter	`0.94`
`JSON_Object_At_Last_Key/512`	`295.81444546814873` ns/iter	`304.2148198672903` ns/iter	`0.97`
`JSON_Fast_Hash_Helm_Chart_Lock`	`56.11680541610688` ns/iter	`54.77125880535911` ns/iter	`1.02`
`JSON_Equality_Helm_Chart_Lock`	`124.76398981205345` ns/iter	`142.24031584893302` ns/iter	`0.88`
`JSON_Divisible_By_Decimal`	`195.4879135181859` ns/iter	`254.75203499242042` ns/iter	`0.77`
`JSON_String_Equal/10`	`4.365021485731448` ns/iter	`4.325690295423628` ns/iter	`1.01`
`JSON_String_Equal/100`	`5.1840753227925225` ns/iter	`5.483453606202525` ns/iter	`0.95`
`JSON_String_Equal_Small_By_Perfect_Hash/10`	`0.8180369729848661` ns/iter	`0.6916532219474825` ns/iter	`1.18`
`JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10`	`9.68649588212979` ns/iter	`9.511940530222896` ns/iter	`1.02`
`JSON_String_Fast_Hash/10`	`1.9134592928631762` ns/iter	`2.0264674484935994` ns/iter	`0.94`
`JSON_String_Fast_Hash/100`	`1.9104519867295675` ns/iter	`2.020412543836115` ns/iter	`0.95`
`JSON_String_Key_Hash/10`	`2.1833805509982773` ns/iter	`2.017400941349533` ns/iter	`1.08`
`JSON_String_Key_Hash/100`	`6.269686544251458` ns/iter	`6.632584977695258` ns/iter	`0.95`
`JSON_Object_Defines_Miss_Same_Length`	`2.2961117983134636` ns/iter	`3.4679442391910373` ns/iter	`0.66`
`JSON_Object_Defines_Miss_Too_Small`	`2.2994733722096456` ns/iter	`3.0262226213009416` ns/iter	`0.76`
`JSON_Object_Defines_Miss_Too_Large`	`2.486290405123627` ns/iter	`3.4588483629613873` ns/iter	`0.72`
`Pointer_Object_Traverse`	`23.748957898707232` ns/iter	`29.97079789204014` ns/iter	`0.79`
`Pointer_Object_Try_Traverse`	`25.515708696829524` ns/iter	`39.070394184337964` ns/iter	`0.65`
`Pointer_Push_Back_Pointer_To_Weak_Pointer`	`140.48815275921126` ns/iter	`142.60584925113952` ns/iter	`0.99`
`Pointer_Walker_Schema_ISO_Language`	`1159305.579207954` ns/iter	`1942483.1264367294` ns/iter	`0.60`
`Pointer_Maybe_Tracked_Deeply_Nested/0`	`973031.8723993986` ns/iter	`1137770.5048700322` ns/iter	`0.86`
`Pointer_Maybe_Tracked_Deeply_Nested/1`	`1262336.3799283046` ns/iter	`2014772.1983002343` ns/iter	`0.63`
`Pointer_Position_Tracker_Get_Deeply_Nested`	`542.9666574525783` ns/iter	`696.0436555155467` ns/iter	`0.78`
`JSONPath_Descendant_Filter_Nested`	`1194.8163299278212` ns/iter	`1375.6372306089067` ns/iter	`0.87`
`URITemplateRouter_Create`	`16066.664654288918` ns/iter	`20230.157401244895` ns/iter	`0.79`
`URITemplateRouter_Match`	`138.64458669970028` ns/iter	`155.1787175189985` ns/iter	`0.89`
`URITemplateRouter_Match_BasePath`	`165.12919990500373` ns/iter	`173.2700837432371` ns/iter	`0.95`
`URITemplateRouterView_Restore`	`7645.845951522518` ns/iter	`3732.563660710216` ns/iter	`2.05`
`URITemplateRouterView_Match`	`132.93930831452312` ns/iter	`142.25609013880398` ns/iter	`0.93`
`URITemplateRouterView_Match_BasePath`	`147.72652897829377` ns/iter	`159.5652786959523` ns/iter	`0.93`
`URITemplateRouterView_Arguments`	`351.3796100144984` ns/iter	`428.94197988826676` ns/iter	`0.82`
`JSONL_Parse_Large`	`7151584.59183608` ns/iter	`8395537.25301171` ns/iter	`0.85`
`JSONL_Parse_Large_GZIP`	`8369905.8072290905` ns/iter	`9855448.619718323` ns/iter	`0.85`
`JSONLD_Catalog_Annotation_List_Populate`	`481258.84909596853` ns/iter	`671612.798486277` ns/iter	`0.72`
`JSONLD_Catalog_Materialize`	`2764250.138340017` ns/iter	`3617665.1709844633` ns/iter	`0.76`
`HTML_Build_Table_100000`	`52922678.249994226` ns/iter	`59664265.666659355` ns/iter	`0.89`
`HTML_Render_Table_100000`	`1865955.8710528233` ns/iter	`3524833.4670050703` ns/iter	`0.53`
`GZIP_Compress_ISO_Language_Set_3_Locations`	`27012626.538464703` ns/iter	`31022096.91303909` ns/iter	`0.87`
`GZIP_Decompress_ISO_Language_Set_3_Locations`	`2872814.4918696354` ns/iter	`4964928.089552373` ns/iter	`0.58`
`GZIP_Compress_ISO_Language_Set_3_Schema`	`1607386.7110091266` ns/iter	`1862696.2284945762` ns/iter	`0.86`
`GZIP_Decompress_ISO_Language_Set_3_Schema`	`197270.4832441544` ns/iter	`340010.81589151063` ns/iter	`0.58`
`JOSE_VerifySignature_RS256`	`49848.18037097274` ns/iter	`49221.84342012706` ns/iter	`1.01`
`JOSE_VerifySignature_ES512`	`2174588.78153838` ns/iter	`2108015.5421688813` ns/iter	`1.03`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Benchmark (windows/msvc)

Details

Benchmark suite	Current: `303fa9c`	Previous: `3e1d118`	Ratio
`Regex_Lower_S_Or_Upper_S_Asterisk`	`5.033827678571714` ns/iter	`2.9776652665278633` ns/iter	`1.69`
`Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar`	`5.455522321428573` ns/iter	`2.957495888896274` ns/iter	`1.84`
`Regex_Period_Asterisk`	`6.008474107143716` ns/iter	`2.9165402862726313` ns/iter	`2.06`
`Regex_Group_Period_Asterisk_Group`	`5.145886000000246` ns/iter	`2.935413478211579` ns/iter	`1.75`
`Regex_Period_Plus`	`5.198363000001791` ns/iter	`3.0060845718671407` ns/iter	`1.73`
`Regex_Period`	`5.332103999999163` ns/iter	`2.9263006880456066` ns/iter	`1.82`
`Regex_Caret_Period_Plus_Dollar`	`5.058010999998714` ns/iter	`2.9340241058324295` ns/iter	`1.72`
`Regex_Caret_Group_Period_Plus_Group_Dollar`	`5.142823000001044` ns/iter	`2.9076959559266164` ns/iter	`1.77`
`Regex_Caret_Period_Asterisk_Dollar`	`5.227131999999983` ns/iter	`3.0064946834731328` ns/iter	`1.74`
`Regex_Caret_Group_Period_Asterisk_Group_Dollar`	`5.098741071427478` ns/iter	`2.945267183555622` ns/iter	`1.73`
`Regex_Caret_X_Hyphen`	`8.232997767856988` ns/iter	`6.1490553571417195` ns/iter	`1.34`
`Regex_Period_Md_Dollar`	`45.13200048990407` ns/iter	`28.449787372992386` ns/iter	`1.59`
`Regex_Caret_Slash_Period_Asterisk`	`7.901447544644517` ns/iter	`6.187823214286173` ns/iter	`1.28`
`Regex_Caret_Period_Range_Dollar`	`5.676246999998966` ns/iter	`3.1602060986883393` ns/iter	`1.80`
`Regex_Nested_Backtrack`	`58.2056160714168` ns/iter	`38.99767299107332` ns/iter	`1.49`
`JSON_Array_Of_Objects_Unique`	`633.8204464285013` ns/iter	`536.0658928572499` ns/iter	`1.18`
`JSON_Parse_1`	`8923.64364444904` ns/iter	`7778.414062501935` ns/iter	`1.15`
`JSON_Parse_Real`	`16366.167784963864` ns/iter	`12057.712499999494` ns/iter	`1.36`
`JSON_Parse_Decimal`	`11871.36718749926` ns/iter	`9478.465721136641` ns/iter	`1.25`
`JSON_Parse_Schema_ISO_Language`	`7822303.333333972` ns/iter	`7881426.666666711` ns/iter	`0.99`
`JSON_Parse_Integer`	`6354.144642855886` ns/iter	`4457.204352582859` ns/iter	`1.43`
`JSON_Parse_String_NonSSO_Plain`	`7784.771205357239` ns/iter	`6370.942857143648` ns/iter	`1.22`
`JSON_Parse_String_SSO_Plain`	`3801.4330331552105` ns/iter	`2811.295798529232` ns/iter	`1.35`
`JSON_Parse_String_Escape_Heavy`	`22431.746875000157` ns/iter	`23197.723239696243` ns/iter	`0.97`
`JSON_Parse_Object_Short_Keys`	`13375.276562499039` ns/iter	`10398.467187499706` ns/iter	`1.29`
`JSON_Parse_Object_Scalar_Properties`	`6857.832142858098` ns/iter	`5327.933035713629` ns/iter	`1.29`
`JSON_Parse_Object_Array_Properties`	`11556.855357144155` ns/iter	`9417.622242759146` ns/iter	`1.23`
`JSON_Parse_Object_Object_Properties`	`11686.26785714163` ns/iter	`9436.17796348908` ns/iter	`1.24`
`JSON_Parse_Nested_Containers`	`80379.00669641829` ns/iter	`63425.0669643066` ns/iter	`1.27`
`JSON_From_String_Copy`	`64.28241964284454` ns/iter	`48.99904944867711` ns/iter	`1.31`
`JSON_From_String_Temporary`	`59.44059000000834` ns/iter	`56.09431999998833` ns/iter	`1.06`
`JSON_Number_To_Double`	`121.00969642858671` ns/iter	`89.67931742503959` ns/iter	`1.35`
`JSON_Object_At_Last_Key/8`	`7.564687499999374` ns/iter	`7.6287232142848085` ns/iter	`0.99`
`JSON_Object_At_Last_Key/32`	`23.67363589649951` ns/iter	`18.916896597471155` ns/iter	`1.25`
`JSON_Object_At_Last_Key/128`	`90.1697638316959` ns/iter	`91.59088169644139` ns/iter	`0.98`
`JSON_Object_At_Last_Key/512`	`434.22057677555586` ns/iter	`324.1121428570994` ns/iter	`1.34`
`JSON_Fast_Hash_Helm_Chart_Lock`	`108.40745312499678` ns/iter	`87.09739955357065` ns/iter	`1.24`
`JSON_Equality_Helm_Chart_Lock`	`210.84634375000633` ns/iter	`199.28488396053226` ns/iter	`1.06`
`JSON_Divisible_By_Decimal`	`317.54674107144103` ns/iter	`255.92389312164784` ns/iter	`1.24`
`JSON_String_Equal/10`	`10.828348437499358` ns/iter	`8.817167410715143` ns/iter	`1.23`
`JSON_String_Equal/100`	`11.7219553571439` ns/iter	`11.861587500000326` ns/iter	`0.99`
`JSON_String_Equal_Small_By_Perfect_Hash/10`	`2.6154635714289856` ns/iter	`1.2940823214287482` ns/iter	`2.02`
`JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10`	`15.023765624998678` ns/iter	`10.107518750000821` ns/iter	`1.49`
`JSON_String_Fast_Hash/10`	`6.747501116071776` ns/iter	`4.187808124999037` ns/iter	`1.61`
`JSON_String_Fast_Hash/100`	`6.658116964286721` ns/iter	`4.140939337750452` ns/iter	`1.61`
`JSON_String_Key_Hash/10`	`5.421900892857577` ns/iter	`3.1439392857147856` ns/iter	`1.72`
`JSON_String_Key_Hash/100`	`12.04201428571342` ns/iter	`8.332004427090625` ns/iter	`1.45`
`JSON_Object_Defines_Miss_Same_Length`	`4.723355367684759` ns/iter	`4.7880787500005795` ns/iter	`0.99`
`JSON_Object_Defines_Miss_Too_Small`	`4.881585714284954` ns/iter	`5.294893000000229` ns/iter	`0.92`
`JSON_Object_Defines_Miss_Too_Large`	`4.7875265189245635` ns/iter	`5.243456000000606` ns/iter	`0.91`
`Pointer_Object_Traverse`	`67.73656250000926` ns/iter	`41.97925223214938` ns/iter	`1.61`
`Pointer_Object_Try_Traverse`	`71.20306919643177` ns/iter	`45.6150576284414` ns/iter	`1.56`
`Pointer_Push_Back_Pointer_To_Weak_Pointer`	`171.37591102965482` ns/iter	`147.7447321428928` ns/iter	`1.16`
`Pointer_Walker_Schema_ISO_Language`	`11102229.687498521` ns/iter	`10628518.749999927` ns/iter	`1.04`
`Pointer_Maybe_Tracked_Deeply_Nested/0`	`2441268.5618731407` ns/iter	`2181684.062500011` ns/iter	`1.12`
`Pointer_Maybe_Tracked_Deeply_Nested/1`	`3715254.0106942365` ns/iter	`3772322.4598930706` ns/iter	`0.98`
`Pointer_Position_Tracker_Get_Deeply_Nested`	`557.107031249972` ns/iter	`573.7636160714844` ns/iter	`0.97`
`JSONPath_Descendant_Filter_Nested`	`2386.076466432416` ns/iter	`1754.8537096908562` ns/iter	`1.36`
`URITemplateRouter_Create`	`40519.39527594625` ns/iter	`36993.9118303567` ns/iter	`1.10`
`URITemplateRouter_Match`	`236.0018437499889` ns/iter	`249.79457033542934` ns/iter	`0.94`
`URITemplateRouter_Match_BasePath`	`265.21735335792437` ns/iter	`282.8069528116642` ns/iter	`0.94`
`URITemplateRouterView_Restore`	`32720.911264240818` ns/iter	`20525.523608418076` ns/iter	`1.59`
`URITemplateRouterView_Match`	`201.32557154692375` ns/iter	`179.61486247416633` ns/iter	`1.12`
`URITemplateRouterView_Match_BasePath`	`209.43968261433034` ns/iter	`198.49800676352848` ns/iter	`1.06`
`URITemplateRouterView_Arguments`	`560.363482142837` ns/iter	`450.1229263062773` ns/iter	`1.24`
`JSONL_Parse_Large`	`32541099.999997683` ns/iter	`25440271.428570736` ns/iter	`1.28`
`JSONL_Parse_Large_GZIP`	`34080774.99999536` ns/iter	`25949246.153843958` ns/iter	`1.31`
`JSONLD_Catalog_Annotation_List_Populate`	`3190136.44067778` ns/iter	`3016982.6271181763` ns/iter	`1.06`
`JSONLD_Catalog_Materialize`	`8444007.99999979` ns/iter	`7417498.888887368` ns/iter	`1.14`
`HTML_Build_Table_100000`	`92534314.28572056` ns/iter	`103776616.66666426` ns/iter	`0.89`
`HTML_Render_Table_100000`	`8005505.555555222` ns/iter	`13216244.000000188` ns/iter	`0.61`
`GZIP_Compress_ISO_Language_Set_3_Locations`	`36519642.10526833` ns/iter	`31546690.909088038` ns/iter	`1.16`
`GZIP_Decompress_ISO_Language_Set_3_Locations`	`10144897.333332967` ns/iter	`12510262.50000109` ns/iter	`0.81`
`GZIP_Compress_ISO_Language_Set_3_Schema`	`2092772.753623148` ns/iter	`2118452.1739126905` ns/iter	`0.99`
`GZIP_Decompress_ISO_Language_Set_3_Schema`	`650503.7946428364` ns/iter	`659012.5892858265` ns/iter	`0.99`
`JOSE_VerifySignature_RS256`	`21476.720327862015` ns/iter	`20297.019905981222` ns/iter	`1.06`
`JOSE_VerifySignature_ES512`	`1376127.6785714927` ns/iter	`1066923.750000066` ns/iter	`1.29`

This comment was automatically generated by workflow using github-action-benchmark.

Add a few more UTF-8 utilities

3a1183d

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

augmentcode Bot reviewed Aug 16, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread src/core/unicode/include/sourcemeta/core/unicode.h Outdated

Fix

303fa9c

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

github-actions Bot reviewed Aug 16, 2026

View reviewed changes

jviotti merged commit 0dc34ec into main Aug 16, 2026
13 checks passed

jviotti deleted the unicode-helpers branch August 16, 2026 23:20

Navigation Menu

Uh oh!

Uh oh!

Add a few more UTF-8 utilities - #2735

Add a few more UTF-8 utilities#2735
jviotti merged 2 commits into
mainfrom
unicode-helpers

jviotti commented Aug 16, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

augmentcode Bot commented Aug 16, 2026

Uh oh!

augmentcode Bot left a comment

Uh oh!

augmentcode Bot Aug 16, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

jviotti commented Aug 16, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

augmentcode Bot commented Aug 16, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Benchmark (linux/gcc)

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Benchmark (macos/llvm)

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Benchmark (linux/llvm)

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Benchmark (windows/msvc)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jviotti commented Aug 16, 2026 •

edited by cubic-dev-ai Bot

Loading

augmentcode Bot Aug 16, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading