Add a contrib script for easily computing code coverage on LLVM

[jviotti]

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

[github-actions]

Benchmark (linux/llvm)

Details

Benchmark suite	Current: a6b3c7c	Previous: 307253a	Ratio
Regex_Lower_S_Or_Upper_S_Asterisk	1.1505211942537497 ns/iter	2.495579993864196 ns/iter	0.46
Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar	1.1468526254162272 ns/iter	2.4906250776448444 ns/iter	0.46
Regex_Period_Asterisk	1.1465043332948996 ns/iter	2.4944338826279284 ns/iter	0.46
Regex_Group_Period_Asterisk_Group	1.1482502723077972 ns/iter	2.4920789845738245 ns/iter	0.46
Regex_Period_Plus	1.2892797541779282 ns/iter	3.114870895841509 ns/iter	0.41
Regex_Period	1.288973820155407 ns/iter	3.1166792059983384 ns/iter	0.41
Regex_Caret_Period_Plus_Dollar	1.289887365751487 ns/iter	3.116836017358497 ns/iter	0.41
Regex_Caret_Group_Period_Plus_Group_Dollar	1.294223634605698 ns/iter	3.1118813708243924 ns/iter	0.42
Regex_Caret_Period_Asterisk_Dollar	1.2105590315225587 ns/iter	3.732799265722568 ns/iter	0.32
Regex_Caret_Group_Period_Asterisk_Group_Dollar	1.1454318444877323 ns/iter	3.73368216618657 ns/iter	0.31
Regex_Caret_X_Hyphen	2.401122156656291 ns/iter	5.919587309848701 ns/iter	0.41
Regex_Period_Md_Dollar	15.745966893234053 ns/iter	27.416364377804417 ns/iter	0.57
Regex_Caret_Slash_Period_Asterisk	2.286024260936966 ns/iter	6.538644023792727 ns/iter	0.35
Regex_Caret_Period_Range_Dollar	1.442267443687106 ns/iter	4.356288636507369 ns/iter	0.33
Regex_Nested_Backtrack	25.497653012430437 ns/iter	42.676241441832914 ns/iter	0.60
JSON_Array_Of_Objects_Unique	306.93571731062985 ns/iter	470.7249687098731 ns/iter	0.65
JSON_Parse_1	2882.0093558981944 ns/iter	4469.8773103744 ns/iter	0.64
JSON_Parse_Real	3396.160353685112 ns/iter	5048.170057434267 ns/iter	0.67
JSON_Parse_Decimal	5134.91337402757 ns/iter	7283.617348051341 ns/iter	0.70
JSON_Parse_Schema_ISO_Language	2080531.7319277513 ns/iter	3214042.3899082844 ns/iter	0.65
JSON_Parse_Integer	2193.929312585111 ns/iter	3574.464997502427 ns/iter	0.61
JSON_Parse_String_NonSSO_Plain	2008.458864632907 ns/iter	3094.557511082769 ns/iter	0.65
JSON_Parse_String_SSO_Plain	1606.7468687130183 ns/iter	2634.208624726175 ns/iter	0.61
JSON_Parse_String_Escape_Heavy	9468.21040653049 ns/iter	12061.352299325146 ns/iter	0.79
JSON_Parse_Object_Short_Keys	4563.649770988317 ns/iter	7711.873292157831 ns/iter	0.59
JSON_Parse_Object_Scalar_Properties	2370.224955738508 ns/iter	3951.228876049743 ns/iter	0.60
JSON_Parse_Object_Array_Properties	3222.269614137484 ns/iter	5268.3101894004285 ns/iter	0.61
JSON_Parse_Object_Object_Properties	3473.3639940086914 ns/iter	5249.086499771545 ns/iter	0.66
JSON_Parse_Nested_Containers	21399.243169649704 ns/iter	34632.51141371667 ns/iter	0.62
JSON_From_String_Copy	6.221453882810139 ns/iter	10.814582306133591 ns/iter	0.58
JSON_From_String_Temporary	5.7004687818948305 ns/iter	9.877679983388921 ns/iter	0.58
JSON_Number_To_Double	14.648355281147836 ns/iter	22.908251539527257 ns/iter	0.64
JSON_Object_At_Last_Key/8	2.5876555370720666 ns/iter	3.9300633688864552 ns/iter	0.66
JSON_Object_At_Last_Key/32	9.156149842533459 ns/iter	12.326828710365382 ns/iter	0.74
JSON_Object_At_Last_Key/128	40.44732546937961 ns/iter	49.241394079026456 ns/iter	0.82
JSON_Object_At_Last_Key/512	138.43019051522188 ns/iter	382.22652926837605 ns/iter	0.36
JSON_Fast_Hash_Helm_Chart_Lock	37.10427427885715 ns/iter	62.88060346854943 ns/iter	0.59
JSON_Equality_Helm_Chart_Lock	89.02576193397677 ns/iter	159.13562260831185 ns/iter	0.56
JSON_Divisible_By_Decimal	185.60143958977622 ns/iter	244.84191731611935 ns/iter	0.76
JSON_String_Equal/10	2.049461421634804 ns/iter	5.921069340187636 ns/iter	0.35
JSON_String_Equal/100	2.8587286032144914 ns/iter	6.549879544406178 ns/iter	0.44
JSON_String_Equal_Small_By_Perfect_Hash/10	0.4817718578845434 ns/iter	0.9353299130350242 ns/iter	0.52
JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10	7.71224580069366 ns/iter	14.958352650624624 ns/iter	0.52
JSON_String_Fast_Hash/10	1.1481936066100942 ns/iter	2.177231200521529 ns/iter	0.53
JSON_String_Fast_Hash/100	1.1264219433199718 ns/iter	2.1782935803964336 ns/iter	0.52
JSON_String_Key_Hash/10	1.0943615376479376 ns/iter	2.738995128834428 ns/iter	0.40
JSON_String_Key_Hash/100	5.579833880099911 ns/iter	9.028601975200173 ns/iter	0.62
JSON_Object_Defines_Miss_Same_Length	1.6677879384897576 ns/iter	3.4493405811851696 ns/iter	0.48
JSON_Object_Defines_Miss_Too_Small	1.6141708217695738 ns/iter	3.1045656732863907 ns/iter	0.52
JSON_Object_Defines_Miss_Too_Large	1.597916173609124 ns/iter	2.7070469456842092 ns/iter	0.59
Pointer_Object_Traverse	17.38653809869367 ns/iter	28.227552988155075 ns/iter	0.62
Pointer_Object_Try_Traverse	25.465031067529672 ns/iter	30.521465648125563 ns/iter	0.83
Pointer_Push_Back_Pointer_To_Weak_Pointer	98.09506730514954 ns/iter	171.19523970172557 ns/iter	0.57
Pointer_Walker_Schema_ISO_Language	1635165.1705608184 ns/iter	1504320.867549698 ns/iter	1.09
Pointer_Maybe_Tracked_Deeply_Nested/0	913948.0326481638 ns/iter	1258447.1869488733 ns/iter	0.73
Pointer_Maybe_Tracked_Deeply_Nested/1	1205869.7395264145 ns/iter	1665359.5060241318 ns/iter	0.72
Pointer_Position_Tracker_Get_Deeply_Nested	518.637456910559 ns/iter	628.9619399693175 ns/iter	0.82
JSONPath_Descendant_Filter_Nested	913.5843450875775 ns/iter	1529.6327522599313 ns/iter	0.60
URITemplateRouter_Create	12842.157709186295 ns/iter	23568.909454017128 ns/iter	0.54
URITemplateRouter_Match	112.2085972431723 ns/iter	175.71259540535377 ns/iter	0.64
URITemplateRouter_Match_BasePath	122.70714927034159 ns/iter	207.28291173155804 ns/iter	0.59
URITemplateRouterView_Restore	2361.5111887567955 ns/iter	8241.652124457823 ns/iter	0.29
URITemplateRouterView_Match	94.00162807525216 ns/iter	141.60547300939177 ns/iter	0.66
URITemplateRouterView_Match_BasePath	99.71661242475508 ns/iter	162.04465692109505 ns/iter	0.62
URITemplateRouterView_Arguments	331.1208405527963 ns/iter	421.34357307712736 ns/iter	0.79
JSONL_Parse_Large	5728206.204918065 ns/iter	9047340.25973997 ns/iter	0.63
JSONL_Parse_Large_GZIP	6940651.058823509 ns/iter	10726817.833332974 ns/iter	0.65
JSONLD_Catalog_Annotation_List_Populate	480751.22886598913 ns/iter	587513.0033277945 ns/iter	0.82
JSONLD_Catalog_Materialize	2463764.6453899494 ns/iter	3420747.956097579 ns/iter	0.72
HTML_Build_Table_100000	41602670.230772145 ns/iter	56874076.36363352 ns/iter	0.73
HTML_Render_Table_100000	2896252.4262294476 ns/iter	1814545.6345178164 ns/iter	1.60
GZIP_Compress_ISO_Language_Set_3_Locations	21554719.48484753 ns/iter	32558978.409089044 ns/iter	0.66
GZIP_Decompress_ISO_Language_Set_3_Locations	2900942.6473030127 ns/iter	4060287.3352601184 ns/iter	0.71
GZIP_Compress_ISO_Language_Set_3_Schema	1346771.9307692721 ns/iter	1862248.2127660592 ns/iter	0.72
GZIP_Decompress_ISO_Language_Set_3_Schema	251832.6359066415 ns/iter	356142.8516260092 ns/iter	0.71
JOSE_VerifySignature_RS256	39210.92490000539 ns/iter	59746.05963654549 ns/iter	0.66
JOSE_VerifySignature_ES512	1519055.2754880094 ns/iter	2445372.2517482894 ns/iter	0.62

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark (macos/llvm)

Details

Benchmark suite	Current: a6b3c7c	Previous: 307253a	Ratio
Regex_Lower_S_Or_Upper_S_Asterisk	2.062499972927501 ns/iter	2.0575725495354327 ns/iter	1.00
Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar	2.3878547946429136 ns/iter	2.1172220266509747 ns/iter	1.13
Regex_Period_Asterisk	2.5606372103429527 ns/iter	2.255097832353288 ns/iter	1.14
Regex_Group_Period_Asterisk_Group	2.271209521271975 ns/iter	2.162814596867904 ns/iter	1.05
Regex_Period_Plus	2.5666393084905383 ns/iter	2.439908058211187 ns/iter	1.05
Regex_Period	4.463863711100912 ns/iter	2.7913785673327296 ns/iter	1.60
Regex_Caret_Period_Plus_Dollar	2.642740631413546 ns/iter	2.805151542539622 ns/iter	0.94
Regex_Caret_Group_Period_Plus_Group_Dollar	2.7178988841310936 ns/iter	2.531646340452302 ns/iter	1.07
Regex_Caret_Period_Asterisk_Dollar	1.8685275436424256 ns/iter	2.3590329756251784 ns/iter	0.79
Regex_Caret_Group_Period_Asterisk_Group_Dollar	1.8797881341403802 ns/iter	2.110680497131926 ns/iter	0.89
Regex_Caret_X_Hyphen	6.600734239507281 ns/iter	7.148036908676206 ns/iter	0.92
Regex_Period_Md_Dollar	20.673191813064175 ns/iter	25.044545145181257 ns/iter	0.83
Regex_Caret_Slash_Period_Asterisk	6.176709636942799 ns/iter	6.289371330474764 ns/iter	0.98
Regex_Caret_Period_Range_Dollar	2.127514509977072 ns/iter	3.7375920119758357 ns/iter	0.57
Regex_Nested_Backtrack	31.611457062626915 ns/iter	37.297763644187036 ns/iter	0.85
JSON_Array_Of_Objects_Unique	391.0496034384973 ns/iter	541.9567658625933 ns/iter	0.72
JSON_Parse_1	3546.622785922561 ns/iter	4568.240154717983 ns/iter	0.78
JSON_Parse_Real	5807.0499473438385 ns/iter	6809.654804088727 ns/iter	0.85
JSON_Parse_Decimal	6515.19429044944 ns/iter	9583.073994925066 ns/iter	0.68
JSON_Parse_Schema_ISO_Language	2461447.6483518276 ns/iter	3338608.870967828 ns/iter	0.74
JSON_Parse_Integer	3416.0203454145203 ns/iter	5121.220604789159 ns/iter	0.67
JSON_Parse_String_NonSSO_Plain	4215.535174088376 ns/iter	3617.0037981579667 ns/iter	1.17
JSON_Parse_String_SSO_Plain	2402.62430424647 ns/iter	2339.060452563039 ns/iter	1.03
JSON_Parse_String_Escape_Heavy	22934.60571391187 ns/iter	26853.90906775922 ns/iter	0.85
JSON_Parse_Object_Short_Keys	7527.503254963291 ns/iter	7679.121296487485 ns/iter	0.98
JSON_Parse_Object_Scalar_Properties	3969.8854798988823 ns/iter	3973.0087540476366 ns/iter	1.00
JSON_Parse_Object_Array_Properties	5513.574719573922 ns/iter	6096.559295788246 ns/iter	0.90
JSON_Parse_Object_Object_Properties	5026.1203219990975 ns/iter	7585.136760426253 ns/iter	0.66
JSON_Parse_Nested_Containers	34950.41080005767 ns/iter	35416.637849733306 ns/iter	0.99
JSON_From_String_Copy	15.424252668201975 ns/iter	13.857640984528443 ns/iter	1.11
JSON_From_String_Temporary	9.054292552579748 ns/iter	8.068024515004828 ns/iter	1.12
JSON_Number_To_Double	50.17332500000293 ns/iter	52.4326187170113 ns/iter	0.96
JSON_Object_At_Last_Key/8	6.267924131038002 ns/iter	6.104699999999639 ns/iter	1.03
JSON_Object_At_Last_Key/32	18.009344638935115 ns/iter	18.270054644250973 ns/iter	0.99
JSON_Object_At_Last_Key/128	82.77358407776732 ns/iter	72.54469824293635 ns/iter	1.14
JSON_Object_At_Last_Key/512	300.38206218281925 ns/iter	219.58366990504737 ns/iter	1.37
JSON_Fast_Hash_Helm_Chart_Lock	86.60292733503753 ns/iter	71.82300778779378 ns/iter	1.21
JSON_Equality_Helm_Chart_Lock	219.50008343996492 ns/iter	226.97022725647133 ns/iter	0.97
JSON_Divisible_By_Decimal	252.19030235639423 ns/iter	223.36631135579327 ns/iter	1.13
JSON_String_Equal/10	9.669029124736108 ns/iter	7.192899472872286 ns/iter	1.34
JSON_String_Equal/100	9.84842236607846 ns/iter	7.354650852933218 ns/iter	1.34
JSON_String_Equal_Small_By_Perfect_Hash/10	0.5025195978612272 ns/iter	0.4501226014659435 ns/iter	1.12
JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10	6.600907371871662 ns/iter	3.7997953359661496 ns/iter	1.74
JSON_String_Fast_Hash/10	4.615450837093928 ns/iter	3.417640618438581 ns/iter	1.35
JSON_String_Fast_Hash/100	3.5491446622661877 ns/iter	2.419654892094896 ns/iter	1.47
JSON_String_Key_Hash/10	2.8119487881761316 ns/iter	1.9432286286590381 ns/iter	1.45
JSON_String_Key_Hash/100	3.92154116204142 ns/iter	2.415469256323311 ns/iter	1.62
JSON_Object_Defines_Miss_Same_Length	4.061496874884089 ns/iter	3.4646376663943492 ns/iter	1.17
JSON_Object_Defines_Miss_Too_Small	4.503980555385392 ns/iter	3.9849446313440406 ns/iter	1.13
JSON_Object_Defines_Miss_Too_Large	4.231111772355852 ns/iter	3.214441002207001 ns/iter	1.32
Pointer_Object_Traverse	30.93701839125663 ns/iter	23.5939546397485 ns/iter	1.31
Pointer_Object_Try_Traverse	58.354859535629856 ns/iter	38.43643700647921 ns/iter	1.52
Pointer_Push_Back_Pointer_To_Weak_Pointer	291.61718232909743 ns/iter	160.81067139097718 ns/iter	1.81
Pointer_Walker_Schema_ISO_Language	2747969.91891874 ns/iter	1647206.0276676891 ns/iter	1.67
Pointer_Maybe_Tracked_Deeply_Nested/0	1814291.7746114705 ns/iter	1042326.8688295489 ns/iter	1.74
Pointer_Maybe_Tracked_Deeply_Nested/1	2423653.317152051 ns/iter	1696011.2936348787 ns/iter	1.43
Pointer_Position_Tracker_Get_Deeply_Nested	575.0741739407646 ns/iter	504.3787002515388 ns/iter	1.14
JSONPath_Descendant_Filter_Nested	2294.940074142669 ns/iter	1601.4545868614225 ns/iter	1.43
URITemplateRouter_Create	26307.055992654474 ns/iter	25136.733643628653 ns/iter	1.05
URITemplateRouter_Match	239.01923039541398 ns/iter	207.085476095864 ns/iter	1.15
URITemplateRouter_Match_BasePath	255.95889836828067 ns/iter	248.999116276661 ns/iter	1.03
URITemplateRouterView_Restore	16137.113817400177 ns/iter	11808.216880076574 ns/iter	1.37
URITemplateRouterView_Match	201.2709463404798 ns/iter	154.54716486653774 ns/iter	1.30
URITemplateRouterView_Match_BasePath	230.61108358713062 ns/iter	171.6093299392016 ns/iter	1.34
URITemplateRouterView_Arguments	691.4716646440362 ns/iter	594.0044632107558 ns/iter	1.16
JSONL_Parse_Large	13530903.466666663 ns/iter	7167514.165049282 ns/iter	1.89
JSONL_Parse_Large_GZIP	11951861.111109944 ns/iter	8192124.378789837 ns/iter	1.46
JSONLD_Catalog_Annotation_List_Populate	656350.3456376367 ns/iter	609294.0462374428 ns/iter	1.08
JSONLD_Catalog_Materialize	3732445.3280424676 ns/iter	2890086.394494649 ns/iter	1.29
HTML_Build_Table_100000	56765708.38461605 ns/iter	42857562.500008345 ns/iter	1.32
HTML_Render_Table_100000	2139957.1890243827 ns/iter	1962973.5070033227 ns/iter	1.09
GZIP_Compress_ISO_Language_Set_3_Locations	53365388.91666483 ns/iter	28332744.79166903 ns/iter	1.88
GZIP_Decompress_ISO_Language_Set_3_Locations	5565060.410000342 ns/iter	3763730.133721046 ns/iter	1.48
GZIP_Compress_ISO_Language_Set_3_Schema	1993991.3952804012 ns/iter	2124701.352499869 ns/iter	0.94
GZIP_Decompress_ISO_Language_Set_3_Schema	378093.131188122 ns/iter	334446.7750132079 ns/iter	1.13
JOSE_VerifySignature_RS256	29402.38802643317 ns/iter	26188.599445216616 ns/iter	1.12
JOSE_VerifySignature_ES512	1199032.1455397864 ns/iter	1503433.969465378 ns/iter	0.80

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark (windows/msvc)

Details

Benchmark suite	Current: a6b3c7c	Previous: 307253a	Ratio
Regex_Lower_S_Or_Upper_S_Asterisk	4.962684000000195 ns/iter	2.617790357142878 ns/iter	1.90
Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar	4.966330090393948 ns/iter	2.6217377208736132 ns/iter	1.89
Regex_Period_Asterisk	5.0007348214288685 ns/iter	2.5698617857143193 ns/iter	1.95
Regex_Group_Period_Asterisk_Group	5.048855000000003 ns/iter	2.6116439708824415 ns/iter	1.93
Regex_Period_Plus	4.699361394417961 ns/iter	2.900379683615522 ns/iter	1.62
Regex_Period	4.69268547954925 ns/iter	2.6353953994326185 ns/iter	1.78
Regex_Caret_Period_Plus_Dollar	4.657099563966831 ns/iter	2.803769774282196 ns/iter	1.66
Regex_Caret_Group_Period_Plus_Group_Dollar	4.643808180561961 ns/iter	2.9395162907059955 ns/iter	1.58
Regex_Caret_Period_Asterisk_Dollar	5.024359599335598 ns/iter	2.7250578770312 ns/iter	1.84
Regex_Caret_Group_Period_Asterisk_Group_Dollar	4.9597360000001345 ns/iter	2.645095644959814 ns/iter	1.88
Regex_Caret_X_Hyphen	7.8047678571430925 ns/iter	6.543028571428521 ns/iter	1.19
Regex_Period_Md_Dollar	55.428870000002924 ns/iter	30.9553660714279 ns/iter	1.79
Regex_Caret_Slash_Period_Asterisk	7.439707589285456 ns/iter	6.567862723213683 ns/iter	1.13
Regex_Caret_Period_Range_Dollar	5.409958999999843 ns/iter	3.142487183291524 ns/iter	1.72
Regex_Nested_Backtrack	84.55196428571097 ns/iter	42.85303086744762 ns/iter	1.97
JSON_Array_Of_Objects_Unique	676.9860714285869 ns/iter	518.3275999999637 ns/iter	1.31
JSON_Parse_1	9236.631979321479 ns/iter	8461.63633198047 ns/iter	1.09
JSON_Parse_Real	17937.744002749838 ns/iter	12505.585714285822 ns/iter	1.43
JSON_Parse_Decimal	11201.282812499436 ns/iter	9210.395089285861 ns/iter	1.22
JSON_Parse_Schema_ISO_Language	8288994.666666743 ns/iter	8184676.000000005 ns/iter	1.01
JSON_Parse_Integer	6007.595535714013 ns/iter	4615.784856662832 ns/iter	1.30
JSON_Parse_String_NonSSO_Plain	8172.813960651658 ns/iter	9657.56696428599 ns/iter	0.85
JSON_Parse_String_SSO_Plain	3845.1322408353462 ns/iter	3038.1459821430412 ns/iter	1.27
JSON_Parse_String_Escape_Heavy	22279.26474034914 ns/iter	23284.772491378935 ns/iter	0.96
JSON_Parse_Object_Short_Keys	12938.79625537352 ns/iter	10427.045312500382 ns/iter	1.24
JSON_Parse_Object_Scalar_Properties	6665.091964285758 ns/iter	6026.326496719882 ns/iter	1.11
JSON_Parse_Object_Array_Properties	11703.656249999916 ns/iter	9792.161195708479 ns/iter	1.20
JSON_Parse_Object_Object_Properties	11869.416071429026 ns/iter	9505.611582090769 ns/iter	1.25
JSON_Parse_Nested_Containers	84514.1741071436 ns/iter	65671.89285714648 ns/iter	1.29
JSON_From_String_Copy	63.34873214286115 ns/iter	49.706001676076546 ns/iter	1.27
JSON_From_String_Temporary	87.83376840028471 ns/iter	58.35402000000158 ns/iter	1.51
JSON_Number_To_Double	140.2166607142874 ns/iter	91.38993342009447 ns/iter	1.53
JSON_Object_At_Last_Key/8	7.289338169642613 ns/iter	7.667637276786035 ns/iter	0.95
JSON_Object_At_Last_Key/32	25.040578962324865 ns/iter	19.19430293563825 ns/iter	1.30
JSON_Object_At_Last_Key/128	94.91796004830273 ns/iter	92.6967547903257 ns/iter	1.02
JSON_Object_At_Last_Key/512	430.35451367147397 ns/iter	334.7926929363373 ns/iter	1.29
JSON_Fast_Hash_Helm_Chart_Lock	116.4848593750012 ns/iter	76.96814732142596 ns/iter	1.51
JSON_Equality_Helm_Chart_Lock	203.34924092192003 ns/iter	218.8042187499839 ns/iter	0.93
JSON_Divisible_By_Decimal	358.4881811825059 ns/iter	269.8904743425916 ns/iter	1.33
JSON_String_Equal/10	9.943244152038996 ns/iter	9.666414062500017 ns/iter	1.03
JSON_String_Equal/100	11.534131249999469 ns/iter	13.045951785714344 ns/iter	0.88
JSON_String_Equal_Small_By_Perfect_Hash/10	2.1462526984973733 ns/iter	1.5514195312499979 ns/iter	1.38
JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10	20.465830631238976 ns/iter	12.492328125000007 ns/iter	1.64
JSON_String_Fast_Hash/10	6.4742946428571555 ns/iter	4.975475889941144 ns/iter	1.30
JSON_String_Fast_Hash/100	6.514705357142857 ns/iter	4.056155133928462 ns/iter	1.61
JSON_String_Key_Hash/10	5.013901943083627 ns/iter	3.7496997486053236 ns/iter	1.34
JSON_String_Key_Hash/100	11.019735937499497 ns/iter	8.673103086279546 ns/iter	1.27
JSON_Object_Defines_Miss_Same_Length	4.654149787174367 ns/iter	4.8794242198394775 ns/iter	0.95
JSON_Object_Defines_Miss_Too_Small	4.781281483887004 ns/iter	5.073383999999805 ns/iter	0.94
JSON_Object_Defines_Miss_Too_Large	4.608885278144847 ns/iter	4.927324000000226 ns/iter	0.94
Pointer_Object_Traverse	68.597660714284 ns/iter	42.6442437499972 ns/iter	1.61
Pointer_Object_Try_Traverse	67.7519999999987 ns/iter	45.443912317465184 ns/iter	1.49
Pointer_Push_Back_Pointer_To_Weak_Pointer	178.15792710879893 ns/iter	164.05212036061792 ns/iter	1.09
Pointer_Walker_Schema_ISO_Language	13894055.35714364 ns/iter	10402103.124999229 ns/iter	1.34
Pointer_Maybe_Tracked_Deeply_Nested/0	2525269.915254352 ns/iter	2242921.212121363 ns/iter	1.13
Pointer_Maybe_Tracked_Deeply_Nested/1	3728848.2558139106 ns/iter	3905851.39664813 ns/iter	0.95
Pointer_Position_Tracker_Get_Deeply_Nested	557.6020089285626 ns/iter	552.4101562499288 ns/iter	1.01
JSONPath_Descendant_Filter_Nested	2408.8305705016396 ns/iter	1741.139481379695 ns/iter	1.38
URITemplateRouter_Create	39519.98178604091 ns/iter	36150.96694701925 ns/iter	1.09
URITemplateRouter_Match	218.0598429437571 ns/iter	263.4877550664341 ns/iter	0.83
URITemplateRouter_Match_BasePath	247.3301271129435 ns/iter	293.40500743245286 ns/iter	0.84
URITemplateRouterView_Restore	26265.605433916968 ns/iter	18592.207966141526 ns/iter	1.41
URITemplateRouterView_Match	174.2390295249373 ns/iter	181.15957686336836 ns/iter	0.96
URITemplateRouterView_Match_BasePath	199.2172142158175 ns/iter	212.1850624999766 ns/iter	0.94
URITemplateRouterView_Arguments	594.6572999999944 ns/iter	457.1760739706366 ns/iter	1.30
JSONL_Parse_Large	33444535.000000995 ns/iter	34660420.000000164 ns/iter	0.96
JSONL_Parse_Large_GZIP	34090138.09523674 ns/iter	34272238.09523553 ns/iter	0.99
JSONLD_Catalog_Annotation_List_Populate	3314100.000000053 ns/iter	2922060.5932204947 ns/iter	1.13
JSONLD_Catalog_Materialize	10696047.999999791 ns/iter	6916630.357143357 ns/iter	1.55
HTML_Build_Table_100000	94622300.00000155 ns/iter	98833742.8571445 ns/iter	0.96
HTML_Render_Table_100000	8262209.999999944 ns/iter	12488005.357143704 ns/iter	0.66
GZIP_Compress_ISO_Language_Set_3_Locations	38027533.33333284 ns/iter	31930936.363639 ns/iter	1.19
GZIP_Decompress_ISO_Language_Set_3_Locations	13095739.062499944 ns/iter	11713094.642857414 ns/iter	1.12
GZIP_Compress_ISO_Language_Set_3_Schema	2647104.694835676 ns/iter	1980188.2037531515 ns/iter	1.34
GZIP_Decompress_ISO_Language_Set_3_Schema	634843.6607143221 ns/iter	686351.4508928604 ns/iter	0.92
JOSE_VerifySignature_RS256	24415.12706331395 ns/iter	19414.598344627855 ns/iter	1.26
JOSE_VerifySignature_ES512	1625173.4375000636 ns/iter	1045495.3125000444 ns/iter	1.55

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark (linux/gcc)

Details

Benchmark suite	Current: a6b3c7c	Previous: 307253a	Ratio
JOSE_VerifySignature_RS256	22008.102950427186 ns/iter	21124.50912736474 ns/iter	1.04
JOSE_VerifySignature_ES512	582494.3120331952 ns/iter	545324.2871595323 ns/iter	1.07
GZIP_Compress_ISO_Language_Set_3_Locations	36512801.105261944 ns/iter	32406903.77272673 ns/iter	1.13
GZIP_Decompress_ISO_Language_Set_3_Locations	4368422.911949103 ns/iter	5021383.484848592 ns/iter	0.87
GZIP_Compress_ISO_Language_Set_3_Schema	2028442.547826223 ns/iter	2071749.1424332198 ns/iter	0.98
GZIP_Decompress_ISO_Language_Set_3_Schema	385077.7410665226 ns/iter	348085.60517155705 ns/iter	1.11
HTML_Build_Table_100000	55078917.16666791 ns/iter	54754428.90000011 ns/iter	1.01
HTML_Render_Table_100000	1834763.1263156328 ns/iter	2844880.897959117 ns/iter	0.64
JSONLD_Catalog_Annotation_List_Populate	696092.6766467168 ns/iter	671535.6570881308 ns/iter	1.04
JSONLD_Catalog_Materialize	3746387.917582884 ns/iter	3328320.218009601 ns/iter	1.13
JSONL_Parse_Large	9648506.35616492 ns/iter	9288097.013333451 ns/iter	1.04
JSONL_Parse_Large_GZIP	11051949.460317975 ns/iter	10722967.81538503 ns/iter	1.03
URITemplateRouter_Create	21086.42130844667 ns/iter	18434.716852100537 ns/iter	1.14
URITemplateRouter_Match	157.83251087680136 ns/iter	146.7722387443369 ns/iter	1.08
URITemplateRouter_Match_BasePath	187.15578438760392 ns/iter	165.77170091556772 ns/iter	1.13
URITemplateRouterView_Restore	8267.088350955493 ns/iter	3682.6634270342465 ns/iter	2.24
URITemplateRouterView_Match	123.76135068189043 ns/iter	116.59562772076373 ns/iter	1.06
URITemplateRouterView_Match_BasePath	143.94824153199494 ns/iter	132.27295627205797 ns/iter	1.09
URITemplateRouterView_Arguments	467.7529880344847 ns/iter	541.6550884092952 ns/iter	0.86
JSONPath_Descendant_Filter_Nested	1647.5048209209504 ns/iter	1571.1625067396114 ns/iter	1.05
Pointer_Object_Traverse	23.52577227349279 ns/iter	25.043286691793668 ns/iter	0.94
Pointer_Object_Try_Traverse	25.857914217842193 ns/iter	26.290408966471762 ns/iter	0.98
Pointer_Push_Back_Pointer_To_Weak_Pointer	128.99853025818518 ns/iter	128.25436124059536 ns/iter	1.01
Pointer_Walker_Schema_ISO_Language	1498203.3303769394 ns/iter	1736090.1670823107 ns/iter	0.86
Pointer_Maybe_Tracked_Deeply_Nested/0	1244567.4685313161 ns/iter	1184081.7436333122 ns/iter	1.05
Pointer_Maybe_Tracked_Deeply_Nested/1	1705494.7469587491 ns/iter	1939105.3833332863 ns/iter	0.88
Pointer_Position_Tracker_Get_Deeply_Nested	579.4128074956743 ns/iter	623.7529411442937 ns/iter	0.93
JSON_Array_Of_Objects_Unique	456.86853831444387 ns/iter	412.9048521144397 ns/iter	1.11
JSON_Parse_1	5331.654234742628 ns/iter	4796.112355336047 ns/iter	1.11
JSON_Parse_Real	5686.7802073679395 ns/iter	4901.035785847448 ns/iter	1.16
JSON_Parse_Decimal	10476.963817168862 ns/iter	7765.6320263108055 ns/iter	1.35
JSON_Parse_Schema_ISO_Language	3684952.9259256395 ns/iter	3384313.7766989986 ns/iter	1.09
JSON_Parse_Integer	3706.017787075118 ns/iter	3602.478932960173 ns/iter	1.03
JSON_Parse_String_NonSSO_Plain	3419.993266454244 ns/iter	3351.9925378849643 ns/iter	1.02
JSON_Parse_String_SSO_Plain	2902.5030882786746 ns/iter	2587.686193313138 ns/iter	1.12
JSON_Parse_String_Escape_Heavy	17135.98928196267 ns/iter	15963.069536499284 ns/iter	1.07
JSON_Parse_Object_Short_Keys	8700.673468879337 ns/iter	7761.881524459389 ns/iter	1.12
JSON_Parse_Object_Scalar_Properties	4382.131046140319 ns/iter	3915.1280097960516 ns/iter	1.12
JSON_Parse_Object_Array_Properties	6872.459898128628 ns/iter	5615.821101727825 ns/iter	1.22
JSON_Parse_Object_Object_Properties	6609.5097179802215 ns/iter	5540.633840047959 ns/iter	1.19
JSON_Parse_Nested_Containers	40319.51907913183 ns/iter	35361.138439771945 ns/iter	1.14
JSON_From_String_Copy	8.653095945286761 ns/iter	7.777571770547034 ns/iter	1.11
JSON_From_String_Temporary	8.961525598771262 ns/iter	7.2586064657036165 ns/iter	1.23
JSON_Number_To_Double	19.973716662780912 ns/iter	19.471846940483566 ns/iter	1.03
JSON_Object_At_Last_Key/8	5.6292868742183355 ns/iter	5.934625667717926 ns/iter	0.95
JSON_Object_At_Last_Key/32	21.54248379650553 ns/iter	19.59423966839971 ns/iter	1.10
JSON_Object_At_Last_Key/128	87.82259463760386 ns/iter	74.95884725229267 ns/iter	1.17
JSON_Object_At_Last_Key/512	395.10058111390134 ns/iter	304.56299502901174 ns/iter	1.30
JSON_Fast_Hash_Helm_Chart_Lock	75.0945294905804 ns/iter	64.41990930415045 ns/iter	1.17
JSON_Equality_Helm_Chart_Lock	147.63381129048406 ns/iter	126.13094210760553 ns/iter	1.17
JSON_Divisible_By_Decimal	283.1718024710028 ns/iter	273.91439296509776 ns/iter	1.03
JSON_String_Equal/10	5.065906109624974 ns/iter	4.434693042407398 ns/iter	1.14
JSON_String_Equal/100	5.693459988855253 ns/iter	7.507871186136169 ns/iter	0.76
JSON_String_Equal_Small_By_Perfect_Hash/10	0.6278666418139447 ns/iter	0.8257918412646853 ns/iter	0.76
JSON_String_Equal_Small_By_Runtime_Perfect_Hash/10	25.20703505004156 ns/iter	17.01464641173116 ns/iter	1.48
JSON_String_Fast_Hash/10	3.1779435778847955 ns/iter	3.091875619448593 ns/iter	1.03
JSON_String_Fast_Hash/100	3.1859377424505912 ns/iter	3.097594517319751 ns/iter	1.03
JSON_String_Key_Hash/10	1.2572419587134493 ns/iter	1.4431988659208619 ns/iter	0.87
JSON_String_Key_Hash/100	12.452630882250531 ns/iter	11.840237035888435 ns/iter	1.05
JSON_Object_Defines_Miss_Same_Length	3.1166584158622643 ns/iter	2.9153684054553692 ns/iter	1.07
JSON_Object_Defines_Miss_Too_Small	3.4273488617824865 ns/iter	3.429351730183524 ns/iter	1.00
JSON_Object_Defines_Miss_Too_Large	3.1191300101954225 ns/iter	2.9032866446745698 ns/iter	1.07
Regex_Lower_S_Or_Upper_S_Asterisk	0.6233378089124547 ns/iter	0.86505946018048 ns/iter	0.72
Regex_Caret_Lower_S_Or_Upper_S_Asterisk_Dollar	0.6230446503969304 ns/iter	0.8644839328625109 ns/iter	0.72
Regex_Period_Asterisk	0.6250745689422464 ns/iter	0.8653812215636244 ns/iter	0.72
Regex_Group_Period_Asterisk_Group	0.6238561193753116 ns/iter	0.8650437104189088 ns/iter	0.72
Regex_Period_Plus	0.6231184682525351 ns/iter	0.8652434728587546 ns/iter	0.72
Regex_Period	0.6228375023826183 ns/iter	0.8645817338302227 ns/iter	0.72
Regex_Caret_Period_Plus_Dollar	0.6229621654388166 ns/iter	0.8648171469886242 ns/iter	0.72
Regex_Caret_Group_Period_Plus_Group_Dollar	0.6235016665279914 ns/iter	0.8654462471597625 ns/iter	0.72
Regex_Caret_Period_Asterisk_Dollar	0.6239420462802416 ns/iter	0.8649036695385133 ns/iter	0.72
Regex_Caret_Group_Period_Asterisk_Group_Dollar	0.6232403941725586 ns/iter	0.8654057185754322 ns/iter	0.72
Regex_Caret_X_Hyphen	3.7403314075755723 ns/iter	2.8843426716890006 ns/iter	1.30
Regex_Period_Md_Dollar	26.70824289982389 ns/iter	22.69091451563375 ns/iter	1.18
Regex_Caret_Slash_Period_Asterisk	4.048518506529603 ns/iter	2.8836638765671814 ns/iter	1.40
Regex_Caret_Period_Range_Dollar	1.2613530186186808 ns/iter	1.4408901724973875 ns/iter	0.88
Regex_Nested_Backtrack	36.553297098738085 ns/iter	37.346838230203616 ns/iter	0.98

This comment was automatically generated by workflow using github-action-benchmark.

[augmentcode]

🤖 Augment PR Summary

Summary: This PR adds a POSIX contrib script for producing LLVM source-based coverage reports.



Changes:

• Configures a Debug, static build with Clang coverage instrumentation flags.
• Builds and runs the Core test suite with a per-process LLVM raw-profile path.
• Discovers executable paths from CTest's JSON test listing.
• Exports an LCOV trace per test executable and merges line/branch observations with AWK.
• Generates an HTML report with llvm-cov show and copies it to the requested destination.
• Prints summary, trace, and report locations for consumers of the script.
• Forwards LLVM_PROFILE_FILE through CLITest child environments.
• Adjusts the process test helper's environment display for LLVM-profile entries.
• Documents Makefile parallelism environment variables and excludes contrib from vendor pulling.

Technical notes: The script retains profile artifacts beside the build directory and excludes build, vendor, and test sources from the rendered coverage view.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

[augmentcode]

Makefile:11: make test still invokes CTest with bare --parallel on line 36, which counts as a command-line parallel setting and prevents CTest from reading CTEST_PARALLEL_LEVEL. Exporting the variable therefore does not control test parallelism as this new documentation promises.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augmentcode]

contrib/coverage.sh:198: This closes each merged LCOV record without writing any of the accumulated branches as BRDA records. Consequently coverage.lcov silently loses the branch coverage that llvm-cov export supplied, so downstream LCOV consumers cannot report branch coverage.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augmentcode]

contrib/coverage.sh:254: After a failing ctest, this branch only prints a diagnostic and the script reaches EOF successfully. Automation invoking the coverage script will therefore accept a coverage report generated from a failed test suite.

Severity: high

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[cubic-dev-ai]

3 issues found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="contrib/coverage.sh">

<violation number="1" location="contrib/coverage.sh:70">
P1: When `CC` and `CXX` are unset on a host whose default compiler is GCC, CMake sends these Clang-only flags to GCC and the build fails before coverage runs. Select `clang`/`clang++` or validate the compiler before applying LLVM instrumentation.</violation>

<violation number="2" location="contrib/coverage.sh:99">
P1: With the standard CTest `json-v1` output, this pattern never starts collecting command arguments, so every run exits without generating a report. Match the opening `[` on the `command` line, while retaining support for an array on the following line.</violation>
</file>

<file name="test/process/process_spawn_and_capture_main.cc">

<violation number="1" location="test/process/process_spawn_and_capture_main.cc:171">
P2: The filter excludes entries whose name starts with `__LLVM_PROFILE`, but the coverage variable is `LLVM_PROFILE_FILE` (single underscore, no leading underscore), which does not match that prefix. This repo itself establishes the name: `contrib/coverage.sh` runs ctest with `LLVM_PROFILE_FILE=...` and `test/process/.../clitest.h` reads `clitest_inherited("LLVM_PROFILE_FILE")`. Under instrumentation the helper's `environment` output still contains `LLVM_PROFILE_FILE=...`, so the exclusion never fires and the intended isolation does not happen. Match the real variable instead, e.g. `starts_with("LLVM_PROFILE_FILE=")`.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When CC and CXX are unset on a host whose default compiler is GCC, CMake sends these Clang-only flags to GCC and the build fails before coverage runs. Select clang/clang++ or validate the compiler before applying LLVM instrumentation.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At contrib/coverage.sh, line 70:

<comment>When `CC` and `CXX` are unset on a host whose default compiler is GCC, CMake sends these Clang-only flags to GCC and the build fails before coverage runs. Select `clang`/`clang++` or validate the compiler before applying LLVM instrumentation.</comment>

<file context>
@@ -0,0 +1,257 @@
+  -DCMAKE_COMPILE_WARNING_AS_ERROR:BOOL=ON \
+  -DSOURCEMETA_CORE_TESTS:BOOL=ON \
+  -DBUILD_SHARED_LIBS:BOOL=OFF \
+  -DCMAKE_C_FLAGS:STRING="-fprofile-instr-generate -fcoverage-mapping" \
+  -DCMAKE_CXX_FLAGS:STRING="-fprofile-instr-generate -fcoverage-mapping" \
+  -DCMAKE_EXE_LINKER_FLAGS:STRING="-fprofile-instr-generate" \
</file context>

[cubic-dev-ai]

P1: With the standard CTest json-v1 output, this pattern never starts collecting command arguments, so every run exits without generating a report. Match the opening [ on the command line, while retaining support for an array on the following line.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At contrib/coverage.sh, line 99:

<comment>With the standard CTest `json-v1` output, this pattern never starts collecting command arguments, so every run exits without generating a report. Match the opening `[` on the `command` line, while retaining support for an array on the following line.</comment>

<file context>
@@ -0,0 +1,257 @@
+OBJECT_LIST="$OUTPUT_DIRECTORY/objects.txt"
+: > "$OBJECT_LIST"
+ctest --test-dir "$BUILD_DIRECTORY" --show-only=json-v1 | awk '
+/"command" : *$/ { collecting = 1; next }
+collecting && /^[[:space:]]*\[[[:space:]]*$/ { next }
+collecting && /^[[:space:]]*\],?[[:space:]]*$/ { collecting = 0; next }
</file context>

Suggested change

	/"command" : *$/ { collecting = 1; next }
	/"command"[[:space:]]*:[[:space:]]*(\[[[:space:]]*)?$/ { collecting = 1; next }

[cubic-dev-ai]

P2: The filter excludes entries whose name starts with __LLVM_PROFILE, but the coverage variable is LLVM_PROFILE_FILE (single underscore, no leading underscore), which does not match that prefix. This repo itself establishes the name: contrib/coverage.sh runs ctest with LLVM_PROFILE_FILE=... and test/process/.../clitest.h reads clitest_inherited("LLVM_PROFILE_FILE"). Under instrumentation the helper's environment output still contains LLVM_PROFILE_FILE=..., so the exclusion never fires and the intended isolation does not happen. Match the real variable instead, e.g. starts_with("LLVM_PROFILE_FILE=").

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At test/process/process_spawn_and_capture_main.cc, line 171:

<comment>The filter excludes entries whose name starts with `__LLVM_PROFILE`, but the coverage variable is `LLVM_PROFILE_FILE` (single underscore, no leading underscore), which does not match that prefix. This repo itself establishes the name: `contrib/coverage.sh` runs ctest with `LLVM_PROFILE_FILE=...` and `test/process/.../clitest.h` reads `clitest_inherited("LLVM_PROFILE_FILE")`. Under instrumentation the helper's `environment` output still contains `LLVM_PROFILE_FILE=...`, so the exclusion never fires and the intended isolation does not happen. Match the real variable instead, e.g. `starts_with("LLVM_PROFILE_FILE=")`.</comment>

<file context>
@@ -166,6 +166,12 @@ auto main(int argc, char *argv[]) -> int {
          entry != nullptr && ((*entry) != nullptr); ++entry) {
+      // Coverage instrumentation sets this on the program itself before it
+      // starts, so it is not part of what the caller handed over
+      if (std::string_view{*entry}.starts_with("__LLVM_PROFILE")) {
+        continue;
+      }
</file context>