Results

Technique comparison

What switching one technique on or off changes, on the same puzzles.

Comparisons


network+aspiration=0verified	-0.16-0.23 to -0.10	outside the noise	5.91e-07
network+check_extension=0verified	-0.77-0.86 to -0.69	outside the noise	1.64e-135
network+correction_history=0verified	+0.01-0.06 to +0.08	inside the noise	0.725
network+counter_history=0verified	-0.04-0.08 to +0.01	inside the noise	0.0886
network+delta_pruningverified	+0.04+0.01 to +0.07	outside the noise	0.0174
network+futility=0verified	-1.09-1.17 to -1.02	outside the noise	7.42e-194
network+iirverified	-0.01-0.02 to +0.01	inside the noise	0.276
network+killers_history=0verified	-1.20-1.28 to -1.12	outside the noise	1.71e-212
network+late_move_pruningverified	+0.22+0.13 to +0.30	outside the noise	5.18e-08
network+lmr=0verified	-0.28-0.35 to -0.20	outside the noise	1.13e-14
network+null_move=0verified	-0.15-0.20 to -0.09	outside the noise	4.63e-11
network+probcut=0verified	-0.05-0.08 to -0.03	outside the noise	4.88e-05
network+razoringverified	-0.30-0.35 to -0.25	outside the noise	5.05e-72
network+reverse_futility=0verified	+0.13+0.05 to +0.21	outside the noise	0.00403
network+see=0verified	-0.67-0.73 to -0.61	outside the noise	1.6e-121
network+singularverified	-0.02-0.05 to +0.00	inside the noise	0.0899

Change is the second run minus the first, in points of puzzles solved, with the 95% interval below it. A run whose interval holds 0 is inside the noise.

Run details

Every result has a run. A run records the engine commit and the hash of the binary, the machine, the settings, the data that it used, the commit of this framework, and the conditions when it ran. Select a run in a table to open its details.

network+aspiration=0change -0.16 points

Result
A solved	93.4% (None)
B solved	93.2% (network+aspiration=0)
Change, B minus A	-0.16 points (-0.23 to -0.10)
Only A solved	1554
Only B solved	1287
p (McNemar)	5.91e-07
Puzzles	200,000
Run
Run	20261018T224620Z-compare-8ee31762
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:20Z
Config	default against network+aspiration=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T212643Z-puzzles-bec0f575
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.52734375

Back to the table

network+check_extension=0change -0.77 points

Result
A solved	93.4% (None)
B solved	92.6% (network+check_extension=0)
Change, B minus A	-0.77 points (-0.86 to -0.69)
Only A solved	3335
Only B solved	1610
p (McNemar)	1.64e-135
Puzzles	200,000
Run
Run	20261018T224621Z-compare-43d4dc48
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:21Z
Config	default against network+check_extension=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T213426Z-puzzles-dce451f2
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.52734375

Back to the table

network+correction_history=0change +0.01 points

Result
A solved	93.4% (None)
B solved	93.4% (network+correction_history=0)
Change, B minus A	+0.01 points (-0.06 to +0.08)
Only A solved	1607
Only B solved	1628
p (McNemar)	0.725
Puzzles	200,000
Run
Run	20261018T224611Z-compare-a7aad0cd
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:11Z
Config	default against network+correction_history=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T204021Z-puzzles-6aa69704
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.44091796875

Back to the table

network+counter_history=0change -0.04 points

Result
A solved	93.4% (None)
B solved	93.4% (network+counter_history=0)
Change, B minus A	-0.04 points (-0.08 to +0.01)
Only A solved	832
Only B solved	763
p (McNemar)	0.0886
Puzzles	200,000
Run
Run	20261018T224622Z-compare-3dff963c
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:22Z
Config	default against network+counter_history=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T214159Z-puzzles-775be4d8
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.6455078125

Back to the table

network+delta_pruningchange +0.04 points

Result
A solved	93.4% (None)
B solved	93.4% (network+delta_pruning)
Change, B minus A	+0.04 points (+0.01 to +0.07)
Only A solved	278
Only B solved	338
p (McNemar)	0.0174
Puzzles	200,000
Run
Run	20261018T224618Z-compare-e5a23eac
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:18Z
Config	default against network+delta_pruning
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T211859Z-puzzles-e78cd9d1
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.52734375

Back to the table

network+futility=0change -1.09 points

Result
A solved	93.4% (None)
B solved	92.3% (network+futility=0)
Change, B minus A	-1.09 points (-1.17 to -1.02)
Only A solved	2837
Only B solved	1026
p (McNemar)	7.42e-194
Puzzles	200,000
Run
Run	20261018T224604Z-compare-b73b7eed
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:04Z
Config	default against network+futility=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T200159Z-puzzles-660f8914
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.39208984375

Back to the table

network+iirchange -0.01 points

Result
A solved	93.4% (None)
B solved	93.4% (network+iir)
Change, B minus A	-0.01 points (-0.02 to +0.01)
Only A solved	103
Only B solved	87
p (McNemar)	0.276
Puzzles	200,000
Run
Run	20261018T224615Z-compare-4460bd79
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:15Z
Config	default against network+iir
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T210333Z-puzzles-f07776ee
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.48583984375

Back to the table

network+killers_history=0change -1.20 points

Result
A solved	93.4% (None)
B solved	92.2% (network+killers_history=0)
Change, B minus A	-1.20 points (-1.28 to -1.12)
Only A solved	3090
Only B solved	1112
p (McNemar)	1.71e-212
Puzzles	200,000
Run
Run	20261018T224624Z-compare-b60c06b8
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:24Z
Config	default against network+killers_history=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T214938Z-puzzles-26018564
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.6455078125

Back to the table

network+late_move_pruningchange +0.22 points

Result
A solved	93.4% (None)
B solved	93.6% (network+late_move_pruning)
Change, B minus A	+0.22 points (+0.13 to +0.30)
Only A solved	1996
Only B solved	2356
p (McNemar)	5.18e-08
Puzzles	200,000
Run
Run	20261018T224614Z-compare-9585ccb8
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:14Z
Config	default against network+late_move_pruning
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T205542Z-puzzles-02b8ee0e
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.48583984375

Back to the table

network+lmr=0change -0.28 points

Result
A solved	93.4% (None)
B solved	93.1% (network+lmr=0)
Change, B minus A	-0.28 points (-0.35 to -0.20)
Only A solved	2075
Only B solved	1606
p (McNemar)	1.13e-14
Puzzles	200,000
Run
Run	20261018T224607Z-compare-8537a32d
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:07Z
Config	default against network+lmr=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T201717Z-puzzles-7ea13abd
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.44091796875

Back to the table

network+null_move=0change -0.15 points

Result
A solved	93.4% (None)
B solved	93.2% (network+null_move=0)
Change, B minus A	-0.15 points (-0.20 to -0.09)
Only A solved	1215
Only B solved	911
p (McNemar)	4.63e-11
Puzzles	200,000
Run
Run	20261018T224605Z-compare-d4132e81
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:05Z
Config	default against network+null_move=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T200929Z-puzzles-aeac08f5
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.39208984375

Back to the table

network+probcut=0change -0.05 points

Result
A solved	93.4% (None)
B solved	93.3% (network+probcut=0)
Change, B minus A	-0.05 points (-0.08 to -0.03)
Only A solved	269
Only B solved	182
p (McNemar)	4.88e-05
Puzzles	200,000
Run
Run	20261018T224610Z-compare-2f13327e
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:10Z
Config	default against network+probcut=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T203239Z-puzzles-ee359638
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.44091796875

Back to the table

network+razoringchange -0.30 points

Result
A solved	93.4% (None)
B solved	93.1% (network+razoring)
Change, B minus A	-0.30 points (-0.35 to -0.25)
Only A solved	1362
Only B solved	581
p (McNemar)	5.05e-72
Puzzles	200,000
Run
Run	20261018T224613Z-compare-0404643a
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:13Z
Config	default against network+razoring
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T204805Z-puzzles-c9782aad
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.48583984375

Back to the table

network+reverse_futility=0change +0.13 points

Result
A solved	93.4% (None)
B solved	93.5% (network+reverse_futility=0)
Change, B minus A	+0.13 points (+0.05 to +0.21)
Only A solved	2021
Only B solved	2209
p (McNemar)	0.00403
Puzzles	200,000
Run
Run	20261018T224603Z-compare-2b5d5ba0
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:03Z
Config	default against network+reverse_futility=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T195414Z-puzzles-6f3411af
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.39208984375

Back to the table

network+see=0change -0.67 points

Result
A solved	93.4% (None)
B solved	92.7% (network+see=0)
Change, B minus A	-0.67 points (-0.73 to -0.61)
Only A solved	1802
Only B solved	660
p (McNemar)	1.6e-121
Puzzles	200,000
Run
Run	20261018T224608Z-compare-6f78f172
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:08Z
Config	default against network+see=0
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T202500Z-puzzles-88d9a59a
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.44091796875

Back to the table

network+singularchange -0.02 points

Result
A solved	93.4% (None)
B solved	93.4% (network+singular)
Change, B minus A	-0.02 points (-0.05 to +0.00)
Only A solved	244
Only B solved	207
p (McNemar)	0.0899
Puzzles	200,000
Run
Run	20261018T224617Z-compare-93c29048
Protocol	compare version 1
Grade	verified Made from a clean state.
Started	2026-10-18T22:46:17Z
Config	default against network+singular
Limits	a: 20261018T194649Z-puzzles-65e441ed b: 20261018T211117Z-puzzles-a78039a7
Data
Dataset	lichess-puzzles-sample 506b1f835374
Machine
CPU	13th Gen Intel(R) Core(TM) i9-13900HX, 32 logical CPUs
Kernel	7.0.0-38-generic
Framework
Commit	4b41d6414940
State	clean
Conditions
Power source	ac
Governor	powersave
Load at start	0.52734375

Back to the table