BETA Cutting Room
Download
Menu

Honest, repeatable results

Benchmark

We publish how BETA markers compare with a commercial engine’s recorded results and with published research. Every number comes with its method, its assumptions and the commands to repeat it.

In short

+0.42 ppMean vs Lectra with Lectra’s own time per task (60 tasks)median −0.30 pp · 25 wins, 3 ties, 32 losses
−0.73 ppMean vs Lectra at 60 seconds per task (300 tasks)median −1.22 pp · 84 wins, 12 ties, 204 losses
−1.51 ppvs the sparrow paper’s 20-minute means, with 60 seconds
360 / 360BETA layouts on Lectra tasks that passed the independent validator (300 at 60 s, 60 at Lectra’s time)

No general win over Lectra. With Lectra’s recorded time per task, BETA’s mean was +0.42 pp, but the median task was −0.30 pp, and BETA won 25, tied 3 and lost 32 of 60 tasks. With only 60 seconds per task, BETA’s mean was −0.73 pp (BETA won 84, tied 12 and lost 204 of 300 tasks). Equal seconds do not mean equal computing power: Lectra’s engine version and hardware are unknown, and these are sampled tasks, not all Lectra customers. At 60 seconds, small markers (up to 25 pieces) range from +0.05 to +4.62 pp across the efficiency groups. Markers with more than 75 pieces range from −3.70 to +0.07 pp. Large markers are where BETA needs the most work.

A percentage point (pp) is the difference between two efficiencies: 85% against 86% is 1 pp. At 85% efficiency, 1 pp is about 1.2% of marker length.

We have no matched tests against Tukatech, Gerber or Optitex. Their markers are not in any public dataset. Send us your markers and we will compare on your styles.

Lectra dataset: real client markers

The Lectra nesting dataset has 100,000 real nesting tasks from about 100 fashion customers. For each task it records the efficiency that Lectra’s own nesting engine reached. It is the only public source of commercial-engine results on real garment markers. We take a fixed, balanced sample, run BETA on the same pieces and fabric width, and compare.

Lectra dataset: 60 seconds

Final

BETA minus Lectra: mean -0.7296 pp; median -1.2210 pp. 84 wins, 12 ties and 204 losses. Equal seconds do not imply equal compute.

−0.73 ppMean difference. Tier B (includes Tier A). On average BETA is 0.73 points behind Lectra. Median task: −1.22 pp.
−1.17%Mean marker length saving. Negative means BETA’s marker was longer.
300 / 300BETA layouts that passed the independent validator

Wins 84 · ties 12 · losses 204. A tie is within 0.1 points.

Mean difference by marker size and Lectra efficiencyBETA minus Lectra, in percentage points. 25 tasks in each row. Bars to the left mean BETA’s marker was longer.
  • 1–25 pieces, Lectra 75–85%+0.05 pp
  • 1–25 pieces, Lectra 85% or more+0.28 pp
  • 1–25 pieces, Lectra below 75%+4.62 pp
  • 151+ pieces, Lectra 75–85%−3.70 pp
  • 151+ pieces, Lectra 85% or more−3.57 pp
  • 151+ pieces, Lectra below 75%+0.07 pp
  • 26–75 pieces, Lectra 75–85%−1.25 pp
  • 26–75 pieces, Lectra 85% or more−1.12 pp
  • 26–75 pieces, Lectra below 75%+2.25 pp
  • 76–150 pieces, Lectra 75–85%−1.31 pp
  • 76–150 pieces, Lectra 85% or more−2.58 pp
  • 76–150 pieces, Lectra below 75%−2.48 pp

Correlation with the reference over 300 pairs: Pearson 0.878, Spearman 0.883. Per-task differences range from −26.85 to +37.49 pp; 90% of tasks fall between −6.17 and +5.77 pp.

Conditions and assumptions
Time budget
60 s
Machine
Shared Linux server, AMD Ryzen 9 9900X3D, 12 cores / 24 logical CPUs
Parallel work
4 tasks at a time, 3 search threads each
Seeds
1
Background load
Shared machine; see archived load.log and conditions.json. No claim of exclusive CPU use.
BETA version
0.1.0 (revision a2d5de276e5d679acd00a023476ed84209924c57)
Engine
sparrow 0.3.0 on jagua-rs 1.0.0 (unmodified)
Date
2026-10-19
Notes
Equal seconds do not imply equal compute. Runtime measures the whole nesting call and evidence I/O; validation is also timed separately.

Compared with: Lectra recorded efficiency (CC BY 4.0).

  • Search budgets exclude construction, repair and final validation overhead.
  • Shared hardware and wall-clock cutoffs can change results despite fixed seeds.
  • Sheet type 0 is assumed to be an ordinary open rectangular strip; no verified codebook is available.
  • flip_x is assumed to reflect the x coordinate before rotation.
  • Tier B includes Tier A and retains a subset of permitted orientations; unsupported constraints stay excluded.
  • The balanced stratified sample is unweighted and does not estimate population performance.
  • Lectra hardware, engine version, worker count and stopping semantics are unknown.
All numbers (table)
Lectra dataset: 60 seconds: groups
GroupValid / attemptsBETA meanReference meanMean diff.Median diff.Win / tie / lossMean length saving
Tier B (includes Tier A)300 / 30077.30%78.03%−0.73 pp−1.22 pp84 / 12 / 204−1.17%
Tier A296 / 29677.19%77.90%−0.71 pp−1.22 pp84 / 12 / 200−1.15%
Tier B only4 / 485.25%87.20%−1.95 pp−1.67 pp0 / 0 / 4−2.30%
Lectra dataset: 60 seconds: by stratum
GroupValid / attemptsBETA meanReference meanMean diff.Median diff.Win / tie / lossMean length saving
1–25 pieces, Lectra 75–85%25 / 2580.20%80.15%+0.05 pp−0.14 pp10 / 2 / 13+0.01%
1–25 pieces, Lectra 85% or more25 / 2587.74%87.46%+0.28 pp−0.01 pp11 / 3 / 11+0.25%
1–25 pieces, Lectra below 75%25 / 2564.03%59.41%+4.62 pp+1.37 pp22 / 3 / 0+6.49%
151+ pieces, Lectra 75–85%25 / 2577.67%81.37%−3.70 pp−3.77 pp0 / 0 / 25−4.82%
151+ pieces, Lectra 85% or more25 / 2584.16%87.73%−3.57 pp−3.75 pp1 / 0 / 24−4.31%
151+ pieces, Lectra below 75%25 / 2568.82%68.75%+0.07 pp−2.06 pp8 / 0 / 17−0.86%
26–75 pieces, Lectra 75–85%25 / 2580.09%81.33%−1.25 pp−1.22 pp7 / 0 / 18−1.63%
26–75 pieces, Lectra 85% or more25 / 2586.27%87.39%−1.12 pp−0.98 pp5 / 3 / 17−1.36%
26–75 pieces, Lectra below 75%25 / 2566.83%64.58%+2.25 pp−0.77 pp8 / 0 / 17+2.41%
76–150 pieces, Lectra 75–85%25 / 2579.48%80.79%−1.31 pp−1.49 pp5 / 0 / 20−1.87%
76–150 pieces, Lectra 85% or more25 / 2586.31%88.89%−2.58 pp−2.64 pp1 / 1 / 23−3.10%
76–150 pieces, Lectra below 75%25 / 2565.97%68.44%−2.48 pp−1.43 pp6 / 0 / 19−5.22%

Task IDs (300): 10236, 1057, 10582, 11012, 11734, 12009, 1331, 14495, 14560, 15268, 15918, 16118, 16557, 16628, 16849, 17290, 17525, 17677, 1816, 18453, 18562, 18791, 19148, 19321, 19589, 21054, 21445, 21968, 22106, 22684, 22849, 23598, 23743, 23776, 23787, 24038, 24063, 24496, 24912, 25299, 25999, 26522, 26533, 2689, 27040, 27472, 27626, 27711, 27749, 27855, 28611, 28840, 28844, 29474, 29735, 29936, 30314, 30388, 31797, 3197, 32439, 32538, 32541, 32890, 33075, 33203, 33224, 33311, 33513, 33566, 33672, 3371, 33718, 33831, 34215, 34287, 34356, 34453, 34685, 34892, 36035, 36118, 36372, 36408, 36695, 37325, 37398, 37664, 37729, 37917, 38083, 38101, 38174, 38205, 38442, 38704, 38728, 38730, 3914, 39324, 39513, 39737, 39744, 40309, 40407, 40462, 41006, 41281, 41493, 41692, 42273, 4232, 42808, 43166, 43735, 43847, 4408, 45069, 45098, 45509, 45617, 46162, 46919, 47077, 47802, 48, 49743, 50026, 50552, 50887, 50888, 51003, 51976, 52542, 52779, 52868, 53060, 53135, 53153, 53283, 53287, 53410, 53418, 53629, 53658, 53739, 54188, 54218, 54281, 54355, 54773, 55478, 55697, 55932, 5613, 5716, 57241, 57335, 57525, 57989, 57991, 58340, 58427, 58562, 58674, 58844, 59279, 59710, 6059, 60778, 60855, 60895, 61034, 61052, 61412, 61720, 61871, 61953, 62461, 62607, 62850, 6308, 63299, 63488, 63641, 64935, 65817, 65898, 66304, 66379, 66440, 67036, 67070, 67126, 6718, 67375, 67574, 6762, 67666, 6770, 67720, 68349, 6838, 68841, 69671, 7022, 70290, 70514, 7198, 72414, 72598, 72676, 7288, 72895, 73614, 73988, 74611, 74995, 74999, 75675, 75899, 76834, 7740, 77595, 77724, 77779, 7829, 78479, 78745, 78813, 7929, 79307, 79347, 79540, 7989, 8000, 80135, 80516, 80541, 80590, 81334, 81522, 81959, 82222, 8307, 84852, 85363, 85876, 86565, 86600, 86643, 86692, 86869, 8729, 87417, 87574, 87586, 87587, 87869, 88293, 88866, 89012, 89273, 89475, 89627, 8986, 89884, 90303, 90416, 90649, 911, 91340, 91535, 91873, 91993, 92225, 92357, 92808, 93662, 9413, 94361, 9443, 95026, 95256, 95338, 95863, 95910, 96058, 9644, 96869, 97167, 97170, 97458, 97607, 9792, 98215, 98469, 99194, 99356, 99698

Source file: data/benchmark/lectra-60s.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/public/README.md (external site).

cargo run --release --manifest-path bench/public/Cargo.toml -- --out <fresh-output-directory>

Reports: Method and results (external site)

Lectra dataset: recorded Lectra time

Final

BETA minus Lectra: mean +0.4208 pp; median -0.3044 pp. 25 wins, 3 ties and 32 losses. Equal seconds do not imply equal compute.

+0.42 ppMean difference. Tier B (includes Tier A). On average BETA is 0.42 points ahead of Lectra. Median task: −0.30 pp.
+0.61%Mean marker length saving. Negative means BETA’s marker was longer.
60 / 60BETA layouts that passed the independent validator

Wins 25 · ties 3 · losses 32. A tie is within 0.1 points.

Mean difference by marker size and Lectra efficiencyBETA minus Lectra, in percentage points. 5 tasks in each row. Bars to the left mean BETA’s marker was longer.
  • 1–25 pieces, Lectra 75–85%+0.98 pp
  • 1–25 pieces, Lectra 85% or more+0.70 pp
  • 1–25 pieces, Lectra below 75%+1.34 pp
  • 151+ pieces, Lectra 75–85%−2.77 pp
  • 151+ pieces, Lectra 85% or more−1.98 pp
  • 151+ pieces, Lectra below 75%+0.64 pp
  • 26–75 pieces, Lectra 75–85%+0.74 pp
  • 26–75 pieces, Lectra 85% or more−0.13 pp
  • 26–75 pieces, Lectra below 75%+4.85 pp
  • 76–150 pieces, Lectra 75–85%+2.04 pp
  • 76–150 pieces, Lectra 85% or more−0.72 pp
  • 76–150 pieces, Lectra below 75%−0.63 pp

Correlation with the reference over 60 pairs: Pearson 0.959, Spearman 0.945. Per-task differences range from −3.77 to +22.20 pp; 90% of tasks fall between −2.97 and +3.49 pp.

Conditions and assumptions
Time budget
Recorded Lectra duration, 244-608 s
Machine
Shared Linux server, AMD Ryzen 9 9900X3D, 12 cores / 24 logical CPUs
Parallel work
4 tasks at a time, 3 search threads each
Seeds
1
Background load
Shared machine; see archived load.log and conditions.json. No claim of exclusive CPU use.
BETA version
0.1.0 (revision a2d5de276e5d679acd00a023476ed84209924c57)
Engine
sparrow 0.3.0 on jagua-rs 1.0.0 (unmodified)
Date
2026-10-19
Notes
Equal seconds do not imply equal compute. Runtime measures the whole nesting call and evidence I/O; validation is also timed separately.

Compared with: Lectra recorded efficiency (CC BY 4.0).

  • Search budgets exclude construction, repair and final validation overhead.
  • Shared hardware and wall-clock cutoffs can change results despite fixed seeds.
  • Sheet type 0 is assumed to be an ordinary open rectangular strip; no verified codebook is available.
  • flip_x is assumed to reflect the x coordinate before rotation.
  • Tier B includes Tier A and retains a subset of permitted orientations; unsupported constraints stay excluded.
  • The balanced stratified sample is unweighted and does not estimate population performance.
  • Lectra hardware, engine version, worker count and stopping semantics are unknown.
All numbers (table)
Lectra dataset: recorded Lectra time: groups
GroupValid / attemptsBETA meanReference meanMean diff.Median diff.Win / tie / lossMean length saving
Tier B (includes Tier A)60 / 6077.31%76.89%+0.42 pp−0.30 pp25 / 3 / 32+0.61%
Tier A59 / 5977.24%76.80%+0.44 pp−0.30 pp25 / 3 / 31+0.64%
Tier B only1 / 181.42%82.37%−0.94 pp−0.94 pp0 / 0 / 1−1.16%
Same tasks at 60 s60 / 6075.80%76.89%−1.09 pp−1.09 pp20 / 2 / 38−1.38%
Lectra dataset: recorded Lectra time: by stratum
GroupValid / attemptsBETA meanReference meanMean diff.Median diff.Win / tie / lossMean length saving
1–25 pieces, Lectra 75–85%5 / 580.89%79.91%+0.98 pp+0.65 pp3 / 0 / 2+1.15%
1–25 pieces, Lectra 85% or more5 / 587.33%86.64%+0.70 pp+0.63 pp3 / 0 / 2+0.79%
1–25 pieces, Lectra below 75%5 / 552.42%51.09%+1.34 pp+1.12 pp4 / 1 / 0+2.55%
151+ pieces, Lectra 75–85%5 / 577.41%80.18%−2.77 pp−2.91 pp0 / 0 / 5−3.60%
151+ pieces, Lectra 85% or more5 / 584.96%86.93%−1.98 pp−2.29 pp1 / 0 / 4−2.37%
151+ pieces, Lectra below 75%5 / 570.68%70.04%+0.64 pp−0.42 pp2 / 0 / 3+0.89%
26–75 pieces, Lectra 75–85%5 / 581.48%80.74%+0.74 pp−0.44 pp2 / 0 / 3+0.89%
26–75 pieces, Lectra 85% or more5 / 586.92%87.05%−0.13 pp+0.37 pp3 / 0 / 2−0.18%
26–75 pieces, Lectra below 75%5 / 568.07%63.23%+4.85 pp+1.14 pp3 / 1 / 1+6.71%
76–150 pieces, Lectra 75–85%5 / 582.13%80.09%+2.04 pp+3.38 pp3 / 0 / 2+2.38%
76–150 pieces, Lectra 85% or more5 / 587.64%88.36%−0.72 pp−0.44 pp0 / 1 / 4−0.83%
76–150 pieces, Lectra below 75%5 / 567.77%68.40%−0.63 pp−0.72 pp1 / 0 / 4−1.03%

Task IDs (60): 14560, 15268, 17677, 22106, 24912, 2689, 27711, 28840, 32439, 32890, 33203, 34287, 34453, 37664, 37917, 38174, 38442, 39744, 43735, 4408, 45098, 52542, 53153, 53410, 53418, 54188, 57241, 58562, 6059, 60855, 60895, 61412, 63488, 6718, 67375, 67574, 6770, 70514, 72414, 72676, 7288, 74999, 75675, 75899, 76834, 7989, 80516, 80590, 81959, 86869, 8729, 87417, 87574, 87586, 89012, 90416, 91993, 9443, 95863, 96058

Source file: data/benchmark/lectra-matched.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/public/README.md (external site).

cargo run --release --manifest-path bench/public/Cargo.toml -- --out <fresh-output-directory>

Reports: Method and results (external site)

Lectra dataset, 60 seconds per task (round-2 laptop)

Final

Completed round-2 laptop measurement. This uses the earlier converter and engine; compare hardware, budgets and assumptions before attributing the change.

−2.36 ppMean difference. Tier B (includes Tier A). On average BETA is 2.36 points behind Lectra. Median task: −2.97 pp.
−3.62%Mean marker length saving. Negative means BETA’s marker was longer.
300 / 300BETA layouts that passed the independent validator

Wins 68 · ties 10 · losses 222. A tie is within 0.1 points.

Mean difference by marker size and Lectra efficiencyBETA minus Lectra, in percentage points. 25 tasks in each row. Bars to the left mean BETA’s marker was longer.
  • 1–25 pieces, Lectra below 75%+4.55 pp
  • 1–25 pieces, Lectra 75–85%−0.30 pp
  • 1–25 pieces, Lectra 85% or more0.00 pp
  • 26–75 pieces, Lectra below 75%+0.01 pp
  • 26–75 pieces, Lectra 75–85%−2.13 pp
  • 26–75 pieces, Lectra 85% or more−2.03 pp
  • 76–150 pieces, Lectra below 75%−5.72 pp
  • 76–150 pieces, Lectra 75–85%−3.67 pp
  • 76–150 pieces, Lectra 85% or more−3.36 pp
  • 151+ pieces, Lectra below 75%−3.10 pp
  • 151+ pieces, Lectra 75–85%−6.85 pp
  • 151+ pieces, Lectra 85% or more−5.68 pp

Correlation with the reference over 300 pairs: Pearson 0.852, Spearman 0.856. Per-task differences range from −28.88 to +37.14 pp; 90% of tasks fall between −9.78 and +5.66 pp.

Conditions and assumptions
Time budget
60 s per task
Machine
Shared 16-thread Intel laptop (i9-11900H), other programs running
Parallel work
5 tasks at a time, 3 search threads each
Seeds
1
Background load
not controlled
BETA version
0.1.0 (round-2 workspace) (revision 3022b1285ff7)
Engine
sparrow 0.3.0 on jagua-rs 1.0 (unmodified), BETA garment rules, independent validator
Date
2026-10-18
Notes
Sampling seed 7030786; 25 tasks in each of 12 strata. Import, construction, repair and validation add time outside the 60 s search. Source revision 3022b1285ff7 had recorded local benchmark changes. Dates use UTC; the matched run ended on 19 October in Asia/Kolkata.

Compared with: Lectra recorded efficiency (Lectra's own nesting engine) (CC BY 4.0).

  • Sheet type 0 is an ordinary open rectangular strip of the stated width (single layer). There is no public codebook for this code.
  • Tier B keeps a subset of the rotations and mirrors that Lectra was allowed. BETA never gets more freedom than Lectra; sometimes it gets less (a handicap for BETA).
  • flip_x = 1 is read as a reflection of the x coordinate before rotation.
  • Lectra's engine version, hardware and worker count are unknown. Equal seconds do not mean equal computing power.
  • The sample balances piece count and Lectra efficiency. Its means describe the sample, not all Lectra customers.
All numbers (table)
Lectra dataset, 60 seconds per task (round-2 laptop): groups
GroupValid / attemptsMean diff.Median diff.Win / tie / lossMean length saving
Tier B (includes Tier A)300 / 300−2.36 pp−2.97 pp68 / 10 / 222−3.62%
Tier A (exact orientation sets)293 / 293−2.33 pp−2.90 pp68 / 10 / 215−3.61%
Lectra dataset, 60 seconds per task (round-2 laptop): by stratum
GroupValid / attemptsMean diff.Median diff.Win / tie / lossMean length saving
1–25 pieces, Lectra below 75%25 / 25+4.55 pp+1.37 pp21 / 2 / 2+6.37%
1–25 pieces, Lectra 75–85%25 / 25−0.30 pp−0.13 pp10 / 1 / 14−0.44%
1–25 pieces, Lectra 85% or more25 / 250.00 pp−0.06 pp11 / 2 / 12−0.08%
26–75 pieces, Lectra below 75%25 / 25+0.01 pp−2.95 pp5 / 0 / 20−1.28%
26–75 pieces, Lectra 75–85%25 / 25−2.13 pp−2.09 pp4 / 2 / 19−2.78%
26–75 pieces, Lectra 85% or more25 / 25−2.03 pp−1.79 pp3 / 1 / 21−2.46%
76–150 pieces, Lectra below 75%25 / 25−5.72 pp−3.75 pp2 / 0 / 23−11.09%
76–150 pieces, Lectra 75–85%25 / 25−3.67 pp−3.98 pp5 / 0 / 20−5.17%
76–150 pieces, Lectra 85% or more25 / 25−3.36 pp−3.52 pp1 / 1 / 23−4.07%
151+ pieces, Lectra below 75%25 / 25−3.10 pp−4.83 pp5 / 1 / 19−6.10%
151+ pieces, Lectra 75–85%25 / 25−6.85 pp−6.82 pp0 / 0 / 25−9.31%
151+ pieces, Lectra 85% or more25 / 25−5.68 pp−5.16 pp1 / 0 / 24−7.04%

Task IDs (300): 10236, 1057, 10582, 11012, 11734, 12009, 1331, 14495, 14560, 15268, 15918, 16118, 16557, 16628, 16849, 17290, 17525, 17677, 1816, 18453, 18562, 18791, 19148, 19321, 19589, 21054, 21445, 21968, 22106, 22684, 22849, 23598, 23743, 23776, 23787, 24038, 24063, 24496, 24912, 25299, 25999, 26522, 26533, 2689, 27040, 27472, 27626, 27711, 27749, 27855, 28611, 28840, 28844, 29474, 29735, 29936, 30314, 30388, 31797, 3197, 32439, 32538, 32541, 32890, 33075, 33203, 33224, 33311, 33513, 33566, 33672, 3371, 33718, 33831, 34215, 34287, 34356, 34453, 34685, 34892, 36035, 36118, 36372, 36408, 36695, 37325, 37398, 37664, 37729, 37917, 38083, 38101, 38174, 38205, 38442, 38704, 38728, 38730, 3914, 39324, 39513, 39737, 39744, 40309, 40407, 40462, 41006, 41281, 41493, 41692, 42273, 4232, 42808, 43166, 43735, 43847, 4408, 45069, 45098, 45509, 45617, 46162, 46919, 47077, 47802, 48, 49743, 50026, 50552, 50887, 50888, 51003, 51976, 52542, 52779, 52868, 53060, 53135, 53153, 53283, 53287, 53410, 53418, 53629, 53658, 53739, 54188, 54218, 54281, 54355, 54773, 55478, 55697, 55932, 5613, 5716, 57241, 57335, 57525, 57989, 57991, 58340, 58427, 58562, 58674, 58844, 59279, 59710, 6059, 60778, 60855, 60895, 61034, 61052, 61412, 61720, 61871, 61953, 62461, 62607, 62850, 6308, 63299, 63488, 63641, 64935, 65817, 65898, 66304, 66379, 66440, 67036, 67070, 67126, 6718, 67375, 67574, 6762, 67666, 6770, 67720, 68349, 6838, 68841, 69671, 7022, 70290, 70514, 7198, 72414, 72598, 72676, 7288, 72895, 73614, 73988, 74611, 74995, 74999, 75675, 75899, 76834, 7740, 77595, 77724, 77779, 7829, 78479, 78745, 78813, 7929, 79307, 79347, 79540, 7989, 8000, 80135, 80516, 80541, 80590, 81334, 81522, 81959, 82222, 8307, 84852, 85363, 85876, 86565, 86600, 86643, 86692, 86869, 8729, 87417, 87574, 87586, 87587, 87869, 88293, 88866, 89012, 89273, 89475, 89627, 8986, 89884, 90303, 90416, 90649, 911, 91340, 91535, 91873, 91993, 92225, 92357, 92808, 93662, 9413, 94361, 9443, 95026, 95256, 95338, 95863, 95910, 96058, 9644, 96869, 97167, 97170, 97458, 97607, 9792, 98215, 98469, 99194, 99356, 99698

Source file: data/benchmark/lectra-60s-laptop-r2.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/results/2026-10-18-round2-manifest.md (external site).

Use the archived round-2 source and inputs identified in marker/bench/results/2026-10-18-round2-manifest.md. Current public-suite commands use the corrected v0.3 converter.

Reports: Round-2 60-second report (external site) · Round-2 policy audit (external site)

Lectra dataset, Lectra's own time per task (round-2 laptop)

Final

Completed round-2 laptop measurement. This uses the earlier converter and engine; compare hardware, budgets and assumptions before attributing the change.

−1.09 ppMean difference. Tier B (includes Tier A). On average BETA is 1.09 points behind Lectra. Median task: −1.13 pp.
−1.37%Mean marker length saving. Negative means BETA’s marker was longer.
60 / 60BETA layouts that passed the independent validator

Wins 20 · ties 2 · losses 38. A tie is within 0.1 points.

Mean difference by marker size and Lectra efficiencyBETA minus Lectra, in percentage points. 5 tasks in each row. Bars to the left mean BETA’s marker was longer.
  • 1–25 pieces, Lectra below 75%+1.33 pp
  • 1–25 pieces, Lectra 75–85%+0.70 pp
  • 1–25 pieces, Lectra 85% or more+0.33 pp
  • 26–75 pieces, Lectra below 75%+3.15 pp
  • 26–75 pieces, Lectra 75–85%−0.60 pp
  • 26–75 pieces, Lectra 85% or more−2.00 pp
  • 76–150 pieces, Lectra below 75%−2.12 pp
  • 76–150 pieces, Lectra 75–85%−2.28 pp
  • 76–150 pieces, Lectra 85% or more−2.29 pp
  • 151+ pieces, Lectra below 75%−1.32 pp
  • 151+ pieces, Lectra 75–85%−4.92 pp
  • 151+ pieces, Lectra 85% or more−3.02 pp

Correlation with the reference over 60 pairs: Pearson 0.959, Spearman 0.928.

Conditions and assumptions
Time budget
Lectra's recorded time (244-608 s, median 304 s)
Machine
Shared 16-thread Intel laptop (i9-11900H), other programs running
Parallel work
5 tasks at a time, 3 search threads each
Seeds
1
Background load
not controlled
BETA version
0.1.0 (round-2 workspace) (revision 3022b1285ff7)
Engine
sparrow 0.3.0 on jagua-rs 1.0 (unmodified), BETA garment rules, independent validator
Date
2026-10-18
Notes
On the same 60 tasks at 60 s the mean was -2.5276 pp. The extra time added 1.4411 pp on average. Source revision 3022b1285ff7 had recorded local benchmark changes. Dates use UTC; the matched run ended on 19 October in Asia/Kolkata.

Compared with: Lectra recorded efficiency (Lectra's own nesting engine) (CC BY 4.0).

  • Sheet type 0 is an ordinary open rectangular strip of the stated width (single layer). There is no public codebook for this code.
  • Tier B keeps a subset of the rotations and mirrors that Lectra was allowed. BETA never gets more freedom than Lectra; sometimes it gets less (a handicap for BETA).
  • flip_x = 1 is read as a reflection of the x coordinate before rotation.
  • Lectra's engine version, hardware and worker count are unknown. Equal seconds do not mean equal computing power.
  • The sample balances piece count and Lectra efficiency. Its means describe the sample, not all Lectra customers.
All numbers (table)
Lectra dataset, Lectra's own time per task (round-2 laptop): groups
GroupValid / attemptsMean diff.Median diff.Win / tie / lossMean length saving
Tier B (includes Tier A)60 / 60−1.09 pp−1.13 pp20 / 2 / 38−1.37%
Tier A (exact orientation sets)59 / 59−1.08 pp−1.07 pp20 / 2 / 37−1.36%
Same 60 tasks at 60 s60 / 60−2.53 pp−2.91 pp18 / 1 / 41−3.51%
Lectra dataset, Lectra's own time per task (round-2 laptop): by stratum
GroupValid / attemptsMean diff.Median diff.Win / tie / lossMean length saving
1–25 pieces, Lectra below 75%5 / 5+1.33 pp+1.12 pp4 / 1 / 0+2.55%
1–25 pieces, Lectra 75–85%5 / 5+0.70 pp+0.64 pp3 / 0 / 2+0.81%
1–25 pieces, Lectra 85% or more5 / 5+0.33 pp+0.46 pp3 / 0 / 2+0.38%
26–75 pieces, Lectra below 75%5 / 5+3.15 pp+0.99 pp3 / 0 / 2+4.82%
26–75 pieces, Lectra 75–85%5 / 5−0.60 pp−1.32 pp2 / 0 / 3−0.77%
26–75 pieces, Lectra 85% or more5 / 5−2.00 pp−2.53 pp0 / 1 / 4−2.38%
76–150 pieces, Lectra below 75%5 / 5−2.12 pp−2.04 pp1 / 0 / 4−3.29%
76–150 pieces, Lectra 75–85%5 / 5−2.28 pp−1.54 pp2 / 0 / 3−3.50%
76–150 pieces, Lectra 85% or more5 / 5−2.29 pp−2.64 pp0 / 0 / 5−2.70%
151+ pieces, Lectra below 75%5 / 5−1.32 pp−1.19 pp1 / 0 / 4−2.03%
151+ pieces, Lectra 75–85%5 / 5−4.92 pp−3.64 pp0 / 0 / 5−6.61%
151+ pieces, Lectra 85% or more5 / 5−3.02 pp−3.93 pp1 / 0 / 4−3.67%

Task IDs (60): 14560, 15268, 17677, 22106, 24912, 2689, 27711, 28840, 32439, 32890, 33203, 34287, 34453, 37664, 37917, 38174, 38442, 39744, 43735, 4408, 45098, 52542, 53153, 53410, 53418, 54188, 57241, 58562, 6059, 60855, 60895, 61412, 63488, 6718, 67375, 67574, 6770, 70514, 72414, 72676, 7288, 74999, 75675, 75899, 76834, 7989, 80516, 80590, 81959, 86869, 8729, 87417, 87574, 87586, 89012, 90416, 91993, 9443, 95863, 96058

Source file: data/benchmark/lectra-matched-laptop-r2.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/results/2026-10-18-round2-manifest.md (external site).

Use the archived round-2 source and inputs identified in marker/bench/results/2026-10-18-round2-manifest.md. Current public-suite commands use the corrected v0.3 converter.

Reports: Round-2 matched-duration report (external site)

Academic instances

Research papers use a small set of standard nesting instances, including shirts, trousers and swimwear. The best open-source result is the sparrow paper. BETA uses sparrow as its search engine, so this test checks that BETA’s adapter does not lose quality, at a much shorter time.

Academic instances: 60 seconds

Final

One fixed suite, with every scored marker independently validated. See the assumptions before comparing results.

−1.51 ppMean difference. All. On average BETA is 1.51 points behind the paper. Median task: −1.75 pp.
−1.78%Mean marker length saving. Negative means BETA’s marker was longer.
21 / 21BETA layouts that passed the independent validator

Wins 0 · ties 1 · losses 20. A tie is within 0.1 points.

Efficiency by instanceMean of three 60-second BETA runs against the paper’s mean of 100 runs at 1200 seconds. Further right is better.
  • AllBETA, 60 s 86.41% vs 87.92% Paper, 1200 s

Correlation with the reference over 21 pairs: Pearson 0.992, Spearman 0.955. Per-task differences range from −2.57 to −0.03 pp; 90% of tasks fall between −2.28 and −0.11 pp.

Conditions and assumptions
Time budget
60 s
Machine
Shared Linux server, AMD Ryzen 9 9900X3D, 12 cores / 24 logical CPUs
Parallel work
4 tasks at a time, 3 search threads each
Seeds
1, 2, 3
Background load
Shared machine; see archived load.log and conditions.json. No claim of exclusive CPU use.
BETA version
0.1.0 (revision a2d5de276e5d679acd00a023476ed84209924c57)
Engine
sparrow 0.3.0 on jagua-rs 1.0.0 (unmodified)
Date
2026-10-20
Notes
Equal seconds do not imply equal compute. Runtime measures the whole nesting call and evidence I/O; validation is also timed separately.

Compared with: Sparrow paper means, 100 runs at 1200 s.

  • Search budgets exclude construction, repair and final validation overhead.
  • Shared hardware and wall-clock cutoffs can change results despite fixed seeds.
  • BETA uses Sparrow itself. Published references use 1200 seconds, 100 runs and Ryzen 9 7950X hardware; this is an adapter and budget comparison.
All numbers (table)
Academic instances: 60 seconds: groups
GroupValid / attemptsBETA meanReference meanMean diff.Median diff.Win / tie / lossMean length saving
All21 / 2186.41%87.92%−1.51 pp−1.75 pp0 / 1 / 20−1.78%
Academic instances: 60 seconds: by stratum
GroupValid / attemptsBETA meanReference meanMean diff.Median diff.Win / tie / lossMean length saving
albano3 / 388.08%89.47%−1.39 pp−1.44 pp0 / 0 / 3−1.58%
dagli3 / 387.15%89.26%−2.11 pp−2.12 pp0 / 0 / 3−2.42%
mao3 / 384.61%86.14%−1.53 pp−1.81 pp0 / 0 / 3−1.82%
marques3 / 390.30%90.93%−0.63 pp−0.11 pp0 / 1 / 2−0.71%
shirts3 / 387.92%89.66%−1.74 pp−1.74 pp0 / 0 / 3−1.98%
swim3 / 376.12%78.26%−2.14 pp−2.11 pp0 / 0 / 3−2.82%
trousers3 / 390.73%91.73%−1.00 pp−0.89 pp0 / 0 / 3−1.10%

Task IDs (7): albano, dagli, mao, marques, shirts, swim, trousers

Source file: data/benchmark/academic-60s.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/public/README.md (external site).

cargo run --release --manifest-path bench/public/Cargo.toml -- --out <fresh-output-directory>

Reports: Method and results (external site)

Whole garments (FreeSewing)

FreeSewing publishes open-source garment patterns. We use them to test complete garments with real garment rules (grain, pairs, size ratio). There is no commercial result for these exact jobs, so the numbers show BETA alone.

FreeSewing: 60 seconds

Final

One fixed suite, with every scored marker independently validated. See the assumptions before comparing results.

81.45%Mean BETA efficiency. All.
3 / 3BETA layouts that passed the independent validator
Conditions and assumptions
Time budget
60 s
Machine
Shared Linux server, AMD Ryzen 9 9900X3D, 12 cores / 24 logical CPUs
Parallel work
4 tasks at a time, 3 search threads each
Seeds
1
Background load
Shared machine; see archived load.log and conditions.json. No claim of exclusive CPU use.
BETA version
0.1.0 (revision a2d5de276e5d679acd00a023476ed84209924c57)
Engine
sparrow 0.3.0 on jagua-rs 1.0.0 (unmodified)
Date
2026-10-20
Notes
Equal seconds do not imply equal compute. Runtime measures the whole nesting call and evidence I/O; validation is also timed separately.

Compared with: BETA results only (MIT (FreeSewing fixtures)).

  • Search budgets exclude construction, repair and final validation overhead.
  • Shared hardware and wall-clock cutoffs can change results despite fixed seeds.
  • Generated tailoring drafts on open 1500 mm fabric, S1 M2 L2 XL1, no nap and zero gap. No commercial reference is available.
All numbers (table)
FreeSewing: 60 seconds: groups
GroupValid / attemptsBETA meanReference mean
All3 / 381.45%–
FreeSewing: 60 seconds: by stratum
GroupValid / attemptsBETA meanReference mean
simon-ratio-as-is1 / 189.30%–
teagan-ratio-as-is1 / 178.64%–
titan-ratio-as-is1 / 176.40%–

Task IDs (3): simon-ratio-as-is, teagan-ratio-as-is, titan-ratio-as-is

Source file: data/benchmark/freesewing-60s.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/public/README.md (external site).

cargo run --release --manifest-path bench/public/Cargo.toml -- --out <fresh-output-directory>

Reports: Method and results (external site)

FreeSewing: 300 seconds

Final

One fixed suite, with every scored marker independently validated. See the assumptions before comparing results.

82.09%Mean BETA efficiency. All.
3 / 3BETA layouts that passed the independent validator
Conditions and assumptions
Time budget
300 s
Machine
Shared Linux server, AMD Ryzen 9 9900X3D, 12 cores / 24 logical CPUs
Parallel work
4 tasks at a time, 3 search threads each
Seeds
1
Background load
Shared machine; see archived load.log and conditions.json. No claim of exclusive CPU use.
BETA version
0.1.0 (revision a2d5de276e5d679acd00a023476ed84209924c57)
Engine
sparrow 0.3.0 on jagua-rs 1.0.0 (unmodified)
Date
2026-10-20
Notes
Equal seconds do not imply equal compute. Runtime measures the whole nesting call and evidence I/O; validation is also timed separately.

Compared with: BETA results only (MIT (FreeSewing fixtures)).

  • Search budgets exclude construction, repair and final validation overhead.
  • Shared hardware and wall-clock cutoffs can change results despite fixed seeds.
  • Generated tailoring drafts on open 1500 mm fabric, S1 M2 L2 XL1, no nap and zero gap. No commercial reference is available.
All numbers (table)
FreeSewing: 300 seconds: groups
GroupValid / attemptsBETA meanReference mean
All3 / 382.09%–
FreeSewing: 300 seconds: by stratum
GroupValid / attemptsBETA meanReference mean
simon-ratio-as-is1 / 189.57%–
teagan-ratio-as-is1 / 179.03%–
titan-ratio-as-is1 / 177.66%–

Task IDs (3): simon-ratio-as-is, teagan-ratio-as-is, titan-ratio-as-is

Source file: data/benchmark/freesewing-300s.json

How to repeat this run

Run from the marker/ folder of the source code. Full steps: marker/bench/public/README.md (external site).

cargo run --release --manifest-path bench/public/Cargo.toml -- --out <fresh-output-directory>

Reports: Method and results (external site)

One factory style (provisional, private)

On one real knitwear style from an Indian factory, BETA’s marker was 4.6% to 6.7% shorter than the marker from the factory’s Gemini CAD. This is one style. The data is private, so you cannot repeat the test, and it does not show that BETA beats Gemini in general. We need more factory comparisons: send us your markers.

Method

What we measure

  • Marker efficiency = total piece area ÷ (fabric width × marker length). Higher is better.
  • Difference = BETA efficiency minus the reference efficiency, in percentage points. A tie is within 0.1 points.
  • Length saving = 100 × (1 − reference efficiency ÷ BETA efficiency). Positive means BETA’s marker is shorter for the same pieces.
  • Means and medians use valid layouts only. A layout that is too long for a fixed sheet counts as a loss.

Every BETA marker is checked

An independent validator checks every BETA layout: all pieces present, no overlaps, grain and allowed rotations, fabric width and gaps. A layout that fails gets no score.

Fair rules: Tier A and Tier B

Each Lectra task says which rotations and mirror images each piece may use. Tier A tasks give BETA exactly the same choices. Tier B tasks give BETA the same choices or fewer, never more. The headline uses Tier B, which includes Tier A. Where BETA has fewer choices, that is a handicap for BETA, not for Lectra. Tasks with constraint types we cannot apply exactly are left out (3,253 of 100,000).

The sheet-type assumption

Each task has a sheet_type code. There is no public codebook. We assume code 0 means an ordinary open strip of fabric (one layer, the stated width). Code 0 covers 99,403 of 100,000 tasks. The strong correlation between BETA and Lectra results supports this, but does not prove it. Codes 2 and 3 are tested separately and never enter the headline.

Sample and time

The 300-task sample uses seed 7030786 and has 25 tasks in each of 12 groups (4 piece-count bands × 3 Lectra-efficiency bands). The 60-task subset for the matched-time run was chosen before any results were seen. Lectra’s engine version, hardware and number of workers are unknown, and its recorded times vary (244 to 608 seconds in our subset). Equal seconds do not mean equal computing power. Our machines were shared with other work.

What we do not claim

  • We do not claim BETA beats Lectra, Tukatech, Gerber, Optitex or any other commercial system.
  • The balanced sample describes those tasks. It is not an estimate for all Lectra customers.
  • Vendor fabric-saving claims (often 1–5%) are their own tests on their own data. Nobody has published a fair test between these tools.

Repeat the benchmark yourself

  1. Get the source code and install Rust. Python is needed once, to convert the Lectra dataset files.
  2. Choose a data folder outside the source code. Set BETA_DATA_DIR and CARGO_TARGET_DIR to folders on a disk with space.
  3. Convert the dataset. The converter checks the published checksums and reads the files with a restricted loader: 
    python -I bench/tools/lectra_convert.py download
python -I bench/tools/lectra_convert.py inspect
python -I bench/tools/lectra_convert.py self-test
python -I bench/tools/lectra_convert.py convert --sample-size 300 --seed 7030786
  4. Run the tests, then the benchmark commands shown under each result (“How to repeat this run”). Use a new output folder for each run. Reduce --jobs on a smaller computer.
  5. Each run writes run.json, results.csv and summary.md. The website reads summary files in the format described in marker/apps/site/data/benchmark/SCHEMA.md (external site).

Full instructions: marker/docs/benchmarks.md (external site). No pattern geometry is published on this site; the files contain totals and task IDs only.

Data and attribution