Commit Graph
230 Commits
Author SHA1 Message Date
Mark Greene c5fea49f5d Fix: Faster tank and legendary camouflage model builds
## Why

In CI, `models_legendary` was the slowest chunk of the website build (65 min on 2026-11-08), slower than `models_tanks` (49 min) even though there are far fewer camouflages (443) than tanks (738). In the CI log, each camouflage takes longer than the last, from about 8 s at the start to about 50 s near the end, which looked like a leak.

It isn't a leak:

- Nothing is kept between routes (no module-level caches) and the file reader caches nothing.
- The camouflages later in the list are heavier: newer tanks with more parts and textures.
- All the work is JavaScript on one thread, so Astro's 4 routes at a time take turns on one core. Each route's time also includes waiting for the other three. The build only kept about 1 core busy.

A CPU profile of one heavy camouflage (1224, 52 s) showed where the time goes:

- **About 60% in `readTexture`:** decoding DDS textures in our own decoder (`core/src/streams/dds.ts`).
- **About 16% in `crc32`:** checksumming every `.dvpl` game file with the pure-JavaScript `crc` package.
- **Repeated work:** that camouflage decoded 32 textures, but only 8 different files. Every part decodes the camouflage's shared atlas again (4 textures × 7 parts).

## What changed

### DDS decoder (`packages/core/src/streams/dds.ts`)

The old decoder allocated at almost every step:

- for every pixel, a new 4-value array (`[...color, alpha]`), then `forEach` + `Math.round`;
- for every block, a closure plus palette objects built with `.map()` and spread;
- for every small read (2–8 bytes), a byte copy through `consumeUint8Array`;
- every loop through lodash `times()`, adding a function call per block and per pixel.

A 2048×2048 texture has 4 million pixels in 262,000 blocks, so that meant tens of millions of allocations and calls per texture, all on the single JavaScript thread.

`decodeBlocks` now:

- reads the file's bytes in place;
- works out each block's palette once into two reused 16-byte buffers;
- writes the pixels straight into the output with plain loops.

The output is unchanged. Palette values use the same formulas in the same order (for example `(2 * c0 + c1) / 3`) and are rounded with the same `Math.round(value * 255)`. Rounding once per palette entry gives the same bytes as rounding once per pixel. Two old quirks are kept on purpose:

- BC1 is treated as transparent when `color0 < color1` (the standard says `<=`).
- BC3's 6-alpha mode stores `255` for index 7, which wraps to `1` as a byte (the standard value is opaque). This is marked in a comment and left for a separate fix, because fixing it changes some textures.

### Texture reuse (`[...chunk_models_tanks]/[id]/model.glb.ts`)

`cachedImage` keeps the last 64 finished textures (encoded images), keyed by reader and file path, and wraps `readBaseColor`, `readOcclusion`, `readRoughnessMetallic` and `readNormal`. Repeat requests reuse the same image instead of decoding and encoding again. The cap keeps memory from growing over the build. The logic inside the four readers is unchanged; they're only re-indented.

### DVPL checksum (`packages/core/src/blitz/readDVPL.ts`)

`Bun.hash.crc32` replaces the `crc` package and gives the same values. `readDVPL` only runs under Bun (every caller reads through `LocalVFS`, which already uses `Bun.file`). `crc` is removed from core's dependencies; the scripts package still uses it.

## Timings

All local timings are on the same Mac, one after another, with nothing else heavy running.

### Full builds

"Before" for a local build is CI time ÷ 1.5, which matches the real ~40-minute local camouflage build of main.

| Build | CI on main (2026-11-08) | Before, local (CI ÷ 1.5) | After, local | Speed-up |
|---|---|---|---|---|
| models_tanks | 49 min | ~33 min | 6 min 37 s | ~4.9× |
| models_legendary | 65 min | ~43 min | 4 min 52 s | ~8.9× |

With the 2026-11-10 production run (58 and 57 min), it's ~5.8× for tanks and ~7.8× for camouflages.

The camouflage build after each step, locally:

| Stage | models_legendary |
|---|---|
| main (real local build) | ~40 min |
| Texture reuse + checksum | 32 min 24 s |
| + DDS decoder | 4 min 52 s |

### Samples (main against this change, same machine)

| Sample | main | After | Speed-up | Identical |
|---|---|---|---|---|
| 30 DDS textures (BC1/BC2/BC3, incl. sRGB) | 6.1 s | 0.3 s | ~20× | 30/30 |
| 5 heavy late camouflages from CI | 124.6 s | 13.4 s | ~9.3× | 5/5 |
| 45 camouflages (every 10th in build order) | 258.5 s | 77.2 s | ~3.35× | 45/45 |
| 29 tanks (15 slowest in CI + 14 spread) | 105.3 s | 43.6 s | ~2.4× | 29/29 |

The 5 heavy camouflages after each step:

| Camouflage | main | Texture reuse | + checksum | + DDS decoder |
|---|---|---|---|---|
| 1224 | 49.7 s | 6.2 s | 6.4 s | 3.2 s |
| 2146 | 17.2 s | 6.3 s | 6.4 s | 2.9 s |
| 2122 | 31.4 s | 7.5 s | 5.9 s | 2.6 s |
| 2143 | 16.9 s | 10.1 s | 6.1 s | 2.8 s |
| 2059 | 9.6 s | 5.8 s | 4.1 s | 1.9 s |
| **Total** | **124.6 s** | **36.0 s** | **28.9 s** | **13.4 s** |

The single-camouflage speed-up (3.35× on the 45-camouflage sample) is smaller than the full-build one (~8×). With decoding no longer holding the JavaScript thread, Astro's 4 routes at a time overlap instead of queuing, and sharp's threads run alongside them.

### Full site build (`CHUNK=*`, all chunks one after another)

36 min 36 s, 0 errors, 21,613 routes, peak memory 4.6 GB.

| Chunk | Time |
|---|---|
| api | ~4.3 min |
| models_tanks | ~6.1 min |
| models_legendary | ~4.6 min |
| default (pages, 9 locales) | ~21 min |

### CPU and memory

- The tank build used 1,183 s of CPU in 397 s, so about 3 cores busy on average; before, it was about 1.
- Peak memory was 3.5 GB for the tank build and 4.6 GB for the full site.
- CI runners have 4 cores and 16 GB, so this fits. No threads or processes were added: Astro's concurrency (4) and sharp's threading are unchanged.

## What this means for CI

The chunks run as parallel jobs, so the website build takes as long as its slowest chunk. Before, that was `models_legendary` at about 65 min. Using CI ≈ 1.5× local, tank models should take about 10 min and camouflages about 7–8 min. `default` (about 36 min in CI) becomes the slowest chunk, so the whole build should drop from about 65 min to about 36. This is an estimate until the next CI run.

## Checked

- Byte-identical GLBs (SHA-1) for 45 camouflages and 29 tanks against main, and decoded pixels for 30 DDS textures.
- A full site build finishes with no errors.
- Animations and particles are unaffected: the camouflage GLBs, including their animation data, are byte-identical.
2026-11-13 17:06:17 +13:00
tresabhi ad0c8b3431 open source WOTB_GLOSSARY 2026-11-08 18:38:46 -06:00
Mark Greene 99373872ed Merge branch 'main' into animated-legendary-camos 2026-11-07 15:02:21 +13:00
Mark Greene fddf407cc0 Fix: Address review on animated legendary camouflages
- Inline the camouflage preset lookup instead of a one-off `camouflagePreset` helper
- Move the `primitive` override into the `jsxTree` mergers, so `skinTree` takes plain mergers and falls back to `<primitive>` like the mesh and group defaults
2026-11-05 17:26:56 +13:00
tresabhi 88346308cb more hacks for this stupid ai slop wargaming is shoving down the client 2026-11-04 16:14:42 -06:00
tresabhi ebedac0642 some hacks to get some tier 11's compiling 2026-11-04 16:04:02 -06:00
Mark Greene 682f039c6c Fix: Hide unused states of legendary camouflage parts (fixes maus) 2026-11-01 14:46:21 +13:00
Mark Greene 8566b2cc2f Feature: Looping garage animations for legendary camouflages
Legendary camo add-ons now loop their garage animation in the Tankopedia viewer. It's always on, with no settings.

- The garage replay now also works out which motions end up looping, and the export writes only those clips
- Customization parts export skeletons, skin weights and sockets
- Parts render as primitives so their bones can animate, with attachments layered on top
- Base tank models are unchanged
2026-10-31 22:15:57 +13:00
tresabhi bddcf10e90 assert 2026-10-29 11:07:33 -05:00
tresabhi 484d4ae3da please dont make name undefinedable, just default to the string id 2026-10-29 11:07:11 -05:00
Mark Greene 0f299335d6 Fix: Legendary camouflage parts, materials, names and attachments
Fixes a batch of legendary camouflage problems in the Tankopedia viewer, adds three missing unique camouflages and lets attachments be shown together with a camouflage.

## Camouflage parts (state switchers)
- Camouflages now choose which tank pieces show the same way the game does. Each switcher's load-time scenario script runs at export time (checks for attached camouflage items, or compares the camouflage preset), along with its state-switch actions. This replaces the old guess from part names (new `core/blitzkit/switcherStates.ts`).
- Fixes missing roof MGs, fenders and fuel tanks (IS-3 Bear, Obj. 140 Terror, IS-7, IS-4, FV215b 183, AMX 50 B, KV-1S) and default parts mixed into camouflages (Pershing Speedfire, Forest Witch Fullmetal, Waffenträger Goldstreifen).
- Switchers inside camouflage parts now show one state instead of all of them stacked (Obj. Kust double net, Goldstreifen stowage, Tiger II cat).
- States are matched by original node names, because the GLTF loader renames duplicates.
- Camouflage part meshes with no LODs (`lodIndex -1`) are kept, which restores the Tiger II cat body and missing pieces on about 11 other camouflages.

## Materials
- `AlphaBlend` materials export as transparent. `AlphaBlendAdditive` materials export as additive and are applied in `jsxTree`, and unlit `Textured` materials export as unlit (IS-8 Raroh flames, dinosaur holograms, and glass and screens on a few base tanks).
- Camouflage part materials that share a name with the tank's material no longer replace the tank's camouflage material (Bisonte C45 Unbreakable).

## Names
- With a legendary camouflage selected, the title shows the tank's in-battle name ("GuP Type 95 Ha-Go", "KV-1 Warrior", "Shinobi").
- Camouflages without a name in the game files yet show "Unreleased" in orange instead of their raw string key, and no longer crash the tank names list.

## Attachments
- Attachments (`kind: customization`) are listed in their own group under the camouflages and can be turned on together with a camouflage. They load the same way as camouflages and fade with the same transition.

## Other
- Liberté, You Rock! and Ozzie (single-tank camouflages that only set a preset through `override`) are now available.
- Mastery gun mark meshes (`custom_gun_marks_*`) are no longer drawn (Helsing, HWT20 FalconFire and two camouflage guns).

## Not included
- Pixel Wave, Digital Trace and Forest Shadow are tiled pattern camouflages, which the viewer can't render yet.
- FV301 Corbie's drone position and the dinosaur heads' glow depend on camouflage animations and emissive materials, which are separate PRs.

Fixes #417
Fixes #419
Fixes #421
2026-10-29 15:28:11 +13:00
tresabhi 7f77b4006d shit ass wargaming legacy systems 2026-10-25 13:31:12 -05:00
tresabhi 9236c54b4b rgba8888 2026-10-24 21:29:45 -05:00
Mark Greene a495a5f208 Feature: Legendary camouflages in the Tankopedia viewer 2026-10-22 16:12:03 +13:00
tresabhi 007d4e9540 make tier 11's test tanks 2026-10-21 09:56:21 -05:00
Mark Greene 85e03d3b42 Feature: Legendary camouflages in the Tankopedia viewer 2026-10-20 18:46:29 +13:00
tresabhi b9388cc544 add basic xi compatibility 2026-10-19 19:08:25 -05:00
TrèsAbhi 73080fd60a Merge branch 'main' into remove-imgur-references 2026-10-18 14:03:19 -05:00
TrèsAbhi 8132f2d91e Merge branch 'main' into calculated-bounding-boxes 2026-10-18 13:59:49 -05:00
Mark Greene a365fa5e50 Fix: migrate Imgur images to BlitzKit hosting 2026-10-08 20:28:02 +13:00
Mark Greene 57d77bf2b4 Fix: calculate tank dimensions from collision models 2026-10-08 20:25:03 +13:00
Mark Greene 940c3c537b Feature: track repair time in Tankopedia
Adds track repair time to the Tankopedia survivability stats, matching the value the game lists.

- Added a required `repair_time` to track definitions, filled from each chassis's hysteresis health divided by its health regen per second
- Applied the toolbox, the repair crew skill and the protective kit as multiplicative repair speed bonuses
- Added a Track repair time row under fire chance; it appears once the definitions are regenerated on the next build

Fixes #308
2026-10-08 18:02:05 +13:00
TrèsAbhi aa6268a86b Merge branch 'main' into stable-tank-sort-order 2026-10-07 16:30:42 -05:00
TrèsAbhi de03a2ab95 Merge branch 'main' into tier-eleven-compatibility 2026-10-07 16:23:11 -05:00
tresabhi c70fae8162 game strings 2026-10-05 11:24:29 -05:00
tresabhi d79e0ddb5d caching for strings 2026-10-05 11:24:01 -05:00
Mark Greene e4e92d76b4 Feature: tier XI compatibility
Prepares BlitzKit for tier XI tanks ahead of their release. Tier X was hardcoded as the maximum in the tank search filters, the guesser and the tech tree, and had no roman numeral beyond X.

- Added `XI` to `TIER_ROMAN_NUMERALS`
- Added an awaitable tier list derived from the highest tier in the tank definitions, so nothing changes until tier XI tanks exist
- Used it for the tank search tier filters and the guesser tier picker and defaults
- Made the tech tree check for successors instead of assuming tier X is the last tier

Fixes #302
2026-10-05 18:18:19 +13:00
Mark Greene 273881c460 Fix: keep tank order stable on select
Tanks sharing the same tier, nation, class and type were shown in an arbitrary order that changed after selecting a tank. The meta sort had no tiebreaker, and the filtered list was sorted and reversed in place, so each re-sort started from the previous reversed order and flipped tied tanks around.

- Added tank ID as the first sort key in `metaSortTank` so tied tanks always come out in the same order
- Sorted a copy of the tank search results, pre-sorted by ID, instead of the cached list
- Added an `exclude` option to tank search and used it in Compare so tanks already being compared can't be added again, including through select all or by swapping another slot

Fixes #313
2026-10-05 18:00:43 +13:00
tresabhi f3a8f2faab only supply full name if it's different 2026-10-04 14:03:32 -05:00
Mark Greene 6153beced6 Fix: search redirecting early and missing full tank names
**Enter key**
- Enter redirected to the previous results' first tank while a search was still buffering, because the key handler captured a stale `searching` value.
- Enter on an empty field redirected to the first tank of the unfiltered list (Sheridan).
- The handler now reads live filter state and only acts when there is a settled search, matching the quick-link button.

**Full names in search**
- Search only knew short tank names, so queries like "super pershing" found nothing when the game stores a short form.
- Add an optional `name_full` to `TankDefinition`, filled from the game's full name when a short name exists, and index it (plus an accent-stripped copy) in `fetchTankNames` / `SEARCH_KEYS`.
- Displayed names are unchanged. Full names become searchable once the definitions are regenerated on the next build.
2026-10-04 14:24:02 +13:00
tresabhi 1a2caf120b remove some dead code 2026-10-02 16:55:53 -05:00
tresabhi 347efa75fe add compatibility for string game mode ids 2026-09-23 16:31:35 -05:00
tresabhi 84f950ade4 fix tag0 assumption 2026-09-02 16:32:46 -05:00
tresabhi 3152c18840 simplify robots.txt 2026-08-12 11:46:45 -05:00
tresabhi 877ba9408d some more recording 2026-08-11 20:29:50 -05:00
tresabhi b3e439669b start of performance 2026-08-11 20:03:57 -05:00
tresabhi dcda627518 dvpl free dir 2026-08-11 09:45:17 -05:00
tresabhi ab87282103 use PUBLIC_ENVIRONMENT 2026-08-07 13:33:05 -05:00
tresabhi 408417ca68 bruh i am ignorant 2026-08-07 13:30:25 -05:00
tresabhi b26c5137f3 aliases 2026-08-06 20:01:50 -05:00
tresabhi bc2b9f1267 glossary 2026-08-05 13:01:51 -05:00
tresabhi 9e4d9c7968 tank model and collision 2026-08-04 20:18:05 -05:00
tresabhi 129a2c52f9 camo icon 2026-08-04 19:49:53 -05:00
tresabhi 33ebbbeb65 imports 2026-08-04 14:02:52 -05:00
tresabhi 94c3608ca7 fix missing tank map 2026-08-04 13:35:51 -05:00
tresabhi 7e9be14a9a label all chunks 2026-08-04 13:22:44 -05:00
tresabhi 5f773931bc split giant class 2026-08-04 13:19:17 -05:00
tresabhi b2fd85a5b0 some api paths 2026-08-04 13:07:27 -05:00
tresabhi b060e65af5 dynamically import client api 2026-08-02 20:28:57 -05:00
tresabhi 59108d7637 update (4) 2026-08-01 15:08:33 -05:00