Commit Graph
1775 Commits
Author SHA1 Message Date
Mark Greene c5fea49f5d Fix: Faster tank and legendary camouflage model builds
## Why

In CI, `models_legendary` was the slowest chunk of the website build (65 min on 2026-11-08), slower than `models_tanks` (49 min) even though there are far fewer camouflages (443) than tanks (738). In the CI log, each camouflage takes longer than the last, from about 8 s at the start to about 50 s near the end, which looked like a leak.

It isn't a leak:

- Nothing is kept between routes (no module-level caches) and the file reader caches nothing.
- The camouflages later in the list are heavier: newer tanks with more parts and textures.
- All the work is JavaScript on one thread, so Astro's 4 routes at a time take turns on one core. Each route's time also includes waiting for the other three. The build only kept about 1 core busy.

A CPU profile of one heavy camouflage (1224, 52 s) showed where the time goes:

- **About 60% in `readTexture`:** decoding DDS textures in our own decoder (`core/src/streams/dds.ts`).
- **About 16% in `crc32`:** checksumming every `.dvpl` game file with the pure-JavaScript `crc` package.
- **Repeated work:** that camouflage decoded 32 textures, but only 8 different files. Every part decodes the camouflage's shared atlas again (4 textures × 7 parts).

## What changed

### DDS decoder (`packages/core/src/streams/dds.ts`)

The old decoder allocated at almost every step:

- for every pixel, a new 4-value array (`[...color, alpha]`), then `forEach` + `Math.round`;
- for every block, a closure plus palette objects built with `.map()` and spread;
- for every small read (2–8 bytes), a byte copy through `consumeUint8Array`;
- every loop through lodash `times()`, adding a function call per block and per pixel.

A 2048×2048 texture has 4 million pixels in 262,000 blocks, so that meant tens of millions of allocations and calls per texture, all on the single JavaScript thread.

`decodeBlocks` now:

- reads the file's bytes in place;
- works out each block's palette once into two reused 16-byte buffers;
- writes the pixels straight into the output with plain loops.

The output is unchanged. Palette values use the same formulas in the same order (for example `(2 * c0 + c1) / 3`) and are rounded with the same `Math.round(value * 255)`. Rounding once per palette entry gives the same bytes as rounding once per pixel. Two old quirks are kept on purpose:

- BC1 is treated as transparent when `color0 < color1` (the standard says `<=`).
- BC3's 6-alpha mode stores `255` for index 7, which wraps to `1` as a byte (the standard value is opaque). This is marked in a comment and left for a separate fix, because fixing it changes some textures.

### Texture reuse (`[...chunk_models_tanks]/[id]/model.glb.ts`)

`cachedImage` keeps the last 64 finished textures (encoded images), keyed by reader and file path, and wraps `readBaseColor`, `readOcclusion`, `readRoughnessMetallic` and `readNormal`. Repeat requests reuse the same image instead of decoding and encoding again. The cap keeps memory from growing over the build. The logic inside the four readers is unchanged; they're only re-indented.

### DVPL checksum (`packages/core/src/blitz/readDVPL.ts`)

`Bun.hash.crc32` replaces the `crc` package and gives the same values. `readDVPL` only runs under Bun (every caller reads through `LocalVFS`, which already uses `Bun.file`). `crc` is removed from core's dependencies; the scripts package still uses it.

## Timings

All local timings are on the same Mac, one after another, with nothing else heavy running.

### Full builds

"Before" for a local build is CI time ÷ 1.5, which matches the real ~40-minute local camouflage build of main.

| Build | CI on main (2026-11-08) | Before, local (CI ÷ 1.5) | After, local | Speed-up |
|---|---|---|---|---|
| models_tanks | 49 min | ~33 min | 6 min 37 s | ~4.9× |
| models_legendary | 65 min | ~43 min | 4 min 52 s | ~8.9× |

With the 2026-11-10 production run (58 and 57 min), it's ~5.8× for tanks and ~7.8× for camouflages.

The camouflage build after each step, locally:

| Stage | models_legendary |
|---|---|
| main (real local build) | ~40 min |
| Texture reuse + checksum | 32 min 24 s |
| + DDS decoder | 4 min 52 s |

### Samples (main against this change, same machine)

| Sample | main | After | Speed-up | Identical |
|---|---|---|---|---|
| 30 DDS textures (BC1/BC2/BC3, incl. sRGB) | 6.1 s | 0.3 s | ~20× | 30/30 |
| 5 heavy late camouflages from CI | 124.6 s | 13.4 s | ~9.3× | 5/5 |
| 45 camouflages (every 10th in build order) | 258.5 s | 77.2 s | ~3.35× | 45/45 |
| 29 tanks (15 slowest in CI + 14 spread) | 105.3 s | 43.6 s | ~2.4× | 29/29 |

The 5 heavy camouflages after each step:

| Camouflage | main | Texture reuse | + checksum | + DDS decoder |
|---|---|---|---|---|
| 1224 | 49.7 s | 6.2 s | 6.4 s | 3.2 s |
| 2146 | 17.2 s | 6.3 s | 6.4 s | 2.9 s |
| 2122 | 31.4 s | 7.5 s | 5.9 s | 2.6 s |
| 2143 | 16.9 s | 10.1 s | 6.1 s | 2.8 s |
| 2059 | 9.6 s | 5.8 s | 4.1 s | 1.9 s |
| **Total** | **124.6 s** | **36.0 s** | **28.9 s** | **13.4 s** |

The single-camouflage speed-up (3.35× on the 45-camouflage sample) is smaller than the full-build one (~8×). With decoding no longer holding the JavaScript thread, Astro's 4 routes at a time overlap instead of queuing, and sharp's threads run alongside them.

### Full site build (`CHUNK=*`, all chunks one after another)

36 min 36 s, 0 errors, 21,613 routes, peak memory 4.6 GB.

| Chunk | Time |
|---|---|
| api | ~4.3 min |
| models_tanks | ~6.1 min |
| models_legendary | ~4.6 min |
| default (pages, 9 locales) | ~21 min |

### CPU and memory

- The tank build used 1,183 s of CPU in 397 s, so about 3 cores busy on average; before, it was about 1.
- Peak memory was 3.5 GB for the tank build and 4.6 GB for the full site.
- CI runners have 4 cores and 16 GB, so this fits. No threads or processes were added: Astro's concurrency (4) and sharp's threading are unchanged.

## What this means for CI

The chunks run as parallel jobs, so the website build takes as long as its slowest chunk. Before, that was `models_legendary` at about 65 min. Using CI ≈ 1.5× local, tank models should take about 10 min and camouflages about 7–8 min. `default` (about 36 min in CI) becomes the slowest chunk, so the whole build should drop from about 65 min to about 36. This is an estimate until the next CI run.

## Checked

- Byte-identical GLBs (SHA-1) for 45 camouflages and 29 tanks against main, and decoded pixels for 30 DDS textures.
- A full site build finishes with no errors.
- Animations and particles are unaffected: the camouflage GLBs, including their animation data, are byte-identical.
2026-11-13 17:06:17 +13:00
tresabhi ad0c8b3431 open source WOTB_GLOSSARY 2026-11-08 18:38:46 -06:00
tresabhi be9dd9721d non transparent ground 2026-11-08 18:30:07 -06:00
tresabhi 47cb0f36f9 download wg parts 2026-11-08 18:10:00 -06:00
tresabhi 7918a88666 fetch urls 2026-11-08 18:07:12 -06:00
tresabhi cd2ef98e22 code split download client 2026-11-08 17:38:55 -06:00
tresabhi ce27388c00 organize 2026-11-07 20:48:08 -06:00
tresabhi e8b2651595 move to global scripts 2026-11-07 20:44:23 -06:00
tresabhi 01445cf845 move download-smart-dlc to closed 2026-11-07 20:20:36 -06:00
tresabhi b121147b43 undo optional dep 2026-11-07 20:17:10 -06:00
tresabhi b3b56462d1 move @blitzkit/closed to optional 2026-11-07 20:07:30 -06:00
Mark Greene 232b3e7770 Fix: Address karelpak1's review on animated legendary camouflages
- Sockets bound to a joint now sit right on the joint, as in game. Only the skeleton's own children are bound. This fixes the WZ-114 gun barrels collapsing.
- Clips keep their full duration when it runs past their last key.
- Part states are kept per entity and exported on each part's own switchers, so parts with matching names can't overwrite each other.
- A finished one-shot animation now stops on whatever it switched to, as in game. It only keeps looping if it changes nothing.
- Delayed actions run in delay order.
2026-11-07 15:35:12 +13:00
Mark Greene 99373872ed Merge branch 'main' into animated-legendary-camos 2026-11-07 15:02:21 +13:00
Mark Greene fddf407cc0 Fix: Address review on animated legendary camouflages
- Inline the camouflage preset lookup instead of a one-off `camouflagePreset` helper
- Move the `primitive` override into the `jsxTree` mergers, so `skinTree` takes plain mergers and falls back to `<primitive>` like the mesh and group defaults
2026-11-05 17:26:56 +13:00
tresabhi 88346308cb more hacks for this stupid ai slop wargaming is shoving down the client 2026-11-04 16:14:42 -06:00
tresabhi ebedac0642 some hacks to get some tier 11's compiling 2026-11-04 16:04:02 -06:00
TrèsAbhi a2b97380ac Update countdown text to reflect 3 tanks 2026-11-04 03:50:49 +00:00
tresabhi de4094554e countdown 2026-11-03 09:26:32 -06:00
tresabhi 7fb73edb87 Merge branch 'main' into pr/Hitjack007/283 2026-11-01 13:29:12 -06:00
tresabhi c82d2ccf1c Merge branch 'main' into pr/Hitjack007/378-2 2026-11-01 13:26:16 -06:00
tresabhi f4894655b0 yeah this is fine 2026-11-01 13:23:35 -06:00
tresabhi e4b49e1600 use canvas 2026-11-01 13:20:02 -06:00
tresabhi 7bfd930cea Revert "remove granuality"
This reverts commit b65229dc80.
2026-11-01 13:15:06 -06:00
tresabhi 71e9e6fd5b Revert "use canvas"
This reverts commit ff507ed4e6.
2026-11-01 13:15:06 -06:00
tresabhi be0be90bdb Revert "no gray picking"
This reverts commit 62637418e5.
2026-11-01 13:15:06 -06:00
tresabhi 62637418e5 no gray picking 2026-11-01 13:07:22 -06:00
tresabhi ff507ed4e6 use canvas 2026-11-01 13:05:23 -06:00
tresabhi b65229dc80 remove granuality 2026-11-01 13:03:37 -06:00
tresabhi 4a6a5b3d34 Merge branch 'main' into pr/Hitjack007/285-1 2026-11-01 12:55:25 -06:00
Mark Greene 682f039c6c Fix: Hide unused states of legendary camouflage parts (fixes maus) 2026-11-01 14:46:21 +13:00
tresabhi 0632899ecf disable halloween 2026-10-31 18:14:10 -05:00
Mark Greene 8566b2cc2f Feature: Looping garage animations for legendary camouflages
Legendary camo add-ons now loop their garage animation in the Tankopedia viewer. It's always on, with no settings.

- The garage replay now also works out which motions end up looping, and the export writes only those clips
- Customization parts export skeletons, skin weights and sockets
- Parts render as primitives so their bones can animate, with attachments layered on top
- Base tank models are unchanged
2026-10-31 22:15:57 +13:00
tresabhi e216b4cd6b fix 1 pixel lights 2026-10-30 16:49:43 -05:00
Mark Greene c535811c1d Fix: Fade glowing camouflage parts with the lights
Unlit `Textured` materials loaded as `MeshBasicMaterial`, so they stayed bright while the lights receded during a camouflage switch (E 100 legendary camouflage).

- `Textured` materials now export as standard materials with the base color texture as emissive and a black base color, so they look the same as before.
- The Tankopedia model fades their emissive intensity with the lighting transition.
- Additive blending applies to these parts again, since they are standard materials.
2026-10-30 17:26:50 +13:00
Mark Greene c772d47a07 Fix: Unify Tankopedia model, camouflage and attachment loaders
- Tankopedia state uses `loaders: { model, skin, attachment }` with `requested` and `loaded`
- useSkin swaps the camouflage halfway through the lighting transition for both cached and downloaded skins
- Remove the leftover "Unreleased" camouflage name handling
2026-10-30 07:11:31 +13:00
tresabhi 523fee5010 its only ever going to be standard material 2026-10-29 11:15:28 -05:00
tresabhi e5be69dd37 no unreleased! 2026-10-29 11:08:20 -05:00
tresabhi bddcf10e90 assert 2026-10-29 11:07:33 -05:00
tresabhi 484d4ae3da please dont make name undefinedable, just default to the string id 2026-10-29 11:07:11 -05:00
Mark Greene 0f299335d6 Fix: Legendary camouflage parts, materials, names and attachments
Fixes a batch of legendary camouflage problems in the Tankopedia viewer, adds three missing unique camouflages and lets attachments be shown together with a camouflage.

## Camouflage parts (state switchers)
- Camouflages now choose which tank pieces show the same way the game does. Each switcher's load-time scenario script runs at export time (checks for attached camouflage items, or compares the camouflage preset), along with its state-switch actions. This replaces the old guess from part names (new `core/blitzkit/switcherStates.ts`).
- Fixes missing roof MGs, fenders and fuel tanks (IS-3 Bear, Obj. 140 Terror, IS-7, IS-4, FV215b 183, AMX 50 B, KV-1S) and default parts mixed into camouflages (Pershing Speedfire, Forest Witch Fullmetal, Waffenträger Goldstreifen).
- Switchers inside camouflage parts now show one state instead of all of them stacked (Obj. Kust double net, Goldstreifen stowage, Tiger II cat).
- States are matched by original node names, because the GLTF loader renames duplicates.
- Camouflage part meshes with no LODs (`lodIndex -1`) are kept, which restores the Tiger II cat body and missing pieces on about 11 other camouflages.

## Materials
- `AlphaBlend` materials export as transparent. `AlphaBlendAdditive` materials export as additive and are applied in `jsxTree`, and unlit `Textured` materials export as unlit (IS-8 Raroh flames, dinosaur holograms, and glass and screens on a few base tanks).
- Camouflage part materials that share a name with the tank's material no longer replace the tank's camouflage material (Bisonte C45 Unbreakable).

## Names
- With a legendary camouflage selected, the title shows the tank's in-battle name ("GuP Type 95 Ha-Go", "KV-1 Warrior", "Shinobi").
- Camouflages without a name in the game files yet show "Unreleased" in orange instead of their raw string key, and no longer crash the tank names list.

## Attachments
- Attachments (`kind: customization`) are listed in their own group under the camouflages and can be turned on together with a camouflage. They load the same way as camouflages and fade with the same transition.

## Other
- Liberté, You Rock! and Ozzie (single-tank camouflages that only set a preset through `override`) are now available.
- Mastery gun mark meshes (`custom_gun_marks_*`) are no longer drawn (Helsing, HWT20 FalconFire and two camouflage guns).

## Not included
- Pixel Wave, Digital Trace and Forest Shadow are tiled pattern camouflages, which the viewer can't render yet.
- FV301 Corbie's drone position and the dinosaur heads' glow depend on camouflage animations and emissive materials, which are separate PRs.

Fixes #417
Fixes #419
Fixes #421
2026-10-29 15:28:11 +13:00
TrèsAbhi 590f4a5df3 Merge pull request #407 from blitzkit/main-i18n
New Crowdin updates
2026-10-25 18:41:43 -05:00
TrèsAbhi dcd77d5666 Merge branch 'main' into beg 2026-10-25 18:38:48 -05:00
tresabhi 46fd061aff simplify 2026-10-25 18:37:28 -05:00
Mark Greene 049ef58691 Fix legendary camos
Fixed https://github.com/blitzkit/blitzkit/commit/b540ab781419cbaa693597bf89a2f62962d8295f @tresabhi
2026-10-26 12:22:51 +13:00
TrèsAbhi bb64d718c2 Merge branch 'main' into beg 2026-10-25 18:11:37 -05:00
tresabhi 0d6c508382 remove skins from api 2026-10-25 15:00:17 -05:00
tresabhi 7f77b4006d shit ass wargaming legacy systems 2026-10-25 13:31:12 -05:00
tresabhi c11285e985 exit code retry 2026-10-25 13:03:35 -05:00
tresabhi a4ee9cf2bd automatic url 2026-10-25 12:50:48 -05:00
tresabhi 41fb822571 apply smart dlc 2026-10-25 12:27:57 -05:00