perf(defpacks): make the item packs again without holding the game up

A custom item change made the item and name packs again on the next
fetch, on the one thread every game shares: ~360 ms at 5000 items,
~1.6 s at 50000, after every change, players making them many times a
minute. Nearly all of it was the AMF3 writer.

- the writer fills one growing buffer instead of a Buffer per value
  (byte-identical output, checked against the old one on every cached
  pack): the item pack 230 -> 52 ms at 5000 items, 1524 -> 231 ms at 50000.
- a changed pack is made again at most every AJ_PACK_REMAKE_MS (15 s);
  until then the one made before goes out with the custom item version
  it has, and customItems.sendAll sends a game logging in what its packs
  lack by that version. one remake at a time; deflated on libuv's threads.
- tests make the packs again at once (AJ_PACK_REMAKE_MS=0), and a run of
  the items test checks the throttled case.

Busy evening simulated in-process (a change every 3 s, packs fetched
6 a second, 30 s): longest hold-up 329 -> 61 ms at 5000 items, 1648 ->
359 ms at 50000 (p99 240 -> 6 ms).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
BurrowdevandClaude Opus 5.5 committed 2026-10-20 14:44:07 +03:00
1 parent c5b1bc37d9
commit 04b8d325ea
7 files changed
+178 -64

No files matched your search

+12 -13
View File
@@ -53,11 +53,11 @@ const SCHEMA = `CREATE TABLE IF NOT EXISTS custom_items (
const items = new Map(KINDS.map(kind => [kind, new Map()])); // kind -> id -> entry
let made = 0; // entries in the order they were made (`seq`), for lists
let changes = 0; // goes up with every change a game must be told of; an entry's `changed` is the count at its last
let broadcast = () => {}; // (cmd, ...params) to every game playing; set by lib/world.js
// address key -> {at, clothing, den, names}: `changes` when that address
// last fetched the packs with custom items in, the oldest within
// PACKS_SEEN_MS (players behind one address fetch at different times).
// address key -> {at, clothing, den, names}: the version (defpacks
// customVersion) of the packs with custom items in that address last
// fetched, the oldest within PACKS_SEEN_MS (players behind one address
// fetch at different times). an entry's `changed` is the version of its last change.
const packsSeen = new Map();
const db = () => accounts.database();
@@ -87,7 +87,7 @@ const rowOf = entry => {
const apply = entry => {
defpacks.setCustomItem(entry.kind, rowOf(entry), entry.name, entry.inShop);
entry.changed = ++changes;
entry.changed = defpacks.customVersion();
};
// what "xdi" carries of an entry: the fields the client reads, its name and
@@ -317,7 +317,7 @@ const setArtMode = (kind, id, mode) => {
}
entry.artMode = mode;
db().prepare("UPDATE custom_items SET art_mode = ? WHERE kind = ? AND id = ?").run(mode, kind, entry.id);
entry.changed = ++changes;
entry.changed = defpacks.customChanged();
announce(entry);
return {item: entry};
};
@@ -332,7 +332,7 @@ const setArt = (kind, id, art, pending = false) => {
entry.art = art || null;
entry.artPending = Boolean(art && pending);
db().prepare("UPDATE custom_items SET art = ?, art_pending = ? WHERE kind = ? AND id = ?").run(entry.art, entry.artPending ? 1 : 0, kind, entry.id);
entry.changed = ++changes;
entry.changed = defpacks.customChanged();
announce(entry);
};
@@ -400,20 +400,19 @@ module.exports = {
const def = baseDef(entry.kind, entry.base);
return def ? defpacks.string(entry.kind === "den" ? def.nameStrId : def.titleStrId) || String(def.abbrName || entry.base) : String(entry.base);
},
// an address fetched a pack with custom items in: "clothing", "den" or
// "names" (lib/defpacks.js customPackKind). called before the pack is made,
// so that a change while it is made counts as after.
sawPack(key, packKind) {
// an address was sent a pack with custom items in: "clothing", "den" or
// "names" (lib/defpacks.js customPackKind), made at `version`.
sawPack(key, packKind, version) {
const time = Date.now();
const seen = packsSeen.get(key);
if (seen && time - seen.at < PACKS_SEEN_MS) {
if (seen[packKind] === undefined) {
seen[packKind] = changes;
seen[packKind] = version;
}
return;
}
packsSeen.delete(key);
packsSeen.set(key, {at: time, [packKind]: changes});
packsSeen.set(key, {at: time, [packKind]: version});
// the oldest are first
for (const [oldKey, old] of packsSeen) {
if (packsSeen.size <= MAX_PACKS_SEEN && time - old.at < PACKS_SEEN_MS) {