Vibe-HD-2D/docs/ASSET_GUIDE.md

# Asset guide: pixel art, textures and 3D models

How to make art for this engine, written for humans and AI agents alike.
Everything here is reproducible from scripts in `tools/`; never hand-edit a
generated file without also changing the script that produces it.

## 1. What is pixel art and what is 3D

HD-2D mixes two kinds of art. Pick the right one before making anything.

| Thing | Kind | Where it lives | How it is drawn |
| --- | --- | --- | --- |
| Characters, NPCs, monsters | Pixel-art **sprite** | `assets/sprites/` | `BillboardBatch` (camera-facing card) |
| Small props: flowers, grass tufts, tall grass, signs, pickups | Pixel-art **sprite** | `assets/textures/` | `BillboardBatch` |
| Ground, paths, cliff faces | Pixel-art **tiles** (32 × 32) | `assets/textures/ground_*` | `Terrain` texture array |
| Trees, bushes, rocks, bridges, buildings, fences, braziers | **3D model** with higher-res painted textures | `assets/models/scenery.glb` | `ModelBatch` (instanced) |
| Fire, rain, fog, light, water, sparkles | **Effect**, no asset | shaders / `ParticleSystem` / `Weather` | engine code |

Rule of thumb: if it is smaller than the player or is a character, it is a
sprite. If it is scenery the player walks around, it is a 3D model.

## 2. Units and scale

- 1 world unit = 1 terrain tile = **32 pixel-art pixels**. Keep all pixel art
  at this density so pixels line up (`BillboardBatch::sizeForDensity(32)`).
- Art that was upscaled (e.g. the 256 px test sprite is 64 px art × 4) is drawn
  at `32 × factor` px per unit (`sizeForDensity(128)`).
- The engine is **Y-up**; Blender is **Z-up**. The glTF exporter converts, so
  model in Blender with +Z up and the base of the object at Z = 0.
- Reference sizes (world units): player sprite ≈ 1.5 tall; oak ≈ 5.5 tall,
  canopy ≈ 5 wide; pine ≈ 6 tall; bush ≈ 0.9 tall, 1.6 wide; rock ≈ 0.6 tall;
  bridge deck 17 long, 2.6 wide; cliff step 0.5 high per terrain level.
- High-res model textures: aim for roughly 256–512 texels per world unit
  (1024² leaf cards, 512 × 1024 bark, 1024² stone).

## 3. Pixel art (PixelLab)

All pixel art comes from the [PixelLab API](https://api.pixellab.ai/v2/llms.txt)
through `tools/pixellab.py`, driven by the manifest `tools/assets.json`. The
key is `PIXELLAB_API_KEY` in `.env` (never commit it; `.env` is gitignored).

```sh
python3 tools/pixellab.py --list       # x = file exists
python3 tools/pixellab.py              # generate everything missing
python3 tools/pixellab.py flowers      # regenerate one entry (costs credits)
python3 tools/pixellab.py --balance    # check credits before big batches
```

### Manifest entries

Every key except `kind`, `out`, `pick` and `tile_id` is sent to the API as-is,
so any documented endpoint parameter can be used.

**Sprite** (`"kind": "image"`, default) → `POST /create-image-pixflux`, synchronous:

```json
"lantern": {
  "out": "lantern.png",
  "description": "small iron street lantern on a short post, warm glass, game sprite",
  "image_size": {"width": 32, "height": 64},
  "no_background": true,
  "view": "side",
  "outline": "selective outline",
  "shading": "medium shading",
  "detail": "highly detailed",
  "seed": 40
}
```

- Max 400 × 400. Use sizes that are multiples of 16 and match the 32 px/unit
  density (a 1-unit-tall prop is 32 px tall).
- Always `"no_background": true`; sprites must be transparent.
- `"view": "side"` reads best for billboards (the camera looks from the side
  and slightly above). Enums: outline `single color black outline | single
  color outline | selective outline | lineless`; shading `flat | basic | medium
  | detailed | highly detailed shading`; detail `low detail | medium detail |
  highly detailed`.
- Fix the `seed` so regenerating is reproducible.

**Seamless tiles** (`"kind": "tiles"`) → `POST /create-tiles-pro`, async:

```json
"ground_snow": {
  "kind": "tiles",
  "out": "ground_snow.png",
  "pick": 0,
  "description": "1). fresh snow, soft blue shadows 2). snow with footprints 3). snow over grass 4). icy snow",
  "tile_type": "square_topdown",
  "tile_size": 32,
  "tile_view": "top-down",
  "outline_mode": "segmentation",
  "seed": 14
}
```

- Numbered descriptions give one variation group each; the API returns ~16
  tiles saved as `<stem>_<n>.png` (4 per numbered item, in order).
- After the first run the script prints a `tile_id`; **add it to the entry**
  so later runs re-download for free instead of paying again.
- The storage host rejects default user agents; the script already sends one.
- Use only `square_topdown`, 32 px, `top-down` for terrain.

### Sprite rules the engine relies on

- **Feet on the lowest opaque row.** Billboards are planted by the lowest
  pixel with alpha ≥ 128 (`Texture::opaqueBounds`), so empty space below the
  feet is fine but stray pixels below them make the sprite float. Sheet
  frames and atlas cells use a per-instance `anchor` instead (below).
- **No baked ground shadows.** The engine casts real shadows and draws a
  contact blob under the player. PixelLab sometimes paints a grey ellipse under
  objects; regenerate or erase it (alpha 0) before use.
- Hard alpha (0 or 255) is ideal; soft edges are thresholded at 0.5.
- Sprites face **left** by default; the demo mirrors them when walking right.

### Using pixel art in the engine

- Sprite: `BillboardBatch b; b.loadTexture(path);` then push
  `BillboardInstance{pos, sway, size, flip, tint}` and `b.upload()`.
  `sway` 0 = rigid, 1 = grass-like wind and player push. The remaining
  fields default to "whole texture, no effects": `uv` (sub-rect), `anchor`
  (feet row v), `turn` (extra yaw, radians; paper-turn / facing hints, shadow
  cards stay sun-facing), `alpha` (0..1 fade, dithered in color and shadow
  passes; there is no blending) and `flash` (0..1 toward bright white for hit
  or capture flashes).
- Terrain layer: add the file stem to `kLayers` in `demo/DemoScene.cpp` (all
  layers must be the same size) and reference its index from a
  `TerrainMaterial`.

### Sprite sheets and atlases

One instance draws one sub-rect of its batch's texture: `uv` = (minU, minV,
maxU, maxV) in GL orientation (v = 0 is the image **bottom**), `size` = world
size of that rect (`BillboardBatch::cellSize(cellPixels, 32)`), and `anchor`
= v of the frame's feet row (`anchor < 0` falls back to the texture's lowest
opaque row, which is wrong for any frame but the bottom row). `flip`
mirrors inside the rect.

- **Grid sheets** (PNG authored top-down, equal cells, no spacing):
  `uv = Texture::cellUV(tex.size(), cell, col, rowFromTop)` and
  `anchor = Texture::cellAnchor(tex.size(), cell, rowFromTop, feetRowFromCellTop)`,
  where `feetRowFromCellTop` is the pixel row (0 = top of the cell) the feet
  stand on. Keep the feet on the same row in every frame of an animation or
  the sprite bobs. Leave at least one transparent pixel between a frame's
  art and its cell edge, otherwise filtering (and minified mips) bleeds the
  neighbour frame in.
- **Runtime atlas** (`hd2d::SpriteAtlas`, for sprites created or downloaded
  while the game runs): `atlas.init({64, 64}, cols, rows)`, `int slot =
  atlas.add(image)` (bottom-centered in its cell, cropped to the cell, -1
  when full), `set` / `remove` to replace or free a slot. Every cell has its
  own 1 px transparent gutter. Share the page with
  `batch.sharedTexture = &atlas.texture()` and use `uv = atlas.uv(slot)`,
  `anchor = atlas.anchor(slot)` (the image's lowest opaque row). Images you
  compose on the CPU should go through `bleedTransparentColors` (the atlas
  and `loadImage` already do).
- `Texture::create/load(..., nearest = true)` makes a `GL_NEAREST`, mip-less
  copy for crisp ImGui previews; billboards should keep the default.
- `hd2d_demo --sprite-test` shows a labelled row of atlas / sheet sprites
  (flip, turn, alpha, flash, frame cycling) in front of the player.

### Characters and animation

PixelLab can make consistent characters with 4/8 directions and animations
(`POST /create-character-v3`, `/characters/animations`, then
`GET /characters/{id}/zip` or `/spritesheet`). `tools/pixellab.py` does not
download them yet: add a new `kind` (submit, poll `GET /background-jobs/{id}`
every 5–10 s until `completed`, download) and play the frames through the
sprite-sheet path above. Keep the 32 px/unit density.

## 4. High-resolution textures for 3D models

`tools/scenery_textures.py` paints them procedurally with numpy + Pillow into
`assets/models/src/`:

```sh
python3 tools/scenery_textures.py
```

| File | Size | Used for | Notes |
| --- | --- | --- | --- |
| `leaves_oak.png`, `leaves_bush.png` | 1024² RGBA | canopy cards | cluster of leaves on transparency, **hard alpha** |
| `leaves_dense.png` | 1024² RGB | canopy cores | opaque, tileable leaf mass that fills gaps |
| `pine_frond.png` | 1024 × 512 RGBA | pine branches | stem along +u from the left edge |
| `bark.png` | 512 × 1024 | trunks | vertical fibres, tiles horizontally |
| `rock.png` | 1024² | rocks | grey; moss comes from vertex color |
| `cobble.png`, `stone_blocks.png` | 1024² | bridge deck / walls | Voronoi flagstones, staggered blocks |
| `iron.png`, `coals.png` | 512² / 256² | braziers | coals are emissive |

Guidelines:

- Stylized, saturated, painterly (Dragon Quest III HD-2D / Octopath), not
  photographic. Bake no lighting except gentle large-scale variation; the
  engine lights everything.
- Tileable textures must wrap (use `fft_noise`, which is periodic, or draw
  shapes at ±size offsets like `leaf_dense`).
- Cutout textures: threshold alpha to 0/255 (see `leaf_cluster`); the
  transparent pixels' color is bled from neighbours automatically on load.
- Add a new texture: write a function returning a PIL image, save it in
  `main()`, then reference the file name from a Blender `material()`.

## 5. 3D models (Blender)

`tools/blender_scenery.py` builds every model procedurally and exports
**one** `assets/models/scenery.glb` (textures embedded once, one root node per
model). Run it either way:

```sh
blender -b -P tools/blender_scenery.py                  # headless
```

```python
# inside a running Blender, e.g. via the Blender MCP `execute_blender_code` tool
import os; os.environ["HD2D_REPO"] = "/path/to/repo"
p = "/path/to/repo/tools/blender_scenery.py"
exec(compile(open(p).read(), p, "exec"), {"__name__": "__main__", "__file__": p})
```

It only touches its own `HD2D_Scenery` collection and lays models out side by
side along X for inspection (take a viewport screenshot to check). Re-run
`scenery_textures.py` first if textures changed (and `img.reload()` the images
in a live session).

### Building blocks

- `MeshBuilder.face(points, uvs, colors, normals, material_index)` adds a
  polygon with per-corner data; `build(name, materials)` makes the object.
- Helpers: `tube` (tapered trunks/branches with bark UVs), `ico` (icosphere),
  `clump` (leaf core + alpha-tested leaf cards), `box`, `quad_face` (world
  projected UVs), `lathe` (surfaces of revolution: bowls, posts, pots).
- `material(name, image, alpha=False, emission=0.0)` creates the Principled
  material the exporter understands (alpha → `MASK`, emission →
  `emissiveTexture` + strength).

### Adding a model

1. Write `make_thing(name, seed)` returning `mb.build(...)`.
2. Add `("thing", make_thing, seed)` to `MODELS`.
3. Run the script, then load it in the demo:
   `ModelBatch b(library.get("thing"))`, push `ModelInstance{pos, yaw, scale, tint}`, `b.upload()`, add to `scene.models`.

### Conventions the engine depends on

- **Origin at the ground contact point**, +Z up, base at Z = 0 (slightly below
  is fine to avoid gaps on slopes). Long objects run along +X.
- **Vertex color `COLOR_0`**: rgb multiplies the texture (use it for baked
  ambient occlusion and tint, e.g. darker at the base, moss on top); alpha is
  the **wind weight** (0 rigid, 1 fully swaying; scales with height).
- Materials named **`foliage*`** are alpha-tested, double-sided, get wrap
  lighting, sun translucency and soft self-shadowing. Everything else is
  single-sided and back-face culled.
- **Winding must be CCW seen from outside.** Single-sided faces are invisible
  from behind, so a wrongly wound or misplaced face shows up from one side
  only. Check models from several angles, and never put a face across an
  opening (the first bridge had an end wall across its entrance that was only
  visible from outside).
- Foliage: set custom normals pointing away from the canopy center (`clump`
  does this) for soft, rounded shading instead of flat cards.
- **Sockets**: an empty parented to the model becomes `Model::sockets["name"]`
  (model space). Use them for attach points (the bridge exports `fire_0`,
  `fire_1` in its braziers; the demo spawns fires there with
  `ModelInstance::toWorld`).
- Budget: a few thousand faces per model is fine; instancing handles counts.

### Per-batch options (`ModelBatch`)

| Field | Use |
| --- | --- |
| `castShadows`, `inReflections` | turn off for tiny props |
| `windScale` | multiplies the vertex wind weight |
| `groundOcclusion` | strength of the baked contact shadow under each instance (trees 1.0, bushes 0.8, rocks 0.7); rebuild `GroundOcclusion` after moving instances |
| `occluderFade` | dither away where it hides the player (tall trees: on; low props like bridges, rocks, bushes: off) |

Collision is gameplay code, not part of the asset: the demo keeps a list of
circular obstacles and special-cases the bridge deck (its numbers mirror
`BRIDGE_*` in the Blender script; keep them in sync).

## 6. Checking your work

Always look at the result; don't assume it looks right.

```sh
cmake -S . -B build -G Ninja && ninja -C build
./build/hd2d_demo --screenshot out.png --frames 90 --pos X,Z --dist 8 --yaw 30 --set dof=0
./build/hd2d_demo --screenshot rain.png --frames 90 --pos X,Z --rain 1
./build/hd2d_demo --screenshot night.png --frames 90 --pos X,Z --hour 23   # also try --hour 6 / 18
./build/hd2d_demo --screenshot ui.png --frames 420 --ui-demo   # game UI (hd2d::ui)
```

- `--pos` puts the player (and camera target) at world X,Z; `--yaw` and
  `--dist` orbit the camera; `--set dof=0` removes blur for inspection.
- Check every new model from at least two opposite yaws, in rain (wet look,
  wind), at night (`--hour 23`: moonlight + fire light, emissives) and near
  water (reflection).
- Check sprites at default zoom next to the player for scale and pixel size.
- UI fonts (`assets/fonts/`, loaded by `hd2d::ui::loadFonts`): OFL only, with
  the font's `OFL.txt` saved as `<Family>-OFL.txt`; never modify or re-instance
  a font with a Reserved Font Name (Cinzel). Dear ImGui's stb_truetype renders
  a variable font's default instance, so choose fonts whose default weight
  reads well. Check UI changes with `--ui-demo` at `--frames 120` (mid-reveal)
  and `--frames 420`, at `--size 1920x1080` as well as the default size.