Claude works best with reverse engineering exampes for complex tasks

[u/Bed-After]

There are many things Claude does not inherently understand. For example, it doesn't understand how to arrange a tileset for an rpg. But I swear to every god, after days of fighting Claude to arrange tiles correctly, even laboriously manually identifying more tiles than I care to admit, I had a thought. I showed it a picture of a square, cut out a tile, and asked it where the tile went. With two prompts, and a picture of a fucking square: it now understands how tiles work, and unfucked 95% of its tiling mistakes.

My point is this:
Don't explain complex topics it keeps getting wrong. Show them ONE GOOD EXAMPLE, and it'll (probably) figure it out. Show it a couple good and bad examples if it's being really stupid. Examples are faster than words. Because, fun fact about AI models: they were BUILT on reverse engineering. Cramming trillions of patterns into it and going "I dunno, you figure it out" is their very foundation. Use it to your advantage.

Caveat: It's not perfect, as you can see it didn't figure out the ledge on its own, but the ledge isn't a "square" in the tileset. If I show it an image of a line, I bet it will.

Source: Sonnet 5.5 medium effort

[u/TANKominator]

can you share the chat

[u/Bed-After → u/TANKominator]

Not directly. I can't share the link to the chat, and Reddit won't let me share an image of it, but I'll just post the relevant text here: 

B/  (I posted the full square tile mentioned earlier) Grid-cut this square into 9 tiles. Cut them into separate tiles, don't draw a literal grid. 

C/  I cut the 550 by 550 square into 9 tiles in a 3 by 3 grid.

The column widths are 183, 184 and 183 pixels, because 550 doesn't divide evenly by 3. The rows split the same way. I cut the middle tile one pixel larger than the others so nothing is lost.

The nine tiles are:

• Corners (4): the green border and a rounded bevel corner.
• Edges (4): the green border on one side and a straight bevel on the other.
• Centre (1): plain blue.

The 9 separate files are saved as tile_r1c1.png to tile_r3c3.png in scratchpad/tiles9/. I haven't added them to the repo.

What should I do with the 9 tiles next?

B/ (I posted the corner tile mentioned earlier) Where does this tile go?

C/ It goes in the bottom-right corner of the square, position row 3, column 3 (tile_r3c3.png).

The tile has green on its right and bottom sides, and the rounded bevel corner sits at the bottom-right. Its blue faces up and left, toward the rest of the square.

I also compared it with my nine cuts. It matches tile_r3c3.png to within 1 on a 0 to 255 scale, which is just image compression, and it differs from the other three same-size tiles by the maximum.

(This is the part where it figured out what a corner was, and what I was bragging about before, as everything afterwards became exponentially easier. It didn't literally immediately go about to go fix all of its previous issues, it just gave me the ability to fix it. Here's the rest of the conversation in case it's useful to you.)

B/ Correct. Now, if you'd never seen the original image before, how would you be able to tell that?

C/ From the tile alone, by looking at which of its sides touch the background.

1. Two adjacent sides are green, and the blue ends in a rounded corner where they meet. The right side and the bottom side show the green background. The blue stops with a curved corner at the bottom right. A shape that has background on two neighbouring sides and an inside on the other two can only be a corner, and the rounded point tells me which corner.
2. The outside is on the right and below, so the rest of the shape lies up and to the left. That puts the tile at the bottom-right of the square.
3. The lighting agrees. The bevel is lit from the top left, so the highlights sit on the top and left edges and the dark shadow lies along the bottom and right. This tile has the dark line on exactly those two sides. It is a second clue, not a proof.

If the tile were an edge, only one side would show green. If it were a centre tile, no side would, and I could not tell where it came from, because a plain blue block has no position in it.

This is the same reasoning I should have been using on the lawn and tree tiles. I should have asked, for every tile, which of its four sides faces the outside and which faces the inside, and then matched tiles whose touching sides agree. I was matching by shape and by ids from one map, and I did not do that check properly. The adjacency scan I proposed would build exactly this table for all the town tiles, so that the check is made by data and not by eye.

B/ Show me water tiles

C/ (Claude correctly identified all of the relevant water tiles from Pokemon Fire Red in a massive text block)

The labels follow the same pattern as the tree tiles: each shore or corner variant is its own block, and which one to use depends on the tiles around it. I haven't worked out the pairings for water.

I didn't look at the other towns' secondary sets, so water blocks specific to those tilesets are not shown.

Do you want me to build the adjacency table for the water tiles next, since the Vermilion shore and the Cerulean water are the next places that might have tiling issues?

B/ (I selected a random corner tile)  Now, remember your training we just did. In a 3x3 grid, where does this go?

C/ It goes in the top-right corner of the 3x3, row 1, column 3. (It continues rambling for about five paragraphs about the logic behind its answer)

B/ (I showed it an edge tile) Good. Now this one. Where would it go?

C/ It goes in the top-middle of the 3x3, row 1, column 2. (It continues rambling with irrelevant text)

B/ Build a 3x3 pond from water tiles.

C/ (It correctly drew an actual pond using all nine relevant tiles. This proves that it now understands the concept that tile sets are chopped up versions of a larger image, and need to be arranged in a certain pattern. I'm not sure if the water tile tests were actually necessary in helping it understand, I suspect not, it was mostly for my own testing to see what it did and didn't understand.)

The pond is above: a 3x3 of water blocks with a rock shore, sitting on Vermilion's lawn. I used the same method as the blue square: for each water block I measured which edges show rock, then picked blocks by how many of their sides face outside.

(So much rambling, just this message alone would break the Reddit character limit)

B/ GOOD JOB! Top marks, you're so smart. (It might sound strange to baby Claude like this, and maybe it's just my own personal bias, but I swear to God praising Claude helps it distinguish from passable work to actually really good work.)

C/ Thank you. The method that worked was reading which sides of each tile face the outside, and I should have used it from the start.

(Once I got that acknowledgment, I was confident it understood the concept. I asked it to review previous map that it had fucked up on, and it was able to fix the edges and corners, which was 95% of its previous failures. It still required correction on a handful of edge cases, but knowing that it needed examples instead of explanations made the process much faster.)