[OT][FUN] Indiana BOFH ed i Predatori del CED Perduto

[banryu79]

Questa "storiella" non so bene come classificarla, perchè non ha a che fare con bug e/o tecnicismi vari, rientra piuttosto nella "gestione del personale" e la vedo come una sorta di trucco o hack psicologico

---

You Wouldn't Like Me When I'm Angry

I once worked at THQ studio Relic Entertainment on The Outfit, which some may remember as one of the earlier games for the Xbox 360. We started with a PC engine (single-threaded), and we had to convert it to a complete game on a next-gen multi-core console in about 18 months. About three months before shipping, we were still running at about 5 FPS on the 360. Obviously this game needed some severe optimization.

When I did some performance measurements, it became clear that as much as the code was slow and very "PC," there were also lots of problems on the content side as well. Some models were too detailed, some shaders were too expensive, and some missions simply had too many guys running around.

It's hard to convince a team of 100 people that the programmers can't simply "fix" the performance of the engine, and that some of the ways people had gotten used to working to needed to be changed. People needed to understand that the performance of the game was everybody's problem, and I figured the best way to do this is with a bit of humor that had a bit of hidden truth behind it.

The solution took maybe an hour. A fellow programmer took four pictures of my face -- one really happy, one normal, one a bit angry, and one where I am pulling my hair out. I put this image in the corner of the screen, and it was linked to the frame rate. If the game ran at over 30fps, I was really happy, if it ran below 20, I was angry.

After this change, the whole FPS issue transformed from, "Ah, the programmers will fix it." to, "Hmm, if I put this model in, Nick is going to be angry! I'd better optimize this a little first." People could instantly see if a change they made had an impact on the frame rate, and we ended up shipping the game at 30fps.

- Nick Waanders

---

Della serie: emoticons on steroid!

[banryu79]

Altra storia di bug fixing "by chance", per così dire...

---

This is not the bug you're looking for

Brett Douville, LucasArts

In early 2002, we were readying Star Wars: Jedi Starfighter for submission to Sony. One niggling TCR bug remained, which was that the controller's analog stick functionality would shut off while we were loading our post-mission cutscenes, causing the red light in the center of the controller to go off as well. This bug had shown up when we updated to a library version required by Sony, and the programmer who had originally written both the movie-loading code and the IOP logic for the controller itself had left LucasArts some months before.

In the hopes of narrowing down the problem quickly in code I didn't understand, I inserted seven screen clears in different colors in the code I determined to be the likely source of the problem, hoping that I could at least narrow it down to one or two sections of code by checking what color the screen was when the analog controls turned off. But when I then tried to reproduce the bug, it had disappeared.

It's an old programming adage that if you don't understand the cause, you can't be said to have fixed the bug. But in this case, we were two or three days away from our intended submission date and missing it would have been a big deal. So I changed all the screen clears to black, marked the bug fixed, and called it a day. We shipped on time, with no TCR showstoppers.

---

e questa invece, racconta di un hack della madonna (buffer overflow design!)...

---

(s)elf-exploitation

Jonathan Garrett, Insomniac Games

Ratchet and Clank: Up Your Arsenal was an online title that shipped without the ability to patch either code or data. Which was unfortunate.

The game downloads and displays an End User License Agreement each time it's launched. This is an ascii string stored in a static buffer. This buffer is filled from the server without checking that the size is within the buffer's capacity.

We exploited this fact to cause the EULA download to overflow the static buffer far enough to also overwrite a known global variable. This variable happened to be the function callback handler for a specific network packet. Once this handler was installed, we could send the network packet to cause a jump to the address in the overwritten global. The address was a pointer to some payload code that was stored earlier in the EULA data.

Valuable data existed between the real end of the EULA buffer and the overwritten global, so the first job of the payload code was to restore this trashed data. Once that was done things were back to normal and the actual patching work could be done.

One complication is that the EULA text is copied with strcpy. And strcpy ends when it finds a 0 byte (which is usually the end of the string). Our string contained code which often contains 0 bytes. So we mutated the compiled code such that it contained no zero bytes and had a carefully crafted piece of bootstrap asm to un-mutate it.

By the end, the hack looked like this:

1. Send oversized EULA
2. Overflow EULA buffer, miscellaneous data, callback handler pointer
3. Send packet to trigger handler
4. Game jumps to bootstrap code pointed to by handler
5. Bootstrap decodes payload data
6. Payload downloads and restores stomped miscellaneous data
7. Patch executes

Takeaways: Include patching code in your shipped game, and don't use unbounded strcpy.

---

[grigor91]

Originariamente inviato da biffuz

...
Alla fine la palla è passata a me. La mia soluzione:

query = Quey("SELECT <campi che ci servono> FROM MegaTabella LEFT JOIN Festivita ON MegaTabella.data = Festivita.data WHERE YEAR(MegaTabella.data) = $1 ORDER BY MegaTabella.data DESC"); // Nota: la tabella Festivita è sempre esistita
query.setParameter(1, anno);
result = query.execute(); // Pochi secondi, qualche centinaio di righe, pochi kB
while (result.hasNext()) {
  FaiIlTuoBelCalcolo(result.next());
}

OT:
Scusa ma non ci dovrebbe essere una INNER JOIN invece che una LEFT JOIN?

[biffuz]

Originariamente inviato da grigor91

OT:
Scusa ma non ci dovrebbe essere una INNER JOIN invece che una LEFT JOIN?

Sì, giusto, non è il codice vero d'altronde, non ci ho pensato troppo quando ho scritto il post. Neanche mi aspettavo che qualcuno lo leggesse a fondo

[banryu79]

E per chiudere in bellezza, prima di partire per le vacanze (che faccio rigorosamente off-line) posto questa perla. Quanti di voi conoscono il famoso Dalton allocator per la memoria?

---

The Dalton Allocator

Jonathan Adamczewski, Insomniac Games

Toward the end of one project, we discovered that after playing through the game for many hours, movies would not trigger when they were supposed to. Fragmentation within one of our memory allocation heaps had made it impossible to reliably allocate the large blocks of memory that were needed for full-screen movie playback.

We needed to find a way to be sure that we could always get the memory we needed. However, it was too late in the project and too risky to consider defragmentation of the heap, we didn't have enough spare memory available to set aside exclusively for movie playback, and taking space away from other systems at that stage was impractical.

It was clear that when movies were playing, there were many other systems sitting idle, so we gave some consideration to "borrowing" memory from them. However, these other potential sources were either too small, or else also suffered from the same kind of fragmentation problems that we were already seeing. There was plenty of free space available for the GPU, but for various reasons we could not use that for the movie-playback buffers. So we needed another source of space.

While looking into another problem, a particular pattern of memory allocations within our main game-asset heap stood out: When the game started up, a number of assets were loaded from disk, and once in memory were never modified. Some of those assets were very large. None of them were needed while a movie was playing.

This gave us an idea: What if we were to copy the contents of one of the larger asset files to somewhere else in memory? The GPU's memory would work just fine as a place to temporarily stash the asset, and then we could temporarily reuse the space in the asset heap for movie playback, copying the asset data back from GPU memory when the movie finished.

So that's what we did. We chose the largest set of animation clips used for one of the game's heroes (named Dalton). The animation system was switched off, animation clips copied into the GPU's memory space, and the memory handed to the movie-playback system. At the end of the movie, the animation data was copied back to where it belonged, the animation system restarted, and the game proceeded as if nothing horrible had happened. The implementation ended up being fairly straightforward, though the process does span a number of frames to ensure that all the systems involved are safely synchronized with each other through each step of the process.

The memory for the movie system has since been described as coming from "the Dalton Allocator" after the character whose animation clip memory was usurped.

(I've since discovered that there's some history of this kind of thing at the studio -- of both stashing random things in GPU memory, and failing to set aside enough space earlier in the project...)

---

Pare che diversi problemi relativi alla gestione della memoria siano cosa comune; seguono alcuni commenti in calce all'articolo relativi a questa tematica.

Amir Ebrahimi
Re: The Dalton Allocator, I did something similar at Naughty Dog because we had to fit code and art into a 3MB (IIRC) memory block and quite frequently artists would go up to the very limit and us programmers would be scrounging around for every KB to get the level to fit within memory. Frustrating? Yes. How to solve? Mis-report the memory by 256KB

Aaron San Filippo
I've heard similar stories before - reserve some "shipping time" memory and don't tell the art team about it. Very efficient solution to the problem

Joseph Moore
Yeah, that's a very common technique. On a DS title I worked on, we reserved a rather sizable chunk of memory on the side as a "just in case" buffer. Whenever a level designer needed a bit more memory for levels, an artist a bit more for textures, or a sound designer a bit more for audio, we could dole it out. (We, of course, made a great show of scrounging and optimizing to obtain the requested memory.) By the end of our development cycle, the buffer was nearly gone.

Brian Schmidt
Joseph--that's funny...
When I was doing Genesis/SNES music and sound, I'd do the same thing, though in reverse. My system was designed to submit a single black-box binary file to the developer. I'd always make sure my first delivery filled up my entire audio budget, even though most of the binary was empty. That way, I could "save the day" by being able to add in sounds late in the game, or by "optimizing" my audio to give some back to the developer...

[kwb]

Noto che questo "trucco" di mettere da parte della memoria prima di iniziare il progetto è stato molto usato
Dite che si fa ancora coi PC/Console di oggi?

[banryu79]

Jamming the cartridge

Michael A. Carr-Robb-John, Monolith Productions

In 1993 I was finishing off Desert Strike; I was doing the conversion from the 16-bit Mega Drive / Genesis to its humble little brother, the 8-bit Master System. The game was exceeding the desired cartridge size by about 12K, and going to the next size cartridge was out of the question. Today, 12K sounds incredibly small, but back then it was a major deal. During development, I had budgeted and planned all the sound and graphic resources and they were within their limits. The only section I hadn't been so strict on was the code. In those days, games were written in assembly language -- in this specific case, Z80 assembly -- so I had only one option left. I spent a week going through and finding redundant code and rewriting things to use a smaller memory footprint (usually at the cost of being more processor-intensive.)

By the time I had finished, the game fit onto a cartridge with just 98 bytes! The game was burned onto ROM and tested for a few days by the chaps in QA before being submitted to Sega for certification. Unfortunately it didn't pass on the first time, and the required fixes pretty quickly used up those 98 bytes. I think when we did publish, there were only 6 bytes free!

[mone.java]

Originariamente inviato da banryu79

Jamming the cartridge

Michael A. Carr-Robb-John, Monolith Productions

In 1993 I was finishing off Desert Strike; I was doing the conversion from the 16-bit Mega Drive / Genesis to its humble little brother, the 8-bit Master System. The game was exceeding the desired cartridge size by about 12K, and going to the next size cartridge was out of the question. Today, 12K sounds incredibly small, but back then it was a major deal. During development, I had budgeted and planned all the sound and graphic resources and they were within their limits. The only section I hadn't been so strict on was the code. In those days, games were written in assembly language -- in this specific case, Z80 assembly -- so I had only one option left. I spent a week going through and finding redundant code and rewriting things to use a smaller memory footprint (usually at the cost of being more processor-intensive.)

By the time I had finished, the game fit onto a cartridge with just 98 bytes! The game was burned onto ROM and tested for a few days by the chaps in QA before being submitted to Sega for certification. Unfortunately it didn't pass on the first time, and the required fixes pretty quickly used up those 98 bytes. I think when we did publish, there were only 6 bytes free!

[banryu79]

Painting sounds

Edward J. Douglas, Flying Helmet Games

I was running cinematics on a long-running racing series. Our scenes were a mix of straightforward startgrids and fancier action sequences. As we iterated on further sequels, our ambitions with the scenes got greater and greater, but our technology iterated at a slower pace.

You see, the cars would be animated by a QA "stunt driver" to get the base motion, then hacked up in a 3D animation program to adjust the timing and placement of the action, and re-exported to our in-game cinematic tool where playback would simulate all the physics and engine behavior. Lots of gameplay data was captured, including gas and brake information from the controller and represented in metadata in the 3D file, then reproduced in-game. The idea was that this would drive the car audio system as well.

The problem came a few sequels down, where our scenes were very complicated, with a mix of hand-animated car action and recorded capture. The old tricks of using the engine metadata to drive simple audio rev samples for our cars wouldn't work anymore -- the data just wasn't there! The audio team couldn't post-process the audio like a movie, because any scene could have any car in it, depending on player's choice and modifications, so the audio needed to be procedural. But by this time, our games had won numerous awards for audio, especially for the car engine sounds, so we were determined to make it work.

Coming on to beta, things were looking dicey, but a combination of ingenuity and madness between our cinematics, audio, and AI engineering teams found the solution. The gas and brake metadata was represented by a float scale on a cube in the 3D scene. If an artist went in and "drew" curves in a keyframe editor like 3DS Max, they could draw in the car engine sounds they wanted. A few members of the audio team rushed to learn 3DS Max, and by using their intuition of how rev patterns should look, they drew in the animation, exported all the scenes, and squeezed it all in in time. It sounded great, but after this last-minute hack, we knew we'd need something more robust if we'd continue with the same tech-base for the next sequel.

...Or so I thought. I left the studio after that game, and a few years later I met an audio guy who had joined that team after I left. It wasn't long before I realized they never upgraded the tech, and he was the "engine rev painter" guy for the latest sequel.