performance decrease over time
I don't know if this is the right forum - I posted under programming questions and got no response for a couple of weeks, so...
For fun, I've been running the Creatures demo for hours. At the start, and with a small window, it can do about 120 generations per hour. Leave it on overnight, and it's doing maybe 3 / hour. It's not a step function, so it's not the graphics card going to software mode. The decrease is rapid at first, then the speed seems to be asymptotically approaching zero later on. I actually have graph of it if anyone's interested.
I'm running on a 3GHzx4 quad-xeon mac pro 4GB RAM, NVIDIA GeForce 7300 GT, 256MB VRAM, Mac OS X 10.4.10, breve 2.5. Using Activity Monitor shows nearly ~85% CPU use whether running fast in the beginning or slow later. Memory usage isn't an issue either - it stays steady at 60 or 70 MB real memory, 400 MB virtual. The machine is definitely not running out.
To be honest, the above speeds are with a modified version that doesn't log every individual, just the generation report (best and average). I thought at first it was a matter of the log getting full. But other than speeding up the fast/early rate, the abbreviated log hasn't helped.
I'd assume this behavior might happen with my own simulations in the future, and I'd like to avoid it.
Any ideas? Thank you!
Jim Grimes
looking into a memory leak...
Hi Jim,
based on your comments and some made by the previous thread in this forum (about the destroy method never being called), it's looking to me like there might be an object leak in the simulation that's causing an accumulation of objects over time.
The fact that you're not seeing a huge increase in memory is a little puzzling, but depending on the nature of the objects being leaked, it's possible that the slowdown is not coming from the memory consumed by the objects, but due to other object-dependent computation costs (like collision detection, physical simulation, iteration, etc).
If there is in fact an object leak, then it's a bug in the Creatures demo code, and not something that should effect other simulations, assuming that they're written carefully. Creatures is one of the few demo simulations that does constant creation/destruction of large numbers of objects, so that's probably why this problem cropped up there.
In any case, I'm investigating the problem and hope to get it worked out soon.
- jon
Re: looking into a memory leak
I wonder if it has something to do with a phenomenon I've noticed before, which has been ascribed to differences in memory management. I had coded up a simulation in both java and C++ separately, and decided to time a long run, just to see how close to C++ speed java was. I used similar optimization settings. To my surprise the java code actually ran faster. This was a simulation that freely made and destroyed lots of small objects in the course of every step - probably millions over the course of a medium-length run. I ran across an article that described the effect of C++'s scattering new objects all over the place and thus rendering the machine's L2 cache almost useless. Also, the memory management in C++ has to do fairly complex maneuvering for every object creation, whereas java just keeps working out in the heap in a very straightforward manner, then does a single fast garbage collection every once in a while.
So is this a possible cause?
Thanks again,
Jim Grimes
Iteration Time?
I was planning to write a GP system, what happens if the push stack gets stuck in an infinite loop or very long regression?
Does the simulation wait until each + to iterate: is complete?
I have not failed! I have only tried 100,000 ways that will not work. —Thomas Edison
Generation Size and Complexity
How large are the individual push stacks growing to? Over hundreds of generations an individual may have ten fold the number of instructions. How would you put a limit on the object run time?
I have encountered similar performance measures with over 100,000,000 computations per object iteration. Breve runs fine, but extensive computation that grows over time has brought my simulations to a halt. This is in the Breve rendering engine, also turned of draw-every-frame in the controller. Steve does complete the run but nothing is updated to the screen.
Steve is a scripting language, (e.g We can't reinvent Photoshop in Python!) It's just not practical.
I have not failed! I have only tried 100,000 ways that will not work. —Thomas Edison