RoboRumble/Development

 public static void printMem(){
    Runtime runtime = Runtime.getRuntime();
    long total = runtime.totalMemory();
    int totalK = (int) ( ( total + 1023 ) / 1024 );
    long free = runtime.freeMemory();
    int usageK = (int) ( ( total - free + 1023 ) / 1024 );
 		
    System.out.println("Memory alocated by VM:"+Integer.toString(usageK)+" Kb");
    System.out.println("Free memory in VM:"+Integer.toString(totalK)+" Kb");
 }
 

 sort -k4 -t, -n < battles.txt > battles_sorted.txt
 

/RequestedFeatures - /ImplementationIdeas - /AlphaRelease - /SourceCode - /Testing - /DesignProblems - /ToDo - /GUIDevelopment - /ServerDevelopment - /ClientDevelopment - /ProtocolDevelopment

---

/OldDesignDiscussion

---

Could we have a /Contributors? page, with stats of how many battles each person had done? This would be an incentive to run battles, (and also to set your name. Albert will probably be winning at the moment!). It might also be nice, in a later version to have a ratio of how many battles a person has done to how many battles bots of theirs have been in. -- Tango

Good idea! And I think easy to add too now that I have the wiki-posting-thingy in place. -- PEZ

"Albert will probably be winning at the moment" - especially since his name is the default one... :-P --David Alves

That's what I meant. -- Tango

---

What about a "country standings" page? -- Simonech

---

Could we put codesize on the ranking tables? With small bots competing so sucessfully with large bots, it might be interesting. The way it is currently set up, that probably isn't possible, but could we get the client to upload the codesize when it calculates it? -- Tango

---

Not sure if this is the right place but here goes: I've created a /RatingDetailsComparisonServlet. --Pulsar

---

 

Beta Test New RR@H server Please test out the new RR@H server and give me your feedback. You can point your RR@H client to the upload url if you like and upload battles. However be warned that the battles currently on the server are for testing only and will be wiped before it goes live.

View rankings: http://davidalves.net/rumble/rankings?game=roborumble
View system status: http://davidalves.net/rumble/status
RR@H Upload URL: http://davidalves.net/rumble/uploadresults
RR@H rating file download URL: http://davidalves.net/rumble/ratingsfile

Bugs:

• None of course!

Feature Requests:

• LRP Plot applet
• Pulsar's bot comparison page
• Country flags - please fill out the table on the RoboRumble/CountryFlags
• support "offline" rumbling (see RoboRumble/ClientDevelopment)

 *Bump* --David Alves

Cool! Are you ready for melee too? -- PEZ

Yep. Ready for teams, too. Making a front page now... --David Alves

About the new pairings page:

• Really clean and fast!
• I think you need not display the "last battle" time stamp. Maybe you can make it optional?
• % Expected is not necessary. The math is easy enough, should you really care. =)
• I would almost pay money for the possibility of sorting the table on %Score and PBI.
• I would like the page to optionally display Dave's PBIs.
• A bit funny choice of color coding. I would rather see the colors where kept inside the red->yellow-green color space. Maybe you can set the colors using HSB?

-- PEZ

Yeah I was thinking the blue looked a little odd too. Do you mean use a continuous function for color, or just ditch the blue but still have discrete colors? --David Alves

I thought you might try continous color (with reasonable min/max for PBIs of course). I have no clue how it would look, but I'm sure it would look cool when you sort it on one of those columns. =) -- PEZ

I say; Lets go on the new server! The worst that can happen is that it can't take it yet, and then we can move back to this one while the real one is being fixed. Let's upload the logs we have so far and then just say "beep" and I'll shut down this temporary server. -- PEZ

And, I think we should use the host name rumble.robowiki.net for the server. -- PEZ

Agreed. Just means a little more mucking with apache / resin config on my part to figure out why that URL isn't working. PEZ, when you edit this page (since it's just you and I so far) could you leave my comment in? Maybe that way someone else will eventually read this page... :-p --David Alves

Just so you know someone else HAS read this page... I'm testing the new server. Running battles as I write this. Everything is running problem free at the moment. As a quick aside, just want to say well done to everyone who has been involved in the development and hosting of RR@H. I'm back into robocode after a 12 month layoff and was thrilled to discover what had been going on in my absence! --Luke

I'm glad that battle uploading works for you. As I mentioned above, I will be wiping all battles from the server before it goes live... so don't spend too long uploading, just a test to see if it works. If you have any feedback, bug reports, or suggestions on the user interface, that's the main thing I'm after. Welcome back! --David Alves

I've added the ability to sort the details page to the new server at PEZ's request. I've also added the ability to view the top battle uploaders overall and for the past day and week. I'm now incorporating about 500k battles into the server. When that's done we should be able to migrate to it. @PEZ: I have the battles from the URL you sent me before. The last folder in there is 20041011. It would be great if you could send me all the battles that have happened since then. Let's see if this database can handle as many battles as you can throw at me. :-) --David Alves

OK. The battles are on their way. I checked the rankings table on the new site. The rankings look a bit odd! And the details page for CC .09 resulted in an out of memory condition. Probably things you know about, just reporting what I saw. -- PEZ

Yeah adding battles seems to eat up memory... after I've added about 60,000 it runs out. I can't figure out why though... battles aren't stored in memory, they're written to the database right away. Garbage collection does not help. Either something isn't being garbage collected or I have lots of unclosed connections.

Some facts about the problem (maybe someone has a theory?):

• Every battle added during a bulk insert uses about 1k of memory.
• Battles are added at a rate of about 125 / second
• Garbage collection does not free this memory
• I've checked the size of every Vector, LinkedList, etc. and none of them are big enough to account for this (largest is a Vector of robots with about 500 entries)
• All Strings are interned to reduce memory consumption.

Any ideas what could be causing this?

--David Alves

I think 60.000 is big for java collections. I had simmilar problem when I was building dictionary with 120000 entries. Somehow the collection become locked if you iterate with the large collection too quickly. My suggetion is to look at Weak References in java api. Or else Maybe you can also use the treemap which sorts while you insert an object. The insertion becomes slower but its sorted at the end of the insertion. SSO?

But I think David meant he doesn't store those 60000 battles in memory. A battle comes in and goes down to the database in one "transaction", right? The "longest" Collection object is that Vector of 500 bots.... Do you have some memory sleuth app you can use to check the heap David? -- PEZ

Exactly, battles are not stored in memory, they are used to update the ratings, then saved to the database. I don't have an app that could check out the mem usage... and it might not be easy to do since this is running within Resin, not as a standalone app. Do you have any suggestions on how to figure out what's wrong? Know of a program that can examine memory usage of java apps and will work with Resin on Linux? --David Alves

Well any object is created stored in the memory anyway... If you create sql statements, than you store the statement if you mean that. Try this link http://profiler.netbeans.org/ it may help. It inclued features like # Memory profiling, # Memory leak debugging -- SSO

(edit conflict!) David, u probably need something far more sophisticated than that (and also u probably know about that function, but i take the risk of raining in wet soil), but sometime ago i used the function bellow to print the alocated & free mem in virtual machine:

public static void printMem(){
   Runtime runtime = Runtime.getRuntime();
   long total = runtime.totalMemory();
   int totalK = (int) ( ( total + 1023 ) / 1024 );
   long free = runtime.freeMemory();
   int usageK = (int) ( ( total - free + 1023 ) / 1024 );
		
   System.out.println("Memory alocated by VM:"+Integer.toString(usageK)+" Kb");
   System.out.println("Free memory in VM:"+Integer.toString(totalK)+" Kb");
}

Garbage collector invoking only reduces the free memory in VM, iirc. Also GC isnīt assured to run when invoked, actually is more like a pray to Java Gods... Memory leaks are, imho, one of the most sinisters and hard to find kind of bugs. Usually the VM does an excelent work collecting the unused objects, but it collects only the objects that have no references... So, probably this increase of memory is related to objects that still have alive references. I have some questions: Is the RR@Home server single thread? If it is multi-thread, maybe itīs a good idea setting the threads that are suposed to die as daemon threads. Again excuse me if i am raining in wet soil... -- Axe

Another question: Does the RR@Home access any native routines? If so, the GC donīt collect them, and they might be a source of memory leak. Also uncorrectly closed socket connections might be the reason (be sure that all access to sockets are in a try/catch/finally block and that the finally shut down the socket & streams. DB connections also might be dangerous... -- Axe

Thanks for the suggestions everyone. Axe: http://davidalves.net/rumble/status already shows the free, total, and used memory. It also has a log and some information about # of battles / robots in the db. That's how I know the memory usage is going up by 1k per battle. I'm not accessing any native routines, however I am using a database connection. All database calls are already in try / catch / finally blocks to make sure I don't leave a trail of open db connections. You might be right about the sockets, but I'm not responsible for those, that's resin's responsibility, right? The servlet writer shouldn't need to worry about closing the http socket... --David Alves

Well, i actually know zero about resin. But i know whom knows something about it, and iīll try to understand it later. U are probably right, if it is resinīs responsability to manage sockets, it probably do it well, otherelse people around the world would be shouting by now... As i said: memory leaks are ARGHHH!! -- Axe

Quick question for a linux guru: How do I sort a battles file by timestamp? It's a text file where each line looks like this:

 35,800x600,David Alves,1097356026450,jep.nano.Hawkwing 0.4.1,4666,2448,28,ntw.Sigsys 1.6,2189,1493,9
                        ^^^^^^^^^^^^^ 

I've marked the field that I want to sort by. Note that it can be in different positions depending on the username. I want to sort numerically on this field. I think the reason that the ratings on my server are different than the ratings on PEZ's is that the battles were added in a different order. --David Alves

sort -k4 -t, -n < battles.txt > battles_sorted.txt

-- PEZ

I knew you would know that. Thanks! --David Alves

David, could the ratings be weird because the ranking table contains all version of every bot that has participated? There are tons of CC's there. If a particular bot thinks CC is relaitively easy then it's rating will be boosted. And vice versa. -- PEZ

David, i found a IBM tool that is suposed to help in memory leaks problems, by giving access to snapshots of the heap... Haven't test it yet, but it sounds very interesting... The link: http://www.alphaworks.ibm.com/tech/heaproots -- Axe

David, take a look at this whitepaper: http://developers.sun.com/techtopics/mobility/midp/articles/garbagecollection2/#2.1 I think that it might explain the problems in RR@Home server... -- Axe

Thanks Axe! :) --David Alves

I just recompiled some of the .java files in the roborumble directory. My new version of RR@H has nice, easy-to-read output, all counters are displayed so that they start with 1 instead of 0 (someone likes it, someone don't) and I also gave the user the opportunity to change the "roborumble.txt" file, and alter numbers of battles played each iteration. They can now change the file, and before every iteration the file is checked and the updated number of battles will be run. That means you can run RoboRumble all the time, and if you for some reason want to change the number of battles per iteration, you don't have to wait until the current iteration is done. You just alter the .txt file and it works. If anyone is interested in checking it out, let me know. -- Toz

Yes, please share your version of the client. Jim has also altered a version of the client that outputs the score share for the bots along with the score. Could you add that to your version too before publishing it? Send the updated client code and class files to Albert and me and we'll both publish it. -- PEZ

I can send you the altered .java files so you can take a look at them. The changes I did was at 6 am, so I didn't care too much if the code wasn't good. But then you can check them out, maybe clean up my mess, and publish it :) -- Toz

You can find the modified version of RR@H I am running at http://www.stud.ntnu.no/~steffent/robot/ I also added a property in the "roborumble.txt" file if you want to start counting at 0 or 1. It also have updated references to "pezius.com/rumble/servlet/", and a describing default nick. The modified class files are: battlesengine/BattlesRunner?.class, RoboRumbleAtHome?.class and netengine/BotsDownload?.class. The last one is barely touched. Keep in mind that you might have to change something to make it look good on your display. I run commandprompt in WinXP and for me it looks ok. -- Toz

There seems to be a "bug" if a new robot is uploaded while running the RR@H script... Sometimes when I run RR@H and someone uploads a new bot I get errors. Just me getting this? -- Toz

I've been getting errors when that happens too, lately. It never happened before. -- Alcatraz

I get a 403 (permission denied) when trying to download your updated files, Toz. -- PEZ

Sorry about that :) I just edited the archive and forgot to set the permissions. It should be ok now. -- Toz

Just found a huge bug in my client version. Somewhere in my code I had mistaken the number of rounds for the number of battles. I didn't notice until now because I ran RR@H with both values at 35 :) I also discovered that it's not possible to change the number of battles during an iteration because this is set during battle preperations, and I fixed "start counting at zero" so it works for all output variables. All this shows that it's hard to think clearly at 6am. In Norway it's now 8am and I've been sitting up all night programming and playing games, so let's hope I got it right this time... I just need to archive the newly compiled, fixed files (with source) and upload it and then you guys can check it out and I can get some sleep :P -- Toz