Perhaps the number of rounds and battlefield size should both be given somehow by the server? I think it's also time we consider log-ins and at least basic precautions, since we seem to have our basics settled. I'm also about ready to restart GUI work, things seem reasonably settled, and Albert was kind enough to add some really useful parameters for controlling the program's execution. -- Kuuran
We can use a static html page for the server side decided parameters. My client (alpha5) hangs while trying to download bots. Maybe the repository site slows down on the client like it does on humans at times? Is the client honouring any robots.txt file they might have on the repository site? If not, it might just shut the downöoad down if it feels the client is demanding too much of it. -- PEZ
I think it would be easier to (1) Upload the parameters file into the Wiki so people can download it. (2) Filter the received battles according to the specifications. It would avoid having to connect every time to the server.
I guess the number of battles count is wrong in the RankingTest? page. I counted it for a bot and the number shown is higher than the battles fought. -- Albert
Well a wiki page might do, but anyone can alter it so I thought a static page would be better. The point with making it a web file would be so that both the client and a human can download it.
I think the battles count is really a "rounds" count. Which one would you rather have? I'll either change the heading or the content. =) -- PEZ
It may be important to prevent people from changing parameters and messing up the stats, though. Also, as for the algorithm, I think it's fine, the reason it's taking so long to stabilize is likely because 90%+ of the bots came in together near the 1600 mark, and rankings are relative, so they're taking a long time diffusing. -- Kuuran
Rounds is better. If people can change the number of rounds it will be meaningless if it's battles. -- Tango
I agree, I suspect the battles count is a rounds count, though I've noticed it's always in multiples of 11, I think it's counting an extra match for some reason. -- Kuuran
Oh dear, it appears some bots are throwing exceptions or timing out (it happens, especially in older bots). They and their opponents end up with divisions by zero in their rankings! This may actually be some fault in robocode as these matches seem to be from the last few I ran in a batch of 1000 battles, it's possibly that robocode's known to be buggy memory management (remember SandboxDT crashing?) started frying the bigger more memory-intensive bots by leaving them no memory to play in. This is just a guess, of course. -- Kuuran
Oh! I'w try to fix it ASAP, but it won't be before this night.
When I was running Face2face, I was getting memory problems when I was running long series (that's why I made RoboRumble@home iterative). In any case, we need to filter it!
-- Albert
Indeed, we should probably just discard matches with 0 scores on either side, they're rarely valid. In other news, the NaNs are spreading, any bot fighting something with that rank will get it after the battle. -- Kuuran
Well, I'll leave the totalling script until anyone of you have any suggestions for it or I come up with an idea for it (whichever comes first). Is it OK to use a wiki page you think? I kinda like it. Any comments someone might have about the current rankings or movement in the rankings can be made by just editing the page (the wiki way). And the revision control will let us browse previous rankings with ease. As long as people editing the page remember to check "This change is a minor edit" each revision of the page will correspond to a rankings update. (Provided someone ran Roborumble@Home that day, which seems likely). Let us know what you think. -- PEZ
I certainly like it. -- Kuuran
I think it works well, although the current page is called "test", which will need to be changed when we go live. -- Tango
I like it also. -- Albert
kawigi.SpareParts 0.7.nosnd crashes RoboRumble@home. It happened to me twice. Some else has noticed it? -- Albert
I haven't had this problem yet, and since I've run about 2000 battles I think SpareParts would've come up at least once. So I'm going to guess it might have something to do with your system or your copy of SpareParts. Does SpareParts run properly in a normal match started from within Robocode? -- Kuuran
It was running some battles (Beta 1) and then it stopped:
# # HotSpot Virtual Machine Error, Internal Error # Please report this error at # http://java.sun.com/cgi-bin/bugreport.cgi # # Java VM: Java HotSpot(TM) Client VM (1.4.0_01-b03 mixed mode) # # Error ID: 43113F2652414D452D41503F491418160E435050005B # # Problematic Thread: prio=5 tid=0x009D2348 nid=0xa98 runnable #
Should i report it to that url? -- Tango
Typo: "Uloading Results..." should be "Uploading Results..." -- Tango
Never mind, it's just JVM crashing. No need to report it. --
The JVM seems to be doing this quite alot overhere ... I already lost alot of good data this way :/ Any idea what's causing it? -- FnH
Just noticed you don't lose data when the JVM crashes. Well done Albert :) -- FnH
I've had some problems with the JVM slowing my system to a crawl after running RoboRumble overnight, but since the CPU consumption was minimal (actually less than normal when RoboRumble was running) I believe it was a memory issue. It's possible you're having some memory issues of your own. I know RoboRumble was made iterative to avoid alot of these problems, but the java process doesn't die between iterations, only the robocode one on the JVM does, so not all the mess seems to be cleaned up. Could your issue be something related (is this happening after alot of hard running, possibly concurrent with robocode?)?
And yes, it all gets saved to a text file match by match, Albert's idea of disintegrating everything was a good one. So all your partials will be uploaded next time you complete a run. -- Kuuran The aim of releasing the server with limited functionality is to install it and see how it works. PEZ, could you install in on the server? It needs a directory called "rankings" under the root application directory. In my computer, I installed it as part of the "examples" application. My file structure is like follows:
UploadedResults.class and RankingDetails.class are placed in Tomcat 4.1\webapps\examples\WEB-INF\classes\ directory. The new "rankings" directory is Tomcat 4.1\webapps\examples\rankings The URL to post the results is http://localhost:8080/examples/servlet/UploadedResults The URL to read the results is http://localhost:8080/examples/servlet/RankingDetails?game=RoboRumble&name=apv.Aspid 1.7 (it works fine from my browser)I hope it makes installation easy. When the server is ready, we will have to modify the roborumble.txt file to point the RESULTSURL to the right URL. -- Albert
I have installed the server classes on this server now. (Thanks heaven for mod_rewrite, it rules!) Now the RESULTSURL in roborumble.txt should be set like so:
RESULTSURL=http://rumble.robowiki.dyndns.org/servlet/UploadedResultsThis works for me, but I'm not sure if this host name works outside my LAN yet. Anyone who would like to verify that can click here: I get an error message about data missing for this bot. I have checked the "rankings" directory and the file is there for Sedan. Since Linux is case sensitive with file names I tried writing the "game" parameter with lower case (same as the file in "rankings") but it didn't help. Anyways, please let me know if you too get this error message, because that means you can reach Albert's servlet and that would be way cool. I think Albert must have set a world record in the time it took him from knowing zilch about server side Java to downloading Tomcat and have a useful servlet running. Ya'da'man Albert! -- PEZ
I'm running the client right now, and uploading a lot of results. It seems to work fine for me. But it fails to read the results once they are uploaded :-( I'w investigate. -- Albert
I GOT IT!!!! You don't have to put ".txt" at the end of the name. Use the following URL:
-- Albert
This is cooler than ice! Very good work Albert. How's the ranking calculated as it stands now? -- PEZ
I'm ,using the formula from Paul Evans based on ELO rankings, with the following considerations:
Of course, needs some testing. Probably I'w have to introduce some rule to make the new bots to move faster.
-- Albert
Just for fun I made a little shell script that produces a total from the files available. Check: http://rumble.robowiki.dyndns.org/rankings/roborumble_Totals.txt
The script:
![[Home]](/images/RoboWiki.png)
#!/bin/bash
RESULT_FILE="roborumble_Totals.txt"
cat /dev/null > $RESULT_FILE
for file in roborumble_*_*.txt
do
awk -F"[ =]" '
BEGIN {
OFMT = "%.2g"
}
/^#Rankings file for/ {
name = $4" "$5
}
/^ranking=/ {
ranking = $2
}
/^battle/ {
battles += $2
}
END {
print ranking+0 " - " sprintf("%7d", battles) " - " name
}
' $file
done | sort -gr | awk '
BEGIN {
print "RANKING TOTALS\n"
print "GAME = roborumble\n"
print "RANK: RATING - BATTLES - BOT"
}
{
print sprintf("%4d", NR) ": " $0
}
' >> $RESULT_FILEI've just made a very quick reverse engineering on the rankings files, but I think I got it right. I think that this should become two servlets:
Could you post a sample datafile? Maybe I'll have a go at a few XSLT's transforming it in something pleasing to look at :) -- FnH
The data file format is not yet designed. Please feel free to make one up for the purpose of the XSLT development. You can assume about the same contents as the ER result table to start with I think. The XSLT need only deal with the result table I think. And no actual styling information, but instead "class=..." and then we can add the actual styling from the surrounding JSP. -- PEZ
The server should create the result pages once in a while, this way all pageviews are static, and won't have too much of a performance impact on the server (one of the problems ER is having) ... -- FnH
That's the purpose of the update-total-rankings servlet. I'm thinking that any client that has uploaded new rankings should consider running it. -- PEZ
A good idea would be to have the rankings updated once a day. Also would be nice to have the old rankings stored in the server, so you can take a look on the past and see how it evolves. Because it is hosted in the Wiki, may be a "Wiki" look and feel would be interesting. -- Albert
Maybe we could post the results on the wiki. That way the revision control would take care of the history thingy and the look and feel would solve itself. =) -- PEZ
Hrm, this data files thing is odd. Look at the detailed rankings for NanoSatan. He beat both TityusMega and Teancum, presumably on computers where they have no stored data on him, yet he also lost near shut-outs to them, presumably on computers where they do have stored data. -- Kuuran
The rankings are [here], oh, and now is a good time to update your preferences files with your usernames, so everything isn't being run by Albert :P -- Kuuran
We should decide how many rounds per match we use. I think 10 rounds is to low, as it handicaps bots that don't store data files and also it introduces excesive variance. Because we have now more power to run the battles, 20 or 30 rounds would be better? What do you think? -- Albert
It doesn't really introduce any more variance if you run 10 rounds over and over, but yes, it handicaps non-persistent bots. Maybe the 35 used in MiniBot? Most of the non-persistent MiniBots are already optimized with this in mind anyway. -- Kuuran
We have been able to run 40.000 matches in 12 hours!!! RoboRumble@home is powerful! -- Albert
Powerful indeed! While you guys have been running all those matches I have changed my totalling script so that it now posts the sorted rankings on the /RankingsTest page. It wasn't as easy as I thought at first. The wiki is a picky about the page that is posted. It must contain a reference to the old page and such so the script must parse the old page for this reference first before posting the new page. Also, all this robocoding has made my Perl head slow ... =) I won't dig in to cookie handling just yet, so for now you'll have to live with seeiing the wrong user name in the "Changes" listing. The script is just a very quick 'n dirty hack, but it might do for now and we can make a serious program for this later. I can put this script in a cron job (if you're not a Unix head this means an automatically batched job). I'll go for each half hour to begin with. Then we can change this to a less frequent run when things have matured some. I'll try to get those flags into the listing (I really like them=). Meanwhile you others can follow the excellent excample of whoever updated the participants list with all those bots. All bots in the ER would be a good start. =) -- PEZ
That'd be Albert's work. I'll get on adding any major omissions I find. -- Kuuran
If it's on a wiki page are we trusting people not to edit it? Is there some way you can password protect certain pages, a bit like wikipeadia's home page? While we are testing, i think we can trust everyone, but once people that don't use the wiki come to see how their bots are doing, trust may not be enough. -- Tango
I don't think that's a problem. All changes to the page will show in the revision control. If anyone is acting a fool like that it is as simple as restoring the correct version of the page. (That goes for the whole wiki by the way.) And, every once in a while my script will post an update anyway. -- PEZ
True. Is there any plan yet to get the server to send lists of battles to be done? The current rankings list is very uneven. Also, what conclusion did we come to for handling data files? -- Tango
Can you please add the Canadian flag as the icon for package 'arthord'? -- Kuuran
60,000 battles now. Once we get it all worked out, i don't think we are going to have any problems running enough battles to keep it stable. :-) -- Tango
I think we can leave as is. It will settle down for itself. Also, first we must make sure the ranking system works fine. Have you noticed the rating needs a lot of battles to settle down? Probably we should make it a little bit faster. About numbering, no problem. I'w add it with some minor changes (ie. delete temp directory) in the next releases. -- Albert
Perhaps the number of rounds and battlefield size should both be given somehow by the server? I think it's also time we consider log-ins and at least basic precautions, since we seem to have our basics settled. I'm also about ready to restart GUI work, things seem reasonably settled, and Albert was kind enough to add some really useful parameters for controlling the program's execution. -- Kuuran
We can use a static html page for the server side decided parameters. My client (alpha5) hangs while trying to download bots. Maybe the repository site slows down on the client like it does on humans at times? Is the client honouring any robots.txt file they might have on the repository site? If not, it might just shut the downöoad down if it feels the client is demanding too much of it. -- PEZ
I think it would be easier to (1) Upload the parameters file into the Wiki so people can download it. (2) Filter the received battles according to the specifications. It would avoid having to connect every time to the server.
I guess the number of battles count is wrong in the RankingTest? page. I counted it for a bot and the number shown is higher than the battles fought. -- Albert
Well a wiki page might do, but anyone can alter it so I thought a static page would be better. The point with making it a web file would be so that both the client and a human can download it.
I think the battles count is really a "rounds" count. Which one would you rather have? I'll either change the heading or the content. =) -- PEZ
It may be important to prevent people from changing parameters and messing up the stats, though. Also, as for the algorithm, I think it's fine, the reason it's taking so long to stabilize is likely because 90%+ of the bots came in together near the 1600 mark, and rankings are relative, so they're taking a long time diffusing. -- Kuuran
Rounds is better. If people can change the number of rounds it will be meaningless if it's battles. -- Tango
I agree, I suspect the battles count is a rounds count, though I've noticed it's always in multiples of 11, I think it's counting an extra match for some reason. -- Kuuran
Oh dear, it appears some bots are throwing exceptions or timing out (it happens, especially in older bots). They and their opponents end up with divisions by zero in their rankings! This may actually be some fault in robocode as these matches seem to be from the last few I ran in a batch of 1000 battles, it's possibly that robocode's known to be buggy memory management (remember SandboxDT crashing?) started frying the bigger more memory-intensive bots by leaving them no memory to play in. This is just a guess, of course. -- Kuuran
Oh! I'w try to fix it ASAP, but it won't be before this night.
When I was running Face2face, I was getting memory problems when I was running long series (that's why I made RoboRumble@home iterative). In any case, we need to filter it!
-- Albert
Indeed, we should probably just discard matches with 0 scores on either side, they're rarely valid. In other news, the NaNs are spreading, any bot fighting something with that rank will get it after the battle. -- Kuuran
Well, I'll leave the totalling script until anyone of you have any suggestions for it or I come up with an idea for it (whichever comes first). Is it OK to use a wiki page you think? I kinda like it. Any comments someone might have about the current rankings or movement in the rankings can be made by just editing the page (the wiki way). And the revision control will let us browse previous rankings with ease. As long as people editing the page remember to check "This change is a minor edit" each revision of the page will correspond to a rankings update. (Provided someone ran Roborumble@Home that day, which seems likely). Let us know what you think. -- PEZ
I certainly like it. -- Kuuran
I think it works well, although the current page is called "test", which will need to be changed when we go live. -- Tango
I like it also. -- Albert
kawigi.SpareParts 0.7.nosnd crashes RoboRumble@home. It happened to me twice. Some else has noticed it? -- Albert
I haven't had this problem yet, and since I've run about 2000 battles I think SpareParts would've come up at least once. So I'm going to guess it might have something to do with your system or your copy of SpareParts. Does SpareParts run properly in a normal match started from within Robocode? -- Kuuran
It was running some battles (Beta 1) and then it stopped:
# # HotSpot Virtual Machine Error, Internal Error # Please report this error at # http://java.sun.com/cgi-bin/bugreport.cgi # # Java VM: Java HotSpot(TM) Client VM (1.4.0_01-b03 mixed mode) # # Error ID: 43113F2652414D452D41503F491418160E435050005B # # Problematic Thread: prio=5 tid=0x009D2348 nid=0xa98 runnable #
Should i report it to that url? -- Tango
Typo: "Uloading Results..." should be "Uploading Results..." -- Tango
Never mind, it's just JVM crashing. No need to report it. --
The JVM seems to be doing this quite alot overhere ... I already lost alot of good data this way :/ Any idea what's causing it? -- FnH
Just noticed you don't lose data when the JVM crashes. Well done Albert :) -- FnH
I've had some problems with the JVM slowing my system to a crawl after running RoboRumble overnight, but since the CPU consumption was minimal (actually less than normal when RoboRumble was running) I believe it was a memory issue. It's possible you're having some memory issues of your own. I know RoboRumble was made iterative to avoid alot of these problems, but the java process doesn't die between iterations, only the robocode one on the JVM does, so not all the mess seems to be cleaned up. Could your issue be something related (is this happening after alot of hard running, possibly concurrent with robocode?)?
And yes, it all gets saved to a text file match by match, Albert's idea of disintegrating everything was a good one. So all your partials will be uploaded next time you complete a run. -- Kuuran
For PEZ: I just uploaded the server classes modified to fix the NaN bug (I hope). You can download the file from the same page you download the client. Now it will skip any result with a 0 score in either side, and check for NaNs. No need to delete the existing files. Bots with a NaN will be replaced to a 1600 rating. -- Albert
I've installed the updated servlets now. -- PEZ
It seems servlets are the old ones (I added some formating to the details but it's not working). May be Tomcat must be reestarted? -- Albert
Duh! I always forget that. Now fixed. -- PEZ
OK, got first XSLT's finished ... takes xml input, then sorts and ranks results and writes this in another xml-file. This XML-file can then be transformed into wiki-language (finished) or a html-page (in progress) Who do I send them to? -- FnH
Send them my way. I can change my script so that it produces the XML for it. But not tonight, I'm starting to get really, really sleepy here. ... -- PEZ
Sent -- FnH
You shouldn't be doing stuff like messing with your crontab when you can hardly keep your eyes open... It was a beginners error. Now /RankingsTest will be updated every half hour so keep those @Home clients running! I can't participate because the only machine I have that will run the client is the server and I want it to be responsive to Tomcat and wiki requests. -- PEZ
What do you exactly mean by that? Another option would be to increase the change rate constant, or make it deppend on the number of battles fought (the less the faster). Also, note that new versions will start with the old version ranking.
That takes me to another question. How is the script dealing with versions change? If more than one version of a bot is present, it should consider the one which has fought the last battle. Also, it should not show a bot if it has no matches for, lets say, 2 days. In this way the server would eliminate the bots removed from competition (or updated to new versions). -- Albert
A low constant won't be as big of a problem (it actually helps promote stability) when most of the bots are at their fair rating and we just need to work with new bots. The problem is that now every bot is at an artificial rating near 1600 they're all slowly adjusting in the right direction, and the fact that the ratings are so far off just makes the process that much more painful (because we'll have a fluctuation back in the opposite direction later, most likely). We probably should up the constant, just to get them all near their actual rating quickly, then drop it again. -- Kuuran
Rather than removing bots which haven't played for 2 days, can't you just get the server to look at the /Participents? page and remove bots that aren't on it (don't delete the data, just don't display it) -- Tango
I don't know what I mean with that. =) I just thought that it might be possible to make a rough estimate on a new bot. But now I realize it would solve nothing as long as all bots are off in their ratings. As for how the totalling script deals with new versions... It doesn't. At first I just made the script because I was curious on how my bots ranked. And then I tried post it on the wiki and to make the ranking table fancy. =) But it shouldn't be very hard to make it consider this new version thingy. -- PEZ
It was a bit harder to patch the "consider only the newest version" functionality on the current script. Right now I have made it so that files for bots that haven't run the last 5 days (chicken me) are moved away so that they don't appear in the total ranking. Albert, maybe you could check if it's easy for you to make sure only the relevant data file for a given bot stays in the "rankings" folder? If it's not easy for you I'll make a second try on "my" side. -- PEZ
One more thing. The wiki is currently set to keep old revisions of the pages for 14 days. Is that enough of history you think? Right now the total rankings page is starting to consume more disk space than the rest of the wiki pages together. But that will not be a problem once we update the rankings once a day instead of once every half hour. So I could set the wiki to save more revisions if you think it would be useful. -- PEZ
Ok, I'w try by reading the participants list from the upload servlet. If it is not able to retrieve it, it will not change anything. if it finds it, the servlet will check both participants to see if they are in the competition. -- Albert
So you're saying if a bot is removed from the participants page and someone trys to upload a battle using it (having not updated their particpants1v1.txt file since the removal) the server will ignore it? -- Tango
About revisions: What about marking the revisions as minor changes, and then uncheck, lets say, the ones made at 12:00? -- Albert
Albert, good idea about checking the participants list. I'll still have a problem with my script though but I think that if you just skip the version number in the data file name stored in the "rankings" folder this would do the trick and reflect the fact that we consider the old and new versions ranking the like. What about minor revisions? I was thinking we should only do one update each day anyway. Also I was thinking we could use the "minor change" box for when adding comments to the page manually. -- PEZ
That's what i was going to say, but i got one of those annoying Edit Conflict things, and you said it first. I'll say it anyway:
There was a plan to use the minor changes to discuss the ratings on the page. When we aren't testing the rankings will only be updated daily anyway, i think. -- Tango
Roborumble@Home has really boosted the wiki usage. I have gotten edit conflicts at least 20 times the last days and I haven't got 10 in total before Roborumble. =) -- PEZ
I know. I'd never had an edit conflict until a few days ago. I didn't even know what happened when there was one. In the last few days every few edits is conflicted. -- Tango
#!/bin/bash
# $Id: totalrumble,v 1.5 2003/09/02 22:42:34 peter Exp $
rankingsWikiPage='RoboRumble/RankingsTest';
DATE=$(date)
cd /var/lib/tomcat4/webapps/rumble/rankings/
find . -maxdepth 1 -type f -mtime +5 | xargs -i mv {} old/
for file in roborumble_*.txt
do
flag=${file#roborumble_}
flag=${flag%%.*}.gif
flag="http:/robocode/rumble/pkg-flag/${flag}"
bot=${file#roborumble_}
bot=${bot%.txt}
version=${bot#*_}
wikiName=${bot%_*}
package=${wikiName%.*}
wikiName="[[${wikiName##*.}]]"
details="[http://rumble.robowiki.dyndns.org/servlet/RankingDetails?game=roborumble&name=${bot/_/%20} Details]"
awk -F"[ =]" -v bot="${package}.${wikiName} ${version}" -v details="${details}" -v flag="${flag}" '
BEGIN {
OFMT = "%.2g"
}
/^ranking=/ {
rating = $2
}
/^battle/ {
battles += $2
}
END {
print rating+0 "|" bot "|" details "|" flag " |" battles
}
' $file
done | sort -gr | awk -F"|" -v date="`date`" '
BEGIN {
print "=== RANKING TOTALS ===\n"
print "LAST UPDATE: " date "\n"
print "|<b>RANK</b>||<b>BOT</b>||<b>RATING</b>|<b>ROUNDS</b>"
}
{
print "| " NR " | " $4 " | " $2 " | " $3 " | " $1 " | " $5
}
' | post-rumble-results $rankingsWikiPageThat last line will read something like:| to-results-xml | to-rankings-listing | to-wiki-syntax | post-rumble-results(and I can remove the "sort -gr" in the pipeline) when I have gotten FnHs XSLT's in place. I'll also need to modularize things a bit if I should be able to add the contributors listing page withouth breaking the OAOO rule. -- PEZ
And here's the post-rumble-results script. Feel free to use it as a start if you ever automate updates of a usemod wiki. Please consult the Robowiki community in general and me in particular before you start script updating any page on this site. It's easy to fill my hard disks with scripts like this.
#!/usr/bin/perl -w
# $Id: post-rumble-results,v 1.4 2003/09/03 06:12:39 peter Exp $
use strict;
use LWP;
use HTML::TokeParser;
my $browser = LWP::UserAgent->new;
my $wikiPage = $ARGV[0];
shift @ARGV;
my $url = 'http://robowiki.dyndns.org/perl/robowiki/?action=edit&id=' . $wikiPage;
my ($title, $oldtime, $oldconflict) = wikiFields($browser->request(HTTP::Request->new('GET', $url))->content());
my @pageData = <>;
$browser->post($url, [
'title' => $title,
'oldtime' => $oldtime,
'oldconflict' => $oldconflict,
'summary' => 'New results! (Testing)',
'text' => join('', @pageData)
]);
sub wikiFields {
my $stream = HTML::TokeParser->new(\$_[0]) or die $!;
while (my $tag = $stream->get_tag("form")) {
$tag = $stream->get_tag('input');
$title = $tag->[1]{value} || "--";
$tag = $stream->get_tag('input');
$oldtime = $tag->[1]{value} || "--";
$tag = $stream->get_tag('input');
$oldconflict = $tag->[1]{value} || "0";
}
return ($title, $oldtime, $oldconflict);
}-- PEZ
New RR@H beta2 and new Server Classes released.
I just released the new client and the new server classes with the new ranking system and smaller data files.
-- Albert
Albert, can you write a quick note on how you process the results to the ratings please. -- Paul Evans
In the /RankingSystem page. --Albert
Server v.b3 released
PEZ, I just release the v.b3 for the server. The changes are:
-- Albert
Cool. I'll install it right away. For the next version maybe you can put the momentum figure into the total ratings file so I can display it in the rankings table? -- PEZ
I'w try. Let me check. -- Albert
New servlets in operation. But now Marshmallow's details page is very short. Only the first ten bots or so. -- PEZ
Seems there is a problem with some bots. Please, could you send me by mail the ratings file and the MarshMallow? details file so I can check what happens (albert_pv at terra.es? -- Albert
I believe it was Fermat whose details are now a server error about a null String being passed to parseDouble. I think all the results passed to the previous version server might be not showing. I still can't upload. The joys of beta ;) -- Kuuran
I can upload now, and i'm uploading a lot of battles as i type. That might explain why you can't, i'm overloading the server. ;-) -- Tango
I've added a "total battles fought" figure to the end or the rankings page. It's close to 40k battles already! -- PEZ
Nice. We could add also a rating average to see which is the current medium rating. -- Albert
Server v.b4 released
I just released server b4:
I have not been able to test it with multiple clients (since my installation is local) so I don't know what will happen!!! -- Albert
We'll soon find out. I have activated the new servlets. My guess is that since the upload is a rather lengthy process we might see some timeouts at the client side. I assume the upload session isn't atomic? Then the problem shouldn't be too awful to live with. -- PEZ
Should not bee a problem. It keeps track of unloaded results battle by battle, so if one time out, it will be uploaded later. -- Albert
@PEZ: Are scripts prepared to deal with different games? (ie. roborumble, teamsrumble, minirumble, microrumble, nanorumble). -- Albert
Not really. But I don't think that should be too hard to add. (Though I have said so before and been wrong...). Let me check. -- PEZ
OK, now it might be prepared for those games you mentioned there. Whenever a new game is introduced, just tell me and I'll add it to the list. -- PEZ
Maybe I was too quick to say it was ready. Now the rankings pages doesn't look like much. =) Strange, because I think it worked the first time I tested. I'll investigate some and you can start a new game if you like and I should be finished when you have some data for the new game. -- PEZ
Gah, it was some stupid wiki-linking thingy. Obviously it accepts name for pages posted that it later refuses to display. Well, now I think we are ready for a test. Which nano is best anyway? -- PEZ
I'm fixing an small (and anoying) problem I found when CodeSize tries to read the .jar file. As soon as it is fixed, I'w release the new client and we test. -- Albert
Will the new client offer/set %ages for each battle type - e.g. 2% team, 5% melee, 5% 1v1Nano etc? -- Paul Evans
I'm thinking about it. For teams, it's sure it will not, because participants are different (so it uses a different properties file) - but there are some possible work-arounds modifying the ".bat" file. Also, server will need some adjustments, since now it filters out any result with field size different from 800x600. For minis, micros and nanos, I hope it will in a later release (or we will have nanos starvating). Right now what it does is to check if a battle corresponds to a certain size and then upload the results twice (or 3, or 4 times): once for each competition. -- Albert
Distributed computing is way cool! Though it looks a bit strange that all bots have index 1600.00. -- PEZ
Yeah! It looks strange to me also. I'm checking to see why it happens, but I don't see anything strange in the program. -- Albert
Aren't you not counting results for the opponents of new bots? All the bots are new, so they're all facing new bots, and all having their ratings not adjusted, unless I'm missing something? I just assumed we didn't see this in the big rumble because of the rate matchs were run at. -- Kuuran
That's a very good point. It could well be that. It shouldn't be hard to change the new bot rule so it doesn't count if they are both new bots. BTW, Now the results are summerised, do you have any record of who ran what battles? Would it be easy to just keep a running total of how many battles each client has run? -- Tango
I fear that I screewed up something, and no ranking is beeing updated right now! (nor mini, nor micro, nor nano, nor normal) -- Albert
I was beginning to think that the roborumble index had settled. But a bit too much since the ratings didn't change at all. I'm ready to install new servlets as soon as you have downloadable. -- PEZ
New Server servlets are ready (Server v.b5). Sorry about that. The filters I added to prevent wrong data to be readed were filtering everything. -- Albert
Now they are installed and activated. -- PEZ
Question, is it possible to run all the battles we uploaded to the b4 servlet through the b5 servlet so all that data isn't lost (and so the codesize restricted leagues come to some semblance of an update)? -- Kuuran
I doubt it. The data probably got filtered away and is lost. But the previous rankings are there and all pairings data and such. So in theory it should be possible to prime the mini rumbles. I think. -- PEZ
If you stocked a results file with all the existing match data and set a client to upload only, with only the minirumble addresses, then it should on it's own sort all the data out and upload. Would be a fairly big load, though, and you'd probably have to do it locally to stop from bogging down the server. -- Kuuran
In fact it's not completely lost (we can say only 50% lost). Note that ratings are calculated using the %score and the enemies rating. The rating was not updated, but the % score was. Because every time you upload results for a bot, the server reevaluates against ALL BOTS (not only the enemy) the % scores updated between b4 and b5 will act on bot rating, even if it will take some more time (one round against a single enemy per bot). The programs are not rubust, but the model is very tolerant to faults :-) -- Albert
Very nice work, Albert. :) -- Kuuran
If I understand correctly implementing "single threaded model" means you are guaranteed that a service method of the servlet (like doGet() or doPost()) is executed in serie and not in parallell. However this does not mean that the ratings files doesn't risk concurrent updates. The servlet container might very well choose to create a new instance of the servlet. (In fact it probably does if a new request arrives before a previous request is finished processing). I can start figuring on a "data manager" object that the servlet can use if you like. -- PEZ
If you look at the code, you will see that the servlet "implements singleThreadModel?" which (according to documentation) shoudl guarantee that there is just one instance of the servlet, running in a single thread. Of course, I can be completely wrong (I have not experience with it). So if someone has experience in cases like this, please take a look into the servlet. -- Albert
That's what I'm talking about. I think it only guarantees that a single thread executes that servlets service methods at single time. But it says nothing about only creating one servlet instance for what I can see. Can you point me to where you have read about it? -- PEZ
http://www.depi.itch.edu.mx/examples/misc_servlets/singlethread.html -- Albert
This is not how I have been tought this thing works. I must read up on the subject it seems. I'll be back. -- PEZ
Well, in the (oldish) book "JAVA Servlet Programming" by Jason Hunter it quite clearly says the opposite. It says about what I have said above (which is not too strange since this book is where I started with servlets). Implementing the single thread model means Tomcat will use a pool of servlet instances instead of sending multiple threads through a single servlet instance. And the fact that we still get data corruption seems to prove the book is right here. I'll go ahead write a data manager and we can try it to see if it helps. Following our quick-and-dirty route for the first implementations this shouldn't take too long from what I see in the current servlet code. -- PEZ
Aternativly it may be time to switch to a database - it should be designed for concurrency issues. -- Paul Evans
Indeed, but we can take it in small steps. If I can free the upload servlet from knowledge about how data is stored it won't care later when we use a database. -- PEZ
The total number of battles being reported is twice the actual number. Look at the team rumble for a very quick to see example, it either needs to stop counting the battle once for each bot participating in it (or wow will we have a lot of melee battles run ;) or it needs to divide the total by the number of bots per battle. -- Kuuran
I don't quite follow. The DT team has 5 battles reported. How can that be twice the actual number? -- PEZ
On the team league both the battle count and the ranking does not correlate between the ranking page and the details page. The 1v1 league is having problems with correlating battle count between the ranking page and the details page. -- Paul Evans
The ranking page is static, generated every two hours. The details page is dynamic, and shows the current status of a bot. The difference between both pages is just the number of pages (and rating change) since the last time ranking page was generated. -- Albert
I'm referring to the total battles in the league figure at the bottom of the ranking page. -- Kuuran
The first, tiny step towards a safer data handling is installed. It's as bad as before in terms of not stopping concurrent writes (and reads while writing). But now all disk access is done by a Singleton object called DataManager which, being a Singleton, will be able to ensure the data integrity. Most of the data storage knowledge is removed from the servlet which should allow Albert or anybody else to update the servlet service while I or someone else continue trying to stop the data corruption we are currently experiencing. -- PEZ
Shouldn't the total battle count be the sum of all individual battles? I guess not in melee, coming to think of it, but we have no melee game yet. Remind me when we start that game if I seem to have forgot. -- PEZ
Wait! Now I understand what you mean. Twice as many. Each pairing leads to two bots getting a battle count. I'll fix it. -- PEZ
OK. Dirty fix is that I divide by 2 before reporting the figure. This is because there's no information for me in the ratings file how many bots are participating in the battles. Since the melee games will probably also have a set number of bots I can fix those in the same way. -- PEZ
Thanks :) -- Kuuran
I've scheduled the publishing script so that it runs every morning 07:00 CET sharp. What will be a good update frequency? Every other day or once a week? Let's vote. -- PEZ
I think I'd prefer once a day, even if the rankings are less than stable. -- Kawigi
Publishing what exactly? -- Paul Evans
Sorry. Publishing the current ranking tables. -- PEZ
Once a day better. What makes this interesting is that changes fast. -- Albert
Can this page not be done dynamically? (the upload code may have to update a summary file, but it would keep all pages in sync). -- Paul Evans
The upload code already updates the summary file. It's just the publishing of it that's done only now and then. Of course it could be done manually. This is not a technical limitations issue. It's about how often we want the rankings table to be updated. Since the rankings are reasonably stable it won't matter all that much, in some sense. But I, for one, would like the table to be static for periods of time so that I can title myself this days/week #18 (or wherever I'll end up when Marshmallow has stopped falling like a stone). I see no problem at all with the asynchrisity (is that a wrod?) of the different pages. As long as you know one is dynamic and the other frozen for a given period of time. A point with keeping the rankings frozen a while is that we can browse some in the history of the rankings. The wiki keeps the old revisions of it's pages for 14 days (as the settings are now).
Of course, we could alwyas do both dynamic and static. Something along the lines of always keeping a dynamic rankings table and the every week publish the rankings to a page named after the month it is run (like /RankingsSeptember03?). Then the wiki would "forever" keep 12 rankings a year and, if I up the number of days the wiki keeps old revisons, the current month holds four weekly revisions of the rankings. -- PEZ
I would like to see the dynamic ratings table - it has the advantage of not clogging up the wiki 'changes' page - and I agree that it would be nice to have weekly/monthly snapshots kept for posterity (presumably the snapshots would simply be the 'output' from the servlet). The other advantage of a dynamic page is that problems can be detected much earlier, new bots won't have to wait for a week to see how good they are, and client contributions can be seen immediatly encoraging partcipation in the @home project. -- Paul Evans
OK, then I vote for this suggestion. Even if one of the reasons we went for static ranking tables was to not bogg the server down with this transformation for all requests, but that could be taken care of in the servlet if it shows to be problem. -- PEZ
I'w check and will see. -- Albert
I suggest each new version should start from 1600, there is no law to state that new bot versions are a goos as, or even related to previous versions. -- Paul Evans
Works for me. But it better be intentional. =) -- PEZ
The question is. Where do you expect a new version be closer to: 1600 or its previous ranking? Note that misplacing a bot can perjudicate bots that fight against him until it stabilizes (can you imagine what will happen to bots figthing DT 3.0 until it gets up in the rankings?) -- Albert
What if you initialize the rating to 1600 and exaggerate the rating change speed in a bot with really few battles fought and while it is in that state you do the opposite with bots it meets (that is, don't let it contaminate the enemy bot until you have a reasonable guess about where the "new" bot belongs). Unless they both are in this "find neighbourhood" state. If so you treat it as a normal battle result. -- PEZ
Could you colour code the "score %" column too maybe? Something like:
share < 40 (red) 40 <= share < 49 (light red) 49 <= share < 51 (white) 51 <= share < 60 (light green) share >= 60 (green)-- PEZ
I just uploaded the new servlets. The changes are:
-- Albert
I've installed the new servlets. Also changed the sceduling for the rankings wiki pages so thet they run once a week (Sundays, early mornings CET) and posts to a page named after the ucrrent month e.g. RoboRumble/Rankings roborumble 200309 This gives us the ranking snapshots for each month archived on the wiki. If someone wants a particular weeks snapshot saved it's just a matter of copying the wiki text source to another page. -- PEZ
Cool! But the daily rankings update will be generated daily as usual, isn't it? -- Albert
Not as I have set it up right now. I thought your dynamic rankings servlet would take care of that need. -- PEZ
OK. I think it's fine. One more question: are then the ranking pages not ending with 200309 valid or not?
They wont be updated any more anyways. I think that makes the deprecated. Another thing. The flag gifs are in the folder "pkg-flags" side by side with the "rankings" folder. In case you decide to include flags in the dynamic rankings page. I also think that servlet should be named "Rankings". -- PEZ
Great, I'w include them together with a link to the details page. -- Albert
I just uploaded the new server beta 7. It includes some changes:
Server classes beta 7 are in operation. -- PEZ
The server keeps doing weird things. Now it shows a "Duelisst" bot in the rankings. Any idea why it happens? Also, I tried to access battles_roborumble.txt to investigate the problem but the file seems not to be there. -- Albert
Hmm, I e-mailed you about this a couple of days ago. Didn't you get it? The battles_ files disappearance might be my fault entirely. I'll check it out tonight. -- PEZ
Do you think that synchronizing also the methods could fix the concurrent update problem? Take a look into:
http://livedocs.macromedia.com/jrun/4/Programmers_Guide/techniques_servlet15.htm
-- Albert
It might, especially if there's only one point in the code that writes to a file. -- Kawigi