[Home]ABC/TemporaryRRServer

Robo Home | ABC | Changes | Preferences | AllPages

I'm closing down this server as of tomorrow (2008/11/12), we now have a better rumble server solution developed by Darkcanuck. Please configure your clients: http://testwiki.roborumble.org/w/index.php?title=Darkcanuck/RRServer.

I had a lot of fun. I experimented with Albert's code and made some small improvements, I even have a working version that uses mySQL instead of text files. Unfortunately I don't currently have the time for further developments, and Darkcanuck is doing a great job with his server.

-- ABC

Thanks for your work! Any plan to release the code behind some of your changes/improvements? I think some of those would be nice to have merged into Darkcanuck's work, like the nicer table formatting and your really neat LRP implementation. :) -- Rednaxela

Thanks for putting up this server and mobilizing the community to contribute so many battles! In fact, I'm still importing battle data from your server -- can you keep the files online even after you shut the servlets down? You're welcome to help out and/or steer the direction for the new server. Also, if you're willing to pass on the LRP js code, I'd love to add it to the website. It looks it just needs a URL that provides the battle details table in JSON format? -- Darkcanuck

I'll leave it online for a few days, I'll just disable the results uploading, feel free to take anything you need. The LRP and comparing javascript just needs the details data in this format: http://abchome.aclsi.pt:8080/rumble/RatingDetailsJson?game=roborumble&name=jk.mega.DrussGT 1.3.0 . The previous versions comparison is still a bit rough, it expects an URL /GetOlderVersionsJson? that lists the previous versions of a bot in json format... I'll see if I can still improve it in the next few days. -- ABC


Ok, I can't take it anymore, life without a working RR server is too boring... :)

I've installed a rumble server on my home computer and have been running battles for the last couple of days. I also did a few experiments with the code, to better understand how it works. You can check the current rankings here:

http://abchome.aclsi.pt:8080/

Looks quite good even with only around 20 battles per bot. If you want to contribute just change you client's configuration accordingly, let's see how long it takes to get a truly stable ELO ranking.

I'm planning on adding an APS ranking next weekend, and then maybe put it on dedicated hardware.

-- ABC

Awesome, you're the man!! -- Voidious

Nice! Any plans to put melee or maybe team up there (though team is still working fine on Pulsar's server for now I think)? Hmm, I wonder how temporary this will be considering how thing were last time someone talked of a "temporary" RR server... :) (Also, I just noticed RougeDC got an extremely unusual result against step.nanoPri_1.0 though I'm not sure if it's a bad client problem, something eating lots of the client's CPU, or if RougeDC somehow went and crashed really bad) -- Rednaxela

Good catch, very strange result. So far I'm the only one submitting results, and both my clients are 1.5.4, so no "bad" client. Let's wait and see if it is an isolated case, I can always delete that result and recompute de the ranking. I made a small utility that re-submits all the battles in the log, but it's only feasible while the number of battles is relatively small. I haven't come up with an easy way of undoing a bad result, the iterative nature of the rumble makes it hard. The only way I see is having periodical backups to revert to and re-submitting the good results since then... -- ABC

Ah great work man! I think I'll update DrussGT, and leave a client running =) -- Skilgannon

Finally! You truly are the man! You know, an APS ranking wouldn't have the same trouble with deleting bad results ... ;) -- Simonton

This may take a while to get a workable field, with this many bots. For each bot to have a battle against every other, it is n + (n-1) + (n-2) +... + 1 = n/2*(n+1) battles, which makes 205761 battles. For each bot to have 2000 battles it is (2000/641)X more battles, 642000 battles. Each battle produces the results for 2 bots, therefore 321000 run battles. Assuming the average battle takes 1 minute, we have 5350 client-hours of work. 223 client-days. If we're running (best case) 4 clients 24/7 it will take 55 days. If we just want 1 battle per enemy it will still take ~18 days, but realistically more than that because some battles will happen twice. I suggest we lower the battles-per-bot constant to ~650 in the roborumble client so that it will fill in the PL gaps earlier, rather than running duplicates while PL slots are still missing. -- Skilgannon

I've used a battles-per-bot setting of 10, then increased it to 20, and will will now set it to 50. I'm hoping it will prevent a lot duplicates that way. 18 days for full pairing would be great! But I believe the (ELO) ranking will be "stable" before that, like with 100-200 battles per bot. -- ABC

Sorry - I missed changing one of the URLs in my properties files. My clients are the culprits of giving over 200 battles to those handful of bots. I wish it was a harmless mistake, but now those bots' rankings may not stabilize until everyone else catches up and they start running again unless someone (ABC) intervenes by re-calculating their scores after the other bots have settled into a more stable state. -- Simonton

No problem, until everyone has full pairings the probability of anyone fighting is the same, I think. I'm more concerned about the bad results still happening. I just found two more: stelo.UnfoolableNano? against Dookious and DrussGT. Both battles submitted by Nfwu. Could there be other problematic Robocode versions? -- ABC

Version 1.6.1.1. Stopped client for now. Should I switch to 1.5.4? --Nfwu

Yep, I believe 1.5.4 and 1.6.0 are the only safe versions. But first I need to find an easy way to revert the bad results, please use a different name for submitting battles for now, I will probably have to delete all battles with your user name (nfwu). -- ABC

UnfoolableNano? vs. Ascendant, not Dookious, I think? How about just removing these pairings from the results files of these bots? -- Skilgannon

Yes, and yes. But I want to make it at least semi-automatic. I'm also considering refusing results with >50 PBIs. -- ABC

Looking for all the bad results to remove the pairings was too much work. I had to reset the rankings, remove all of Nfwu's battles from the log and resubmit all the battles (multiple times, it's fun seeing the ranking stabilise even with a small number battles :)). I also did a backup of the ranking and details files and cleaned the battle logs. Maybe I'll set an automatic periodical backup up so that I can revert it back a few days in the case of corruption. -- ABC

How "temporary" do you intend this server to be? For a long-term solution, I would recommend storing all results in a database rather than a log, so that queries can find all kinds of fun things much easier than looking through the log manually, or writing a new script to extract data you want (from bad results to new scoring schemes ... maybe a simulated bracket tourney if you like). I plan to have the RoboResearch server keep all the results that pass through it, including those bound for the rumble, so maybe that database could be a good place to turn in the future. I figured I'd let you know, so you can plan the best use of your time. -- Simonton

Excellent question! With my second child coming at the end of this month I don't think I'll have a lot of time in the next few months(years?) :). I just thought I would have some fun with java application servers instead of tweaking Shadow without a rumble to test it on. Anyway, if you are developing a battle results database, that's where the future rumble must come from, not from my (very) modest java servlet skills. I'll just keep this one running until someone comes up with a better solution. And in the meantime I'll try adding some simple tweaks of my own, the priority being always that it stays functional and online for people to use. -- ABC

I've been toying with the idea of creating a database-backed rumble server to address some of the recent problems. It wouldn't be difficult to extend the client to send additional info (e.g. client version, match type) to add more error checking too. But this all relies on free time...

Thanks for getting this temporary server up -- now that I've fixed my rumble machine, I'll start helping out with the crunching and remove Gaff 1.28a from the list. -- Darkcanuck

I just found one of my clients stuck: it would run 30 battles of Okami vs other bots, then try uploading the same 30ish results from some previous run, hit an exception, and repeat. So some battles got submitted many many times. I deleted all the files in "files/" and "temp/", and now it seems to be working again. Not sure if you'll want to roll back results? I'm using version 1.6.0. -- Simonton

As long as the results are valid I see no need to revert them. I'm really liking the look of the ranking, btw, even with under 100 battles per bot the numbers and positions are very close to what I remember from when the old server was 100%. I'll add an APS sorted ranking table later today. -- ABC

As everybody seems to run battles here at the moment, those 'corrupted file' bots could be added again to the participants I think. -- GrubbmGait

Or maybe we could make an alternative participants page to include them. APS ranking: add &type=APS to the rankings URL.

Looking at the latest Robocode version changelog I think it would be safe to use for rumble. All those 'funny' results could definitely have been from a 'swapping' of scores of the 2 bots. Opinions? -- Skilgannon

Yes, the sooner we test it the better. Please use a different user name for submitting results. GrubbmGait is currently submitting results as [GrubbmGait 160]?, a very nice way to differentiate client versions. -- ABC

Great idea -- done! I've been working on my database-backed server idea the past few days, uploading is working and just need to put together some polished ranking pages. No ELO yet though. The concept is to keep the originally submitted data so that it's fairly easy to remove bad results and rebuild the rankings. The current rumble client doesn't send its version number (just "1") but future clients could send the actual version and we could put new releases on probation. I'll put up a page about it when the server goes live. Questions/suggestions welcome! -- Darkcanuck

My experimental server is now live. But keep your clients pointed to ABC's server unless you really, really like hunting bugs. --Darkcanuck

Nice work, good luck with the bug hunting. Meanwhile I added a cool javascript LRP generator to my server, check it out if you have a decent (non-IE) browser. :) -- ABC

Progress report: The LRP graph now works on Microsoft browsers, but it takes a _long_ time to render the graph. I made a few changes to the general layout, and the generated HTML is now more correct. I've also added a few missing flags (our king has a flag, finally :)). Next comes the details comparison page, it wasn't included in the source files I got so I'll have to add a few things of my own, mainly another nice graph to replace the excel file I use to compare Shadow's versions. :) -- ABC

It's nice to see rankings pages that are easy on the eyes... Can I get a Canadian flag while you're at it? -- Darkcanuck

Ok, my server is now relaying results to ABC's. Now my 4 clients are involved in both efforts. :) -- Darkcanuck

Cool, this is going to be even faster than I thought :). Lots of server processing power needed, though, and the file locking mechanism is definitely not good enough for this amount of incoming data. I see lots of file access exceptions in the server log at peak times. -- ABC

Neat server I just spied it, I love your LRP implimentation, I was originally attempting to do something like the popup right next to the circle that your hovering over but IE didn't cooperate so I compromized, glad to see you got it working. I am also happy to see rating drift hasn't reared its ugly head yet and my robots are rating up where they once did. --Chase

100k battles done in a week! Full pairings, here we come. :) -- ABC


Sorry to say this, but it seems that SilverSurfer (maybe more bots?) has had some problematic battles. I don't know if this bot is extra sensitive for skipped turns or that the 1.6.0 version of the client is not so robust as we thought. I will stop my client as long as I do other things on my PC, and will start it again when I go to sleep. -- GrubbmGait

I've checked most of those battles in the log, it seems that they all come from 1.6.0 clients, different uploaders. I don't have 1.6.0 installed here to verify, but it looks to me like it could be SilverSurfer is throwing exceptions in 1.6.0. -- ABC

Ugh... RougeDC is still getting lots of bad battles too... see step.nanoPri, mz.NanoDeath, gh.micro.GrubbmThree and a fair number of other ones, some battles being as recent as today... I haven't checked what versions/uploaders are giving those yet. It is looking like some of those basically have the score reversed possibly (which I thought was a bug in 1.6.1 that was fixed recently) -- Rednaxela

Well this is strange.... RougeDC has a score of 0.0 against Loki [here] (update: same with SandboxLump too) with a massive problembot index of over 90. Doesn't the server reject all 0.0 scores? It's looking like someone's client is having strange exceptions on RougeDC and the server is for some reason accepting bad 0.0 results. -- Rednaxela

I am always surprised how close Seraphim got to the 2000 Club. According to this server its at 1998.65, oh well maybe I should finish the updated version. --Chase

It's looking like 1.6.1.3 may be a good version for rumble once it's released. After 1.6.1.3 is released, think it will be fine to try uploading here (after brief testing with upload disabled)? :) -- Rednaxela

Is somebody using a bad participants URL? Because the old jk.mega.DrussGT 1.2.7 keeps being uploaded by someone despite the newer 1.3.0 being the one in the participants list. Could whoever this is fix their client please? :) -- Rednaxela

It was me, one of my clients had DOWNLOAD=NOT in its properties. Fixed. -- ABC

Woah there, looks like a 4/5ths of the bots just disappeared from the ranking table there! Was that a misconfigured client with too short of a participants list? Or was it something on the server's end? Strange... -- Rednaxela
Update: And now the server went down :( -- Rednaxela

My net connection was down for the last hour or so, and for around 4 hours yesterday. But that doesn't explain the disappearing of the ranking. It's something that used to happen sometimes in Pulsar's server, probably some small design flaw in the client. -- ABC

Ahh I see. I only seem to recall it happening on Pulsar's server at times with other things (disk space or corrupted results) were being wonky too, or maybe it was just coincidence and I don't remember correctly. In any case, thanks again for hosting this :) -- Rednaxela

Ohh, when refreshing the rankings here, I realized that the browser must not be caching results at all and the server is forced to fully regenerate it each time. I don't know how easy it is to do with the java web pages, but I think it would be nice if you could add a code snippet to add "Cache-Control: must-revalidate" and "Last-Modified:" headers, where "Last-Modified" is set to the time of the most recent battle on the server, and then send the browser a "HTTP/1.1 304 Not Modified" if the browser sent the server a "If-Modified-Since" header not older than the last modified time. A quick function in PHP to do that is [here]. That would make refreshes both less taxing on clients and the server, then again... with Darkcanuck's new server code, it might just be worth leaving the old server code as-is and just eventually implementing this in Darkcanuck's server. Probably not a bit deal, but throwing out one idea of how to make it run just a tiny bit smoother. Feel free to ignore my incessant blather :) -- Rednaxela

Thanks, those are some good advices. Even with a highly dynamic page like the ranking, it is probable that some eager mind (like my own, sometimes :)) refreshes the page faster than the clients sumbmit battles. It should be easy enough to set the http headers in the ranking servlet code, and the file has the last update time in its first line. I'll look into it. -- ABC

I think that the pairings are complete :D My client was running non-mega-battles exclusivly and every bot had at least 690 battles, so I increased the BATTLESPERBOT from 650 to 1000. -- GrubbmGait

Unfortunately I believe pairings are drawn randomly. Bots with less battles than BATTLESPERBOT will be randomly paired with another bot out of the 600, regardless of how many times it has or has not already faced that bot. -- Simonton


Robo Home | ABC | Changes | Preferences | AllPages
Edit text of this page | View other revisions
Last edited October 12, 2008 8:52 EST by ABC (diff)
Search: