I've installed a rumble server on my home computer and have been running battles for the last couple of days. I also did a few experiments with the code, to better understand how it works. You can check the current rankings here:
Looks quite good even with only around 20 battles per bot. If you want to contribute just change you client's configuration accordingly, let's see how long it takes to get a truly stable ELO ranking.
I'm planning on adding an APS ranking next weekend, and then maybe put it on dedicated hardware.
-- ABC
Awesome, you're the man!! -- Voidious
Nice! Any plans to put melee or maybe team up there (though team is still working fine on Pulsar's server for now I think)? Hmm, I wonder how temporary this will be considering how thing were last time someone talked of a "temporary" RR server... :) (Also, I just noticed RougeDC got an extremely unusual result against step.nanoPri_1.0 though I'm not sure if it's a bad client problem, something eating lots of the client's CPU, or if RougeDC somehow went and crashed really bad) -- Rednaxela
Good catch, very strange result. So far I'm the only one submitting results, and both my clients are 1.5.4, so no "bad" client. Let's wait and see if it is an isolated case, I can always delete that result and recompute de the ranking. I made a small utility that re-submits all the battles in the log, but it's only feasible while the number of battles is relatively small. I haven't come up with an easy way of undoing a bad result, the iterative nature of the rumble makes it hard. The only way I see is having periodical backups to revert to and re-submitting the good results since then... -- ABC
Ah great work man! I think I'll update DrussGT, and leave a client running =) -- Skilgannon
Finally! You truly are the man! You know, an APS ranking wouldn't have the same trouble with deleting bad results ... ;) -- Simonton
This may take a while to get a workable field, with this many bots. For each bot to have a battle against every other, it is n + (n-1) + (n-2) +... + 1 = n/2*(n+1) battles, which makes 205761 battles. For each bot to have 2000 battles it is (2000/641)X more battles, 642000 battles. Each battle produces the results for 2 bots, therefore 321000 run battles. Assuming the average battle takes 1 minute, we have 5350 client-hours of work. 223 client-days. If we're running (best case) 4 clients 24/7 it will take 55 days. If we just want 1 battle per enemy it will still take ~18 days, but realistically more than that because some battles will happen twice. I suggest we lower the battles-per-bot constant to ~650 in the roborumble client so that it will fill in the PL gaps earlier, rather than running duplicates while PL slots are still missing. -- Skilgannon
I've used a battles-per-bot setting of 10, then increased it to 20, and will will now set it to 50. I'm hoping it will prevent a lot duplicates that way. 18 days for full pairing would be great! But I believe the (ELO) ranking will be "stable" before that, like with 100-200 battles per bot. -- ABC
Sorry - I missed changing one of the URLs in my properties files. My clients are the culprits of giving over 200 battles to those handful of bots. I wish it was a harmless mistake, but now those bots' rankings may not stabilize until everyone else catches up and they start running again unless someone (ABC) intervenes by re-calculating their scores after the other bots have settled into a more stable state. -- Simonton
No problem, until everyone has full pairings the probability of anyone fighting is the same, I think. I'm more concerned about the bad results still happening. I just found two more: stelo.UnfoolableNano? against Dookious and DrussGT. Both battles submitted by Nfwu. Could there be other problematic Robocode versions? -- ABC
Version 1.6.1.1. Stopped client for now. Should I switch to 1.5.4? --Nfwu
Yep, I believe 1.5.4 and 1.6.0 are the only safe versions. But first I need to find an easy way to revert the bad results, please use a different name for submitting battles for now, I will probably have to delete all battles with your user name (nfwu). -- ABC
UnfoolableNano? vs. Ascendant, not Dookious, I think? How about just removing these pairings from the results files of these bots? -- Skilgannon
Yes, and yes. But I want to make it at least semi-automatic. I'm also considering refusing results with >50 PBIs. -- ABC
Looking for all the bad results to remove the pairings was too much work. I had to reset the rankings, remove all of Nfwu's battles from the log and resubmit all the battles (multiple times, it's fun seeing the ranking stabilise even with a small number battles :)). I also did a backup of the ranking and details files and cleaned the battle logs. Maybe I'll set an automatic periodical backup up so that I can revert it back a few days in the case of corruption. -- ABC
How "temporary" do you intend this server to be? For a long-term solution, I would recommend storing all results in a database rather than a log, so that queries can find all kinds of fun things much easier than looking through the log manually, or writing a new script to extract data you want (from bad results to new scoring schemes ... maybe a simulated bracket tourney if you like). I plan to have the RoboResearch server keep all the results that pass through it, including those bound for the rumble, so maybe that database could be a good place to turn in the future. I figured I'd let you know, so you can plan the best use of your time. -- Simonton
Excellent question! With my second child coming at the end of this month I don't think I'll have a lot of time in the next few months(years?) :). I just thought I would have some fun with java application servers instead of tweaking Shadow without a rumble to test it on. Anyway, if you are developing a battle results database, that's where the future rumble must come from, not from my (very) modest java servlet skills. I'll just keep this one running until someone comes up with a better solution. And in the meantime I'll try adding some simple tweaks of my own, the priority being always that it stays functional and online for people to use. -- ABC
![[Home]](/images/RoboWiki.png)