ABC/TemporaryRRServer

Ok, I can't take it anymore, life without a working RR server is too boring... :)

I've installed a rumble server on my home computer and have been running battles for the last couple of days. I also did a few experiments with the code, to better understand how it works. You can check the current rankings here:

http://abchome.aclsi.pt:8080/

Looks quite good even with only around 20 battles per bot. If you want to contribute just change you client's configuration accordingly, let's see how long it takes to get a truly stable ELO ranking.

I'm planning on adding an APS ranking next weekend, and then maybe put it on dedicated hardware.

-- ABC

Awesome, you're the man!! -- Voidious

Nice! Any plans to put melee or maybe team up there (though team is still working fine on Pulsar's server for now I think)? Hmm, I wonder how temporary this will be considering how thing were last time someone talked of a "temporary" RR server... :) (Also, I just noticed RougeDC got an extremely unusual result against step.nanoPri_1.0 though I'm not sure if it's a bad client problem, something eating lots of the client's CPU, or if RougeDC somehow went and crashed really bad) -- Rednaxela

Good catch, very strange result. So far I'm the only one submitting results, and both my clients are 1.5.4, so no "bad" client. Let's wait and see if it is an isolated case, I can always delete that result and recompute de the ranking. I made a small utility that re-submits all the battles in the log, but it's only feasible while the number of battles is relatively small. I haven't come up with an easy way of undoing a bad result, the iterative nature of the rumble makes it hard. The only way I see is having periodical backups to revert to and re-submitting the good results since then... -- ABC

Ah great work man! I think I'll update DrussGT, and leave a client running =) -- Skilgannon

Finally! You truly are the man! You know, an APS ranking wouldn't have the same trouble with deleting bad results ... ;) -- Simonton

• Was just thinking that myself about APS ranking... ;) -- Rednaxela
• I don't see how APS would be different, the problem is the rolling average used for the scoring. How do you "unroll" a bad result without checking every single previous battle or knowing the previous score before the bad result? Looking for all the battles between 2 specific bots can become hard when the battles log file becomes too large. -- ABC
• The APS score I'm thinking of would not be a rolling average. Keep "total points" and "num battles" for each pairing, then to remove a bad one subtract its score and 1, respectively. Calculate the change in the average for that pairing and apply 1/num-pairings-th of it to the overall average. -- Simonton
• Ok, that's a solution for the ELO too, the next few battles would put the affected bot in its proper place again, or I could force a rating recalc with an increased constant. The only difference to the current setup is that data saving bots would lose the small advantage of the rumble "forgetting" old results, I'd be ok with that. -- ABC

This may take a while to get a workable field, with this many bots. For each bot to have a battle against every other, it is n + (n-1) + (n-2) +... + 1 = n/2*(n+1) battles, which makes 205761 battles. For each bot to have 2000 battles it is (2000/641)X more battles, 642000 battles. Each battle produces the results for 2 bots, therefore 321000 run battles. Assuming the average battle takes 1 minute, we have 5350 client-hours of work. 223 client-days. If we're running (best case) 4 clients 24/7 it will take 55 days. If we just want 1 battle per enemy it will still take ~18 days, but realistically more than that because some battles will happen twice. I suggest we lower the battles-per-bot constant to ~650 in the roborumble client so that it will fill in the PL gaps earlier, rather than running duplicates while PL slots are still missing. -- Skilgannon

I've used a battles-per-bot setting of 10, then increased it to 20, and will will now set it to 50. I'm hoping it will prevent a lot duplicates that way. 18 days for full pairing would be great! But I believe the (ELO) ranking will be "stable" before that, like with 100-200 battles per bot. -- ABC

Sorry - I missed changing one of the URLs in my properties files. My clients are the culprits of giving over 200 battles to those handful of bots. I wish it was a harmless mistake, but now those bots' rankings may not stabilize until everyone else catches up and they start running again unless someone (ABC) intervenes by re-calculating their scores after the other bots have settled into a more stable state. -- Simonton

No problem, until everyone has full pairings the probability of anyone fighting is the same, I think. I'm more concerned about the bad results still happening. I just found two more: stelo.UnfoolableNano? against Dookious and DrussGT. Both battles submitted by Nfwu. Could there be other problematic Robocode versions? -- ABC

Version 1.6.1.1. Stopped client for now. Should I switch to 1.5.4? --Nfwu

Yep, I believe 1.5.4 and 1.6.0 are the only safe versions. But first I need to find an easy way to revert the bad results, please use a different name for submitting battles for now, I will probably have to delete all battles with your user name (nfwu). -- ABC

UnfoolableNano? vs. Ascendant, not Dookious, I think? How about just removing these pairings from the results files of these bots? -- Skilgannon

Yes, and yes. But I want to make it at least semi-automatic. I'm also considering refusing results with >50 PBIs. -- ABC

Looking for all the bad results to remove the pairings was too much work. I had to reset the rankings, remove all of Nfwu's battles from the log and resubmit all the battles (multiple times, it's fun seeing the ranking stabilise even with a small number battles :)). I also did a backup of the ranking and details files and cleaned the battle logs. Maybe I'll set an automatic periodical backup up so that I can revert it back a few days in the case of corruption. -- ABC