[Home]TestBed/Discussion

Robo Home | TestBed | Changes | Preferences | AllPages

Difference (from prior major revision) (no other diffs)

Added: 76a77
* until last year i also ran Robocode at work during the night. But they tightened up security and now i can no longer FTP (i guess that is the problem because i can download bots but can no longer upload the results). I even have problems posting on this wiki. --Loki

Removed: 82d82


Added: 83a84,85

I guess one of my problems is that the ratings of the bots in the ranking range 75 to 100 are closer together than the bots at say ranking 30. In my tests the latest version of mini.Wodan did 2% better (42.23% to 44.30% against my original testbed), so i released it. With the TestBed, mini.Wodan did 1.5% worse. So first, my testbed was not representative for the whole rumble (but which 10 bot testbed is?) and second, a small change in performance costs me almost 20 places in the ranking. So in the future i will only release a new version which performs significantly better and maybe i should use GrubbmGait's advice and use the Roborumble as a testbed --Loki

This is a good idea - I've added in a few of my favourite test bots. No idea how fast they will run on your PC, 35 seconds is about the time my PII333 with 64MB RAM takes to display the stats after the battle. :p -- Jamougha

Ouch! That's even worse than it was on my old laptop. It doesn't exactly make you like SlowBots, does it? -- PEZ

Not much. Nor long test periods; a 1000 round battle is a fairly reliable way to crash my PC, in addition to taking close to an hour against Shadow. -- Jamougha

I'm not sure but I think that Aleph isn't such a fast bot. At least not nearly as fast as the other bots on this list. I just wanted to keep him away from the SlowBot list. Well, the times will tell... -- rozu

Aleph is pretty fast for a pattern matcher. :-) This list is a good start, but we need more pattern matchers and more low-ranked bots to get a really representative mixture. --David Alves

SmallDevil could be added. it is in the middle of the ranking and has a specialization index of about 35 (not bad for a nano) and I don't think that bots can get faster then this one. Mako 1.5 could also be added. -- rozu

Doesn't Mako 1.5 save data? I don't remember excatly, but many Mako's did save data. I was looking at the Mako code yesterday and found both really funny stuff and quite cool stuff. I think I can make a quite strong Mako now with my added experience. -- PEZ

Anyone know DuelistMicro's specialization index? For some reason it's not showing on the details page for 1-v-1 results. --David Alves

That's weird... it's only loading half the page... :-/ -- Tango

Ok, but i'm almost positive that Tityus and Quest save data.... --Brainfade

I believe you're right, in fact, I think Quest may have been the first to save a CribSheet. -- Kawigi

Tityus doesn't save data, it only *loads* data. :-) PEZ commented out the data saving in the release version to make the codesize limits, and bundled it with a gazillion data files. I'm fairly sure Quest 0.1 doesn't either. On my machine it doesn't even generate a data directory. -- Jamougha

It wasn't to make the codesize limit I think. It could have been, but with some releases I had other reasons not to save data. -- PEZ

FUSiON is relatively fast for a megaBot (sure it's not great, but still, it's mid-range). It saves minimal data between rounds (clears between battles) and runs 10 rounds v.s. itself in about 4 seconds on my computer (XP 2100+, 512M DDR256, winXP pro). The newest version has been uploaded today 02/18/2004 -- Scoob

I sorted the list on fastness. And I really don't think Aleph belongs in this community. -- PEZ

I also broke the formatting of the table. I just hate wiki table syntax... -- PEZ

Aleph belongs here because it's fast for a pattern matcher... I'm trying to make a list of bots that I should test against that will give me a good idea of what my actual rating will be. I want at least one good pattern matcher on the list, to test movement vs. PM bots. I would have named this page TestBot, but that's already taken :-p --David Alves

You can try Parakeet... though it's not a very good Patternmatcher, it's at least fast :-). It's open-source, so you can disable the data-saving yourself :-p. --Dummy

Smog and SmallDevil are both blazing fast and if you can beat them you're pretty safe against all simple strategies. -- Kuuran

Thanks for moving the page, this is a much better name.

Is there really any reason to have RaikoMicro when you have Tityus? Or DuelistMicro when you have FhqwhgadsMicro? -- Kuuran

David. When you have choosen the bots to beincluded in the test bed. Maybe we can run a bot of choice in each CodeSize class against all those bots for, say, 250 rounds, to get a zero measurement? Then we could call this score index 100 and have a cool tool for performance tuning. -- PEZ

The measurement for performance tuning would be your rating after battling all these bots. The idea is to have the measurement be a really good predictor of your RR@H rating, that's why I want bots with a wide variety of ratings, low specialization indices, and using different targeting / movement strategies. --David Alves

Shadow 2.44 does save data. It's a bit strange for me seeing on of my bots actually going up in the rankings after the first batch of battles ;). It saves a different kind of data, though... I'll add a flag in the properties file to disable it in the next release. It's intended mostly to improve my performance against the weak bots in the rumble by not wasting the first few rounds detecting simple guns. -- ABC

That's a good strategy. Kind of what Swiffer does, but with a more defined purpose. -- PEZ

Better? :-p --David Alves

I think Swiffers strategy is better for Swiffer. Since it can have tested several simple movement schemes to exploit a given simple/flawed gun. But for Shadow I think it's enough to know if anti-head-on works. Since it has that excellent implementation of AdaptiveMovement it defaults to exploit simple/flawed guns anyway. -- PEZ

I meant is it better now that I changed Shadow's saved data column from "no" to "minimal" :-p --David Alves

Well, you should have said so. =) Still. Shadow is the SlowBot of SlowBots. I won't put it in my test bed anyway. I rather live with "touch {test-bed-bots}.jar" before I run my tests. That way I can use bots that save data too. -- PEZ

Well, maybe it's not that minimal... ;) It saves a segmented GF statistic on the opponent's successful bullets, very much like Narcissus. It works against any type of simple gun, and doesn't hurt against good guns either. :) -- ABC

The SlowBot of SlowBots? Are you serious? -- ABC

Ummm, I should test against it some first before I say something like that. I haven't run it for very long. I still get a bit sick when I think about Shadow since it used to halt my Robocode environment for three weeks before I could find a cure. In any case I don't need Shadow in my test bed until I start figuring on how to beat it. =) -- PEZ

I also haven't made any specific tests, but it looks to me like DT 2.71 is slower than Shadow. And so are most (if not all) of the "traditional" PM bots. -- ABC


ok, i will retire my 'since long used' testbed and try the bots mentioned on this page. Tests with the latest versions of mini.Wodan and mini.Freya showed a significant progres when a ran them with my current testbed, but yet they tumbled in the rankings in 1-vs-1... I guess is will have to retest a large number of versions :( --Loki

Currently I only test against the MovementChallenge2K6 and TargetingChallenge2K6 bots, together with some very feature-specific bots. And ofcourse I sometimes use the RoboRumble as my testbed ;) -- GrubbmGait

My 'old' testbed consisted of abc.tron3.Shadow 3.59.1, jam.RaikoMX 0.32, pez.mini.Pugilist 2.0.9, pez.mini.Tityus 0.9.6, ad.Neo 0.0, kawigi.sbf.FloodMini 1.4, myl.micro.NekoNinja 1.50, mz.NanoDeath 2.56 and wiki.nano.DevilFISH 1.0. These are bots that are my traditional problem bots and they run reasonably fast. When i can contribute some CPU time myself i may also try using the 'RoboRumble' testbed, but for the moment i am locked behind a firewall :) --Loki

I use the TC2K6 almost exclusively for working on my gun, but the MC2K6 is a smaller factor in my movement work. I do use the MC for testing the extremes - perfecting the WaveSurfing against simple targeters, and quick adapting / curve flattening against the better guns - but there are a lot of stages in between, and a lot of nuances that I can't quite tune without real battles. Some of the intermediate bots I test against are PulsarNano, RaikoNano, Vanessa, ChumbaMini?, Barracuda, Charo, and Ares. -- Voidious

For melee I've got two test beds, but I'm not testing for melee at present. The original test bed is a friend's bot and 8 of Loki's, and the secondary test bed includes 9 robots near me in the ratings, favoring celebrated / classic bots. For one on one I always test against GrubbmGrb and I'll often test against my problem bots and the bots of GrubbmGait and Loki. I don't really get all analytical and take notes on progress against a static roster of opponents. I can see that being more beneficial if I were fixing bugs in my movements, but I usually just make a new movement instead of making an old one work right. (Not recommended.) -- Martin

For melee I use the 9 robots listed on TestBed against me, they are quite fast. At least a lot faster that mine. -- Florent

I guess one of my problems is that the ratings of the bots in the ranking range 75 to 100 are closer together than the bots at say ranking 30. In my tests the latest version of mini.Wodan did 2% better (42.23% to 44.30% against my original testbed), so i released it. With the TestBed, mini.Wodan did 1.5% worse. So first, my testbed was not representative for the whole rumble (but which 10 bot testbed is?) and second, a small change in performance costs me almost 20 places in the ranking. So in the future i will only release a new version which performs significantly better and maybe i should use GrubbmGait's advice and use the Roborumble as a testbed --Loki


Robo Home | TestBed | Changes | Preferences | AllPages
Edit text of this page | View other revisions
Last edited March 9, 2006 11:24 EST by Loki (diff)
Search: