Editing revision 24 of TargetingChallengeRM

Robo Home | Changes | Preferences | AllPages

Editing old revision 24. Saving this page will replace the latest revision with this text.

A challenge to see how well we can do against the RandomMovement bots from TargetingChallenge, TargetingChallenge2K6 and TargetingChallenge2K7.

/Results are over 35 rounds, equivalent to the "fast learning" version of the various TCs. Rules are the same as they are for the TCs.

'''Proposed categories:'''

'''Easy'''
#[[Aspid]]Movement
#[[Sparrow]] 2.5 TC
#[[Fhqwhgads]] 1.1 TC
#YngwieTC
# [[FloodMini]] 1.4 - Kawigi's SandboxFlattener movement

'''Medium'''
#[[Tron]] 2.01TC
#HTTC (GouldingiHT with guns removed and ramming disabled). Download from: http://www.robocoderepository.com/BotDetail.jsp?id=1643
# [[RandomMovementBot]] - fluid, random movement
# [[DuelistMicro]] 2.0 - <nowiki>GoTo</nowiki> style RandomMovement (similar in style to [[Cigaret
# [[GrubbmGrb]] 1.2.4 - MultiMode with Stop 'n' Go, Oscillation

'''Hard'''
#SandboxDT 1.91
# [[Cigaret]] 1.31 - random orbital movement
# Fortune 1.0 - MultiMode of several best in class RandomMovement implementations
# WeeklongObsession 1.5 - strong RandomMovement
# [[RaikoMicro]] 1.44 - classic, very strong RandomMovement

----
----

'''From original TargetingChallenge:'''
#SandboxDT 1.91
#[[Aspid]]Movement
#[[Sparrow]] 2.5 TC
#[[Tron]] 2.01TC
#[[Fhqwhgads]] 1.1 TC
#HTTC (GouldingiHT with guns removed and ramming disabled). Download from: http://www.robocoderepository.com/BotDetail.jsp?id=1643
#YngwieTC

'''From TargetingChallenge2K6 '''
# [[Cigaret]] 1.31 - random orbital movement
# [[RandomMovementBot]] - fluid, random movement
# [[FloodMini]] 1.4 - Kawigi's SandboxFlattener movement

'''From TargetingChallenge2K7'''
# Fortune 1.0 - MultiMode of several best in class RandomMovement implementations
# [[DuelistMicro]] 2.0 - <nowiki>GoTo</nowiki> style RandomMovement (similar in style to [[Cigaret]])
# [[GrubbmGrb]] 1.2.4 - MultiMode with Stop 'n' Go, Oscillation
# [[RaikoMicro]] 1.44 - classic, very strong RandomMovement
# WeeklongObsession 1.5 - strong RandomMovement

'''Removed (and reason)'''
#TheArtOfWar 1.2 TC  (Too easy to hit)
#FunkyChicken (Hits walls often, so max score in a round may be significantly less than 100%)
#[[Butterfly]] 2.0 - MultiMode; non-firing version of [[Ugluk]] (Too easy to hit)
# [[DuelistMicro]] 1.22 - simple GoTo movement, quite good against simple aimers (Removed to avoid ambiguity with 2.0 from TC2K7)
# [[GrubbmGrb]] 1.1.3 - MultiMode with Stop 'n' Go, Oscillation (Removed to avoid ambiguity with 1.2.4 from TC2K7)
----
=== Temporary Comments ===

Someone add more to this, class is over and I'm headed home. =)
P.S. Ideally we want RoboResearch to calculate the overall score as the plain average of the scores against each bot. This means that if the bots are "grouped", the groups need to all be the same size. Or just not have groups. Incidentally I removed the two bots that have had 99+ scores attained against them. If I get 99.97 against TAOW, and Voidious gets 99.98, that doesn't really tell us whose gun is better. --[[David Alves]]

* Great!  This does deserve a page of its own, I think.  I would lobby for keeping TAOW and Butterfly in the mix, though.  Sure, it may not discern between you & Voidious, but it may help others realize the potential that can be achieved against these bots with the right gun.  -- [[Simonton]]
* My vote is for leaving them out. They are good tools for people to test their basic learning guns are functioning... but it's not like you can't try the original TC or TC2K6 if you want to do that. I can't imagine how many battles I've run against both of them that were all 99.9+ =) I like the idea of this challenge, too. The rejected bots from TC2K7 could all be good additions to consider - Archer, Grid, and WeekendObsession. We could also remove Fortune and throw in all 3 of the Fortune bots, as they are all awesome. -- [[Voidious]]
* Only 4 of the 35 bots on the TargetingChallenge/ResultsFastLearning page got less than 99% against TAOW, so I'd argue that they don't really differentiate among anyone very well. Against Butterfly, it's not quite as bad... about 40% of people are above 99%. But still, I don't think that leaves any real room for differentiation. --[[David Alves]]
* Ok, I buy those arguments.  -- [[Simonton]]

If we're removing one of the versions of DuelistMicro because of "ambiguity", shouldn't we also remove one of the copies of GrubbmGrb? -- [[Simonton]]
*Yep, the movement is the same, the only difference is a tuned dive-in protection.  -- [[GrubbmGait]]
Ah yeah, missed that one. --[[David Alves]]

Well, maybe you guys have a good reason I'm not thinking of for this, too, but I think hitting walls is a fine reason to keep a bot ''in''.  As has been mentioned on the wiki before, that means you have to hit them well '''and''' fast to get the best score.  -- [[Simonton]]

Well we could measure how much damage it does to itself per round on average. If it's like 1 or 2 damage then I guess that would be ok. --[[David Alves]]

That would depend on how quickly you kill it.  If your gun is bad it'll have lots of time to hit the wall more. -- [[Simonton]]

Well yeah, but I meant for a good gun. In my tests FunkyChicken does around 5-10 damage to itself per round with Phoenix firing at it, with Phoenix scoring around 90. I consider that too high, so I'd vote against it. Otherwise why not ram into walls on purpose in the MovementChallenge2K6? It would lower their score against you!

Sample rounds:
 444 INFO: Damage inflicted: 3197.92  average per round: 91.37
 444 INFO: Estimated enemy wall damage per round: 5.70

596 INFO: Damage inflicted: 3056.16  average per round: 87.32
 596 INFO: Estimated enemy wall damage per round: 9.51

621 INFO: Damage inflicted: 3156.74  average per round: 90.19
 621 INFO: Estimated enemy wall damage per round: 7.70

--[[David Alves]]

If you'd like to leave out FunkyChicken, I have no problem with that.  But as far as principle is concerned, I still think using a bot that does damage to itself over time only makes it a better reference bot.  In my opinion, the targeting challenges would be better if the reference bot shot .1 power bullets at the challenger, so that a measure of how ''fast'' you do your damage is included in the score.  (Of course, the actual scoring would have to be adjusted, too - perhaps to % total score.)  However, in the end, either way will probably rank bots in the same order.  But as for hitting walls in the movement challenges ... sure, why not!  It might even be a decent strategy in the rumble if you know you're not going to win that round. :P -- [[Simonton]]

I kind of have mixed thoughts on the issue. Most importantly, I think FunkyChicken hitting walls messes up the scaling of his scores relative to the others. Killing him in 10 shots might net you a score of 98 while killing him in 20 gets you a score of 95, while against other bots both would get 100. The truth is the first one ''is'' more impressive, but you are doubly punishing a bot for bad accuracy against FC, so really your score against FC is affecting your overall score more than other bots.

This reminds me of something I have thought about from time to time, though. I think our current TC system is not the best it could be, though I bet a lot of the flaws average out. I think a better system would be: the reference bot has something ridiculous, like 1000 energy, and both bots are just killed after 20 shots each round, with the "score" being straight hit percentage. In our current system, killing a bot in 8 shots or killing him in 35 are worth the same thing. The TAOW and Butterfly scores are a good (and extreme) example of this flaw. I think it's largely averaged out against most other bots, but who knows - maybe a gun that gets on streaks is advantaged or disadvantaged in our current TC system in a way that it shouldn't be. (It would basically take a Robocode mod to implement the above system, though.) But I think our current system is pretty solid, anyway.

-- [[Voidious]]

I'm just running the latest DrussGT on this, and I discovered something: waiting until my gun is properly aligned from last tick with this tick's aim boosts my score a LOT in this challenge, but (from previous experience) reduces my score in the rumble. I think this is due to the fact that there is more 'space' between the shots, and the TC doesn't penalize you for taking a long time to kill the enemy, so you only get the benefits of having to predict one tick less. In the rumble, pumping out the bullets, even with a slightly lower hitrate, kills the enemy quicker because you fire as often as possible. Also,  if I have already depleted my energy, then they are less likely to get bullet damage against me. Additionally, when I am going against surfers, my gun is little better than random, so the more I fire the more often I hit them. But in the challenge, with the 'wait until aimed' enabled, it seems my gun is rivaling [[Dookious]] for the top TCRM score... although it's too early to be sure. And I checked for 3 power bullets ;-) -- [[Skilgannon]]

Well... I think "wait until aimed" is technically against the rules for the challenge. It does say "Rules are the same as they are for the TCs." here and on the TargetingChallenge page it says "The Challenger must try to fire power 3 bullets when it can" which I'd understand to mean that you're not allowed to spend any time waiting at zero gunheat without firing. Interesting discussion on the benefits/disadvantages of waiting though. -- [[Rednaxela]]

* EDIT CONFLICT!! (x2) As far as I know, both [[Phoenix]] and [[Dookious wait]] until their gun is pointed to where it should be from the last tick before firing (a quick check confirms the latter, line 142 of DookiLightning.java). By "when it can" I would interpret this as "when it has enough energy", not "when gunheat is zero", because the few lines of clarification below both seem to be about bullet power rather than time of fire. Regardless, my score would definitely be higher than before due to a bug I found in the previous TC mode, the last bullet was always aimed for a 3 power bullet even if it could only be a 1 power due to lack of energy. -- [[Skilgannon]]

Waiting until aimed is not against the rules, I don't think - at least not in the spirit of the rule. I asked [[PEZ]] about this a while ago, and he said that rule was more addressing bots that wait until an advantageous situation to fire, like when the enemy bot is close, near a wall, or like that. -- [[Voidious]]

Well, I suppose so yes. I do view waiting-till-aimed results less interesting/useful at least though, because, as has been noted, waiting till aimed doesn't really help in the rumble, and most bot results here don't wait till aimed I believe. -- [[Rednaxela]]

I don't understand how they're less interesting / useful? Because they help the scores but don't help the rumble? I absolutely disagree with that being a proven fact. I see Skilgannon says it didn't help his gun. I'm almost certain I gained a few points when I added it, and folks like PEZ and ABC would not likely have it in their bots if they didn't find that it helped. (PEZ, in particular, is notorious for that.) While there are only a few results in the TCRM here, I think yours and Skilgannon's are the only ones that might not wait until aimed. And while DrussGT doesn't wait until aimed, I think just about every other bot in the top 20 does. -- [[Voidious]]

Summary:
Consciousness check: 3 * 2 =
This change is a minor edit. Send email notification that TargetingChallengeRM has been changed.
(Visit Preferences to set your user name.)

View other revisions
Robo Home | Changes | Preferences | AllPages