[Home]TargetingChallengeRM

Robo Home | Changes | Preferences | AllPages

Showing revision 26
A challenge to see how well we can do against the RandomMovement bots from TargetingChallenge, TargetingChallenge2K6 and TargetingChallenge2K7.

/Results are over 35 rounds, equivalent to the "fast learning" version of the various TCs. Rules are the same as they are for the TCs.

Proposed categories:

Easy

  1. AspidMovement
  2. Sparrow 2.5 TC
  3. Fhqwhgads 1.1 TC
  4. YngwieTC
  5. FloodMini 1.4 - Kawigi's SandboxFlattener movement

Medium

  1. Tron 2.01TC
  2. HTTC (GouldingiHT with guns removed and ramming disabled). Download from: http://www.robocoderepository.com/BotDetail.jsp?id=1643
  3. RandomMovementBot - fluid, random movement
  4. DuelistMicro 2.0 - GoTo style RandomMovement (similar in style to [[Cigaret
  5. GrubbmGrb 1.2.4 - MultiMode with Stop 'n' Go, Oscillation

Hard

  1. SandboxDT 1.91
  2. Cigaret 1.31 - random orbital movement
  3. Fortune 1.0 - MultiMode of several best in class RandomMovement implementations
  4. WeeklongObsession 1.5 - strong RandomMovement
  5. RaikoMicro 1.44 - classic, very strong RandomMovement



From original TargetingChallenge:

  1. SandboxDT 1.91
  2. AspidMovement
  3. Sparrow 2.5 TC
  4. Tron 2.01TC
  5. Fhqwhgads 1.1 TC
  6. HTTC (GouldingiHT with guns removed and ramming disabled). Download from: http://www.robocoderepository.com/BotDetail.jsp?id=1643
  7. YngwieTC

From TargetingChallenge2K6

  1. Cigaret 1.31 - random orbital movement
  2. RandomMovementBot - fluid, random movement
  3. FloodMini 1.4 - Kawigi's SandboxFlattener movement

From TargetingChallenge2K7

  1. Fortune 1.0 - MultiMode of several best in class RandomMovement implementations
  2. DuelistMicro 2.0 - GoTo style RandomMovement (similar in style to Cigaret)
  3. GrubbmGrb 1.2.4 - MultiMode with Stop 'n' Go, Oscillation
  4. RaikoMicro 1.44 - classic, very strong RandomMovement
  5. WeeklongObsession 1.5 - strong RandomMovement

Removed (and reason)

  1. TheArtOfWar 1.2 TC (Too easy to hit)
  2. FunkyChicken (Hits walls often, so max score in a round may be significantly less than 100%)
  3. Butterfly 2.0 - MultiMode; non-firing version of Ugluk (Too easy to hit)
  4. DuelistMicro 1.22 - simple GoTo? movement, quite good against simple aimers (Removed to avoid ambiguity with 2.0 from TC2K7)
  5. GrubbmGrb 1.1.3 - MultiMode with Stop 'n' Go, Oscillation (Removed to avoid ambiguity with 1.2.4 from TC2K7)

Temporary Comments

Someone add more to this, class is over and I'm headed home. =) P.S. Ideally we want RoboResearch to calculate the overall score as the plain average of the scores against each bot. This means that if the bots are "grouped", the groups need to all be the same size. Or just not have groups. Incidentally I removed the two bots that have had 99+ scores attained against them. If I get 99.97 against TAOW, and Voidious gets 99.98, that doesn't really tell us whose gun is better. --David Alves

If we're removing one of the versions of DuelistMicro because of "ambiguity", shouldn't we also remove one of the copies of GrubbmGrb? -- Simonton

Ah yeah, missed that one. --David Alves

Well, maybe you guys have a good reason I'm not thinking of for this, too, but I think hitting walls is a fine reason to keep a bot in. As has been mentioned on the wiki before, that means you have to hit them well and fast to get the best score. -- Simonton

Well we could measure how much damage it does to itself per round on average. If it's like 1 or 2 damage then I guess that would be ok. --David Alves

That would depend on how quickly you kill it. If your gun is bad it'll have lots of time to hit the wall more. -- Simonton

Well yeah, but I meant for a good gun. In my tests FunkyChicken does around 5-10 damage to itself per round with Phoenix firing at it, with Phoenix scoring around 90. I consider that too high, so I'd vote against it. Otherwise why not ram into walls on purpose in the MovementChallenge2K6? It would lower their score against you!

Sample rounds:

 444 INFO: Damage inflicted: 3197.92  average per round: 91.37
 444 INFO: Estimated enemy wall damage per round: 5.70

 596 INFO: Damage inflicted: 3056.16  average per round: 87.32
 596 INFO: Estimated enemy wall damage per round: 9.51

 621 INFO: Damage inflicted: 3156.74  average per round: 90.19
 621 INFO: Estimated enemy wall damage per round: 7.70

 --David Alves

If you'd like to leave out FunkyChicken, I have no problem with that. But as far as principle is concerned, I still think using a bot that does damage to itself over time only makes it a better reference bot. In my opinion, the targeting challenges would be better if the reference bot shot .1 power bullets at the challenger, so that a measure of how fast you do your damage is included in the score. (Of course, the actual scoring would have to be adjusted, too - perhaps to % total score.) However, in the end, either way will probably rank bots in the same order. But as for hitting walls in the movement challenges ... sure, why not! It might even be a decent strategy in the rumble if you know you're not going to win that round. :P -- Simonton

I kind of have mixed thoughts on the issue. Most importantly, I think FunkyChicken hitting walls messes up the scaling of his scores relative to the others. Killing him in 10 shots might net you a score of 98 while killing him in 20 gets you a score of 95, while against other bots both would get 100. The truth is the first one is more impressive, but you are doubly punishing a bot for bad accuracy against FC, so really your score against FC is affecting your overall score more than other bots.

This reminds me of something I have thought about from time to time, though. I think our current TC system is not the best it could be, though I bet a lot of the flaws average out. I think a better system would be: the reference bot has something ridiculous, like 1000 energy, and both bots are just killed after 20 shots each round, with the "score" being straight hit percentage. In our current system, killing a bot in 8 shots or killing him in 35 are worth the same thing. The TAOW and Butterfly scores are a good (and extreme) example of this flaw. I think it's largely averaged out against most other bots, but who knows - maybe a gun that gets on streaks is advantaged or disadvantaged in our current TC system in a way that it shouldn't be. (It would basically take a Robocode mod to implement the above system, though.) But I think our current system is pretty solid, anyway.

-- Voidious

I'm just running the latest DrussGT on this, and I discovered something: waiting until my gun is properly aligned from last tick with this tick's aim boosts my score a LOT in this challenge, but (from previous experience) reduces my score in the rumble. I think this is due to the fact that there is more 'space' between the shots, and the TC doesn't penalize you for taking a long time to kill the enemy, so you only get the benefits of having to predict one tick less. In the rumble, pumping out the bullets, even with a slightly lower hitrate, kills the enemy quicker because you fire as often as possible. Also, if I have already depleted my energy, then they are less likely to get bullet damage against me. Additionally, when I am going against surfers, my gun is little better than random, so the more I fire the more often I hit them. But in the challenge, with the 'wait until aimed' enabled, it seems my gun is rivaling Dookious for the top TCRM score... although it's too early to be sure. And I checked for 3 power bullets ;-) -- Skilgannon

Well... I think "wait until aimed" is technically against the rules for the challenge. It does say "Rules are the same as they are for the TCs." here and on the TargetingChallenge page it says "The Challenger must try to fire power 3 bullets when it can" which I'd understand to mean that you're not allowed to spend any time waiting at zero gunheat without firing. Interesting discussion on the benefits/disadvantages of waiting though. -- Rednaxela

Waiting until aimed is not against the rules, I don't think - at least not in the spirit of the rule. I asked PEZ about this a while ago, and he said that rule was more addressing bots that wait until an advantageous situation to fire, like when the enemy bot is close, near a wall, or like that. -- Voidious

Well, I suppose so yes. I do view waiting-till-aimed results less interesting/useful at least though, because, as has been noted, waiting till aimed doesn't really help in the rumble, and most bot results here don't wait till aimed I believe. -- Rednaxela

I don't understand how they're less interesting / useful? Because they help the scores but don't help the rumble? I absolutely disagree with that being a proven fact. I see Skilgannon says it didn't help his gun. I'm almost certain I gained a few points when I added it, and folks like PEZ and ABC would not likely have it in their bots if they didn't find that it helped. (PEZ, in particular, is notorious for that.) While there are only a few results in the TCRM here, I think yours and Skilgannon's are the only ones that might not wait until aimed. And while DrussGT doesn't wait until aimed, I think just about every other bot in the top 20 does. -- Voidious

I've tried several times to implement waiting until I get a botwidth of tolerance from the last shot, and it never seemed to help, instead suppressing my score by around 5 points. Then again, it mostly hurt against surfers, and my targeting against surfers is notoriously bad. Perhaps making it dependent on my hitrate would help, or improving my anti-surfer targeting. -- Skilgannon

I guess I was under the impression that wait-until-aimed was far less common and than it apparently is in some case. I was unaware of it's use in many bots really. I suppose I might give it a try in RougeDC some time and see how that works. Personally, I think that "only angles that are reachable by the gun" during the last tick before firing might also be something worth testing though I'm not sure how that would perform. -- Rednaxela


Robo Home | Changes | Preferences | AllPages
Edit revision 26 of this page | View other revisions | View current revision
Edited June 22, 2008 21:11 EST by Rednaxela (diff)
Search: