Editing Ender/Tests

Robo Home | Changes | Preferences | AllPages

-----
''' Gun tests:'''

How would my new ZoomTargeting gun match up against my existing PatternMatching gun?
Also, am I making progress in developing it further?

* 20 aug 2003 Zoom 2 ([[ZoomTargeting/stage2]])
**added MultipleChoice in lower zoomlevels
* 08 aug 2003 Zoom 1 ([[ZoomTargeting/stage1]])
**ZoomTargeting framework. Only using last recorded Angle.
* 08 aug 2003 PM 1.0
**Basic PatternMatcher. Very accurate,  but no MultipleChoice statistics which makes it weaker against Flatteners.

|'''Gun'''|'''Index'''|'''Fhq'''|'''Art'''|'''Spar'''|'''Ash'''|'''DT'''|'''Asp'''|'''Cig'''|'''Tron'''|'''Tit'''|'''Yng'''|'''HTTC'''|'''Fun'''
|Zoom 2   |'''81.5%'''|86.8%    |100%     |91.5%     |75.8%    |68.7%   |78.1%    |71.4%    |69.7%     |69.6%  |93.1%    |78.2%      |94.8%
|Zoom 1   |'''78.7%'''|85.9%    |99.7%    |86.5%     |63.4%    |66.4%   |79.2%    |61.1%    |73.3%     |65.9%  |93.6%    |80.3%      |89.1%
|PM 1.0   |'''76.5%'''|86.4%    |96.5%    |92.0%     |67.9%    |48.9%   |77.1%    |55.3%    |80.5%     |47.0%  |93.9%    |76.5%      |95.7%

The index is calculated following the rules of the TargetingChallenge:<pre>Index= 100 - ((OpponentScore/MyScore) * 100)</pre>

Legend:

(These are all non-shooting bots)
-----
''' Movement tests:'''

-----
'''Comments:'''

The results of my first test (8 aug 2003) show some very interesting results:
* WOEHOE! My new ZoomTargeting gun gets a better index against DT than DT itself! (DT scores 36.2%, Ender scores 33.6%)
* ZoomTargeting performs significantly better against DT than the PatternMatching gun.
* On average, my ZoomTargeting gun is already an improvement over my PatternMatching gun.
* Against some adversaries PatternMatching still scores better
* Fhqwhgads must have an awesome gun, because its movement is not as good as most other top bots, yet it's a top 10 bot?!
* TheArtOfWar's movement is not up to speed with the current state of the art.

Do you really mean to divide opponent score by your own? Shouldn't it be the other way around? -- [[PEZ]]
* yes: lower=better. This way I can compare better with the results in the MovementChallenge.

[[Fhqwhgads]] 1.1 also depends on its opponent to move away from it - it is likely that it ends up close to you and stays there for whole battles (my MovementChallenge entries wouldn't do that).  But it's gun still does kick butt, it's quite similar functionally to the dev FloodHT's gun. -- [[Kawigi]]
* Yes it does stay quite close. It also oscillates near the walls quite a lot. When do you think your new FloodHT will hit the repository? Maybe FloodHT will be a better reference bot for the Targeting Challenge?-- [[Vic]]
* I don't know if it will be a better one, it looks for predictability in a different way than DT does.  On more testing, it looks like FloodHT does about the same as your two guns against Fhqwhgads, but much better against everyone else except maybe DT.  Notably, ArtOfWar got only 0.1% against FloodHT, Aspid got 6.3%, Cigaret and Ash got around 31%, Sparrow got around 2%...  As far as a release date, I'm just swamped already!  My internship is getting close to ending and I'm trying to finish stuff there, meanwhile, I'm trying to make my minibots kick butt for next week!  I'd release the current dev version, but it still has some half-finished features that might screw it up. -- [[Kawigi]]

Congratulations are in place! This gun breaks truly fertile ground. You might take the lead in the best guns league with this. We don't know yet if Fh... is a top 10 bot. But Marshmallow sure has problems against it. It's actually the dev versions most difficult bot, by a margin. That's why I wanted it in the TargetingChallenge. StatistRobot shows a not-so-flat curve for it. Your zooming gun must be able to exploit that. We didn't need your test to know that TheArtOfWar's 1v1 movement is bad though. That has been known for a while. It's what makes it a good reference bot for the targeting challenge. Certain types of guns find it really easy to hit, others (like Gouldingi's) can't spot the weakness. -- [[PEZ]]
* Thanks [[PEZ]], I think this version is no Best Gun yet though. It still lacks the Statistical part. But this does show that its original design is capable of exploiting some until now unrevealed weaknesses in some top bots, most prominently SandboxDT.
** That weaknes could be the same that [[Kawigi]] talks about maybe? About it's walls movement defeating its purpose in some situations. -- [[PEZ]]
** I think that it's the not-near-a-wall movement that's weaker for DT - at least according to FloodMini, DT is more predictable when he's far from the walls then when he's approaching them, because he still may flip directions to avoid them or he might plow right through them.  I think when he 'tuned' his 1.81 movement into his 1.91 movement, though, he also 'squeezed' his profile onto the positive side a little more to make him just slightly less vulnerable to head-on targeting.  That's just a guess.  All I know is that FloodHT hits it enough to make shooting worthwhile.  If the targeting bug Paul described has been around for some time, it could explain the strength of the CoolMovement against it, too, but I'm not sure. -- [[Kawigi]]
** My observations are that DT is more predictable in the early stages, when it tries to get to a wall. Also, it seems more vulnerable in the corners than along the straight walls. This might have to do with the fact that the challenger does not move. DT may oscillate more in corners than it would do when the opponent is moving. -- [[Vic]]
** All bots are more vulnerable in corners. It's simple less room to move there. But of course some are more vulnerable than others. It would be very interesting to see FloodHT's index against DT under the same conditions by the way. And what index did DT itself have? You better make sure you implement the same restriction as the stationary DT does. Like only firing power 3 bullets and whatever. This is also important with the TC bots. Ramming and such things must be disabled. Maybe it goes without saying, but anyway. =) -- [[PEZ]]
** Yes, I also only fire power 3 bullets. The only difference is that I ran my test over only 100 rounds. Second time around my index was a bit worse, about the same as DT's index (36%) but still quite satisfactory :-p. I just watched the match with DT and another thing struck me: as you noted, in corners DT cannot travel its full angle to either side. To compensate I think DT does more full speed dashes on the straights. My MicroSegments can pick up on that difference (if there is a difference, I'm only guessing by what I observed :-). -- [[Vic]]

I expected my PatternMatcher to do better against [[Tron]] and that turned out to be true. [[Tron]] scores 26.7% against my ZoomTargeting gun, and 19.5% against my PatternMatcher gun. I think the reason is that [[Tron]] moves within the constraints of a grid, with a very consistent wave movement. That means its variations in movement are much more restricted and result in a better chance of finding a good match for the PatternMatcher. Although the zoom gun performed worse, I'm still very satisfied with both results since [[Tron]] is a true top bot.

When watching the battles against [[Tron]] , I noticed that the ZoomTargeting gun can adapt amazingly fast! Sometimes [[Tron]] would decide to oscillate (go back and forth using the same path) for a short while. [[Ender]]'s ZoomTargeting gun picks up on that very fast and hurts Tron real bad in that situation. I am wondering if I can retain that property when I add the MultipleChoice statistics to the gun. Maybe I will decide in the end that this version of the gun deserves a permanent place in Ender's guns array if it proves to exploit certain weaknesses in the enemy's movement that Statistical guns are unable to find. Also, this version of the ZoomTargeting gun is surpisingly small in terms of codesize. It might even fit in a MiniBot (maybe even in a MicroBot). Does anyone have a good MiniBot / MicroBot movement that I can use? ;-)

Would anyone be interested in evaluating this gun at the moment?

-- [[Vic]]

It's not easy to choose among fast and slow learning guns. Paul is the only one I know of that manages this. Which of course says that it is possible. =) I am close to finishing my new movement. It is pretty small, codesize 750, including utility functions and bot structure that you might need anyways. It's probably shrinkable to 300 or some such, but that would violate my rules of clean code. I'll send a bot with this movement to both you and [[ABC]] then and you can add your guns. I think the movement is at least Marshmallow 1.9 class. Maybe tonight it's finished. -- [[PEZ]]

Thanx [[PEZ]], your new movement is really good! At least, against my guns :-) On average, Tityus has the same score as Cigaret and DT. Good company I'd say :-)

About slow/fast learning: managing this is one of the design goals I have with ZoomTargeting. When it's finished, the enemy will need to have a flat profile not only in different situation segments, but also in different time segments. And not just two, but (currently) 8 time segments.

-- [[Vic]]

Oooh~~~,almost the same socre as Cigaret,38.9%--38.8%,44.7%--44.7....~;]  -- [[iiley]]

[[PEZ]] has sent me a new version of Tityus. On average this is now the best movement against my guns. It scores a bit worse against my ZoomTargeting gun, but it is now the most difficult target for my PatternMatcher. [[PEZ]], if you submit it to the movement challenge you might submit a winner! -- [[Vic]]

Yes, if it wasn't for DT thinking it's the most predictable movement since SpinBot. =) But I have a theory about why that could be and will test that theory asap. -- [[PEZ]]

Just updated all indices to reflect the new 100- ... formula. Now I can compare with [[ABC]]'s score in the TargetingChallenge and it shows me that I still have a lot of work to do :-) -- [[Vic]]

Summary:
Consciousness check: 6 - 5 =
This change is a minor edit. Send email notification that Ender/Tests has been changed.
(Visit Preferences to set your user name.)

View other revisions
Robo Home | Ender | Changes | Preferences | AllPages