Editing revision 72 of TargetingChallenge2K7/ResultsChat

Robo Home | Changes | Preferences | AllPages

Editing old revision 72. Saving this page will replace the latest revision with this text.

'''[[/ReferenceBots]] - /HowTo - /FastLearning - [[/Results]] - /ResultsFastLearning - /ResultsChat - /PreChat'''

----

Somebody should run LifelongObsession & post the results; I don't have a computer anymore.  The source is in the jar.  Just open up LifelongObsession.java, comment out the getMovement() method, and make the getNextFirepower() method return 3.  Anybody?  -- [[Simonton]]

GrubbmGait: congradulations on first place!! ;)  -- [[Simonton]]

GrubbmGait, did you just run the battles against WeeksOnEnd manually? If not, what version of Robocode are you using with RoboLeague? -- [[Voidious]]
* I ran them manually with 1.1.2, except for WeeksOnEnd, there I had to use 1.2.5A. At work I don't have RoboLeague, but at home I am still using it with 1.0.7Kawigi. If RoboLeague was not really that big an issue, but now with TC2K7 it becomes one, we could ask [[Fnl]] if he is willing to make a special 1.2.6B for it.  -- [[GrubbmGait]]
** Wait, by 1.0.7Kawigi, do you mean that you're still using that jar I built with a few fixes and enhancements when Robocode first went open-source?  I don't even use that anymore... -- [[Kawigi]]
*** I still use it because it is twice as fast with RoboLeague as 1.1.2, and RoboLeague does not need the enhancements. For normal development I currently use 1.2.5A.  -- [[GrubbmGait]]
** My RoboLeague uses 1.07 ''without'' your fixes, plus a super high CPU constant. =) -- [[Voidious]]

Oh, and I'll gladly run LifelongObsession sometime. Don't have a computer?! =) That's no way to live! Just kidding... -- [[Voidious]]

Oh ... uhh ... I just realized that LifelongObsession doesn't have any memory checks in place ... so ... 500 rounds may not work.  You can try like -Xmx2048M or something, but otherwise I guess only /FastLearning will possible.  -- [[Simonton]]
* -Xmx2048M ? Does dat work when you only have 384Mb of real memory present?  -- [[GrubbmGait]]
* :) You never know until you try ... (by the way, yes, it should, but if Java actually uses a lot your machine will start thrashing like mad.  Believe me.  I know.) -- [[Simonton]]

@Voidious: Thanks for running those bots, now I don't feel like a complete loser anymore ;-)  -- [[GrubbmGait]]

Caffiene b1.1.2 is a rather advanced version, other then being unsegmented, but nothing that would make it an oddity in actual aiming. I think its a good bottom reference. Perhaps someone can do Che? --[[Chase-san]]

I thought it would be so much higher Simonton! Can you send me your challenge bot, i'll run the other targeting challenges (it should pwn the original). --[[Chase-san]]

Well, credit to [[Simonton]] for having sweet and innovative new PatternMatching guns, but I don't think I'm alone in believing that PM is inherently weaker than other stats-based approaches. Of course, CodeSize is a huge factor among non-MegaBots. It's quite surprising how very close it is to [[Che]], actually, overall and divided among surfers / non-surfers. I'll be curious to see the other TC's, too. -- [[Voidious]]

It's one of the first PM's to make the 2000 club, (though i'm not sure but maybe Shaakious, made it first? But looking at the version history, doesn't seem likely). Not only make it, but to stay there instead of crossing the line alot. --[[Chase-san]]

This was a new single tick gun.  An experiment in using advancing/lateral velocity instead of heading/velocity.  Unsegemented.  Every little tweak really throws all the scores around.  It usually seems to average out, though.  I ran some scores for the 2K6 challenge running 2 of them in a VG array & managed to match SeaSerpent's fast learning score (this was a while ago, maybe I can find & post them ... I think there's some others I can post somewhere, too).  I'm still not convinced there isn't more potential for pattern matching - I just haven't figured it out yet.  But yes, they seem to do very well in CodeSize leagues.  -- [[Simonton]]

* Quoth the venerable Captain Picard: "It's only impossible until it isn't!" :-) -- [[Voidious]]
* I think many of the 'dropped' movement and targeting types have alot more that could be done with them, its just alot harder then statistical targeting or movement to prefect. --[[Chase-san]]

Jeez, how is it we have no fast learning results yet? Running [[Dookious]] now, then I'll move onto some of the inactive authors' bots. -- [[Voidious]]

Would somebody mind running Decado (FastLearning and/or otherwise)? Just comment out the setAhead and change bulletPower to always be 3...I'm interested in what it would get. Sorry I haven't managed to get Roboleague running yet, I've been busy with adding segmentation to Druss, and life in general. -- [[Skilgannon]]

No problem. Seems reasonably fast in my test run here, so I should be able to get both (regular and fast learning) posted today. -- [[Voidious]]

Thanks -- [[Skilgannon]]

Hmmn...and I got to 3 in the micro with that...heh. My movement must be good. And the fact that I  set my 'best' distance to 400 :-). Of course, I only keep the last 20 000 'frames', so I expect the fast learning result to be better. 
 -- [[Skilgannon]] -- who will be working on a better gun

I'll try to get the TCCalc script updated with an option for TC2K7 formatting soon. I like the split format we have now, but it's definitely a pain to re-arrange / calculate that stuff yourself. Chase-san, didn't mean anything personal saying it's a pain, just fyi. =) -- [[Voidious]]

Now I see the split resultst and I thank God om my bare knees that there are still a lot more non-surfers around. Now I know where my effort should go whenever I get that frustrating GoToSurfing working like I want it to. I will try to run the FL challenge asap (read mondayevening). -- [[GrubbmGait]]

Interesting...Decado has the best score against Shadow in the fast learning :-) -- [[Skilgannon]]

True, and thats very cool. Though thats only one bot, and... Shadows gun probably rips Decado's movement to shreads. --[[Chase-san]]

Yeah, Shadow does =). It's just surprising considering it gets the lowest of the scores overall. It would be nice to have divisions - ie. Micro, Mini, Mega, at least that way I won't feel like the last cow's tail ;-)-- [[Skilgannon]]

* Don't sweat it - everyone knows the RoboRumble is what really matters. =) -- [[Voidious]]

Its my ANN map system crudely glued ontop of a gutted Raiko (no movement modified firing power), currently the stats do not decay, which is what I attribute to the horrible surfer scores. It does pretty well against  most the non surfers, except the multi-modes at this point, I have some ideas on how to improve it, including adding greater dimensional diversity to my map (its lattice is only currently 3d, I made it infinitely expandable). It only records data on real bullets. --[[Chase-san]]
* That's an excellent start against the non-surfers. -- [[Simonton]]

Prototype seems to have a problem against Fortune, GrubbmGrb and WeeklongObsession. It completely obliterates most the surfers, so I need to find a way to fix it to work against MultiMode robots, but I got nothing on that yet, only a few theories. So if you have a suggestion, it would be appreciatied. I can get around to hitting the random mover later. --[[Chase-san]]

* A couple thoughts. First, some odd segments like time since velocity change or distance last X ticks might help against MultiMode - the different movement types might leave some weird signatures in those segments. I'm not sure what attributes you're currently using. However, my second thought is that MultiMode might not be the problem: WeeklongObsession and GrubbmGrb should (maybe someone can confirm this) be dropping the StopAndGo early in the match, not switching modes all throughout; [[Fortune]] is true MultiMode, but its 3 movements are all pretty much best-in-breed versions of different types of RandomMovement, so it really could just be that much harder to hit than the others. You could try the other Fortune bots that we considered for TC2K7 to see for sure. -- [[Voidious]]
** I can confirm it for WeeklongObsession. -- [[Simonton]]
** Same for GrubbmGrb (after one round)  -- [[GrubbmGait]]
** Well one theory  I have is to expand the update range a little at the start of each round. That might lead to it catching on a bit easier for those MM and the Stop&Go bots, as it deticates a larger section to the new movement that is taking place now. I am also toying with the idea of embedding a traditional GF gun in it. :3 --[[Chase-san]]

* I have timeSinceVelocityChange and distance last 2,4,8,16,32, and 64 ticks. I just added the distance last X, in 005, might be the reason for the 5 point jump in GrubbmGrb. The only true difference between the guns in its VG array is the smoothing method it uses, so really its all the same gun. (Adaptive Gun, Standard Gun and Unetwork gun[It doesn't use the gaussian]. All the guns use my experimental bellcurve binsmoothing. (and from the look of things, it does pretty darn good so far). --[[Chase-san]]

I enjoyed those 87 minutes of glory while DCResearch had the best /FastLearning score against Shadow ... -- [[Simonton]]
* hehe, it seems like Chase-san has an even better version. But luckily he only runs 500 round test. Lets see if my 1&#8240; advance lasts longer than 87 minutes :D EDIT: Or you could try out your great new dual core machine and run a few more seasons! --[[Krabb]]
* N/m.  I'll just keep improving my DC!  -- [[Simonton]]
* Shhh! You guys might wake up the [[MostCompetitiveRobocoderEver]] and he'll run [[Phoenix]]. As I remember, he hits [[Shadow]] pretty well. =) -- [[Voidious]]

Awesome! Nice work, David! =) Very impressive. -- [[Voidious]]

Thanks. I actually didn't make any changes to the gun for 0.9, but apparently the 0.860 gun code works well against the reference bots? =P --[[David Alves]]

I think Dooki's gun is at as strong or stronger than Phoenix's, but maybe Phoenix is better at hitting these 10 particular bots. --[[David Alves]]

[[MostCompetitiveRobocoderEver]], you just want it to be better so you have an excuse to improve your gun some more. ;) :D :P --[[Chase-san]]

"Phoenix DC test" - hmm ... I'm beginning to understand your title.  Was the thought process something like, "I see people competing; I'd better go win!"?  :P -- [[Simonton]]

It's a gun I wrote a year ago but gave up on. Since DC guns are all the rage these days I figured I'd dust it off and see what it can do. It's actually not that bad against the non-surfers but it's pretty poor against surfers. --[[David Alves]]

I would like to see what scores Engineer gets in this. Just for reference. ;) --[[Chase-san]]

[[ABC]] - I hope you don't mind, I fixed your math.  All I changed was the grand total, based on the two subtotals.  But then, maybe they were wrong?  Feel free to check out RoboResearch; it's really handy for running challenges like this (and spitting out wiki output to copy-paste). -- [[Simonton]]
* Ah - I fixed my fix.  The grand total was right, the sub-total for surfers was 2 points low. -- [[Simonton]]

I used TCCalc and did the subtotals by hand, I'll check out RoboResearch when I get the time. BTW, your bots (and RMX) are the ones keeping our scores down when compared to the top 2... -- [[ABC]]

* I dunno if it's a VCS vs. DC thing, because 0038 gets pretty high vs WOE and RMX... -- [[Skilgannon]]

Just as a point of discussion, I am wondering how people feel that the targeting and movement challenges have affected the development of their bots, and what impact it has had on their Roborumble performance.  Alternatively, are the challenges just another way to showcase your hard work? -- Martin

One thing I think is very true is that improvement in the targeting challenges is much more likely to correlate to RoboRumble points than improvement in the movement challenges. I think that MovementChallenge2K7 might have been a big step in the right direction, though. I have had many cases where improving my TC scores resulted in RoboRumble rating points, and I generally expect it to, but I've also had a few where it resulted in nothing.

I definitely don't think the challenges are sufficient as your ''only'' benchmark when improving your bots, but I do think they are good benchmarks that can really help if used "correctly" - like knowing that improving your TC score against surfers will not help your overall rating much, or that improving your movement against Shadow might only help against a handful of bots. I must say that I think the "showcase" aspect has become a really important aspect of the challenges, at least for me. Really, there is nothing to say that RoboRumble performance is the "true" test of a bot (or a gun or a movement) as opposed to carefully constructed benchmarks, IMO, or even a different style of tournament / ranking (like PremierLeague). Interesting topic.

-- [[Voidious]]

It's a very imprecise science, bot improvement. Especially when you're fighting against small details in your gun/movement. Big changes are always easier to evaluate. Personally I've always focused at improving my score against top bots, that's probably why I've always tended to be better ranked in the PremierLeague than in the general rumble. I like the challenges a lot though, and I believe they are very useful for testing changes and new ideas. A big improvement in one of the challenges is always guaranteed to have a positive impact in your rumble rating, smaller ones are not. -- [[ABC]]

Perhaps a challenge that is 500 seasons of 1 round matches, no data saving, to get first round preformance, which can be very important. --[[Chase-san]]

Nice scores for Gaff, Darkcanuck! 74 against Shadow is crazy - that's 5+ better than second best?? Wow. -- [[Voidious]]

* Thanks!  It took over 80 iterations of tweaks and bug-fixes since version 1.10 to get that number.  No idea why my method likes Shadow so much, but it's definitely tuned to hit surfers overall.  I just need to figure out how to adapt it for better results against random movers (scores only 85.52 in the RM challenge).  I suppose I should also work on movement one of these days, but targeting seems to be easier to test.  :)  -- [[Darkcanuck]]

Indeed very nice stuff in Gaff! Yeah, random movers ban be of a pain to hit as well as the top guns can, I also have more trouble there than with the surfers. I'd also agree that targeting is certainly easier to test, of course targeting an movement are rather interrelated (and increasingly so at close range I suspect). LunarTwins interestingly performs better with it's simple rolling-average-turn-rate-circular-targeting gun than if I throw a well proven PM gun like Waylander's gun on it. In any case yes, movement might be good for you to work on though, considering it appears to be hurting Gaff's score far more than the targeting. Personally I tend to bounce between targeting, movement, and teams, changing after one category starts to frustrate me too much, but of course, everyone has their own habits. One thing I noticed is that you noted in Gaff's page that you're no longer using hidden nodes. I find this interesting for a couple reasons: Firstly this indicates my theory that Gaff was taking advantage of more complex relationships between inputs isn't actually the case (to a significant degree anyways). Secondly, I'm thinking data dumps of the weights would be far more comprehensible now that hidden nodes are gone (as the weights now far more directly indicate the strength that an input category is associated with an output category). :) -- [[Rednaxela]]

* I've refined the inputs to the network, simplifying where possible.  I also fixed a nasty lateral velocity bug.  Once all that was done, dropping the hidden layer really helped to speed up learning against surfers -- approximates a "low rolling depth" by encouraging catastrophic interference.  At the expense of learning more complex relationships, but we only have 35 rounds to work with.  Or at least that's my theory...  Any suggestions for good segments to use vs. RM?  -- [[Darkcanuck]]

* Here's what DrussGT has:

Feel free to mix, match and adapt =) But if you figure out any improvements, let us know! Of course, what works for NN won't necessarily work for DC. BTW, DrussGT is ONLY tuned to hit RM, but I think enough of the attributes are wacky enough that surfers have trouble dodging them. -- [[Skilgannon]]

* Here's what [[Dookious]] uses for attributes in its Main Gun (non-[[AntiSurfer]] gun):

Again, this is not NN, but in my case, VisitCountStats; nevertheless, I'd expect these attributes to work well in any statistical system. The AntiSurfer gun doesn't really use any attributes not listed here, just less attributes, less segments, and super fast RollingAverage. The wall distance translates basically to: "at what (non-precise) GuessFactor the opponent will be when he hits the wall, if he simply orbits the wave." Good luck!

-- [[Voidious]]

The nice thing about NN is that the net seems to figure out the best segmentation (I was segmenting the data in earlier versions, but performance is actually better using the raw data) as long as the data is scaled from -1 to +1.  I'll give the distance-since input a try, that might help against RM.

Skilgannon, what do you mean by the "current" GF -- wouldn't that always be 0?  -- [[Darkcanuck]]

Summary:
Consciousness check: 2 * 3 =
This change is a minor edit. Send email notification that TargetingChallenge2K7/ResultsChat has been changed.
(Visit Preferences to set your user name.)

View other revisions
Robo Home | TargetingChallenge2K7 | Changes | Preferences | AllPages