[Home]PatternMatcherChallenge

Robo Home | Changes | Preferences | AllPages

Difference (from prior major revision) (minor diff, author diff)

Changed: 1c1
/PMCIndex
/PMCIndex /Results

Changed: 160c160
http://atlas.et.tudelft.nl/~klli87/images/pmchallenge.gif ( -Dummy)
http://atlas.et.tudelft.nl/klli87/images/pmchallenge.gif ( -Dummy)

Changed: 241c241
http://atlas.et.tudelft.nl/~klli87/images/pmchallenge2.gif

http://atlas.et.tudelft.nl/klli87/images/pmchallenge2.gif


Added: 316a317,326

Quick update:

1st: challenge.PatternBot 1.0 25566 3500 700 18190 2710 465 0 79 29 0
2nd: jekl.mini.BlackPearl 21240 1450 290 18055 1444 0 0 33 70 0
Index: 99.26

-- jim

I guess we're all just posting our scores like this? No - I'm going to start a results page! -- Simonton

/PMCIndex /Results

There are a lot of good PatternMatching guns out there. I've got a nice challenge for those guns:-)

first of all, you'll need PatternBot: http://robocoderepository.com/BotDetail.jsp?id=1547

The challenge is to beat PatternBot in a 100 rounds match on a 1000x1000 battlefield with a bot that has the following restrictions:

- you may only fire 0.5 power bullets (like patternbot)
- your bot may NOT move (unlike patternbot :-)
- of course your bot is expected to have a super duper PatternMatching gun, that's the whole fun of it :-)

Your bot is given a slight advantage: PatternBot only starts firing when it gets into its movement pattern. If you hit it a few times before that, you may have a chance to win. If you don't, then only ONE bullet missed will mean you lose :-)

PatternBot is expected to win from all but the best Pattern Matching Guns. Its movement is completely predictable, BUT it does have some nasty (yet predictable) surprises. On a 1000x1000 battlefield it will NOT hit any walls, so that won't break the predictability..

PatternBot will hit your bot 100% of the time! So obviously, your PatternMatching gun should also get a 100% score (or a little less if you are able to hit PatternBot early).

Who will be able to complete the challenge???? Please submit your score below.

--Vic

btw, it CAN be done, i'm sure of it... I ran a test where PatternBot is also stationary to simulate 100% hit ratio. In that case i beat it on survival with a good margin. I'm guessing 97% to 98% rates would do the job (if you get that initial damage)


For those of you that are interested in a second PM Challenge i have uploaded a bot called [[nano.PatternBot]]. Conditions are similar.

It's movement is also completely deterministic, but all bots who entered the first PM Challenge fail hopelessly because the movement has a pattern that is hard to match because of long straight displacements followed by sharp turns. Everytime it looses, the pattern changes (from a triangular to a hexagonal pattern). Everytime it gets hit, the next direction change also changes.

I am curious how you deal with this one :) --Loki


Question department


Discussion on /PMCIndex

Can a bots gun performance (PMC-wise) somehow be measured in the bullet damage it inflicts on PatternBot in, say, a 100 round match? It seems so to me. If this holds I propose that this be used for a /PMCIndex. Also it would be that this measure discriminates better guns since you by inflicting more damage prolong the matches so that a higher accuracy will have more time to increment your /PMCIndex.

I might get more bullet damage if GlowBlowPMC would stop shooting if he has enough energy left at the end. because it often occurs that it's Energy is above 10 when it had a 100% hitrate during a round. If I'd let PatternBot to come back I could gather more points. other improvements are file saving, to get better results the first rounds, also important might be that I adjust some things when the pattern cuts off (disabling, ramming, changing the round (already done)).

1st: rz.GlowBlowPMC		25459	3400	680	18560	2818	0	0	83	32	0
2nd: challenge.PatternBot 1.0	22558	1600	320	18361	1783	493	0	40	68	0

how about this :), you can see what I mean with the bullet damage. -- rozu

Tricking the Index like this would go beyond the goal for the challenge which is justing beating PatternBot. So maybe (your_damage / patternbot_damage) * 100 might yet be the better index after all. I just calculated it for Albert's last result, your two results and my two last results:

MicroAspid  index 18514   relative damage  99.94 %
GlowBlowPMC index 18752   relative damage 100.81 %
GlowBlowPMC index 18361   relative damage 101.08 %
EnderPMC    index 18734   relative damage 100.43 %
EnderPMC    index 18916   relative damage 100.77 %
This shows that your last GlowBlowPMC has become much better and as mentioned before the index is dropping. It also shows that both your winning bots have a higher relative damage then my two not-winning bots. The current index doesn't reflect that. PEZ, would you agree we change the index to relative damage? --Vic

I agree that the /PMCIndex should be changed to relative damage. I would never had dreamt it would enter pattern matcher so good that they can even consider tricking the simple bullet damage index! -- PEZ

In my last version of EnderPMC there was a bug that resulted in HeadOnTargeting in 1 or 2 % of the matches, meaning EnderPMC lost those rounds with a 30% or so hit rate. This lowered the Index considerably, while EnderPMC did win the match! Also i noticed that there can be big differences in results over 100 round matches. If you're unlucky for example you get a few of those hitwall situations like Dummy describes above. Maybe if we do something like this again we should run 1000 or even 10000 round matches to even out the (bad)luck factor. Any thoughts about that? --Vic


Random chatting

Ender's PatternMatching gun is currently unable to win. I previously boasted a 99.45% accuracy, but i accidently read the percentage for the closest segment only... Actually Ender gets a 93.59% hit rate in its last 100 round match (according to PatternBot dev version) which sounds much more reasonable -- Vic

Cool - I' quite busy now, but I'w try ASAP (may be next week?) -- Albert

Wow, 99.45% is better than WallsKiller gets against Walls I think (closer to 92 to 93%. Of course, that's with power 3 bullets). -- Kawigi

I made a LeachPMC which got humiliated by PatternBot. Maybe a hit rate of 25% or some such... -- PEZ

I'm having lots of trouble hitting PatternBot at the start each round, while PatternBot is getting in position to start its patterns :-(. Perhaps I should tell my gun only to fire when a close-enough match has been found. --Dummy

For symbolic pattern matchers, a special symbol should be used to indicate an empty frame from before the round started. This would allow pattern matching during these frames. -- nano

That's what LeachPMC does. It uses an ArrayList of Frame objects that look like so:

class Frame {
    static double heading;
    double headingDelta;
    double velocity;
    boolean isInitialPhase;

    Frame() {
    }

    Frame(double headingDelta, double velocity, boolean isInitialPhase) {
        this.headingDelta = headingDelta;
        this.velocity = velocity;
        this.isInitialPhase = isInitialPhase;
    }

    static void setHeading(double newHeading) {
        heading = newHeading;
    }

    void advanceHeading() {
        heading += headingDelta;
    }

    char getKey() {
        int key = 3;
        key = key + 100 * (isInitialPhase ? 1 : 0);
        key = key + 11 * (int)((10.0 + Math.toDegrees(headingDelta)) * 3);
        key = key + (int)((8.0 + velocity));
        return (char)(key);
    }

    double deltaX() {
        return Math.sin(heading) * velocity;
    }

    double deltaY() {
        return Math.cos(heading) * velocity;
    }
}

class BreakFrame extends Frame {
    char getKey() {
        return LeachPMC.BREAK_KEY;
    }

    double deltaX() {
        return 0;
    }

    double deltaY() {
        return 0;
    }
}
The BreakFrame? gets added at the beginning of each round. I have experimented some with adding that frame also when the radar slips and I get 4+ ticks to interpolate. Never happens against PatternBot when you are standing still though, but Frankie might try that approach again. -- PEZ

You seem to take it for granted that your radar will slip. I don't think this has to be the case. None of my development robots ever misses a scan unless it skips a turn, and the radar still doesn't slip even if that happens. -- nano

Maybe so. I'm probably thinking about the kind of radar I had a fe weeks ago. How if you print out the time since last scan in your scanned event? Print it only if it's > 1. LeachPMC then prints a few 2s an occational 3 and now and then a 4. About 15 times on average per round it seems it breaks the perfection. I then interpolate the missing tick and it works for what I can judge. Please tell how your robots fare here and we might learn something about radar management as well. -- PEZ

I am thinking how a about a RadomMovementChallenge?,every competitive bots just can fire 0.5 power bullet to against a PatternMatcherBot (it can be bot with the best ranking in PatternMatcherChallenge). This can see which bot's movement is radomer. -- iiley

That would be way cool. But how about PatternMatcherBot can fire but not the challenger? Then it could fire power 1->3 bullets and it would be a better measure on movement since it's not all that productive to tune movement against 0.5 bullets. -- PEZ

Yeah! Shall we discuss this further on the RandomMovementChallenge page? This page is getting cramped :-) -- Vic


Results :

( -Dummy)

Here it go results for MicroAspid:

1st: challenge.PatternBot 1.0	26363	4850	970	16855	3299	388	0	97	3	0
2nd: apv.MicroAspid	        15935	150	30	15647	107	0	0	3	97	0
I just removed a low pass filter from MicroAspid (it makes sense only when fighting against random movement bots) and this are the results :-):
1st: challenge.PatternBot 1.0	24183	2650	530	18525	2174	303	0	58	47	0
2nd: apv.MicroAspid	        23289	2350	470	18514	1954	0	0	52	53	0

WTF! That's truly amazing Albert! How do you fit that in a micro? What are your max and min match lenghts? --PEZ

Whoa!!! That's really good! You're just a pig's hair away from completing the challenge Albert! I'm in awe!! --Vic

This reminds me of a time when I said a fully-featured pattern-matcher could easily fit into a Minibot. -- Kawigi

MicroAspid doesn't have min or max distances (it just tries to match the longest existing pattern). About fitting it into a Micro, remember NanoLauLectrik also has a variable lenght pattern matcher (it just rebuilds the predicted bearing in a more pedestrian way) :-) -- Albert

Could you post MicroAspidPMC on the repository? I'd love to see it in action ... even if only for the sheer joy of watching near perfection :-) --Vic

here are the results of EnderPMC 1.1:

--Vic

1st: rz.GlowBlowPMC2		24554	2750	550	18752	2502	0	0	68	45	0
2nd: challenge.PatternBot 1.0	23855	2250	450	18601	2152	346	55	56	55	0

doesn't it look nice? -- rozu

O MY GOD!!!! Read 'em and weep.... You are the first to complete the challenge!! Congratulations Rozu! Notice that EnderPMC inflicts just the slightest bit less bullet damage. That shouldn't have made the difference.... but what has? I'll be back later today ;-) -- Vic ps: please crown yourself at the /PMCIndex

1st: rz.GlowBlowPMC		25459	3400	680	18560	2818	0	0	83	32	0
2nd: challenge.PatternBot 1.0	22558	1600	320	18361	1783	493	0	40	68	0

The results from EnderPMC 1.2 alpha:

1st: challenge.PatternBot	25739	3150	630	18770	2904	284	0	83	37	0
2nd: lv.EnderPMC	        23285	1850	370	18916	2149	0	0	50	63	0

Look at the bullet damage! I'm clearly losing on survival here... But currently EnderPMC gets a very high hit rate (98.4% over 100 rounds). I found that the period for PatternBot's Heading pattern is 600 ticks, so i just increased my maximum matching size from 200 to 1000. .... ok now let's see why EnderPMC loses on survival.... -- Vic

Results for Nibbler/NibblerPMC

1st: challenge.PatternBot 1.0	25506	3250	650	18666	2686	253	0	70	35	0
2nd: cbot.pmc.NibblerPMC	22299	1750	350	18699	1500	0	0	41	65	0

1st: challenge.PatternBot 1.0	24271	2750	550	18304	2095	513	57	64	45	0
2nd: cbot.pmc.NibblerPMC	23137	2250	450	18432	2005	0	0	47	55	0

Results for MicroAspidPMC:

1st: challenge.PatternBot 1.0	24400	2750	550	18468	2171	460	0	60	45	0
2nd: apv.MicroAspidPMC	        23070	2250	450	18496	1873	0	0	48	55	0
LeachPMC:

Nibbler/NibblerPMC just beat PatternBot! =)

1st: cbot.pmc.NibblerPMC 1.0	24796	2750	550	18770	2725	0	0	64	45	0
2nd: challenge.PatternBot 1.0	23632	2250	450	18608	2036	286	0	62	55	0

LeachPMC again: It's a bit unfair that PatternBot collects ram bonus. =) -- PEZ

Some tweaks here and there, a bugfix or two, a speed-optimization, and I ditched the data-saving feature:
At least I get to win a few rounds now... :-) --Dummy

Lol, I just tried FloodHT's gun package on PatternBot for fun (it does choose the pattern-matcher for it), and it was getting about an 80-84% hit-rate without tweaking, and then it fell almost to nothing, and I realized an interesting flaw in the way I implemented the pattern-buffer (using a mod function to put stuff into a finite-length array). I was matching the exact pattern I just recorded and projecting the next part of the pattern that I was going to overwrite. So after round 60-something, it just started to miss a lot, and the hitrate instantly fell from 80 to somewhere under 40, where various stat guns took over. -- Kawigi

Read 'em and weep:

1st: lv.EnderPMC 1.2		26645	3800	760	18772	3312	0	0	87	24	0
2nd: challenge.PatternBot	21626	1200	240	18504	1339	343	0	36	76	0
index 101.45 --Vic

Here it goes the last fight for MicroAspidPMC:

1st: apv.MicroAspidPMC	        25630	3150	630	18954	2896	0	0	84	37	0
2nd: challenge.PatternBot 1.0	23526	1850	370	18859	2187	212	46	50	63	0

Ok, I'm starting a real go at it with Teancum's gun. It might not be done until tomorrow, I just added a mechanism for preventing any matching or projection across rounds, and I also fixed the mod problem I described before, and I think it's ok. Just fixing the mod problem is enough of an excuse to release a new version of Teancum soon. Now I'm trying to figure out if I should do anything with my projection-into-the-wall mechanism, which fires a faster bullet at the point of impact into the wall in the regular dev version of Teancum. His /PMCIndex right now would fall around 90%-91%, as well as his hit rate, and he wins only occasionally, without really doing anything else that wasn't in the dev version except playing with the match length, or doing anything special to hit him in the beginning. This is probably the best so far for a LateralVelocity-based pattern-matcher. Before I caught that mod problem, I seriously was considering just labeling myself as bad at pattern-matching and sticking with stat guns from here on out. -- Kawigi

You guys realy slayed PatternBot when i bearly beat him! This realy was a good way to improve our pattern matchers. I'm not sure that i can take advantage this in the real Nibbler but at least it was fun! =) EnderPMC shows that you realy can humilliate poor PatternBot, atleast in survival. =) --Crippa

Just out of curiosity, Vic (since you didn't release source with PatternBot), would you think that pattern-matching PatternBot relative to myself should still be fundamentally correct? Does he move relative to his opponent once he gets into position? TeancumPMC's index is climbing... -- Kawigi

YALPMCR (Yet another LeachPMC result):
!!! -- PEZ

-DDDDD (YAEFK)-- Kawigi

Ok, here's my official first-version submission of TeancumPMC:

1st: challenge.PatternBot 1.0	26836	4150	830	18325	3215	315	0	86	17	0
2nd: kawigi.sbf.TeancumPMC	19824	850	170	18059	744	0	0	21	83	0
-- Kawigi

The final(?) version of LeachPMC:
It quite consistently beats PatternBot. It's using a general pattern matcher gun with only a few tweaks to score higher against PatternBot. Mainly a very long "movie", info on pre-pattern-phase in the movie and slightly different matching in the beginning and end of the rounds. -- PEZ

New results of EnderPMC 1.3 after a bugfix:

1st: lv.EnderPMC 1.3		26677	4000	800	18564	3312	0	0	84	20	0
2nd: challenge.PatternBot	20855	1000	200	18245	846	460	102	29	80	0
index 101.75

-- Vic

After fixing BlackPearl's projection error (Thanks Kawigi) Pearl is now getting this result:

1st: challenge.PatternBot 1.0	26935	4800	960	17087	3298	733	57	98	4	0
2nd: jekl.mini.BlackPearl	16606	200	40	16133	233	0	0	4	96	0
Index: 94.42%

Quick update:

1st: challenge.PatternBot 1.0	25566	3500	700	18190	2710	465	0	79	29	0
2nd: jekl.mini.BlackPearl	21240	1450	290	18055	1444	0	0	33	70	0
Index: 99.26
-- jim

I guess we're all just posting our scores like this? No - I'm going to start a results page! -- Simonton


Robo Home | Changes | Preferences | AllPages
Edit text of this page | View other revisions
Last edited October 2, 2006 8:05 EST by AC80A73B.ipt.aol.com (diff)
Search: