[Home]RoboResearch/Archive

Robo Home | RoboResearch | Changes | Preferences | AllPages

Difference (from prior major revision) (no other diffs)

Added: 167a168,238




I have a bug, and I'm wondering if it's a bug in robocode. Maybe you guys can help me answer this. DCResearch keeps getting "forcing a stop, no score will be generated" error, but only for the first battle run in the JVM. After one battle is over, it runs smoothly. The trouble is, RoboResearch only runs one battle per JVM. I hacked in a change to RoboResearch to print out the message when this error happens. Could those of you using RoboResearch please update from SVN and see if your bots do the same thing? You'll see the message print to the console if it does. This will be especially interesting to David, I think, since this possible bug causes targeting challenge scores to be lower than they should. -- Simonton
* I'm definitely getting more forced stops than I ought to, including some from the reference bots. I've noticed those when running Robocode the regular way also, even though my bots rarely ever skip turns. (Edit: I also just noticed I'm getting a ton of FAIL 2.0 messages. That can't be good, can it?) -- AaronR

Hey Simonton, I had an idea that might make the results from RoboResearch a little more reliable. You already have code to basically capture stderr from robocode and print it to the console, so that we can see if something goes wrong. How about checking whether a given process printed anything to stderr before adding its results to the database. If something went wrong, don't add it! --David Alves
* That is certainly doable. I should be able to do that tonight after I get home. I'll un-hack-ify that print-to-stderr, too, & make a regular part of RoboResearch. -- Simonton
* Alrighty, I committed what I think should do it. That is, assuming all "abnormal" messages will come out of Robocode's standard error (rather than standard out). You can give it a shot. If you have a bot that you know will cause errors, that will be the ideal test. -- Simonton
* Thanks, works perfectly! --David Alves




A thread following the call for RoboResearch 2.0:
* I'd like to, but I'm kind of strapped for time at the moment. Sounds like good stuff though, esp. the 'rumble when inactive' part =). -- Skilgannon
* I have a bit of free time over the next couple of days, but I don't know if there is anything I could do to help. I'm not exactly an expert when it comes to networking in Java. By the way, who would be hosting the RoboResearch server? -- AaronR
** There are things that need to be done besides the actual networking. Like getting it to fire up the rumble when there's nothing else to do, or getting it to swap robot jars in & out of the live robots directory, so that only the ones that need to be running are running. Would you like to do something like that? You can contact me via one of the methods on ContactInfo. The server, though, may be able to find a home on the same account to which we move this wiki, when that happens. -- Simonton
*** Whoops, sorry for the slow reply. I'll see what I can do about swapping JARs into the robots directory; I'll contact you if I can get it working. On the other hand, that's assuming I can get RoboResearch itself running - I keep getting java.sql.SQLException: socket creation error. -- AaronR
*** Sweet, thanks. You need to use the "Database.IN_MEMORY_CONNECTION" in your BattleRunner?.main() method. That, or fire up the database.sh script. -- Simonton
*** Thanks! -- AaronR
*** Alright, I have RoboResearch rigged up so that it copies the necessary JARs from an auxiliary directory into robocode_install/robots, runs the battles, and then deletes the copies and clears out the robot cache. My sourceforge username is "arotenbe", by the way. -- AaronR
*** Awesome. I'll add you as a developer tonight when I get home. -- Simonton
*** Can you not do the cache clearing if you're going to make RoboResearch run RRAH battles? Maybe have a robocode_test and a robocode_rrah installation as subdirectories instead of just robocode_install? Those of us who gain points from saved data would appreciate it. =) Since Dookious and Phoenix are within 10 points, and saved data is probably worth about 10 points... --David Alves
*** Hmm, I didn't really think about data saving. I think we should leave it as it is, and have RRAH run from the directory "roborumble_install" or "rrah_install". That way, nothing major has to be changed in the code or the existing installs. -- AaronR
*** My plan was to keep RoboResearch and RRAH as totally separate entities, then just have RoboResearch fire up that totally separate program when desired for one iteration. At the end of each iteration (when the RRAH process exits) it would poll the server again to see if it should run some research battles instead. -- Simonton
*** So basically, the question is: should RoboRumble be kept in a completely separate location, or should it be placed in a subdirectory of the RoboResearch folder? Now that I think about it, it may make more sense to keep RoboRumble separate; people may want to run RRAH without RoboResearch, and having a subfolder would make it harder to start the rumble client by itself (albeit only by one or two mouse clicks =). -- AaronR
*** I agree, let's do it that way. -- Simonton (who is off to add you as a developer right now)
*** Thanks, I've committed the changes. Could someone update their copy of RoboResearch and make sure that it actually works? I've been fiddling with my copy of the code so much that I'm not sure whether a fresh installation will even run with the new updates. Also, there is a bug that I can't seem to track down: it appears that the last bot in any challenge doesn't have its results saved to the database. So if you run a challenge, and then run it again, only the last battle will get run. For example, I just ran a MovementChallenge2K7/FastLearning? and it works perfectly, except that if I run it again, the only battle executed is Chalk (RoboResearch thinks that it has run no battles for that bot). -- AaronR
*** Do you have an IM client? Please contact me via email or IM. (ContactInfo) -- Simonton

AaronR - I've been using RoboResearch with your jar swapping for a little while now, and it seems to be working great! I refactored it a bit, I hope you don't mind. Also made one change: I let robocode deal with the .robocache stuff, since it delete directories automatically. It's nice to be able to just throw my dev bots into the robocode_bots directory without worrying about slowing down the battles! Nothing else has happened so far with RoboResearch 2.0; I guess I've been too interested in getting this WS-GF-GF/DC surfing going :). I hope that doesn't make you feel like your effort was a waste. I still want to do 2.0, and like I said, your mods are useful already! -- Simonton
* Great, thanks! Feel free to refactor and change it all you want, it was more of a quick hack than anything. Some of the code was copy-pasted from my equally messy cache cleaner utility I sent in as a feature request. ;) I might be able to do some more work on RoboResearch over the weekend. -- AaronR

Why does RoboResearch print "FAIL 2.0" to the console over and over when I run a MovementChallenge2K7? Looking at the code, it looks like this happens whenever it calculates the score... not sure why that's in there... --David Alves
* That should be removed from the latest version if you update. The lastest version also does the bot jar swapping (which is so far undocumented in the getting started guide). But, you're the one who added that "FAIL 2.0" - I assumed it was for testing ... -- Simonton
Oh hah, whoops! =) --David Alves

I've just committed a fairly signifigant modification: you now no longer have to modifiy the RoboResearch source code if you are running Windows or if you want to run a local challenge (that is, not over a network). RoboResearch will auto-detect the operating system; you need to pass a new command line flag to run battles over a network. -- AaronR
* Very nice! I just committed a quick fix: the runOverSever? flag was not getting set when a .cfg file was specified. Thanks! -- Simonton
** Thanks! Actually, from the SVN history, it looks like you just committed quite a few quick fixes... what are all of those other changes? -- AaronR

And here I still ahve a rather ancient (in development terms) version of this for doing networked battles. --Chase-san
* Actually, AaronR meant to connect to a database server in a different process (or machine), not running distributed battles. I don't know as I'd recommend updating, I have no idea whether distributed battle running still works. -- Simonton




Hmm, I'm trying to use RoboResarch? here but I'm getting the following error:
Exception in thread "Thread-1" java.lang.IllegalArgumentException: It's not worth storing a battle with less than 2 participants
at roboResearch.Database.makeTweakIdArray(Database.java:256)
at roboResearch.Database.store(Database.java:87)
at roboResearch.BattleRunner$BattleRunnerThread.run(BattleRunner.java:336)
Anyone have any ideas what could be going on?

-- Rednaxela

Hmm, nevermind, it started working when I used a clean robocode_bots directory with the necessary bots instead of symlinking to the main robots directory. -- Rednaxela

Hmm, for some reason, I get problems like "pez.rumble.CassiusClay 2pi.08TC still has not started after 100000 ms... giving up." even though CassiusClay 2pi.08TC seems to work fine outside of RoboResearch. Anyone have any suggestions? -- Rednaxela

Gah, nevermind. I misspelled my own bot's version and that caused the problem... -- Rednaxela

Hmm, well I don't have anything near the nice computer labs of Simonton, but I am trying to run this in a distributed way but I can't seem to work out how. I've successfully set up a shared install of RoboResearch within reach of both computers, but I can't seem to figure out how to get the instances running on each computer to cooperate. Anyone know how to get RoboResearch to work in a distributed mode? -- Rednaxela

I know the earlier version worked for distributed, but the later versions haven't been tested yet, now that multi-threading and such have been added. But take a look at [the code], you can probably figure it out. At first glance it seems it is specifically designed for the computer lab at Simonton and Chase-san's university, but with a bit of tweaking of hostnames I'm sure you could get it to work. From all appearances you have to edit the actual code to get it to load the distributed battle code, which is how all the stuff in RoboResearch was originally. If you want to add a hosts configuration file and some code to load it, email Simonton and he'll add you as a developer. -- Skilgannon

Hmm, the code there looks like a bit of a bother. Actually, I think I'm more interested in trying to get started some time on the "communal RoboResearch server" idea that Simonton brought up above, than trying to fix that old distributed code :) -- Rednaxela
* Still interested in that, Red? If so, drop me an email (ContactInfo). -- Simonton
* Mail sent. :-) -- Rednaxela




If anyone updates from SVN, be warned that the main class is now roboResearch.TUI, instead of roboResearch.BattleRunner?, so your scripts will need to be updated. Also, run roboResearch.GUI and let me know what you think of the design so far! Making the GUI is just preparation for running against a communal server. Rednaxela and I had a long discussion a while back about how the server should be designed; I still need to post a summary of our thoughts here to solicit feedback. -- Simonton

Calling All Developers

Ok, I think it's time to take RoboResearch to the next level. RoboResearch 2.0. I want to network it. We'll have a roboresearch server that will keep track of all the challenges people are wanting to run. Then clients will poll the servers for battles to run, downloading bot jars from the requester as necessary. The results will be sent back to the requester, as well, to be stored on their local database. When there are no battles to run, the clients will fire up a copy of the rumble instead. People's clients will prefer their own bots' battles. Who's willing to help? -- Simonton

I'm not sure a communal RoboResearch server is a good idea. It seems like you'd end up in one of two situations. 1) Some people request way more battles than they contribute, so the people running more than their fair share get annoyed, or 2) Everyone requests about the same number of battles as they contribute, in which case what's the point of having it be networked? --David Alves

The Idea

I've started on a new utility something like RoboLeague. The biggest difference is that it stores battle results in a database, not an XML file. This leaves LOTS of flexibility to analyze data any way you can dream up. The other big reason I decided to start on it was to be able to run battles remotely on separate machines (which I plan to accomplish through SSH). I know RoboLeague can do this with leagues (but not divisions?), but I need it in a different format to accomplish the type of remote operation I have in mind. Would there be anyone interested in making this a community project? Here are the other differences from RoboLeague I can think of right now:


Capabilites So Far

Next might be figuring out if it would be capable of running battles for the rumble. Also adding a GUI.


Getting Started

  1. Get a copy of roboresearch from its [sourceforge home] by pointing an SVN client at https://roboresearch.svn.sourceforge.net/svnroot/roboresearch. You're after the "trunk" folder.
  2. Continue with the (hopefully) helpful instructions found in docs/getting_started_running.txt.


Cool. Let me know if you need a hand or have trouble making it work with Robocode. Then I'll support you as much as possible! --Fnl

I need something like this for running lots of seasons of 35-round matches. If it gets integrated into the main Robocode release, that would be even better. Would you mind sharing the code? -- Skilgannon

Do you not use RoboLeague? I can't live without it in development. Actually, I remember it didn't work right with some newer versions of Robocode, but I use it with 1.07. If it still doesn't work with Robocode, that should really be something Fnl or one of us take care of. -- Voidious

I've never tried RoboLeague. Besides not being compatible with the latest version it looked like a mission to set up as well. I'll grab a copy of 1.07 and give it a try....is the source code for it available?

@Simonton, how about merging it with the SourceForge? Robocode project? Things like restarting robocode each season could be set as an option, I'm assuming? What are you using for database access? SQL/ODBC? Also, is it written in Java as well? I'd like to try a bit of Python, and this sounds like something fun, but even if it's in Java I'm willing to lend a bit of my time. I just have to remember that I'm also meant to be studying =) -- Skilgannon

We could make some options in Robocode to support this project. I will support you with whatever you need. --Fnl

I am wondering. How far is this project? Is there any progress? (I am just curious ;-) --Fnl

Ok - why does robocode run WAY slow when I use a copy of the jar in a directory other than the one in which I installed it? It takes a very long time for it to do something between battles. -- Simonton

Alrighty, I just submitted an application to open a sourceforge project for RoboResearch. I hope you were serious about helping, Skilgannon! --Simonton

Oh, Fnl, I just found one more request this project could ask of RoboCode: make sure that when "-nodisplay" is used, it never tries to open *any* kind of user interface (except the console). I can't run the newest version distributed over the network; it's giving me the "can't connect to your computer" message again (since I'm behind a firewall, if it tries to open a connection back to my computer to display a window or anything like that it fails). I thought maybe it was because of the new human-interaction capabilities, so I installed version 1.3.3, and that one works great. -- Simonton

Well - sourceforge apparently ignored my request to open a project. I never heard back from them. On another note, can someone tell me whether it would work to copy robot jars and their cache directories in & out of the robots folder? I would like to get RoboResearch set up to handle data-saving bots without having to leave those bots in the robots directory (because it takes a long time for robocode to go through them all every time it starts). Would robocode re-unzip the jars & overwrite things in the data saving directory, or leave it the way it was? I suppose it wouldn't be too hard to test, I was just wondering if anyone knew. -- Simonton

Ok, it was a very long time coming - BUT THE SOURCE IS NOW ON SOURCEFORGE!! YAY! So, anyone who wishes to help out, tell me how to let you! -- Simonton

""If I/we ever get around to it, RoboResearch will be able to automatically create tweaks of a bot and run them against a set of competitors,"" This is probably most easily accomplished with a GeneticAlgorithm which are in fact what my project is using, the robot is NeBot, (the one that found the bugs in RoboCode). So its not really that difficult of a concept to implement. It would be some work, having to deal with different parameters that would have different discretizations and the effects of the delta's used for those discretizations would have on the robots. But doable. -- Ntroutman 2007-08-30

I'm sad. 2 hours to run the TargetingChallenge for DCResearch is a lot slower than the 2 minutes it took when I had access to the computer lab at the school ... :*( -- Simonton

I have actually optimised it to use the duo's (faster dual cores :) and it has dropped down to about 1 and a half. --Chase-san


I'm getting this when I try to run:

 C:\robocode\RoboResearch>java -cp bin;hsqldb.jar roboResearch.BattleRunner? 
 -s 5
 -C c:\robocode\RoboResearch\challenges\mc2k7.rrc
 -c "davidalves.Phoenix 0.9"

 Running vs. wiki.mc2k7.HawkOnFireOS? MC2K7 ... java.io.FileNotFoundException?: 
 robocode_install\battles\tmp.battle.results 
 (The system cannot find the file specified)
        at java.io.FileInputStream?.open(Native Method)
        at java.io.FileInputStream?.<init>(FileInputStream?.java:106)
        at java.io.FileInputStream?.<init>(FileInputStream?.java:66)
        at java.io.FileReader?.<init>(FileReader?.java:41)
        at roboResearch.BattleResults?.load(BattleResults?.java:22)
        at roboResearch.BattleRunner?.runBattle(BattleRunner?.java:106)
        at roboResearch.BattleRunner?.runBattle(BattleRunner?.java:76)
        at roboResearch.BattleRunner?.runChallenge(BattleRunner?.java:143)
        at roboResearch.BattleRunner?.main(BattleRunner?.java:34)
 There appears to have been an error in Robocode itself.

I put a copy of Robocode 1.4.5 in the robocode_install directory and put all the MC2K7 bots in it. If run robocode manually and select the tmp.battle file, it properly loads up the two bots. Any ideas? --David Alves

Nevermind, figured it out. I had edited the batch file used to run robocode and had it pointing to a different directory. --David Alves

Ah cool, I was going to add that when I got home tonight. 4x speed improvement would be nice, but I had to go to bed before I could add it last night. =) --David Alves

       java -cp ../hsqldb.jar org.hsqldb.Server -database.0 file:roboresearch -dbname.0 roboresearch

There's no harm in running multiple copies of robocode from the same install directory as long as they are just using .battle files, right? I know from experience that it can cause bad interactions if you run multiple RRAH clients from the same directory, but multiple .battle ones seems safe. I'll make the change after I get home tonight. --David Alves

It's working with multiple threads now. Do you need to add me to the project on sourceforge to allow me to commit my changes? --David Alves

It's been committed now. Enjoy! --David Alves

Very nice work! I'm just about to commit a change to print out # of seasons, if you'd like update. -- Simonton

Updated to delete .results files (they stayed forever before). Also if a thread encountered an exception when running a battle it would previously stop running. Now the thread will skip that battle and move on to the next one. Incidentally, the battles are run in the season order, so the # of seasons complete should be

 battles completed so far / # of battles per season

This doesn't account for any exceptions encountered while running battles though, if you want your season-printing thing to account for those then it gets a bit more complicated.

If you're updating to this version you might want to clear out *.results from your robocode_install/battles folder, there might be a lot of 'em. =) --David Alves

Oops ... ok, NOW I committed the printing of the # of seasons in wiki output. Along with a few other, un-notable tweaks. Thanks again for your work, David - I'm running in two threads as we speak! -- Simonton

Just made a nice change in the way challenge battles are chosen. The number of seasons now represents the number of *completed* season you wish to have in the database altogether. The bot(s) with the least number of battles fought will be run until every pairing in the challenge has at least that many battles. It has been committed. -- Simonton


I have a bug, and I'm wondering if it's a bug in robocode. Maybe you guys can help me answer this. DCResearch keeps getting "forcing a stop, no score will be generated" error, but only for the first battle run in the JVM. After one battle is over, it runs smoothly. The trouble is, RoboResearch only runs one battle per JVM. I hacked in a change to RoboResearch to print out the message when this error happens. Could those of you using RoboResearch please update from SVN and see if your bots do the same thing? You'll see the message print to the console if it does. This will be especially interesting to David, I think, since this possible bug causes targeting challenge scores to be lower than they should. -- Simonton

Hey Simonton, I had an idea that might make the results from RoboResearch a little more reliable. You already have code to basically capture stderr from robocode and print it to the console, so that we can see if something goes wrong. How about checking whether a given process printed anything to stderr before adding its results to the database. If something went wrong, don't add it! --David Alves


A thread following the call for RoboResearch 2.0:

AaronR - I've been using RoboResearch with your jar swapping for a little while now, and it seems to be working great! I refactored it a bit, I hope you don't mind. Also made one change: I let robocode deal with the .robocache stuff, since it delete directories automatically. It's nice to be able to just throw my dev bots into the robocode_bots directory without worrying about slowing down the battles! Nothing else has happened so far with RoboResearch 2.0; I guess I've been too interested in getting this WS-GF-GF/DC surfing going :). I hope that doesn't make you feel like your effort was a waste. I still want to do 2.0, and like I said, your mods are useful already! -- Simonton

Why does RoboResearch print "FAIL 2.0" to the console over and over when I run a MovementChallenge2K7? Looking at the code, it looks like this happens whenever it calculates the score... not sure why that's in there... --David Alves

Oh hah, whoops! =) --David Alves

I've just committed a fairly signifigant modification: you now no longer have to modifiy the RoboResearch source code if you are running Windows or if you want to run a local challenge (that is, not over a network). RoboResearch will auto-detect the operating system; you need to pass a new command line flag to run battles over a network. -- AaronR

And here I still ahve a rather ancient (in development terms) version of this for doing networked battles. --Chase-san


Hmm, I'm trying to use RoboResarch? here but I'm getting the following error:

Exception in thread "Thread-1" java.lang.IllegalArgumentException: It's not worth storing a battle with less than 2 participants
	at roboResearch.Database.makeTweakIdArray(Database.java:256)
	at roboResearch.Database.store(Database.java:87)
	at roboResearch.BattleRunner$BattleRunnerThread.run(BattleRunner.java:336)
Anyone have any ideas what could be going on?
-- Rednaxela

Hmm, nevermind, it started working when I used a clean robocode_bots directory with the necessary bots instead of symlinking to the main robots directory. -- Rednaxela

Hmm, for some reason, I get problems like "pez.rumble.CassiusClay 2pi.08TC still has not started after 100000 ms... giving up." even though CassiusClay 2pi.08TC seems to work fine outside of RoboResearch. Anyone have any suggestions? -- Rednaxela

Gah, nevermind. I misspelled my own bot's version and that caused the problem... -- Rednaxela

Hmm, well I don't have anything near the nice computer labs of Simonton, but I am trying to run this in a distributed way but I can't seem to work out how. I've successfully set up a shared install of RoboResearch within reach of both computers, but I can't seem to figure out how to get the instances running on each computer to cooperate. Anyone know how to get RoboResearch to work in a distributed mode? -- Rednaxela

I know the earlier version worked for distributed, but the later versions haven't been tested yet, now that multi-threading and such have been added. But take a look at [the code], you can probably figure it out. At first glance it seems it is specifically designed for the computer lab at Simonton and Chase-san's university, but with a bit of tweaking of hostnames I'm sure you could get it to work. From all appearances you have to edit the actual code to get it to load the distributed battle code, which is how all the stuff in RoboResearch was originally. If you want to add a hosts configuration file and some code to load it, email Simonton and he'll add you as a developer. -- Skilgannon

Hmm, the code there looks like a bit of a bother. Actually, I think I'm more interested in trying to get started some time on the "communal RoboResearch server" idea that Simonton brought up above, than trying to fix that old distributed code :) -- Rednaxela


If anyone updates from SVN, be warned that the main class is now roboResearch.TUI, instead of roboResearch.BattleRunner?, so your scripts will need to be updated. Also, run roboResearch.GUI and let me know what you think of the design so far! Making the GUI is just preparation for running against a communal server. Rednaxela and I had a long discussion a while back about how the server should be designed; I still need to post a summary of our thoughts here to solicit feedback. -- Simonton


Robo Home | RoboResearch | Changes | Preferences | AllPages
Edit text of this page | View other revisions
Last edited September 13, 2008 21:55 EST by Simonton (diff)
Search: