Robo Home | RoboResearch | Changes | Preferences | AllPages

Calling All Developers

Ok, I think it's time to take RoboResearch to the next level. RoboResearch 2.0. I want to network it. We'll have a roboresearch server that will keep track of all the challenges people are wanting to run. Then clients will poll the servers for battles to run, downloading bot jars from the requester as necessary. The results will be sent back to the requester, as well, to be stored on their local database. When there are no battles to run, the clients will fire up a copy of the rumble instead. People's clients will prefer their own bots' battles. Who's willing to help? -- Simonton

I'm not sure a communal RoboResearch server is a good idea. It seems like you'd end up in one of two situations. 1) Some people request way more battles than they contribute, so the people running more than their fair share get annoyed, or 2) Everyone requests about the same number of battles as they contribute, in which case what's the point of having it be networked? --David Alves

The Idea

I've started on a new utility something like RoboLeague. The biggest difference is that it stores battle results in a database, not an XML file. This leaves LOTS of flexibility to analyze data any way you can dream up. The other big reason I decided to start on it was to be able to run battles remotely on separate machines (which I plan to accomplish through SSH). I know RoboLeague can do this with leagues (but not divisions?), but I need it in a different format to accomplish the type of remote operation I have in mind. Would there be anyone interested in making this a community project? Here are the other differences from RoboLeague I can think of right now:

Capabilites So Far

Next might be figuring out if it would be capable of running battles for the rumble. Also adding a GUI.

Getting Started

  1. Get a copy of roboresearch from its [sourceforge home] by pointing an SVN client at https://roboresearch.svn.sourceforge.net/svnroot/roboresearch. You're after the "trunk" folder.
  2. Continue with the (hopefully) helpful instructions found in docs/getting_started_running.txt.

Cool. Let me know if you need a hand or have trouble making it work with Robocode. Then I'll support you as much as possible! --Fnl

I need something like this for running lots of seasons of 35-round matches. If it gets integrated into the main Robocode release, that would be even better. Would you mind sharing the code? -- Skilgannon

Do you not use RoboLeague? I can't live without it in development. Actually, I remember it didn't work right with some newer versions of Robocode, but I use it with 1.07. If it still doesn't work with Robocode, that should really be something Fnl or one of us take care of. -- Voidious

I've never tried RoboLeague. Besides not being compatible with the latest version it looked like a mission to set up as well. I'll grab a copy of 1.07 and give it a try....is the source code for it available?

@Simonton, how about merging it with the SourceForge? Robocode project? Things like restarting robocode each season could be set as an option, I'm assuming? What are you using for database access? SQL/ODBC? Also, is it written in Java as well? I'd like to try a bit of Python, and this sounds like something fun, but even if it's in Java I'm willing to lend a bit of my time. I just have to remember that I'm also meant to be studying =) -- Skilgannon

We could make some options in Robocode to support this project. I will support you with whatever you need. --Fnl

I am wondering. How far is this project? Is there any progress? (I am just curious ;-) --Fnl

Ok - why does robocode run WAY slow when I use a copy of the jar in a directory other than the one in which I installed it? It takes a very long time for it to do something between battles. -- Simonton

Alrighty, I just submitted an application to open a sourceforge project for RoboResearch. I hope you were serious about helping, Skilgannon! --Simonton

Oh, Fnl, I just found one more request this project could ask of RoboCode: make sure that when "-nodisplay" is used, it never tries to open *any* kind of user interface (except the console). I can't run the newest version distributed over the network; it's giving me the "can't connect to your computer" message again (since I'm behind a firewall, if it tries to open a connection back to my computer to display a window or anything like that it fails). I thought maybe it was because of the new human-interaction capabilities, so I installed version 1.3.3, and that one works great. -- Simonton

Well - sourceforge apparently ignored my request to open a project. I never heard back from them. On another note, can someone tell me whether it would work to copy robot jars and their cache directories in & out of the robots folder? I would like to get RoboResearch set up to handle data-saving bots without having to leave those bots in the robots directory (because it takes a long time for robocode to go through them all every time it starts). Would robocode re-unzip the jars & overwrite things in the data saving directory, or leave it the way it was? I suppose it wouldn't be too hard to test, I was just wondering if anyone knew. -- Simonton

Ok, it was a very long time coming - BUT THE SOURCE IS NOW ON SOURCEFORGE!! YAY! So, anyone who wishes to help out, tell me how to let you! -- Simonton

""If I/we ever get around to it, RoboResearch will be able to automatically create tweaks of a bot and run them against a set of competitors,"" This is probably most easily accomplished with a GeneticAlgorithm which are in fact what my project is using, the robot is NeBot, (the one that found the bugs in RoboCode). So its not really that difficult of a concept to implement. It would be some work, having to deal with different parameters that would have different discretizations and the effects of the delta's used for those discretizations would have on the robots. But doable. -- Ntroutman 2007-08-30

I'm sad. 2 hours to run the TargetingChallenge for DCResearch is a lot slower than the 2 minutes it took when I had access to the computer lab at the school ... :*( -- Simonton

I have actually optimised it to use the duo's (faster dual cores :) and it has dropped down to about 1 and a half. --Chase-san

I'm getting this when I try to run:

 C:\robocode\RoboResearch>java -cp bin;hsqldb.jar roboResearch.BattleRunner? 
 -s 5
 -C c:\robocode\RoboResearch\challenges\mc2k7.rrc
 -c "davidalves.Phoenix 0.9"

 Running vs. wiki.mc2k7.HawkOnFireOS? MC2K7 ... java.io.FileNotFoundException?: 
 (The system cannot find the file specified)
        at java.io.FileInputStream?.open(Native Method)
        at java.io.FileInputStream?.<init>(FileInputStream?.java:106)
        at java.io.FileInputStream?.<init>(FileInputStream?.java:66)
        at java.io.FileReader?.<init>(FileReader?.java:41)
        at roboResearch.BattleResults?.load(BattleResults?.java:22)
        at roboResearch.BattleRunner?.runBattle(BattleRunner?.java:106)
        at roboResearch.BattleRunner?.runBattle(BattleRunner?.java:76)
        at roboResearch.BattleRunner?.runChallenge(BattleRunner?.java:143)
        at roboResearch.BattleRunner?.main(BattleRunner?.java:34)
 There appears to have been an error in Robocode itself.

I put a copy of Robocode 1.4.5 in the robocode_install directory and put all the MC2K7 bots in it. If run robocode manually and select the tmp.battle file, it properly loads up the two bots. Any ideas? --David Alves

Nevermind, figured it out. I had edited the batch file used to run robocode and had it pointing to a different directory. --David Alves

Ah cool, I was going to add that when I got home tonight. 4x speed improvement would be nice, but I had to go to bed before I could add it last night. =) --David Alves

       java -cp ../hsqldb.jar org.hsqldb.Server -database.0 file:roboresearch -dbname.0 roboresearch

There's no harm in running multiple copies of robocode from the same install directory as long as they are just using .battle files, right? I know from experience that it can cause bad interactions if you run multiple RRAH clients from the same directory, but multiple .battle ones seems safe. I'll make the change after I get home tonight. --David Alves

It's working with multiple threads now. Do you need to add me to the project on sourceforge to allow me to commit my changes? --David Alves

It's been committed now. Enjoy! --David Alves

Very nice work! I'm just about to commit a change to print out # of seasons, if you'd like update. -- Simonton

Updated to delete .results files (they stayed forever before). Also if a thread encountered an exception when running a battle it would previously stop running. Now the thread will skip that battle and move on to the next one. Incidentally, the battles are run in the season order, so the # of seasons complete should be

 battles completed so far / # of battles per season

This doesn't account for any exceptions encountered while running battles though, if you want your season-printing thing to account for those then it gets a bit more complicated.

If you're updating to this version you might want to clear out *.results from your robocode_install/battles folder, there might be a lot of 'em. =) --David Alves

Oops ... ok, NOW I committed the printing of the # of seasons in wiki output. Along with a few other, un-notable tweaks. Thanks again for your work, David - I'm running in two threads as we speak! -- Simonton

Just made a nice change in the way challenge battles are chosen. The number of seasons now represents the number of *completed* season you wish to have in the database altogether. The bot(s) with the least number of battles fought will be run until every pairing in the challenge has at least that many battles. It has been committed. -- Simonton

I have a bug, and I'm wondering if it's a bug in robocode. Maybe you guys can help me answer this. DCResearch keeps getting "forcing a stop, no score will be generated" error, but only for the first battle run in the JVM. After one battle is over, it runs smoothly. The trouble is, RoboResearch only runs one battle per JVM. I hacked in a change to RoboResearch to print out the message when this error happens. Could those of you using RoboResearch please update from SVN and see if your bots do the same thing? You'll see the message print to the console if it does. This will be especially interesting to David, I think, since this possible bug causes targeting challenge scores to be lower than they should. -- Simonton

Hey Simonton, I had an idea that might make the results from RoboResearch a little more reliable. You already have code to basically capture stderr from robocode and print it to the console, so that we can see if something goes wrong. How about checking whether a given process printed anything to stderr before adding its results to the database. If something went wrong, don't add it! --David Alves

A thread following the call for RoboResearch 2.0:

AaronR - I've been using RoboResearch with your jar swapping for a little while now, and it seems to be working great! I refactored it a bit, I hope you don't mind. Also made one change: I let robocode deal with the .robocache stuff, since it delete directories automatically. It's nice to be able to just throw my dev bots into the robocode_bots directory without worrying about slowing down the battles! Nothing else has happened so far with RoboResearch 2.0; I guess I've been too interested in getting this WS-GF-GF/DC surfing going :). I hope that doesn't make you feel like your effort was a waste. I still want to do 2.0, and like I said, your mods are useful already! -- Simonton

Why does RoboResearch print "FAIL 2.0" to the console over and over when I run a MovementChallenge2K7? Looking at the code, it looks like this happens whenever it calculates the score... not sure why that's in there... --David Alves

Oh hah, whoops! =) --David Alves

I've just committed a fairly signifigant modification: you now no longer have to modifiy the RoboResearch source code if you are running Windows or if you want to run a local challenge (that is, not over a network). RoboResearch will auto-detect the operating system; you need to pass a new command line flag to run battles over a network. -- AaronR

And here I still ahve a rather ancient (in development terms) version of this for doing networked battles. --Chase-san

Hmm, I'm trying to use RoboResarch? here but I'm getting the following error:

Exception in thread "Thread-1" java.lang.IllegalArgumentException: It's not worth storing a battle with less than 2 participants
	at roboResearch.Database.makeTweakIdArray(Database.java:256)
	at roboResearch.Database.store(Database.java:87)
	at roboResearch.BattleRunner$BattleRunnerThread.run(BattleRunner.java:336)
Anyone have any ideas what could be going on?
-- Rednaxela

Hmm, nevermind, it started working when I used a clean robocode_bots directory with the necessary bots instead of symlinking to the main robots directory. -- Rednaxela

Hmm, for some reason, I get problems like "pez.rumble.CassiusClay 2pi.08TC still has not started after 100000 ms... giving up." even though CassiusClay 2pi.08TC seems to work fine outside of RoboResearch. Anyone have any suggestions? -- Rednaxela

Gah, nevermind. I misspelled my own bot's version and that caused the problem... -- Rednaxela

Hmm, well I don't have anything near the nice computer labs of Simonton, but I am trying to run this in a distributed way but I can't seem to work out how. I've successfully set up a shared install of RoboResearch within reach of both computers, but I can't seem to figure out how to get the instances running on each computer to cooperate. Anyone know how to get RoboResearch to work in a distributed mode? -- Rednaxela

I know the earlier version worked for distributed, but the later versions haven't been tested yet, now that multi-threading and such have been added. But take a look at [the code], you can probably figure it out. At first glance it seems it is specifically designed for the computer lab at Simonton and Chase-san's university, but with a bit of tweaking of hostnames I'm sure you could get it to work. From all appearances you have to edit the actual code to get it to load the distributed battle code, which is how all the stuff in RoboResearch was originally. If you want to add a hosts configuration file and some code to load it, email Simonton and he'll add you as a developer. -- Skilgannon

Hmm, the code there looks like a bit of a bother. Actually, I think I'm more interested in trying to get started some time on the "communal RoboResearch server" idea that Simonton brought up above, than trying to fix that old distributed code :) -- Rednaxela

If anyone updates from SVN, be warned that the main class is now roboResearch.TUI, instead of roboResearch.BattleRunner?, so your scripts will need to be updated. Also, run roboResearch.GUI and let me know what you think of the design so far! Making the GUI is just preparation for running against a communal server. Rednaxela and I had a long discussion a while back about how the server should be designed; I still need to post a summary of our thoughts here to solicit feedback. -- Simonton

Robo Home | RoboResearch | Changes | Preferences | AllPages
Edit text of this page | View other revisions
Last edited September 13, 2008 23:55 EST by Simonton (diff)