RoboResearch/Archive - Robo Wiki -= Collecting Robocode Knowledge =-

Calling All Developers

Ok, I think it's time to take RoboResearch to the next level. RoboResearch 2.0. I want to network it. We'll have a roboresearch server that will keep track of all the challenges people are wanting to run. Then clients will poll the servers for battles to run, downloading bot jars from the requester as necessary. The results will be sent back to the requester, as well, to be stored on their local database. When there are no battles to run, the clients will fire up a copy of the rumble instead. People's clients will prefer their own bots' battles. Who's willing to help? -- Simonton

I'm not sure a communal RoboResearch server is a good idea. It seems like you'd end up in one of two situations. 1) Some people request way more battles than they contribute, so the people running more than their fair share get annoyed, or 2) Everyone requests about the same number of battles as they contribute, in which case what's the point of having it be networked? --David Alves

You're right on one point - we should make sure people would actually be interested in running the client. But I can definitely see benefits. Some people may certainly run way more battles than anyone else (someone named "Simonton" comes to mind), but just like in the rumble, you can always turn off your client if you don't feel like assisting that person anymore (and, in this case, turn on the rumble client by itself ;) ). But as for people requesting the same number of battles - the point is that people can do their research at different times. Skilgannon doing research in the evening in South Africa can see results quickly, then you doing research in the evening in California can see them quickly, too, since you're both helping each other out in your times of "need". The users should be instructed briefly in RoboResearch etiquette, so that for example they only run as many battles as they need to get stable ratings (e.g. 50 seasons instead of 500, or whatever). So, I guess the question is: Who out there would actually run a client like this? -- Simonton
I would use it. As long as I always have the option of giving my own tests highest priority, I don't see how it could do anything but help everyone. As for switching to the RoboRumble when there are no more tests to run - I think it would be equally useful to be able to reverse that: run the RoboRumble until everyone has their required battles, then return to contributing cycles towards anybody who wants to run some tests. (In other words: if there are unstable bots in the rumble, I'd give that higher priority than somebody else's gun tests.) As for the timezones thing - I do run stuff 24/7 sometimes, both while I'm at work and while I'm sleeping, but I usually try to do something that will finish before I'm back at the computer, in which case there would be some idle time at the end to contribute back. -- Voidious
- Great idea! That will require a little more logic, but I'm sure we're up to the task! -- Simonton

The Idea

I've started on a new utility something like RoboLeague. The biggest difference is that it stores battle results in a database, not an XML file. This leaves LOTS of flexibility to analyze data any way you can dream up. The other big reason I decided to start on it was to be able to run battles remotely on separate machines (which I plan to accomplish through SSH). I know RoboLeague can do this with leagues (but not divisions?), but I need it in a different format to accomplish the type of remote operation I have in mind. Would there be anyone interested in making this a community project? Here are the other differences from RoboLeague I can think of right now:

It runs RoboCode in a separate process, interacting only with .battle files and .results files (it does not utilize any of RoboCode's APIs). This has benefits and drawbacks, but I did it so that 1) I could get a fresh JVM with each battle (to avoid memory issues as encountered with my Obsession series of bots) and 2) it's a little easier to code up the remote operation.
It is "tweak-centric" as opposed to "bot-centric". The idea is to allow the modification of a bot's properties file, where each permutation of variable assignments defines a "tweak" of that bot. So, there may be many tweaks to each version. If I/we ever get around to it, RoboResearch will be able to automatically create tweaks of a bot and run them against a set of competitors, so you can see what combination of tuner variables brings the best payoff. Maybe it could even search for a maximum benefit & drill down to optimal values? But that's just a dream down the road.
It should have starters you can create & save for, e.g. MC2K7, where you click on "MC2K7", then on the bot/tweak-set you wish to run, then on "go".
There will not a be a "league" style, and there will always be a "focused competitor". The intent of this utility is to test your bot, not rank a group of bots.
So far I plan to wipe out the robots/ and .robotcache/ directories often. It would use a directory of bot jars elsewhere, then copy them into the robots/ directory when it's time to fight. This eliminates the benefit of file saving in bots, but I'm sick of waiting for RoboCode to rebuild the robot database and robot cache so often. This should eliminate a lot of the overhead that would otherwise come from spawning a new instance of RoboCode for every battle.

Capabilites So Far

Run a battle.
Run any number of battles at once, each on a remote host (requires password-free SSH between the machines and that RoboResearch is installed in a common NSF directory).
Store the results in a database.
Read in a set of reference bots from a "RoboResearch Challenge File" and run a challenge bot against them for any number of rounds.
Query the database and give results in wiki format! =D
Run multiple battles at once on your local machine (for multicore/multiprocessor machines).

Next might be figuring out if it would be capable of running battles for the rumble. Also adding a GUI.

Getting Started

Get a copy of roboresearch from its [sourceforge home] by pointing an SVN client at https://roboresearch.svn.sourceforge.net/svnroot/roboresearch. You're after the "trunk" folder.
Continue with the (hopefully) helpful instructions found in docs/getting_started_running.txt.