Calling All Developers
Ok, I think it's time to take RoboResearch to the next level. RoboResearch 2.0. I want to network it. We'll have a roboresearch server that will keep track of all the challenges people are wanting to run. Then clients will poll the servers for battles to run, downloading bot jars from the requester as necessary. The results will be sent back to the requester, as well, to be stored on their local database. When there are no battles to run, the clients will fire up a copy of the rumble instead. People's clients will prefer their own bots' battles. Who's willing to help? -- Simonton
I'm not sure a communal RoboResearch server is a good idea. It seems like you'd end up in one of two situations. 1) Some people request way more battles than they contribute, so the people running more than their fair share get annoyed, or 2) Everyone requests about the same number of battles as they contribute, in which case what's the point of having it be networked? --David Alves
- You're right on one point - we should make sure people would actually be interested in running the client. But I can definitely see benefits. Some people may certainly run way more battles than anyone else (someone named "Simonton" comes to mind), but just like in the rumble, you can always turn off your client if you don't feel like assisting that person anymore (and, in this case, turn on the rumble client by itself ;) ). But as for people requesting the same number of battles - the point is that people can do their research at different times. Skilgannon doing research in the evening in South Africa can see results quickly, then you doing research in the evening in California can see them quickly, too, since you're both helping each other out in your times of "need". The users should be instructed briefly in RoboResearch etiquette, so that for example they only run as many battles as they need to get stable ratings (e.g. 50 seasons instead of 500, or whatever). So, I guess the question is: Who out there would actually run a client like this? -- Simonton
- I would use it. As long as I always have the option of giving my own tests highest priority, I don't see how it could do anything but help everyone. As for switching to the RoboRumble when there are no more tests to run - I think it would be equally useful to be able to reverse that: run the RoboRumble until everyone has their required battles, then return to contributing cycles towards anybody who wants to run some tests. (In other words: if there are unstable bots in the rumble, I'd give that higher priority than somebody else's gun tests.) As for the timezones thing - I do run stuff 24/7 sometimes, both while I'm at work and while I'm sleeping, but I usually try to do something that will finish before I'm back at the computer, in which case there would be some idle time at the end to contribute back. -- Voidious
- Great idea! That will require a little more logic, but I'm sure we're up to the task! -- Simonton
The Idea
I've started on a new utility something like RoboLeague. The biggest difference is that it stores battle results in a database, not an XML file. This leaves LOTS of flexibility to analyze data any way you can dream up. The other big reason I decided to start on it was to be able to run battles remotely on separate machines (which I plan to accomplish through SSH). I know RoboLeague can do this with leagues (but not divisions?), but I need it in a different format to accomplish the type of remote operation I have in mind. Would there be anyone interested in making this a community project? Here are the other differences from RoboLeague I can think of right now:
- It runs RoboCode in a separate process, interacting only with .battle files and .results files (it does not utilize any of RoboCode's APIs). This has benefits and drawbacks, but I did it so that 1) I could get a fresh JVM with each battle (to avoid memory issues as encountered with my Obsession series of bots) and 2) it's a little easier to code up the remote operation.
- It is "tweak-centric" as opposed to "bot-centric". The idea is to allow the modification of a bot's properties file, where each permutation of variable assignments defines a "tweak" of that bot. So, there may be many tweaks to each version. If I/we ever get around to it, RoboResearch will be able to automatically create tweaks of a bot and run them against a set of competitors, so you can see what combination of tuner variables brings the best payoff. Maybe it could even search for a maximum benefit & drill down to optimal values? But that's just a dream down the road.
- It should have starters you can create & save for, e.g. MC2K7, where you click on "MC2K7", then on the bot/tweak-set you wish to run, then on "go".
- There will not a be a "league" style, and there will always be a "focused competitor". The intent of this utility is to test your bot, not rank a group of bots.
- So far I plan to wipe out the robots/ and .robotcache/ directories often. It would use a directory of bot jars elsewhere, then copy them into the robots/ directory when it's time to fight. This eliminates the benefit of file saving in bots, but I'm sick of waiting for RoboCode to rebuild the robot database and robot cache so often. This should eliminate a lot of the overhead that would otherwise come from spawning a new instance of RoboCode for every battle.
Capabilites So Far
- Run a battle.
- Run any number of battles at once, each on a remote host (requires password-free SSH between the machines and that RoboResearch is installed in a common NSF directory).
- Store the results in a database.
- Read in a set of reference bots from a "RoboResearch Challenge File" and run a challenge bot against them for any number of rounds.
- Query the database and give results in wiki format! =D
- Run multiple battles at once on your local machine (for multicore/multiprocessor machines).
Next might be figuring out if it would be capable of running battles for the rumble. Also adding a GUI.
Getting Started
- Get a copy of roboresearch from its [sourceforge home] by pointing an SVN client at https://roboresearch.svn.sourceforge.net/svnroot/roboresearch. You're after the "trunk" folder.
- Continue with the (hopefully) helpful instructions found in docs/getting_started_running.txt.
Cool. Let me know if you need a hand or have trouble making it work with Robocode. Then I'll support you as much as possible! --Fnl
I need something like this for running lots of seasons of 35-round matches. If it gets integrated into the main Robocode release, that would be even better. Would you mind sharing the code? -- Skilgannon
Do you not use RoboLeague? I can't live without it in development. Actually, I remember it didn't work right with some newer versions of Robocode, but I use it with 1.07. If it still doesn't work with Robocode, that should really be something Fnl or one of us take care of. -- Voidious
- Yeah, RoboLeague does everything most people are after. I have a few special needs, like restarting the JVM between battles. Other than that, I thought saving all results to a database would be handy. Also, if I can get it to work, the distributed environment should be very powerful to anyone with access to a computer lab :). But yeah, I can shoot you the code I have so far. If you're interested in helping we should set up some kind of collaboration environment. -- Simonton
I've never tried RoboLeague. Besides not being compatible with the latest version it looked like a mission to set up as well. I'll grab a copy of 1.07 and give it a try....is the source code for it available?
- I think the source is packaged with it, but I'm not sure on the licensing if it is. And hey, it's Robocode that grew incompatible, not RoboLeague. =) It's painless to setup and I bet 95% of Robocoders use it regularly. Not to say you two can't do better (you surely can), but it's still a super useful tool available right now. -- Voidious
- Hmm, all clients that uses/runs Robocode should use the robocode.control package and access the RobocodeEngine? class. Clients that does not do this will not be able to keep up with newer versions of Robocode. It would be quite hard to make changes to Robocode. The robocode.control package is meant for Robocode clients and is always backwards compatible. If something is missing in the robocode.control package, I will be happy to extend it. ;-) --Fnl
@Simonton, how about merging it with the SourceForge? Robocode project? Things like restarting robocode each season could be set as an option, I'm assuming? What are you using for database access? SQL/ODBC? Also, is it written in Java as well? I'd like to try a bit of Python, and this sounds like something fun, but even if it's in Java I'm willing to lend a bit of my time. I just have to remember that I'm also meant to be studying =) -- Skilgannon
- I'm not sure about merging with the Robocode project. I don't plan to integrate with Robocode's source at any point in the future - only through the command line, .battle files, and .results files. So I don't know if that means it should be some separate project? But anyway, for a database I'm using [HSQLDB]. I don't know anything about sourceforge ... could we open a separate project for it pretty easily? -- Simonton
We could make some options in Robocode to support this project. I will support you with whatever you need. --Fnl
- It relies on command line arguments, .battle files and .results files, so as long as they stay the same / backward compatible I'll be in business. Do you think any of these will ever change? -- Simonton
I am wondering. How far is this project? Is there any progress? (I am just curious ;-) --Fnl
- I just finished getting it to run distributed-style. My test run is pitting the latest pattern matching against all 5 random movers as I type. It's highly un-portable, though, I'm sorry to say.
It is not currently saving to a database, though I think that functionality is in place, I'm just not activating it (if I remember correctly). That's it so far, and none of it's pretty to work with. -- Simonton
Ok - why does robocode run WAY slow when I use a copy of the jar in a directory other than the one in which I installed it? It takes a very long time for it to do something between battles. -- Simonton
- Could it be building .robotcache? --Skilgannon
- Oh, sorry, I meant between rounds. -- Simonton
- I've got no idea, but you could try a process of elimination. Do a fresh robocode install, test. Delete seemingly unnecessary files one at a time, test again. If you find it before Fnl (if he knows the reason) replies, it is time well spent =) -- Skilgannon
- I did that for a while with no fruit, so I just adapted RoboResearch to use a full install :) --Simonton
- Ok, now my normal install is doing the same thing. I upgraded to 1.4beta2. At first I just grabbed the jar out if it & put it over my current install, but then I started getting this "it takes a very long time for robots to die" problem. It's not slamming the hard drive, and it's not slamming the processor. So I ran the whole installer over top of my old install, and that didn't help. I installed in a fresh directory, and it works fine. Is anybody else experiencing anything like this? -- Simonton
- Ah HA! It has to do with the CPU constant. When I set the CPU constant really high, it sometimes waits for that long when a robot dies. -- Simonton
Alrighty, I just submitted an application to open a sourceforge project for RoboResearch. I hope you were serious about helping, Skilgannon! --Simonton
- I'll try and dig up my old SF ID, and yes I am. It might take me a few days to get the layout of the code and find time, but I'll definately help with it. -- Skilgannon
- Are you sure... --Starrynte
- I'm kinda busy with Waylander at the moment (and exams...haha), but I've messaged Simonton to add me as a developer. I'm just reading through the code now. -- Skilgannon
Oh, Fnl, I just found one more request this project could ask of RoboCode: make sure that when "-nodisplay" is used, it never tries to open *any* kind of user interface (except the console). I can't run the newest version distributed over the network; it's giving me the "can't connect to your computer" message again (since I'm behind a firewall, if it tries to open a connection back to my computer to display a window or anything like that it fails). I thought maybe it was because of the new human-interaction capabilities, so I installed version 1.3.3, and that one works great. -- Simonton
![[Home]](/images/RoboWiki.png)