We can use this server (same as runs the wiki) for this. It's not a very fast machine, but it doesn't sound like performance of the server is critical. I will not have much time for this myself though. -- PEZ
Maybe its possible to diribute the load of the server to other grids who are wolutered. Moreover we can get the bots from the direct repository. Technically speeking Java has some grid computing classes. May be its possible to develope such a system. SSO?
The server just needs to store the results into a DB, and then read them periodically and update the rankings. I think it will not take to much CPU. -- Albert
Data files could be stored on the server and sent to the clients as well -- FnH
Why do the rank updates have to be done on the server? Why not upload new ranks along with the match results? The server could just give a list of the next 10 matchs to the client, which runs them (as automated as possible, to avoid mistakes and cheating) and then the client calculates the results and new rankings and upload them. The only problem is people running matches simultaneously, but as long as the current rankings are downloaded after the matches are played so the time is as short as possible, it should be ok. -- Tango
Well, I've been inactive for some time, but rumour of a final minibot has reached me. It's a pity I didn't get a chance to update a new NanoSatan for it, I had a few new things that could've squeezed into him. Anyway, the central server should maintain a database of CRCs for both the client program running the competition, and for each individual bot. Thus, no one renames SandBoxDT? locally and pumps their score, and no one hacks the client. Anytime communication is done, do a CRC verify, anytime bots are run, do a CRC verify (this also helps to ensure version uniformity). Now, this could of course be forged with precalculated CRCs, but it should cut out the bulk of people going 'hey, I could try it.' and just leave the hardcore cheaters, who hopefully have been run out of the community by now ;) -- Kuuran
I must admit, i don't really know what a CRC is, but from context i think its a good idea to use them. -- Tango CRC = Cyclic Redundancy Check, most commonly known as the way to check if a ZIP file is damaged.
The only way to discourage hardcore cheaters would be to do random checks of the results and then, when the results don't match ban them (we'll probably need to introduce something like accounts to make the ban stick though) -- FnH
Let's not worry too much about cheating. The Robocode community has so far not contained one such to my knowledge. And if a bot gets a high ranking we will be many testing against it and we will for sure see if it does not perform up to its ranking. As I see it the potential problems with this lies elsewhere. Roboleague is a bit unstable as it is and to get this automated this could be a problem. When it comes to design of the distrubuted rumble I think we could maybe peek some at "The All Seeing Eye" thingy used for ranking in other computer games. -- PEZ
I think the first order of business is to decide what it is that we are looking for. There are some great ideas posted above and we should finalize that list (RoboRumble/RequestedFeatures). Make them pie in the sky for now. Please avoid discussion of details for a few days as we decide what goes in. Once we have a finalized list of features we can start to look at what tools possibly exist. I will do the best I can to try and push some people into helping, but I can in no way do this all myself. If you can help please do so. -- jim
Robocode isn't a consistent thing, you can have fluke wins, enforcing consistency denies this and is overall not terribly helpful. I was thinking about this, however, and realized that all rank changes in a bot deemed to be stable (ie: version not changed on the server in the last x days, where x is settable in the options until we find a good number) of more than a certain amount (variable y, also settable in options) flag the results and don't integrate them immediately. These results can be personally viewed by one or more people to verify their plausibility. Basically, if I'm uploading results where my bot is always number one, you can assume I'm cheating, if I'm uploading results where Kawigi's bot fluked on DT or whatnot, it's quite likely a legitimate fluke. It wouldn't be hard for people to distinguish legit from cheating. Sure, you could still cheat if you upload slightly fudged results, but all that effort for one or two rank spots is really pathetic, and if you're going to do it at that point, I don't much care anymore. This method of using a program to filter down to a few suspect results that a human then reviews seems like it would work nicely, overall, in combination with file verification, etc. -- Kuuran
I am inclined to say that if this is indeed a concern, then we put hooks in to any developed system for this type of checking and delay the implementation of this process until it becomes absolutely necessary. It may sound a bit too ExtremeProgramming? for some but I would think that if Joe Nobody came a long with MyFirstDTSmasher? everyone and his brother would be downloading this bot to see just exactly what it was he had created. I think the community would quickly ferret this sort of foul play out and route around the event. If it is indeed a problem, then we could work later to implement this, but I feel that to be successful we need to focus on the goal of a working distributed system first. -- jim
Most of the talk here has covered the project goals, requested features and the need to stop cheating. I would like to talk about something which is not technical, but is key to to success. We need a project manager, ie, someone who will keep us focused on the goals, stop scope creep, make sure that the contributors to the project do what they've said they'll do, and maintain the framework into which we all contribute.
I tried project management at work and found that it was like herding cats, so I'm not volunteering, but I raised the idea because I don't want to see RoboRumble@home turn into yet another piece of open source abandonware.
-- Rod Hyde
I like that you seem to have gotten momentum up for this project already. Jim seems to be a natural for this kind of role. And since Albert started it all and and you, Rob, have good experiences from project management I would suggest you three distribute the management amongst you. If it was meant to be this project will deliver. If not, it's better all gone. =) My profession is project management, but I hardly have time for my robots so I won't pretend I will have time for participating very much to this project. Other than with my creativity. =) -- PEZ
I think we should abandon roboleague (to get rid of the gui) and start from the bottom. The competition engines from roboleague could still be useful, though.
Let's see if we can work out how project management is done the wiki way :-) I have given it a start with ProjectRoboGrid
tobe, this is excellent feedback. I would love to learn from your experiences. If anyone can get amarok involved in the discussion that would really helpful too. I agree that an option to run the league without the gui should be available. -- jim
I would not dismiss RoboLeague so quickly. The source of RoboLeague is available, so it should not be too difficult to get it to run without the gui. Significant effort has been put into RoboLeague by Christian. It would make this project much lengthier if we had to rebuild from scratch, increasing the chance to get the status AbandonWare?. Do we have enough people for that? I personally would not take that risk. -- Vic
I suggest we just get rid of the roboleague gui then. I have wanted that for very long since it's often in the way anyways. Could we make a quick experiment if this is easy or hard to do? I think that what's actually can become the problem is the Robocode gui, since I don't know if Roboleague just keeps Robocode hidden or what. -- PEZ
The API robocode has for things like roboleague interfacing with it has a call to hide the client I believe. I'm going to dig up the link -- Kuuran
Here you go: [http://robocode.alphaworks.ibm.com/docs/control/index.html]. The RobocodeEngine? class defines a setVisible(boolean) method. -- Kuuran
I have been trying to think of a way to frame the argument for keeping Roboleague for most of the night. I am glad to see that I am not alone in my desire. I agree with tobe that there needs to be a way to run Roboleague without the gui (crontab anyone?) but I am not prepared to throw Roboleague out. First and foremost it does 70% plus of everything we want right now. If the process to query the repository to update new bots could be automated to occur at run startup then a league of static competitors could be run with the push of a button. That does not seem like it should be all that hard. If we could change the interface for selecting Robots to use in a league to browsing the repository rather than the local file system then you would be able to update the competitor list using all bots available in the repository. The functionality to check the local file system for the most current bots before starting would then retrieve the new bot and your league would be "static" again until someone asked to add a new competitor. If those two things were done I think that running a league, while not completely automated yet, would be much easier. If we could get to that point we could then focus on removing the human from the loop. This would most likely involve some mechanism for manipulating the league xml file. If it were a clearly defined API/Java? Interface then the implementation could be done in any of a million ways. We would then be at a point where a single CPU could run an automated league and the gui could be supressed. We could then step on to distributing the system. I can not see that far into the future right now but I would imagine that it would revolve around the league xml file, maintained by a central server. The server would pass out "leaguelets", which are "neighborhoods" within the overall competition being run, to be processed by the various nodes. The nodes would be responsible for running the the leaguelets and returning the results to the central server. The central server would be responsible for making sure that every thing got done and that the results were published. Publishing the results in some raw form (XML?) would even allow us to use some alternate scoring mechanisms (XSLT) or some such.
My biggest fear is that any attempt to redesign from the ground up will result in nothing ever getting done. jim?
I wholehartedly agree with jim here and strongly vote for using RoboLeague. An additional idea to make things even simpler would be to force the owner of a robot to upload a new or updated robot to the server. I think that's not too much to ask and it's simple to implement. As we've found out before (although none of us tested this), RoboLeague then deals with distributing the jars to the clients. -- Vic
If by server you mean the Roboleague Central server, then I would think there is no need to feed a jar to the client. If you kept using Roboleague (modified as I proposed above) all the server would need to send the client would be the file (xml?) describing the "leaguelet" to run. The client would then need to download the missing robot jars and run the leaguelet. -- Sparafucile-jim?
Default behaviour of RoboLeague is that it DOES send jars to the client, so there's no need to change that. I think that will make this even simpler to make. Only the server needs to have an up-to-date set of jars. -- Vic
I agree. What if the repository went down? If all the clients were downloading bots from there no one would be able to update. If the bots came from the server, then someone could update it manually when people email the new bots to them. -- Tango