|
Another thing is that I want all involved in this to have a fair picture of the commitment I make to keep the server serving this project. * Firstly the "server" is not really a machine built for server purposes. It's a desktop minitower that's built to be silent. I have it in a little room here some foot above the floor (so that any water on the floor won't leak into the computer). I have a tape backup running every day and try to remember to cycle the tapes once a week and also remove the old tape from the house so that a fire or a theft should mean at most a week's loss of data. Since it's a desktop system it's not SCSI disks there. It's one IDE disk. This means that if it breaks it will take a while before I get the server services running again. This actually goes for all major components. Rest assured I will make my best to have the wiki and the rumble services running as stable as I can. * At the current I am not nowhere close to tiring on this Robocode addiction of mine. But in the unlikely event I would tire some day I will keep the services to the community running. And if I ever feel I don't want to do that I will give you all due notice and send the databases and scripts and whatever is needed to whomever volounteers to keep the thing running. I say it again, this is unlikely. I enjoy nursing the server too much. =) What we could do to make a stop in the services (due to hardware failures of any kind) mean as little loss of data as possible is to mirror the databases and crucial data somewhere on a regular basis (maybe twice a day or so). Anyone wants to act a mirror just say so. It's good if you are running Linux or MacOSX or some other system with the rsync tools because this I have on the server and this will allow us to consume as little CPU and bandwidth as possible for the mirroring activity. -- PEZ davidalves.net / mandatoryexplosions.com at your service :-) --David Alves Thanks! I'll get in contact with you and we can test the sync procedure. If you haven't received an e-mail from me in two days or so, send me a reminder. =) -- PEZ |
Some suggested solutions:
This has been discussed elsewhere. It was decided that we'll live with this shortcoming until a good solution hits us in the head. As Roboleague resets the data directory when it starts I think that ER has a similar problem. It's the same conditions for all bots and this will level out the impact on the rankings I think. Your first suggestions there will work in the wrong direction from the goals of the projects and will add a level of complexity that costs more than it tastes I think. The third alternative is boring. =) I say we keep living with it. Though solution two might be feasable if we add a timeout for the locks. But it would still be a bit too complicated to my taste. Let's deal with this problem when we have stabilized the current version and when we do not think we have more important/fun things to add. -- PEZ
In all likelihood the vast majority of matches will be run by clients that are frequently in use, with a small minority being one or two time clients run by people curious to see how our system works. That said, these people are probably also running testing on their computers, so it seems most likely that the bots will all have extensive data directories of their own built up within a few days (if they don't already) on the clients that have the greatest effect on their stats. So maybe this won't be a big issue. -- Kuuran
I like idea 2, but it isn't needed yet. Maybe when we get as far as releasing a real version (rather than alpha's and betas) we can think about it. -- Tango
@Kuuran: Unless they're like me, and have different robocode installations for 1-v-1 testing, melee testing, minibots, microbots, nanobots, robocodeGL, robocodeGL development version, robocode 3d, robocodeGL 3d version, etc... :-P
You could always copy your megabot 1-v-1 testing dir into your new roborumble data dir to solve most of the issues :P I also use multiple installs, but only for different versions of robocode (GL, 3D, classic, soccer, etc). I think for the most part the data dirs will populate themselves enough just from the roborumble runs that the problem will solve itself before we could implement a sane solution. -- Kuuran
Displaying results
I think we might have a coming problem with the result files for each bot getting really huge. We must roll them in some way or it will take forever to process the files and I guess they will consume unproportionally much bandwidth and hard disk space too. It already takes almost a second for my blastingly fast web browser (Safari) to load the details page of Marshmallow. -- PEZ
Could you summerise the data for old battles? Keep all the rankings etc. the same, but only store detailed results for the last 50 battles or something. To summerise you could try username, opponent and total scores. That would only mean the problem would occur later, but it would buy time. 1 second isn't bad. Once it starts getting over a minute we need to do something. -- Tango
When it takes a minute to load a details page I will have shut down the servlets long ago. This need to be addressed now. Summarizing data for old battles sounds like the right way to go. We can save detailed data on the last 100 battles or so. Why would that become a problem later? It would set a max size of the files quite effectively. -- PEZ
To reduce the size of the results pages:
For what it's worth, I Agree with PEZ. -- FnH
If we break it down by user it could again start to become an issue given enough users, but I suspect that's just wishful thinking :) -- Kuuran
I'm working on it. You will be able to specify the number of battles you want to display (we can set it to the last 100 battles). Also, I'm adding some sumarization functions. -- Albert
@PEZ: I just uploaded the modified server classes. It displays the battles in groups of 25, and automatically creates a link to display the next 25, and so on. Also, it includes the first of the sumarization functions which shows your bots WhiteWhale (and for who you are a WhiteWhale). The details page also includes an automatic link to this page. -- Albert
Cool! I already knew Frankie was a WhiteWhale of Marshmallow though. =) This takes care of some of the processing problems and all of the bandwidth problem. Though I still don't like that the data files themselves grow infinitly. I think we must solve this in the upload servlet somehow. -- PEZ
OK, I'w change it so it stores only a number of battles, and deletes the old ones. Which number do you think is correct? -- Albert
As long as there's a stop somewhere I'll be calm. =) How about 150? That way it would be six pages with the current settings. Anyone wants more than that? -- PEZ
150 sounds good. As this project is supposed to be about sharing the work, could we share the data files? ie. pages of 25 after the first 150 could be made into static HTML files, and transfered to other peoples servers where they can be linked to by the wiki page. -- Tango
Are you really going to use all that data? sounds like you've got OceansOfSpareTime :) -- FnH
I don't like the idea of getting rid of data at all if it can be helped. You never know when you might need it. I think the WhiteWhales should be set to only show if the rating change is more than a certain value. You are rarely going to perform exactly as expected. What about +/- 5 for now? It can increase once the ratings stabalise. -- Tango
Perhaps the server could have 'listeners' that get all the data the server gets as well (also through posts for example), so that other people (like Tango) with more HD-space and/or other interests in the data can use it? Problem would probably be the server the user would need to provide to receive the data ... -- FnH
I still prefer static HTML pages of the results being stored on random servers. -- Tango
WhiteWhales only appear if the "smoothed change" has an absolute value greater than 1.5. The "smoothed change" is calculated using an exponential smooth with alfa = 0.2 (ie. lastchange * 0.2 + smoothedchange * 0.8). If it's not working like that, then just let me know, because there is some problem :-) -- Albert
Just thinking... Each battle takes about 150 characters to store (it could be reduced to 100). PEZ, what limit do you think is good for a bot? Then we can calculate the number of matches. Note but, that if you want some history "per enemy" you will need quite a big amount of data, since the average history available will be battles/participants. -- Albert
It is working like that, i think. I just think the 1.5 should be higher. -- Tango
Another option is to go like the ER and just sum up all results between two bots. -- Kawigi
Yeah, that would work. Then you would just have to store one battles worth of data for each enemy. -- Tango
You must be able to handle separation Tango. =) I'm more worried about what Marshmallow is still doing there down on rank #42! How about sorting the WhiteWhales lists? I would rather decide on a number of battle details to save based on usefulness rather than on space consumed. 150 would satisfy my need for details for sure. And we could also save the sum of all fights against each enemy bot. That can be a quite entertaining list to read.
I must be able to what? :-/ I don't understand. For a start, what was that a reply to? Sorting the WhiteWhales is a good idea. 150 battles and then per enemy summeries would be fine by me. The only information lost is stuff that is purely random anyway, so of little help. -- Tango
Sorry, bad joke maybe. It was in reply to: "I don't like the idea of getting rid of data at all if it can be helped." -- PEZ
I think whe should add some check to prevent what is happening right now with DT and Sedan. Posting the same match once and again artificially inflates the rating of the winner bot and deflates the rating of the looser. Because they don't compete against other bots, the difference just increases and increases... something like that could make the second worst bot become first by just fighting aganinst the worst bot.
I have no idea on how to prevent it (suggestions are welcome), but for now, when all matches are posted, I'w make a manual entry to adjust the results of DT and Sedan to the same rating of the 3rd bot. -- Albert
There is no need to change anything. It is still stabalising results. It will even out as more battles are played. The ELO system stops things like that really mucking up results. Just winning doesn't increase your ranking. If you do as expected (as they will have done after a while) your ranking doesn't change. -- Tango
I say that, but there is the matter that for some reason they were run as 50 round battles. Get the server to ignore all non-35 round battles, and get it to recalculate the rankings, and it will fix itself. -- Tango
It can't recalculate ratings. They are done on-line when results are uploaded :-(
Can't you run the whole database through the system again? Is the database not stored in the same format as a results1v1.txt file? Doing that would also get rid of all the 10 round battles from before the default was changed. -- Tango
What's wrong with 50-round battles? Gives those bots that don't save data more of a fair chance, since the initial 10 rounds or so of adapting to their enemy don't affect their final rating as much.
I'd say that when we have got this thing stable we can wipe the entire data directory and make the clients start over from scratch when we go live for real. -- PEZ
Yeah, I agree with PEZ. We are in testing period, aren't we? -- Albert
Wiping everything before we go live sounds like a very good idea. -- Tango
Except that we'll have to wait for stable rankings again, but I suppose that will come. And the second-worse robot couldn't get to first by beating the worst several times, unless it beat the worst robot by a gigantic margin, or a larger margin than the best bot would be expected to beat the worst bot. But the fact remains that Sedan would probably look a few places better right now if DT had played more rounds against the people between DT and Sedan :-p -- Kawigi
I disagree, I don't think Sedan would be in a better position. Sedan has been between 1725 and 1735 every time I've checked for the whole duration of the beta. Right now it's at 1729. --David Alves
Fact remains that your DT vs Sedan session caused Sedan to be pushed down the list and DT to climb more rapid than it would otherwise have done. I think we will see Sedan among the top-3 when the ratings have settled. Right now DT is unbeatable and then comes BlestPain and Sedan who lose against no other bots than eachother and DT. -- PEZ
I agree. Please, stick to the designed process, or we will not be able to test it. If we feel robot ratings change to slow, then lets recognize it and change how the rankings evolve (that's why we are testing). All bots have the same right to get its fair rating, no matter if they are good or bad, and artificially speeding up a bot rating creates only suspicion (the most perjudicated bot right now is DT, because it's not clear it should be there, even if it would have arrived for sure). -- Albert
Is the slow down caused by the number of connections, or is it caused by the big datafiles for each bot?. Note that now, every time a battle is uploaded it opens and loads a big data file, and that can be time consuming? Would reducing the size of datafiles help to improve performance? If so, I propose to first change the rating system to a model using accumulated (or rolling) scores, then check how performance is affected, and then, if problems persist, change to a centralized model. Note that developing the proposed model can take some time (now the netengine has 313 lines - excluding external libraries, and the battles engine has 355 lines, for a total of 668 lines, and that makes maintenance easy :-) Also, I noted that my browser takes a long time to download all the flags included in the rankings. How it relates to server performance? May be we could store the flags in a different server so they are read from there? -- Albert
Let's see where the smaller files takes us. Though I still think it will be something to be dealt with when the number of clients increases. I appreciate your need for maintainability of the code and agree we can take it in steps and only take a certain step if the need really arises. The flags shouldn't be a problem. There are only about 20 flags and it is very small files (200-300 bytes each). Your browser should cache them anyway. I guess the slowliness you experience might have to do with it being a quite large table and I have no "width" and "height" attributes in the html code there, which means your browser might take a little while to figure out how to render the table including all those files. -- PEZ
All of the broken link ones are reloaded on every visit, though, since no valid cache for them exists. It would be useful (and cut down on alot of http requests) if there was a default. Those of us who display changes at the top of all pages are also dying when we load up the results page, I noticed a few times in the results the changes were suppressed for some reason, but most times they weren't, is there anything that can be done about this? Custom settings for that page to not display changes, or something on our end even? -- Kuuran
OK, I just skip the image when there's no flag linked to a "main" package name. Hope that helps. About the changes on top of all pages... I don't understand what you mean. -- PEZ
Enable 'show differences on all pages' in preferences and you'll see what I mean. -- Kuuran
Cheez! I guess the diff feature wasn't added with tables in mind... The wiki code is a bit cryptic to say the least. I guess I could try make an exception for that page. ... But I can't even see where it makes the exception for the "Changes" page. I'll dig some more and maybe it can be fixed. -- PEZ
Cool. If it doesn't work out then I'll live, but hopefully you can do something about it. -- Kuuran
I hardcoded it so that diffs-on-all-pages doesn't include pages maching "RoboRumble/Rankings". Ugly as hell, but I almost never look at that code anyway so it could as well be ugly. =) -- PEZ
Cool, much appreciated! :) -- Kuuran
It did before, but from the new release, it checks the participants list only at the begining of the sesion (I did it to reduce trafic). We can reverse the change if we feel it was better before. -- Albert
If it is possible for a new participant to be in a battle list, but not yet downloaded by the client causing a crash then the client should check if there are new participant each time it gets a battle list - otherwise no problem - or check every 4 hours. -- Paul Evans
It is not possible to have a bot in the battles list which is not in the participants list, since the battles list is built from participants list. Also, it checks for the necessary .jar files before creating the battles list, so if you have it in the participants list, but you don't have it (ie. problems in the RobocodeRepository) downloaded, then it will not appear in the battles list. -- Albert
Sorry, I assumed the server issued the battles list -- Paul Evans
Do the clients make any attempt at running battles between robots of similar ranking? It appeared as if Sedan hasn't actually battled DT yet (which I thought was curious) -- Kawigi
At the moment the server doesn't send any info to the client. The only download is the participents list from the wiki page, and the bots from the repository. The server really should set the battles list in a later version. -- Tango
![[Home]](/images/RoboWiki.png)