A Melee ranking system you're welcome to jump in on

********


Ranked Contest Rules:


- Standard 1v1 play: 4-stock, Handicap OFF, Damage Ratio 1.0, Items OFF, 8 Min

- All levels are selected randomly throughout the contest.

- The following stages begin turned on - Final Destination, Battlefield, Dream Land 64, Yoshi’s Story, Fountain of Dreams, and Pokemon Stadium. Each player may then either ban one stage or protect one stage from being banned. Flip a coin. The player who wins the coin flip chooses whether to ban/protect first or second. Past this, any other stage may be turned on or off so long as both competitors agree.

- Both players must create new names prior to beginning play.

- Players remain the same character throughout each contest and, furthermore, obtain a different rank for each character individually.

- Contest ends after the match that concludes with at least one player at 50 KOs (stars) or more (this means that the greatest KO count possible in any contest is 53, which could occur anytime the final match begins when at least one player has exactly 49 KOs).

- Following the contest, photograph the statistics for the brand new names created to play the contest. Each player's statistics must be photographed twice - once scrolled all the way to the top, which will provide all the relevant data, and once scrolled all the way to the bottom, which will prove that only one character was ever used in correlation with that name. Email these screenshots to smashranks@gmail.com.

(Screenshots of all the contests we've completed thus far can be found here.)

- Each player’s rank is weighted based on his/her win percentage throughout the contest (which determines overall win/loss) and the final KO Spread which is derived from taking the final difference in player's KO (star) counts after subtracting one for each SD throughout the contest (this is the final score).

- If either player finishes with a final KO spread less than -20 or fails to win at least one match, the contest will not be weighted and that player is considered skunked. The screenshots can still be submitted, however, and the data will still be applied to each player's personal stat sheet.

(Screenshots of current player's stat sheets can be found here.)

- In order to obtain rank for a character a player must submit ranked contests for that character against at least 5 different players.




********


Regarding our ranking system:


The goal of this is to create a ranking system that weighs every Melee player against each other using elements of existing ranking systems such as Glicko and TrueSkill. Either of these ranking systems produce each player a numeric value, similar to a chess rating.

Aesthetically, however, the ranking system I’ve chosen for this is a Kyu/Dan system. They use it in Karate and other martial arts as well as other fighting games. For those of you unfamiliar, a low rated player in this ranking system would start at a high-numbered Kyu, such as 20, so a 20k. That player would then work his way up toward 1k. From 1k, the next rank up is 1d, switching the player from the Kyu (amateur) side of things to the Dan (master) side of things. Once on the Dan side, the player would rank up to 2d, 3d, etc, until you get to the best of the best. Were people of Mango or Mew2King’s stature to obtain a rank, I imagine it would land right around 7d (seventh degree master). This means your initial numeric value will be converted based on whichever certain range it falls into to something between 20 Kyu and 7 Dan.



********


If you’d like to participate, simply do so, and email your resultant statistic screen shots to smashranks@gmail.com. We will add your results to our ranking system as well as a personal spreadsheet for you and your opponent. Please follow all the rules.



********


Current player's stat sheets:

Arie, Axe, Burnt Toast, Cuba is Death, DJ Mirror, Dunk, Elesdi, Fox, Guinto, Hella, J-Man, Julian, KOS Kirby, L, Lucien, Mackadotious, Mr. Ganondorf, Sam, Sausage Fest, Signia, Silent Spectre, Smoke 2 Joints, T, Taj, Tang, Tyrael, Vans, West Ballz, Zelgadis, Zhu

so this is like a national rankings system? you might want to make it simpler if you want lots of people to participate... lol

how would this provide more detailed information than, say, local power rankings and national tourney results?

Interesting.

john! said:

so this is like a national rankings system?

Click to expand...

National? Global? Ultimately the goal is to churn out every (participating) player a rank the same way yahoo! Chess does.

john! said:

you might want to make it simpler if you want lots of people to participate... lol

Click to expand...

Done.

john! said:

how would this provide more detailed information than, say, local power rankings and national tourney results?

Click to expand...

I would argue that local rankings are just that, local. And tournament results are not a particularly specific indicator of how one player compares to the next. On a server like yahoo! Chess, anyone can jump in, play a few games, and get an idea of where they stand against everyone else on the entire server. The goal of this is to make Melee like that.

EDIT: Rearranged this post for consistency's sake now that things have changed.

not counting sds is ********

At first I was thinking that not counting SD's was ******** too... But then I realized that if you SD, you lose no stars, but you prevent your opponent from getting one. I could get 100 to 0 stars every time if I just SD'd every stock.

That being said, what is the point of this again? Just to see how good a certain character of yours is vs a certain character of your opponent's? The recording rules are dumb, imo, I feel like the word of both players preset should be good enough, I don't see why this has to be that serious lol.

This is terrible. If Melee were to have some sort of rankings, it'd likely be best to do it the way Rajam does for Brawl.

slartibartfast42 said:

At first I was thinking that not counting SD's was ******** too... But then I realized that if you SD, you lose no stars, but you prevent your opponent from getting one. I could get 100 to 0 stars every time if I just SD'd every stock.

Click to expand...

This. Exactly this. You must understand this about the game to understand why the KO spread is designed the way it is, so if you don't, please reread what is written here until you do.

slartibartfast42 said:

That being said, what is the point of this again? Just to see how good a certain character of yours is vs a certain character of your opponent's? The recording rules are dumb, imo, I feel like the word of both players preset should be good enough, I don't see why this has to be that serious lol.

Click to expand...

Okay, this isn't just a way to rank yourself against only the people you actually play contests against, apparently I'm not stressing this enough. When you first start to play a board game online such as Chess, or Scrabble, or Go, which is what I'm trying to make this like, you begin as an unranked player and must play a certain number of required ranked matches in order to obtain a rank. Once you have obtained a rank, even though you only played a handful of people, the server's math is ranking you against every other person on the entire server. Again, that is the goal here. If we were already all ranked players, then someone brand new to Melee could jump in, play the required ranked matches, and then immediately know right about where they stand against every other person who plays this game.


EDIT: The recording rules have been scrapped.

john! said:

so this is like a national rankings system? you might want to make it simpler if you want lots of people to participate... lol

how would this provide more detailed information than, say, local power rankings and national tourney results?

Click to expand...

Yeah, I couldnt be bothered to read all that..

So what happens when someone submits a video of them 4-stocking their friend who sucks over and over? Won't he be considered the best compared to someone like Mew2King who only has videos against other competent players and at least loses 1-2 stocks per game?

I'm down, this might even get me playing smash again.

See if I can get dat Peach ranked.

I'm hoping that this might turn into something big.

interesting idea. why not use elo ratings though?

tarheeljks said:

interesting idea. Why not use elo ratings though?

Click to expand...

q.
F.
T.

We know that certain players are better than others by tournament success. But what we don't know is how much better/worse they really are? What's the actual difference of skill level between two players? Its too easy to say, "Player A is superior than player B."

This ranking system can achieve that. It may not be the most efficient way but the idea is still in its infancy. The biggest question is getting everyone to participate and getting ranked.

From the stat screens you can tell that Burnt_Toast is better than me. Stats don't lie, we played the matches and got the results. The big picture is that I know where I stand. Can you imagine if there was hundreds possibily thousands added into the system?

**** u ******, suck it rofl

Volroth said:

We know that certain players are better than others by tournament success. But what we don't know is how much better/worse they really are? What's the actual difference of skill level between two players? Its too easy to say, "Player A is superior than player B."

This ranking system can achieve that. It may not be the most efficient way but the idea is still in its infancy. The biggest question is getting everyone to participate and getting ranked.

From the stat screens you can tell that Burnt_Toast is better than me. Stats don't lie, we played the matches and got the results. The big picture is that I know where I stand. Can you imagine if there was hundreds possibily thousands added into the system?

Click to expand...

right, i understand the goal of the system and didn't mean to suggest that you are wasting your time or something like that. i like the idea and am pointing out the elo rating system b/c it measures the same thing and has been developed (someone even mentioned chess itt) so it would be easier to implement

also, i see violence has found this thread-- i know he proposed instituting the elo system for power rankings in norcal

Bones0 said:

So what happens when someone submits a video of them 4-stocking their friend who sucks over and over? Won't he be considered the best compared to someone like Mew2King who only has videos against other competent players and at least loses 1-2 stocks per game?

Click to expand...

This is a very logical question, and I was counting on someone proposing it eventually. The answer is no, of course, otherwise this whole system (as well as countless other ranking systems) wouldn't work. The reason the answer is no is because, once again, the math correlates each of us to every single other one of us. (In fact, this even means that because characters are ranked individually, even your own characters [if you were to rank more than one] are weighted against each other.) This means that the system would "know" that Mew2King has beaten people, who have beaten people, who have beaten people, who have beaten people, who have beaten Mr. Four-Stocks His Friend A Lot. As a result, Mr. FSHFAL would acquire a significantly lesser rank.

I also saw a couple of responses about ELO ratings. I'm vaguely familiar, but not super uper duper familiar - are these ratings capable of rating every player who wishes to participate against every other player who wishes to participate? If not, that would be the main difference between it and this.

EDIT: I looked up ELO ratings. I'm quite familiar with them as it turns out. I'd been checking the math used on the Scrabble and Go clients I use and was using that as a basis for my own formulas. Neither of them mentioned ELO, but after looking up exactly what that was, I found that the math used on both my Scrabble and Go clients are just slight variations of the standard ELO rating. This will be as well. The biggest difference is that once given your normal three or four digit rating, it will be converted based on whichever certain range it falls into to something between 20 Kyu and 7 Dan (just as my Go server does, as it also uses a k/d system).

There are two reasons I'd prefer to do it this way. The first is that ranking up "feels" far more significant when you don't watch your rating climb up by 6 or 7 points with every victory. To put that in context, what I mean is that if the range to be rated a 6k is 1000-1100, and you enter the ranking at 1016, then instead of watching your rating slowly climb (1023....1030...1037....etc), all of that would happen behind the scenes, and then once you breached 1100 you'd see yourself rank up all at once from 6k to 5k.

The second reason is that, to me at least, it just feels far more bad *** to say "I'm a 4d" than it does to say "I'm rated 2163".

Sweet idea, but I think a step-by-step guide on how to set this up would be helpful. Obtaining the rank itself seems to be almost prohibitively tedious. 500 singles? That's going to take a while, even for people who play on a daily basis.

I think the barrier for entry into this needs to be lowered, and the process needs to be VASTLY simplified. As I understand it, this is a ranking system that you hope everyone will adopt, right? Problem is, the more people you want to adopt your standard, the more accessible and easy it'll have to be.

This is a kind of idea that relies upon a lot of people using it in order to persuade others to use it. But at this time, since nobody else is really using this system, there's no incentive for people to use the system because there's nobody else to compare themselves to, even if they DO manage to puzzle out the arcane instructions.

EDIT: What are ELO ratings?

EDIT: Pretty much everything I had written here is different now.

The ELO rankings are outdated. There are systems of rankings that are much better at predicting the outcome of matches between two players.

See: http://kaggle.com/chess?viewtype=leaderboard

You may want to try some of their algorithms. They are public knowledge.

Bump-worthy revisions made. Also, thanks GofG. That's given me quite a bit more to work with than I had before.

I dont even understand how you are expecting people to be matched up.

Yo Burnt_Toast check your PM, I need your #

Sveet said:

I dont even understand how you are expecting people to be matched up.

Click to expand...

You don’t need to meet up with any more people than it takes to get ranked. You will be connected to everyone (eventually).



Let me try to explain with an example that includes actual people. Hella and I have talked about doing one of these soon, and I can probably gain rank against Hella (I already know he’s going to try to skunk me, and now that I’m posting this, ****ing even more so for sure haha). But let’s assume that I can. Hella, in turn, can gain rank against, Zel, Germ, etc, who can gain rank against Silent Spectre, The King, etc, who can gain rank against Mango. Now I am officially connected through the math to Mango. Everyone I play ranked contests against is also connected to Mango. Everyone they play ranked contests against, in turn, are also connected to Mango, and so forth.

Now let’s imagine this is also happening on the East Coast. Mew2King has played people, who have played people, who have played people, and the serious East Coast community has become fairly connected as well. And then at some tournament or something where one or the other traveled, Mew2King plays Mango.

Now everyone Mango’s played and the people they’ve played and on down to me and past me to all the people I’ve played and so on are all now connected to Mew2King and everyone on the East Coast.

And if one of them plays Armada, that connects the US and Europe so long as Armada is actively playing people (who are playing people) in Europe.




Let us remember good old MySpace for a moment (RIP). If you had signed up for MySpace back when it had first started (say when it was 10 minutes old), you would have seen MySpace at a time when not every ****ing person on the planet was in your “extended network”. This is because it took some time to get to the point where once anyone signed up anywhere on the planet, they could add just one or two best friends, and so long as those friends had already been active members with at least somewhat of an established friends list, they were immediately connected to everyone in the world via extended network. This is how it will be for us too eventually.

But because this isn’t something that is actively online such as MySpace or one of the board game servers I keep mentioning, we’re going to be doing a bit more of a manual job as far as getting everyone connected, and it’s going to be more difficult. But we’ll still have the internet’s help. I wasn’t going to divulge this bit of info at this juncture because there’s absolutely nothing here yet, but I’ve registered the domain smashranks.com.


Edit: Actually, there is a bit there now, but still under construction. There's not much there yet that you won't find in this thread.


In addition to providing a place where the leaderboards will be viewable and everyone’s personal statistics will be online and easily sortable, one of our goals with this site will be to provide a zoomable map of the world with a dot on it for every smasher in our system. And if you were to hover over a dot, it would provide that player’s smash name and rank. Furthermore, each dot will be a certain color ranging from blue to red, and this will denote to what percentage of everyone in the system this player is connected.




Just for the sake of providing an example of why any of this **** matters, let’s say there are a group of guys who all play each other in Bum****, CA. They’ve been playing for a while, and they’re not bad (against each other, at least) and they want to know where they stand against the world. So each of them plays the required five contests against each other and gets ranked. The problem is, because they live way out in Bum****, the only people whose math is relative to each other’s (in their cases) IS each other’s, which means that their ranks are hardly valid.

So the Bum****ers would head to our site, take a look at our map, and they’d see themselves as blue dots in the middle of their tiny town, separated from everyone else. So they look around the map for the nearest ranked smasher to them that’s a red dot (which I’m thinking will denote “connected to 98% of everyone in the system or better”), and then they argue about which one of them has to make the trek from Bum**** out to wherever the red dot is and play a single contest against it. This single contest immediately validates all of their ranks.

Also, I want to reiterate the fact that not all of them at this point need to go get rank against a red dot smasher. Once whichever one of them lost the argument goes and gets rank, he will become a red dot, which means that his friends and anyone else they ever play against is automatically a red dot too. And that’s how we spread this: Like the ****ing plague.

I just wanted to say that I am thrilled to be a part of this project and I look forward to it's continuing advancement.

Also, we had some great matches with DBR last week.

Stay healthy gentlemen!

tarheeljks said:

interesting idea. why not use elo ratings though?

Click to expand...

if we're using elo, why not use glicko?

This idea seems pretty awesome, cause i'd like to see where I rank.

The second reason is that, to me at least, it just feels far more bad *** to say "I'm a 4d"

Click to expand...

I like how you said that, since I'm actually a 4d(AGA), haha.

I just really hope you don't use KGS's ranking system as an example, because it's pretty awful.



The idea with the blue dots and red dots are akin to provisional ratings, I think.

Personally, to account for the fact that no rating system can be perfect, I think you need some measure of confidence.

For example, someone who plays regularly with 20 other people, who also play regularly with other people, and so on and so forth, should have a more confident rating than people who play with 4-5 people for most of the time, and then go to a tournament every two months or so. If they're winning the tournament, the amount of matches they've played is much higher than if they lost 0-2, and their rating gets more confidence. However, if their performance is average, they should have a lower confidence than the 20 people who play together more often with a larger web of people.

This happens a lot in Go too. I can go to some other region and meet a guy who says he's 6d and beat him, because all he does is give high handicap games to the same 1k-1d players, and he knows their styles, so he wins. He's only playing the same group of people, so even though his ELO rating is high, the confidence value should be quite low because of the low variety of people he's played.

Does this make sense? I feel like it would fix a big problem which plagues a lot of competitive games, whether they're rated or not.

I mean, I'm sure we've all met the guy who shows up to his first tournament bragging about how he's the "best on his block."

Violence, yes, we have been familiarizing ourselves with several different ranking systems and plan to incorporate different ideas from each of them. A measure of confidence will definitely be included. Another idea I'd like to incorporate is an Anchor system that I first read about on KGS (by the way, always super cool to meet another Go player, I'd offer a game but you'd destroy me. I'm a lowly 6k for the time being).

****

Here is where all screenshot results for contests will be uploaded while we're working on our site - http://picasaweb.google.com/smashranks/MeleeResults#

Currently, the results of the 14 contests we've completed thus far are visible at that link including Hella (Ganondorf) vs Zelgadis (Falco), both of whom ended at 100 KOs, it was super fun to watch.


****

metalreflectslime said:

Interesting.

Click to expand...

tarheeljks said:

right, i understand the goal of the system and didn't mean to suggest that you are wasting your time or something like that. i like the idea...

Click to expand...

Eggm said:

This idea seems pretty awesome, cause i'd like to see where I rank.

Click to expand...

If you're someone who believes that this idea can work, I have great news for you - you're right!

What we need now is more results to help us solidify our mathematics and to get a better idea of just what's in store for us as far as getting everyone connected, so if you're digging the idea of this, please give it a shot sometime soon with someone close enough to you in strength that neither is going to skunk the other and email your screens to smashranks@gmail.com

Also, myself and Tyrael are going to be in Oregon this week (me in Portland and him in Corvallis), so if there's anyone in either of those areas that wants to do this, please PM one of us. This would be a great opportunity to connect the Bay Area and the North-West right off the bat.

Speaking of bats, can anyone get me in touch with Simna? Shoot me a PM if you've got a number or screen name.

Thanks.

Holy ****. I just got done with a bunch of sets vs Toast and this system is the most amazing competitive thing ever. In a tournament, once you feel the match going against you sometimes you slack up, but in these sets you're CONSTANTLY on the edge of your seat all the time because EVERY ****ING KILL MATTERS in the overall set.

This is the best goddamn thing to happen to Melee in 9 years.

Captain Falcon vs Ness
http://picasaweb.google.com/smashranks/MeleeResults#5545793366517151474
http://picasaweb.google.com/smashranks/MeleeResults#5545793361898400162
Captain Falcon vs Marth
http://picasaweb.google.com/smashranks/MeleeResults#5545793734183409298
http://picasaweb.google.com/smashranks/MeleeResults#5545793738846450274
Ganondorf vs Ness
http://picasaweb.google.com/smashranks/MeleeResults#5545795838388535698
http://picasaweb.google.com/smashranks/MeleeResults#5545795839234347970

I think we should have all tournament results put into an online database, and EVERY Melee player can have a ranking.

That would be amazing

Yeah, I would like to do this. I'll try to find out more about this and I'll do it sometime. (with No cash and other friends and my gf)

how would you go about proving the legitimacy of tournament results? what if some unknown in a relatively unknown community put together false results to further his ranking?

Rubyiris said:

how would you go about proving the legitimacy of tournament results? what if some unknown in a relatively unknown community put together false results to further his ranking?

Click to expand...

This was the idea behind the recording and public upload rules that used to exist.

Quite a few people thought that was too much to ask, and after trying it myself, I realized that it was true. These contests last about an hour and half, and the resultant videos from my camera were 4-5 GB.

EDIT: Since being reduced to 50 KOs, these contests have come in at an average of about 40 min.

First I had to find a video converter, and none of that I could find were free so you either have to be cool with piracy or pay money. Then the vid also needed to be broken down into smaller parts for youtube upload.

The clincher though, was that while all this was going on, my computer ran like absolute ****, so it was just a plain pain in the ***.

I honestly don't think people are going to try to fake it that much. Both players must be ranked in order for the contest to count, which means that while one person might be getting a slight rank buff, it'll only be at the expense of whoever s/he's playing against as his/her rank will be adversely affected. Would you agree to that? Me neither.

I'm pretty sure my sister would NOT want to lose to me for an hour, nor would she like to be losing for that long.

@BT: I wasn't mentioning your contest. My post was in response to using tournament results to determine national rankings, like brawl does. I wanted to know what methods they would take to stop collusion.

Ranmaru said:

I'm pretty sure my sister would NOT want to lose to me for an hour, nor would she like to be losing for that long.

Click to expand...

The 100 KO threshold may be lowered, especially depending on how long something like a competitive Samus/Peach matchup would take in this format.

Personally, I've found these to be very fun and extremely competitive, and as such I haven't minded their length, but once a few more heads start doing this and can actually give me an informed opinion, I'll be taking everyone's thoughts into consideration.

The goal is to have the final ruleset worked out before the end of the year.

***

Tyrael, Zel, Hella, Sausage Fest, and T (Volrath on the boards), here are your current stat sheets if you'd like to take a look -

Hella
Sausage Fest
T
Tyrael
Zelgadis

Edit: I've noticed that all these have gotten a few downloads so I thought I'd throw in the most recent version of mine - Burnt Toast. I've been involved in most of the contests thus far so it's got the most stats.

Sorry for the double post, but I did a few things here...

-Vastly simplified the first post and reformatted it.

-Changed the way level selection works (feel free to provide an opinion on it but I'll care a hell of a lot more once you've actually tried it).

-Reduced the number of contests required to 6 but increased the number of different people you must play against to 4.

this is very interesting. but if you want it to take off, you need to talk to some nationally reputed smashers and have one of them make a big, easy to understand post advertising it. something that would catch everybody's attention. (if you're from norcal, i recommend the king. then, if you're homies with hmw, i'd try to get him to post an easy-to-understand video explaining the system)

if this takes off, i'll volunteer to connect japan and america lol. but only if this shows promise and interest from the community, cuz it's too hard to explain all this stuff to japanese players i think haha


also what do you think about lowering the 100 KO's thing to like 50 KO's? >_>

50 I think is too low, but we tried a couple of these with a threshold of 60 last night and it worked well enough that I've gone ahead and changed the rules. The main post has been updated.

I do not foresee any future changes to this system.


***


Also, I keep forgetting to mention this crazy bug we ran into - sometimes after a contest one of the new names created will show an incorrect (and impossible) number of Falls in its statistics. Here's an example of this from our screenshots page.

It doesn't really matter because it's still obvious that the opposing player's stat pages are inverses of each other (Here's my opponent's) and the buggy player's Falls can be determined by his opponent's KOs, but it's happened to us three times now so it'll probably happen to you, too.

We have no idea what could be causing this, but it may be related to swapping out memory cards as we were doing so for the sake of running two setups at the same time.

Also, thanks Toph, both for the interest and for being willing to connect Japan and America.

That will be a great help when the time comes.