• Welcome to Smashboards, the world's largest Super Smash Brothers community! Over 250,000 Smash Bros. fans from around the world have come to discuss these great games in over 19 million posts!

    You are currently viewing our boards as a visitor. Click here to sign up right now and start on your path in the Smash community!

Elo Ratings in Competitive Melee

Nintendude

Smash Hero
Joined
Feb 23, 2006
Messages
5,024
Location
San Francisco
Here's the post with the beta software:
http://www.smashboards.com/showpost.php?p=12563300&postcount=76

Not sure if this was mentioned in this forum yet but I think that the Smash community can learn a lot from competitive chess. That is, we should adopt their Elo ratings system based on tournament performance. Certain tourneys should be "approved" by the BR or some other committee to be eligible for ratings and then all matches that occur in pools or brackets will contribute towards everyone's ratings.

There's some really great things that can come from such a system:

1. Seeding can be based solely on player strength rather than bias. This is particularly useful in locals where you have a big pile of players of roughly the same strength and you don't know how to properly seed them.
2. Amateur brackets can have a certain cutoff for rating (like <1800, for example). Different tiers of amateur brackets or round robins can be created with ease.
3. Amateur players will have a better idea of where they stand within the community, simply based on their Elo rating. When I play people in chess, they will ask what my rating is, not "how good are you?" It's easy to gauge how skilled someone is based on ratings instead of handwaving.
4. The question of ranking the top players can be easily solved.
5. People of all skill levels will have more to play for. Not only are they competing for prizes, but they are competing to boost their rating.
6. Bracket manipulation and splitting might be discouraged since throwing matches will have consequences for your rating. Splitting is effectively a draw which can have negative consequences depending on the circumstances.
7. Implementing this kind of sophisticated system will increase the legitimacy and professionalism of competitive Melee.

I'm currently working to get a really similar system implemented for Smash 64's online play. Melee's much bigger community can gain even more from it.

Also I want to point out that this is different from Brawl's ratings system. Brawl's appears to attempt to measure absolute skill rather than relative skill. They weight things like tournament size and tournament results rather than just the individual matchups that occurred. With Elo new players can achieve high ratings very quickly by defeating good players. In Brawl's system they will have to place well in a bunch of large tournaments.
 

SpaceFalcon

Smash Lord
Joined
Oct 3, 2007
Messages
1,714
Location
604
I'm all for improving competitive melee, the only problem with this is that it will further chase away new players. Melee isn't a growing game, we don't need this on addition to all the active players 3-4 stocking everyone. Then making them feel even more inferior for having **** ranks.
 

Nintendude

Smash Hero
Joined
Feb 23, 2006
Messages
5,024
Location
San Francisco
That's not true at all. I believe chess is very inviting to weaker players because of how they use their ratings system. The ratings system is more than just seeing how you stack up. It's a great tool for allowing amateurs to play people at their own level, it removes bias towards good players in tournaments (this is a big turn-off to me), and it gives everyone something to work for. You know those 95% of players who never have any shot at prize money? And are just entering to try and get more experienced? Well now there's something real for them to play for.
 

0Room

Smash Lord
Joined
Aug 21, 2008
Messages
1,953
Location
Boone, NC
How exactly does this system work?
As in, how do you get ranks, as far as

How many points, etc?
 

Nintendude

Smash Hero
Joined
Feb 23, 2006
Messages
5,024
Location
San Francisco
I'm gonna refer you to wikipedia for this one. They explain it much better than I can:
http://en.wikipedia.org/wiki/Elo_ratings

But to summarize, it computes relative skill on a logistical curve. Defeating highly rated players will improve your rating more than defeating weaker players. If you lose to a player much stronger than you, your rating will barely be affected, but if you lose to someone much weaker you'll take a hit.
 

Bones0

Smash Legend
Joined
Aug 31, 2005
Messages
11,153
Location
Jarrettsville, MD
I'm all for improving competitive melee, the only problem with this is that it will further chase away new players. Melee isn't a growing game, we don't need this on addition to all the active players 3-4 stocking everyone. Then making them feel even more inferior for having **** ranks.
I don't really think it would happen like that. New players are discouraged that they get 3-4 stocked, but what is even more discouraging is that even if they finally get a fair match in loser's bracket and win a tight set 2-1, they get basically no recognition for their improvement. A ranking system that compares the lower placing players of one region to those in another gives them more incentive to improve because there is visual data to give them feedback.
 

0Room

Smash Lord
Joined
Aug 21, 2008
Messages
1,953
Location
Boone, NC
This is interesting, I may implement this in the ASU community and see how it works, at least if I can figure it out.
I might Pm if you if you have questions :p
 

LatexRhombus

Smash Journeyman
Joined
Jan 13, 2007
Messages
271
Location
Penn State
I fully support the notion lol

rating systems not only prevent certain biases in seedings and what not, but they actually automatically create some friendly rivalries which is something than can help new players get involved and stick with it

along with that, they work in the favor of both newcomers who can easily set goals to try and move up the ladder and for very high level players.

and as already mentioned, it would create a number of fairness opportunities with respect to amateur brackets and just players in general

unrelated: i should update the first post of the other thread at some point with all the good ideas found within in and in other threads like this one lol
 

0Room

Smash Lord
Joined
Aug 21, 2008
Messages
1,953
Location
Boone, NC
This would require a lot of thought and input though.
Are we going for like a world-wide leaderboard kind of thing?
 

0Room

Smash Lord
Joined
Aug 21, 2008
Messages
1,953
Location
Boone, NC
Yeah there's a lot more to actually implementing this than I think is available on the surface.
Especially since I don't really understand the whole Elo rating system fully yet. But I haven't really read it to be honest. Skim reading doesn't do well when partnered with Math equations.

But seriously, the ideas of implementation do require a lot of thinking before anything is really done. As far as, what is the long term goal, how would this affect current tournament scenes [realistically, not ideally], and how are we going to compile, and record, the scores and outcomes. I think these are all viable, even required, concepts for discussion.
 

Nintendude

Smash Hero
Joined
Feb 23, 2006
Messages
5,024
Location
San Francisco
Yeah there's a lot more to actually implementing this than I think is available on the surface.
Especially since I don't really understand the whole Elo rating system fully yet. But I haven't really read it to be honest. Skim reading doesn't do well when partnered with Math equations.

But seriously, the ideas of implementation do require a lot of thinking before anything is really done. As far as, what is the long term goal, how would this affect current tournament scenes [realistically, not ideally], and how are we going to compile, and record, the scores and outcomes. I think these are all viable, even required, concepts for discussion.
Actually it's not really that complicated. I believe there's 2 straightforward formulas used to calculate the ratings of players A and B given their initial ratings. You have to select a constant known as the "K-factor" which gives you control over how much your rating can increase from a single match. There's pretty established guidelines for picking K-factors already so we just have to pick one and go with it. There also exists websites that will calculate ratings for you. I think it would be good to program our own though (it's really not that hard), so we have full control over it.

All of the things you brought up are things that have to be discussed, yes. That's the purpose of this thread:)

This would require a lot of thought and input though.
Are we going for like a world-wide leaderboard kind of thing?
That's not really the purpose but it's definitely something that can come out of it. Once you get below the top players, your ranking is pretty irrelevant and should be used to categorize you among other players of a similar strength (<2000, <1800, etc) and to seed at tournaments.

This is interesting, I may implement this in the ASU community and see how it works, at least if I can figure it out.
I might Pm if you if you have questions :p
You may want to wait and see how the community responds before implementing it at ASU. Because, let's say a committee is selected (or just the MBR) to oversee the system, approve tournaments to be rated, etc. If you guys try and implement ratings before there's any infrastructure in place and this idea is realized, your ratings may be invalidated and you'll have to start from scratch, just because the authenticity of the ratings couldn't be guaranteed. Get what I'm saying? Though, if you were just planning on using it as the basis for a local ladder, and not use it for anything with the Smash community as a whole, go for it.
 

Tirno

Smash Journeyman
Joined
Jan 4, 2006
Messages
207
Location
Austin, TX
I actually wrote a Python script for the UT Smash Club that parses match results from a set of tio files and spits out a ranking along with scores. If you wanted something similar on a larger scale, you could have a publicly-visible repository of tio files and just have all the rankings calculated from the tio files in that repository. Make some guidelines for getting tio files accepted into the repository, and you'd have the beginnings of a community-wide ranking system, no?
 

Hadster

Smash Cadet
Joined
Feb 14, 2010
Messages
58
This is a revolutionary idea...something that has never existed in this community, but I am certain would contribute so positively.

I totally disagree with the person above who said it would really discourage new/amateur players. Thats absolute rubbish, all you have to do is look at chess (which also implements this system), and you will immediately realise that rankings are actually a huge drive factor for improvement of players, especially low level players.

From experience, "projects" like these which change the face of the professional scene absolutely MUST have a leader, a mastermind, who is in charge of managing and developing it. Sure its cool to make a thread like this to see the general opinion on the idea itself, but if the implementation is to be successful, it needs a proper head, and several people willing to chip in.

Obviously, Nintendude1189, I'm sure the community will be more than willing to help you in all the various ways nessecary to develop the project, but I would strongly advise that you stay as the head of this project. But it will require a lot of hard work and patience, simply because it has never been done before (there was actually something similar aaaages ago, but only for the top players), and if its not really your desire to be so involved in something like this, then I really strongly think that someone should be found to head this project (but obviously Nintendude1189 is the optimum person for this).
 

SSBM_or_GTFO

Smash Apprentice
Joined
Mar 18, 2011
Messages
92
This would be awesome to have. It'd be better if it were some sort of software to make it easier to view and manage, kind of like online highscores for a game.

Programming something like this would be more work in the short run but would pay off in the long run, imo.

Just an idea. :D
 

Aves

Smash Apprentice
Joined
Dec 14, 2009
Messages
84
I love the idea, my favorite thing about it, is that it gives amateurs something to play for.
When i go into a tournament, i know i'm not going to place top 3. But i also know i can beat a lot of people, so it makes each individual match seem worth it. I think a sytstem like this would be great for melee.
 

0Room

Smash Lord
Joined
Aug 21, 2008
Messages
1,953
Location
Boone, NC
You may want to wait and see how the community responds before implementing it at ASU. Because, let's say a committee is selected (or just the MBR) to oversee the system, approve tournaments to be rated, etc. If you guys try and implement ratings before there's any infrastructure in place and this idea is realized, your ratings may be invalidated and you'll have to start from scratch, just because the authenticity of the ratings couldn't be guaranteed. Get what I'm saying? Though, if you were just planning on using it as the basis for a local ladder, and not use it for anything with the Smash community as a whole, go for it.
Yeah I was going to see if it just works.
I like to do new things and see what happens in the community, and I kind of use ASU as a test subject :p

It was like you said, a local ladder, more of a "how does this work, let's get used to the feel of it".
 

huMps

Smash Ace
Joined
Apr 27, 2010
Messages
708
Location
On the fence
An unbiased useful ranking system sounds amazing. May cause controversy but that's only good for hype. And hype/controversy plus official rankings can really help motivate this community. I like this alot
 

Dimitris

Smash Ace
Joined
Apr 13, 2010
Messages
571
How does it work with being active / inactive. If you have a good score, but then you don't attend tourneys for a year, you keep your points (and sort of distort the ranking)?
 

Cosmo!

nerf zelda's dsmash
Joined
Feb 5, 2008
Messages
2,368
Location
Chicago, Illinois
a national Elo system will take a long *** time to propogate, especially if we can't use a lot of old results (lost brackets, etc).
 

CloneHat

Smash Champion
Joined
Jan 18, 2009
Messages
2,131
Location
Montreal, Quebec
It's too hard. Nobody cares. I don't want to keep track of my tournaments.

^Ignore this garbage! This is a great idea, and we should get started on it ASAP. If you have a program capable of keeping track of calculating ratings, then we can have all players volunteer themselves into the system in a different thread, starting them off at a standard rating.

Next, set criteria for which tournaments count. Look at recent nationals/majors, enter them in, and then have all viable tournaments from then on submitted in the official ranking thread, with bracket image attached.

Any fast indication of progress will keep this project on the map, and it will be a great addition to the community!

@Dimitris
: The way chess handles this is by having a separate ranking depending on the frequency this player plays matches. The less frequently they play, the more their ranking moves from each set they win/lose.
 

Divinokage

Smash Legend
Joined
Aug 6, 2006
Messages
16,250
Location
Montreal, Quebec
How does it work with being active / inactive. If you have a good score, but then you don't attend tourneys for a year, you keep your points (and sort of distort the ranking)?
That doesn't seem to be really an issue because when you go to your next tournament even if you are seeded with your current points, you'll simply lose points lol. Anyways, you don't deserve to win if you can't beat everyone.
 

Rappster

Smash Ace
Joined
Jun 9, 2008
Messages
569
Location
Torrance, CA
This is a great idea, but there's a few problems with this:

Each region would need to have an enthusiastic person to maintain a ratings listing/database. I'm moving out to wheaton il. this fall for school, and i'd be more that happy to assist anyone in the area.

Second, we would need ratings comprability. if i'm a 1500 rated player in the midwest but i go out to California, i'ma be pissed off if I have to come in as unrated. so if each group of people who makes a rating system would be willing to accept other's ratings that would make this a lot better

finally, we would need a provisional rating system. elo systems only work if everyone already has an established rating. http://math.bu.edu/people/mg/ratings/approx/approx.html I would recommend that different regions not accept another region's provisional ratings, because provisional ratings can vary pretty wildly.


How does it work with being active / inactive. If you have a good score, but then you don't attend tourneys for a year, you keep your points (and sort of distort the ranking)?
thats how chess works.
i haven't played a tournament in over a year, but my rating is still the same
 

Dimitris

Smash Ace
Joined
Apr 13, 2010
Messages
571
... when you go to your next tournament even if you are seeded with your current points, you'll simply lose points lol.
See that's the problem, 'when you go...' , but in the mean time your ranking misrepresents you in the list. (see Rappsters' post)
 

Nintendude

Smash Hero
Joined
Feb 23, 2006
Messages
5,024
Location
San Francisco
a national Elo system will take a long *** time to propogate, especially if we can't use a lot of old results (lost brackets, etc).
It'll probably take a year until Elo can be used as a basis for seeding, but that's no reason to not start using the system. I think that our community is incredibly behind the times for not having a ratings system in place already.

This is a great idea, but there's a few problems with this:

Each region would need to have an enthusiastic person to maintain a ratings listing/database. I'm moving out to wheaton il. this fall for school, and i'd be more that happy to assist anyone in the area.

Second, we would need ratings comprability. if i'm a 1500 rated player in the midwest but i go out to California, i'ma be pissed off if I have to come in as unrated. so if each group of people who makes a rating system would be willing to accept other's ratings that would make this a lot better

finally, we would need a provisional rating system. elo systems only work if everyone already has an established rating. http://math.bu.edu/people/mg/ratings/approx/approx.html I would recommend that different regions not accept another region's provisional ratings, because provisional ratings can vary pretty wildly.
Isn't rating comparability one of the great advantages to a universal rating scheme? If you are rated 1500, you are rating 1500, period. You aren't only rated 1500 in your region, even if you travel somewhere else. One of the awesome things about Elo is it is unbiased. Players who end up overrated will eventually start losing and their rating will end up where it should be.

As for provisional ratings, why not use a selected set of past tournaments? The ratings are simply provisional anyway, so minor inaccuracies aren't a problem. I think that it would quickly sort itself out.

See that's the problem, 'when you go...' , but in the mean time your ranking misrepresents you in the list. (see Rappsters' post)
It's important to note that this is a ratings scheme, not a ranking scheme. The goal is NOT to create a giant list of every smasher and sort them by strength. Rather, the system is meant to assess everyone's skill level in an unbiased manner. Overrated players will lose and their rating will be adjusted accordingly. Same goes for underrated players.
 

Cobalt

Smash Journeyman
Joined
Aug 22, 2007
Messages
448
Location
Pittsburgh, PA
Tsk, I started working on a program for doing this a week ago or so. The underlying logic is pretty much complete, I just need to add a GUI to make it usable. I'm writing it in Java for simplicity and platform-compatibility, and since the amount of calculation is minimal enough that speed is pretty irrelevant.
 

Brookman

Smash Hero
Joined
Oct 20, 2005
Messages
6,202
Location
pikachu
That doesn't seem to be really an issue because when you go to your next tournament even if you are seeded with your current points, you'll simply lose points lol. Anyways, you don't deserve to win if you can't beat everyone *on your side of the bracket*.


fxed /10
 

eschemat

Smash Journeyman
Joined
Aug 10, 2010
Messages
241
I'm a chess player and I'd like to just say the rating encourages me to increase my play much more and beat others, reach certain designated titles and also beating my greatest rivals. Just putting out my two cents.
 

Rappster

Smash Ace
Joined
Jun 9, 2008
Messages
569
Location
Torrance, CA
I'm a chess player and I'd like to just say the rating encourages me to increase my play much more and beat others, reach certain designated titles and also beating my greatest rivals. Just putting out my two cents.
agreed.
my school's competitive chess team was my life ... until i moved to socal
 

Zivilyn Bane

Smash Master
Joined
Nov 18, 2004
Messages
3,119
Location
Springfield, MO
I spent six years playing tournament chess and have always thought an elo system would be a good addition to the melee tournament scene. I've even brought it up a few times over the years. However nobody really seemed to want to do it. I personally do not have enough time to get a project like this up and running, however I have a lot of experience and knowledge with it and have several ideas on how to implement it in to melee. PM me if you want to talk more about it and I can help out, I have some ideas.
 

0Room

Smash Lord
Joined
Aug 21, 2008
Messages
1,953
Location
Boone, NC
So what specifically needs to be done, for this to happen?
Like for example, BR says okay let's do it

what's step 1 to step whatever?
I'm just trying to get a more concrete idea of this whole thing.
 

Bones0

Smash Legend
Joined
Aug 31, 2005
Messages
11,153
Location
Jarrettsville, MD
I'm not too sure, but from what I've read it would go something like this:

1. Get some initial ratings based on previous tournaments.

2. Figure out a process for updating ratings (Tirno and Cobalt both seem to have made headway in this area).

3. Give ratings time to flesh out to what is accurate because the initial rankings will most likely be pretty off.
 

azianraven

Smash Journeyman
Joined
Mar 13, 2007
Messages
342
Location
Gainesville, FL
This is done in some regions and in Puerto Rico smash already iirc.

I think it's a more well put together system. Maybe the boards can ask them for their knowledge on how to get it done and make it a standard in the smash scene.
 

Nintendude

Smash Hero
Joined
Feb 23, 2006
Messages
5,024
Location
San Francisco
Here's what I envision:

The MBR, or some other committee, will "approve" tournaments to be rated. An approved tournament will use the official MBR ruleset and Tio, and send a completed Tio file to this committee, which will then go through the bracket and update the results (either manually or with a script). Once the system passes its "incubation period," seeding via rating will become another requirement to be approved. Ratings will be updated chronologically and before each weekend so that updated ratings can be used for those tournaments. There should be some kind of database where TOs can go in and quickly get the ratings of all the entrants.

The most important thing is that everything has to uniform and regulated. This "committee" will have to be diligent and strict make sure that results are all authentic. I'd actually suggest that this community start using REAL NAMES, rather than gamertags, to keep track of who is who (in case people change tags). I believe with Tio you can enter both a real name and a gamertag, so it's not like it would be a big change.

The FIDE website is good to take a look at:
http://www.fide.com/index.php

They have databases of TOs, rated tournaments, and players. If you click a player you get a profile showing some basic personal information (like age and country), his rating, his match history, rating history, and his overall ranking under several criteria. I'm not expecting us to develop something quite as sophisticated, but the more of those features we implement the better.

I'd really like to hear the thoughts of some other prominent names in this community. I also think that people should get in touch with nealdt to see if he can make small tweaks to Tio to make this whole process simpler.
 
Top Bottom