Ranked: Gain/Loss of skill rating, how is it calculated?

Any and all discussion relating to Rocket League. Post feedback and suggestions here!

Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Incredipede on Sat Aug 01, 2015 11:15 pm

Hello,

so I've been wondering how the game calculated how many skill rating you gain or lose after every ranked match. Here's a list of 6 matches I played on ranked, all in the same hour, all on 3 vs. 3, often with the same players:

First game: Won, Skill Rating from 1545 to 1548 (+3, 3 vs. 3 match, ended 3 : 2 with overtime)
Second game: Won, Skill Rating from 1548 to 1552 (+4, 3 vs. 3 match, ended 2 : 1 without overtime)
Third game: Won, Skill Rating from 1552 to 1556 (+4, 3 vs. 3 match, ended 1 : 0 without overtime)
Fourth game: Lost, Skill Rating from 1556 to 1546 (-10, 2 vs. 3 match, ended 1 : 2 with overtime)
Fifth: game: Lost, Skill Rating from 1546 to 1521 (-25, 3 vs. 3 match, ended 0 : 2 without overtime)
Sixth game: Lost, Skill Rating from 1521 to 1519 (-2, 3 vs. 3 match, ended 1 : 5 without overtime)

All of these matches were against good players, I haven't noticed anyone playing bad, so I assume all of them had more than 1000 skill rating. The matchmaking was quite fast, and as matchmaking is looking for players which have the same skill, it's more likely that a big part of the players had a similar skill rating or a skill rating which was close to my skill rating.

It's really frustrating that you often gain between 1 and 5 skill rating from winning a match and then often lose between 5 and 25 (!) skill rating from losing a match. Other players have reported those "issues" with the skill rating too, as example a friend who is in the Top 50 only got 1 skill rating from winning a 2 vs. 2 against two players which are in the Top 50, but then lost around 10 skill rating from losing against the same two players.

It would be great if a Psyonix employee could explain how the skill rating system works. Feel free to share your experiences with the skill rating system.

Regards,
Incredipede
Last edited by Incredipede on Sat Aug 01, 2015 11:32 pm, edited 2 times in total.
Incredipede
Rocket-Powered
User avatar
Rocket-Powered
 
Posts: 228
Joined: Sat Jun 28, 2014 2:15 am
Location: Stardew Valley
Steam ID: Incredipede
Region: EU, Steam

Re: Ranked: How does the game determine the skill rating?

Postby Yakaaa on Sat Aug 01, 2015 11:26 pm

Great reserches. But you just forgot one thing for your hypothesis : maybe some of the good players you encountered could have only a few skill points, since they have been playing ranked match only very recently. Maybe they are good because of SARPBC or lots of unranked matchs. Loosing against them would explain the big loss of points.
It's just an idea, but I think it's to have in mind.

By the way, I really am curious like you and think everyone should see just affter a game how their points are calculated (but this can be difficult if there is a riw feature for the cakculation).
Yakaaa
Rocket-Powered
User avatar
Rocket-Powered
 
Posts: 299
Joined: Tue Jun 23, 2015 1:29 am
Steam ID: Yakaaa

Re: Ranked: How does the game determine the skill rating?

Postby Incredipede on Sat Aug 01, 2015 11:29 pm

Yakaaa wrote:[...] maybe some of the good players you encountered could have only a few skill points, since they have been playing ranked match only very recently. Maybe they are good because of SARPBC or lots of unranked matchs. Loosing against them would explain the big loss of points. [...]


Well, of course, it's possible that some of the players had a low skill rating, however the matchmaking was quite was (less than 20 seconds), so that makes me assume that a big part of the players had a similiar skill rating or a skill rating which was close to my skill rating. Also, two of the names I remember can be found in the Top 100 leaderboard, I'm not sure about the other players as I forgot their names.
Incredipede
Rocket-Powered
User avatar
Rocket-Powered
 
Posts: 228
Joined: Sat Jun 28, 2014 2:15 am
Location: Stardew Valley
Steam ID: Incredipede
Region: EU, Steam

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Yakaaa on Sat Aug 01, 2015 11:41 pm

I'm not so sure that skill rating are so close. Maybe the matchmaking can find players with 300+ of delta, it would not surprise me. But I could be totally wrong.
Yakaaa
Rocket-Powered
User avatar
Rocket-Powered
 
Posts: 299
Joined: Tue Jun 23, 2015 1:29 am
Steam ID: Yakaaa

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby DoctorDango on Sat Aug 01, 2015 11:50 pm

I think my score just rose by losing a match. I'm curious how this system works as well.
Image
DoctorDango
Supersonic
User avatar
 
Posts: 14
Joined: Sun Jul 26, 2015 1:00 am
Steam ID: justcalmdowngamer
Region: Steam

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Psyonix-Corey on Sun Aug 02, 2015 12:29 am

Skill gain/loss is based on your expected win probability which is based on your team's average MMR versus the opponent's. If you're heavily favored to win you don't gain much, but you lose a lot, and vice versa if your opponent is heavily favored. Equal skill games should yield 10-20 skill rating for a win.

The problem as you get into Top 50/etc. is on average, most players online and matchmaking are going to be below your MMR. So you will tend to play people below your MMR and you gain little from winning, but lose more on the occasions that you lose.

This is compounded by the MMR inflation we've seen in the preseason at the top end. When you play a lot you're slowly accruing MMR, but players at your skill (but who don't play as much) may be significantly lower rating than you. When you match against them, your games are close but you're at a disadvantage for rating purposes, since you gain little on a win and lose a lot if you lose.

Ranked was a late addition and this is why we had a preseason because we anticipated there might be issues like this. I'm sorry for the frustration. We're looking at changes for Season 1 and beyond, there's a few ways to tackle it:

- Reduce skill search expansion for Ranked queue so you don't get matched against people as far above or below your skill. From a top100 perspective, this means you'd match against less people below you that you stand to gain little from, but it could also mean longer matchmaking times for Ranked, especially at off-peak hours.
- Switch to a "Points" system, similar to how some other games do their League systems. You gain or lose "Points" scaled by your MMR, similar to how it works now, but it's more tightly controlled so we can let MMR do its thing in the background for matchmaking without having to compromise it for the ideal leaderboard functionality.
- Show Ranking update and opponent skill values on the end game screen so you have better visibility of who you played and what the outcome was.

These are just some things we're talking about, I can't promise any timeline unfortunately, but just wanted to give some insight.
Psyonix-Corey
Psyonix Developer
User avatar
Psyonix Developer
 
Posts: 362
Joined: Fri Sep 19, 2008 12:02 pm
PSN ID: psyonix-corey
Steam ID: Corey
Region: US

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby li_MUSKETEERS_il on Sun Aug 02, 2015 2:47 am

I have very frequently encountered the problem of getting 0 points for ranked matches. I know that as my skill points are pretty high it is likely that the system thinks I am the favourite to win, but when I gain absolutely nothing from winning tough games it is a bit disheartening. Maybe you could include something that means you get a minimum of one point for a win, just to make it seem worth the risk of losing 20?



Image
li_MUSKETEERS_il
Acrobatic
 
Posts: 78
Joined: Sun Sep 11, 2011 2:52 am
PSN ID: li_MUSKETEERS_il
Region: EU

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby JLmmeeh on Sun Aug 02, 2015 4:35 am

Thanks for the response corey; could you guys confirm or deny these two observations?

i played a bit of ranked and here's my feedback :

+) Seems like there isn't a max amount of points gained per match limit or if there is it's too high.

+) Seems like both players of a party queuing 2v2 together are rewarded the same amount of rating based on the average of their ratings; regardless of rating difference between the two.

To me there should be a hidden mmr + team mmr system implemented. Whenever you play with someone you never played with in a certain playlist it should create a "team rating" to determine opponents. The team rating gain would be the same but the hidden mmr wouldn't. Would also be nice to be able to see the values of both ratings as well EDIT: I'm talking about creating a party to q obviously; to me solo q and q should be separated as much as possible
JLmmeeh
Rocket-Powered
Rocket-Powered
 
Posts: 136
Joined: Wed Jan 29, 2014 1:25 pm
PSN ID: jLmmeeh
Region: EU

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby ScoTT on Sun Aug 02, 2015 4:48 am

Psyonix-Corey wrote:Skill gain/loss is based on your expected win probability which is based on your team's average MMR versus the opponent's. If you're heavily favored to win you don't gain much, but you lose a lot, and vice versa if your opponent is heavily favored. Equal skill games should yield 10-20 skill rating for a win.

The problem as you get into Top 50/etc. is on average, most players online and matchmaking are going to be below your MMR. So you will tend to play people below your MMR and you gain little from winning, but lose more on the occasions that you lose.

This is compounded by the MMR inflation we've seen in the preseason at the top end. When you play a lot you're slowly accruing MMR, but players at your skill (but who don't play as much) may be significantly lower rating than you. When you match against them, your games are close but you're at a disadvantage for rating purposes, since you gain little on a win and lose a lot if you lose.

Ranked was a late addition and this is why we had a preseason because we anticipated there might be issues like this. I'm sorry for the frustration. We're looking at changes for Season 1 and beyond, there's a few ways to tackle it:

- Reduce skill search expansion for Ranked queue so you don't get matched against people as far above or below your skill. From a top100 perspective, this means you'd match against less people below you that you stand to gain little from, but it could also mean longer matchmaking times for Ranked, especially at off-peak hours.
- Switch to a "Points" system, similar to how some other games do their League systems. You gain or lose "Points" scaled by your MMR, similar to how it works now, but it's more tightly controlled so we can let MMR do its thing in the background for matchmaking without having to compromise it for the ideal leaderboard functionality.
- Show Ranking update and opponent skill values on the end game screen so you have better visibility of who you played and what the outcome was.

These are just some things we're talking about, I can't promise any timeline unfortunately, but just wanted to give some insight.



This is good news ,

I and many others think the ranking system needs to be re-made , The literally pointless games when you get 0 points for winning? but lose 20-30 for losing


Also i often beat two players in the top 15 or 10 and get 1 or 2 points for winning ? I then look to see if my team mate is in the top 100 and he's not . It just does not really add up most of the time , The points seem a bit random and hard to explain . ( Also not very rewarding at all )

i.e - I have also earned points for losing a match a few times! / + Sometimes you can afk and get kicked and lose no points for it.


Also to add to that people are exploiting the fact that you can party up with a completely new account team mate just to earn tons and tons of points .
UKscottUK?
ScoTT
Rocket-Powered
User avatar
Rocket-Powered
 
Posts: 208
Joined: Thu Nov 26, 2009 8:01 pm
Location: Norwich , UK
PSN ID: UKscottUK
Region: EU

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Devious-Dolphin on Sun Aug 02, 2015 5:12 am

I was at 1350, I won a 2v1 and my rank went up by 100.
Image
Devious-Dolphin
Battle-Car
User avatar
Battle-Car
 
Posts: 978
Joined: Sun Jan 22, 2012 9:33 pm
PSN ID: Devious-Dolphin
Region: US

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby E_M_E_T on Sun Aug 02, 2015 5:17 am

Psyonix-Corey wrote: Equal skill games should yield 10-20 skill rating for a win.


I haven't played ranked that much the last few days, dropping me down to division 2. I played a 2v2 against two players who were both in the top 100, my teammate was not in the top 100, and I won with MVP. I got 1 point for that match. How do you explain that?
Image
E_M_E_T
Aerialistic
User avatar
Aerialistic
 
Posts: 1483
Joined: Sun Feb 02, 2014 11:33 pm
PSN ID: merc-835
Steam ID: E_M_E_T
Region: California

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Incredipede on Sun Aug 02, 2015 9:13 am

Psyonix-Corey wrote:Skill gain/loss is based on your expected win probability which is based on your team's average MMR versus the opponent's. If you're heavily favored to win you don't gain much, but you lose a lot, and vice versa if your opponent is heavily favored. Equal skill games should yield 10-20 skill rating for a win.

The problem as you get into Top 50/etc. is on average, most players online and matchmaking are going to be below your MMR. So you will tend to play people below your MMR and you gain little from winning, but lose more on the occasions that you lose.

This is compounded by the MMR inflation we've seen in the preseason at the top end. When you play a lot you're slowly accruing MMR, but players at your skill (but who don't play as much) may be significantly lower rating than you. When you match against them, your games are close but you're at a disadvantage for rating purposes, since you gain little on a win and lose a lot if you lose.

Ranked was a late addition and this is why we had a preseason because we anticipated there might be issues like this. I'm sorry for the frustration. We're looking at changes for Season 1 and beyond, there's a few ways to tackle it:

- Reduce skill search expansion for Ranked queue so you don't get matched against people as far above or below your skill. From a top100 perspective, this means you'd match against less people below you that you stand to gain little from, but it could also mean longer matchmaking times for Ranked, especially at off-peak hours.
- Switch to a "Points" system, similar to how some other games do their League systems. You gain or lose "Points" scaled by your MMR, similar to how it works now, but it's more tightly controlled so we can let MMR do its thing in the background for matchmaking without having to compromise it for the ideal leaderboard functionality.
- Show Ranking update and opponent skill values on the end game screen so you have better visibility of who you played and what the outcome was.

These are just some things we're talking about, I can't promise any timeline unfortunately, but just wanted to give some insight.


Thank you for your detailed reply, you explained the calculation system really well. I'm glad to see that Psyonix may work on the skill rating system in the future and I hope that it will be less frustrating in the future. :yaypsyonix:

E_M_E_T wrote:
Psyonix-Corey wrote: Equal skill games should yield 10-20 skill rating for a win.


I haven't played ranked that much the last few days, dropping me down to division 2. I played a 2v2 against two players who were both in the top 100, my teammate was not in the top 100, and I won with MVP. I got 1 point for that match. How do you explain that?


I would like to know that too as similar things happened to me before. My team (Me, in the Top 100 with two teammates, not in the Top 100) against the other team (Atleast two players of them were in the Top 100). Just like you, I got 1 skill point from winning that match.
Incredipede
Rocket-Powered
User avatar
Rocket-Powered
 
Posts: 228
Joined: Sat Jun 28, 2014 2:15 am
Location: Stardew Valley
Steam ID: Incredipede
Region: EU, Steam

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby IJx on Sun Aug 02, 2015 9:22 am

To me the worst part is that smurfing is pretty much the only way to gain rating as it is now. I've played a lot solo, but still no win rewards me with pretty much anything, even though its clear the ratings are pretty equal (At worst I'm partnered with a first timer versus two guys at top 150) and win gives me nothing. Actually, at this point by playing I can only lose points, since 5 wins in a row pretty much hand me 0-2 points, whilst each loss costs me far too much. I'd need a 99.99% win rate to progress at all.
Image
Your fellow ball curve lobbyist
IJx
Moderator
User avatar
Moderator
 
Posts: 1227
Joined: Thu Feb 27, 2014 6:00 am
Location: Finland
PSN ID: IJayfunk
Steam ID: IJ
Region: EU

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Magicman_ChrisTUrp on Mon Aug 03, 2015 3:48 am

Yes I guess my issue is related to this somehow.. I made my own Topic though. Do to the fact I have almost 90+hrs in 3v3 Rank'd rated matches.. My Skill level has not progressed from Bronze Skill Level 1pt since the 1st day.. Why is that?? Not loosing not gaining .. :/ How come? :O
Magicman_ChrisTUrp
Supersonic
 
Posts: 6
Joined: Mon Aug 03, 2015 3:17 am
Steam ID: http://steamcommunit
Region: US: PC

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby PlayerOne on Wed Aug 12, 2015 2:01 am

ScoTT wrote:
Also to add to that people are exploiting the fact that you can party up with a completely new account team mate just to earn tons and tons of points .


Is this going to be fixed when the rankings are reset? I'm assuming it wasn't intentional.
PlayerOne
Supersonic
 
Posts: 3
Joined: Thu Jul 30, 2015 3:03 pm
PSN ID: VanHoman
Region: Europe

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Cinnadillo on Wed Aug 12, 2015 11:33 am

IMO, I think anything you do should be reasonably public for scrutiny purposes... but I'm a Ph.D. Statistics guy so that'd be my bias. I don't really want to study this, I just want know what we're subject to in order to understand what's going on. 1-2 page white paper. Just the ratings calculation.

I always just figured it would all be based on average rating the players... if so, maybe the "speed" factor (K in Elo formulae) should be lower if there is significant spread between teammates. That way nobody can benefit from playing with somebody too good. Nobody can demerit by playing with somebody too bad.

Good thing about all this data collection is you can set up a scheme and process through with current results.
Image
Cinnadillo
Acrobatic
User avatar
 
Posts: 87
Joined: Mon Jun 22, 2015 11:52 am
PSN ID: Cinnadillo
Region: US

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Zinn on Wed Aug 12, 2015 1:46 pm

I'm still on the low end of the skill brackets (currently 448 points double, 420 solo) but I think the MM could use some adjustments even here. There are people on both teams who can barely hit the ball and you can really feel the difference when either side has two people who can. For instance, if I get partnered with someone who can hit it reliably, it basically becomes a 2v1 or even 2v0 match, where it's just a matter of how hard the other guys get crushed.

I'm not even talking about fancy goals, rocket jumps, amazing positioning or anything, just literally hitting the ball when you want to hit it is the deciding factor in a lot of these matches. I've seen two opponents and a teammate miss a ball that's sitting perfectly still 3 times in a row, followed by me getting there and poking it into their goal. I know there are some limits to how narrow a matchmaker can be but it should be a requirement to reliably hit the ball X times in a match.
Zinn
Acrobatic
 
Posts: 86
Joined: Wed Jul 22, 2015 9:36 am

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Cinnadillo on Wed Aug 12, 2015 2:04 pm

Zinn wrote:I'm still on the low end of the skill brackets (currently 448 points double, 420 solo) but I think the MM could use some adjustments even here. There are people on both teams who can barely hit the ball and you can really feel the difference when either side has two people who can. For instance, if I get partnered with someone who can hit it reliably, it basically becomes a 2v1 or even 2v0 match, where it's just a matter of how hard the other guys get crushed.

I'm not even talking about fancy goals, rocket jumps, amazing positioning or anything, just literally hitting the ball when you want to hit it is the deciding factor in a lot of these matches. I've seen two opponents and a teammate miss a ball that's sitting perfectly still 3 times in a row, followed by me getting there and poking it into their goal. I know there are some limits to how narrow a matchmaker can be but it should be a requirement to reliably hit the ball X times in a match.


there's also problem in ranking systems when you have a defined floor. Everybody started at 100... so anybody new will have to go up from there... players who are down don't go down so they don't get filtered to their skill.

It'd be better if people started at 1000 or 1500. That way lower skill people filter their way downwards... of course that would be depressing to a player so there may be a reason to keep a rating floor.
Image
Cinnadillo
Acrobatic
User avatar
 
Posts: 87
Joined: Mon Jun 22, 2015 11:52 am
PSN ID: Cinnadillo
Region: US

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Jixxers on Thu Aug 20, 2015 8:43 am

I'm at a loss as well on this. I understand the concept Psyonix is following (high pts gain for winning against higher ranked players and vise verse)

I'm going to put my "TLDR" up top of my post because with my ADHD, i even get lost in my own banter sometimes..
If i played against the same team 3x in a row why would i gain high points first game, high points loss second game and low pts gained third game?
would the second game close the gap that much to affect my third game winnings?

Does the ending game score affect the points gain / loss in anyway?
first game (win) - 7 to 0
second game (loss) - 2 to 3
third game (win) - 4 to 2


Here's the scenario i keep running into
    I'm only ranked 300ish but I win against a team, who honestly was not that good and i won 35 points (which tells me they were at a higher rank than I because that is a lot of points)
    i re-entered the queue and played against the same exact team - We lost in overtime, they had a lot of slow setups were I accidentally pushed the ball in by backing up and also they got those accidental "across the map bounce shots" scores. I ended up loosing 60 points for that match.. (which tells me I was at a LOT higher rank than them now, because that is A LOT of points to lose)
    I re-enter the queue and played against the same team (third time), won and only received 15 points..

It makes no sense,
shouldn't I have won more points for the third game than i did from the first game, or at least closer to the amount i first got ?
Seeing how i lost 60 pts on the 2nd game, i would have to assume that they are alot lower ranked than I but yet, the 35 points i got from the first game would tell me they are higher rank than me. would the second game closed the gap that much to affect my third game winnings?


If the concept is i lose more points against low ranking teams, Why would i win 35 points one game and lose 60 points against the same team?
game one points gain -tells me I'm at a lower rank that they are
game two points lost - tells me I'm a lot higher rank than they are...


on a normal day
i win 5 games in a row and loose 1 and i end up a lower rank than what i started.
Jixxers
Supersonic
 
Posts: 2
Joined: Thu Aug 20, 2015 8:17 am
Region: US

Re: Ranked: Gain/Loss of skill rating, how is it calculated?

Postby Jixxers on Thu Aug 20, 2015 9:01 am

Cinnadillo wrote:
there's also problem in ranking systems when you have a defined floor. Everybody started at 100... so anybody new will have to go up from there... players who are down don't go down so they don't get filtered to their skill.

It'd be better if people started at 1000 or 1500. That way lower skill people filter their way downwards... of course that would be depressing to a player so there may be a reason to keep a rating floor.


i think it should be higher than 100 but i can tell you, you can defiantly hit zero.. I've been down to 20 =P
Jixxers
Supersonic
 
Posts: 2
Joined: Thu Aug 20, 2015 8:17 am
Region: US


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 10 guests