General Discussion

PL7.5 Known Issues and planned fixes

Hutch_TimHutch_Tim Administrator, Hutch Staff Posts: 396 admin

We’re seeing a lot of reports of bugs and unexpected physics in PL7.5. Here’s what we’ve identified, and what we’re doing about it.

Car corrections
Pagani Zonda 760RS: High mid-range acceleration unintentionally got worse. We generate an acceleration curve based on the known points (0-60 time etc). The corrections to 0-60 and Top Speed (and holding our mid-range stats in-place, which are unfortunately for sub-150mph speeds) meant the calculation that estimates the rest of the acceleration curve curve generated a weirdly slow high-end acceleration. This will be fixed, and also in future when adjusting 0-60 times like this we will test the MRA wasn’t affected for this reason

Caterham Levante RS: Addition of traction control has apparently made the car slightly slower on dry asphalt races. We’re investigating why this is. In future, when making corrections like this we will test baseline performance even when it shouldn’t have been affected, as is the case here.

Saved hand visual glitch
On some Android devices when you load a saved hand you see a visual glitch in the bottom half of the screen.

High Speed Bowl anomaly
A stock Ford C-Max can get a higher speed on the High Speed Bowl than it should be able to. We’re investigating this.

Hills are really, really easy to get up
What happened
This is probably the biggest one. In PL7.0 we adjusted hills to be as steep in-game as they actually look (previously the physics simulated them as being less steep). This meant lots of cars DNF. We figured it was more interesting for the game to have less steep hills so more cars could compete, so with PL7.5 we adjusted the physics to treat them as if they are a lot flatter - but so much so that high-performance cars with performance tires are winning.

Specifically, where the game represents hills with gradients that vary from 30% to 60% (much steeper than anything driven in real life), in PL7.5 they are treated as if they vary from around 8% to 18%.

What are we doing about it
We will adjust the physics to treat the hills more like they were, but not the full amount. Deciding exactly what is the “right” amount of steepness is tricky, and whatever we choose there will be disagreements on whether certain cars should or shouldn’t be able to get up them. To get ahead of that, we will put together some example cases and ask the community what they think the results should be; we will then choose a modified steepness on that basis.

How did this even happen though? Don’t you even test things?!
First: yes, we do test things. Here’s what happened with hills:

  • We adjusted how steep hills were treated, and we altered traction for 4WD cars (other than all-surface)
  • We then tested a wide representative array of cars on hills and saw that they were indeed getting up them more easily overall, as expected with the steepness change

  • We tested a range of 4WD cars with different tires and confirmed that the changes in traction meant they were faster or slower by the amounts expected

The tests show that the changes we made had the results we expected. What they don’t show is that unexpected cars would win in some match-ups. This turns out to be extremely difficult to test for.

We can (and do) batch test all results on a given challenge and see how things altered overall - but this won’t tell you if the wrong cars are winning. We can (and do) make hundreds of spot-checks of individual match-ups to see if the results are expected, and sometimes identify issues, but we won’t catch everything. But as soon as we put the update live, within 24 hours literally millions of match-ups have been effectively tested by humans, who can judge if the result is correct far better than any algorithm can, and cover significantly more cases than our spot-checks.

So what we doing about it?
Players have said they could have spotted these issues easily. We can’t distribute a version of the game for testing like that, but what we can do is notify players of what changes are being worked on, and then explicitly ask what they would test to see if it worked as intended. We can add these tests to our own to get more confidence that we’ve caught the potential issues. This is what we will do for the planned hill climb adjustment, and for any future physics alterations. Stay tuned.

«1

Comments

  • TD01055TD01055 Member Posts: 1,354 ✭✭✭✭✭
    Honestly you should kinda go back to the hillclimb before update 7.0. It worked pretty well (not perfectly) back then 
    I don’t remember anyone really complaining about the hills in pl7. It’s good that some cars don’t make it up and the traction changes to 4wd perf and off road tyres should have solved the one or 2 funny cases

    Even 2wd-off road cars getting stuck on tough twisty road was fine with me, it meant more diversity and not having hands full of datsuns.
  • krystiankrystian Member Posts: 480 ✭✭✭
    Hutch_Tim said:

    We’re seeing a lot of reports of bugs and unexpected physics in PL7.5. Here’s what we’ve identified, and what we’re doing about it.

    Car corrections
    Pagani Zonda 760RS: High mid-range acceleration unintentionally got worse. We generate an acceleration curve based on the known points (0-60 time etc). The corrections to 0-60 and Top Speed (and holding our mid-range stats in-place, which are unfortunately for sub-150mph speeds) meant the calculation that estimates the rest of the acceleration curve curve generated a weirdly slow high-end acceleration. This will be fixed, and also in future when adjusting 0-60 times like this we will test the MRA wasn’t affected for this reason

    Caterham Levante RS: Addition of traction control has apparently made the car slightly slower on dry asphalt races. We’re investigating why this is. In future, when making corrections like this we will test baseline performance even when it shouldn’t have been affected, as is the case here.

    Saved hand visual glitch
    On some Android devices when you load a saved hand you see a visual glitch in the bottom half of the screen.

    High Speed Bowl anomaly
    A stock Ford C-Max can get a higher speed on the High Speed Bowl than it should be able to. We’re investigating this.

    Hills are really, really easy to get up
    What happened
    This is probably the biggest one. In PL7.0 we adjusted hills to be as steep in-game as they actually look (previously the physics simulated them as being less steep). This meant lots of cars DNF. We figured it was more interesting for the game to have less steep hills so more cars could compete, so with PL7.5 we adjusted the physics to treat them as if they are a lot flatter - but so much so that high-performance cars with performance tires are winning.

    Specifically, where the game represents hills with gradients that vary from 30% to 60% (much steeper than anything driven in real life), in PL7.5 they are treated as if they vary from around 8% to 18%.

    What are we doing about it
    We will adjust the physics to treat the hills more like they were, but not the full amount. Deciding exactly what is the “right” amount of steepness is tricky, and whatever we choose there will be disagreements on whether certain cars should or shouldn’t be able to get up them. To get ahead of that, we will put together some example cases and ask the community what they think the results should be; we will then choose a modified steepness on that basis.

    How did this even happen though? Don’t you even test things?!
    First: yes, we do test things. Here’s what happened with hills:

    • We adjusted how steep hills were treated, and we altered traction for 4WD cars (other than all-surface)
    • We then tested a wide representative array of cars on hills and saw that they were indeed getting up them more easily overall, as expected with the steepness change

    • We tested a range of 4WD cars with different tires and confirmed that the changes in traction meant they were faster or slower by the amounts expected

    The tests show that the changes we made had the results we expected. What they don’t show is that unexpected cars would win in some match-ups. This turns out to be extremely difficult to test for.

    We can (and do) batch test all results on a given challenge and see how things altered overall - but this won’t tell you if the wrong cars are winning. We can (and do) make hundreds of spot-checks of individual match-ups to see if the results are expected, and sometimes identify issues, but we won’t catch everything. But as soon as we put the update live, within 24 hours literally millions of match-ups have been effectively tested by humans, who can judge if the result is correct far better than any algorithm can, and cover significantly more cases than our spot-checks.

    So what we doing about it?
    Players have said they could have spotted these issues easily. We can’t distribute a version of the game for testing like that, but what we can do is notify players of what changes are being worked on, and then explicitly ask what they would test to see if it worked as intended. We can add these tests to our own to get more confidence that we’ve caught the potential issues. This is what we will do for the planned hill climb adjustment, and for any future physics alterations. Stay tuned.

    Good Luck guys!!
    I hope you can update right this time.
  • HeissRodHeissRod Member Posts: 3,155 ✭✭✭✭✭
    Thanks @Hutch_Tim for playing the messenger in this one.

    Will hutch take urgent action, meaning PL7.6 is planned or will the fixing been done with the regular PL8.0?
    Just as an FYI, the last "Hotfix" was February 1st, 2 days after the release of the January Bugs update.  As this is the only hotfix they've ever released, it's not really a reliable point of reference.
  • evilprofesseurevilprofesseur Member Posts: 336 ✭✭✭
    edited November 21
    TD01055 said:
    Honestly you should kinda go back to the hillclimb before update 7.0. It worked pretty well (not perfectly) back then 
    I don’t remember anyone really complaining about the hills in pl7. It’s good that some cars don’t make it up and the traction changes to 4wd perf and off road tyres should have solved the one or 2 funny cases

    Even 2wd-off road cars getting stuck on tough twisty road was fine with me, it meant more diversity and not having hands full of datsuns.
    There was a lot of complaining, everything started with the dakar being useless at hill climbs. this uncovered an issue after an issue.

    @Hutch_Tim
    Hutch_Tim said:

    What are we doing about it

    We can (and do) batch test all results on a given challenge and see how things altered overall - but this won’t tell you if the wrong cars are winning. We can (and do) make hundreds of spot-checks of individual match-ups to see if the results are expected, and sometimes identify issues, but we won’t catch everything. But as soon as we put the update live, within 24 hours literally millions of match-ups have been effectively tested by humans, who can judge if the result is correct far better than any algorithm can, and cover significantly more cases than our spot-checks.

    Come on guys, this is not THAT difficult to test

    I assume that running every car through every hill climb can be automated easily - seeing as you do automated testing for rqs. Make the simulation return the time along with the information about the car used. plot chosen datasets (for instance everything in rare, then everything doing 0-60 in less than 6s etc) on a graph using different colors for key characteristics (tires, drivetrain) and compare the points to baseline climbers in each category (GE280 in rares for instance). Bam, you've got a way to see odd results instantly without tedious comparisons between each car or offloading testing to us.

    Not to mention some automated tests for things that should never happen and should automatically flag a build as useless - like a low car on slicks making it up the hill. 

    please do take note that ride height doesn't matter at all anymore in hillclimbs. You've also mentioned that the hills are now steeper than they used to be a while before but if that was the case performance tires cars should not be faster up the hill now - especially since they ostensibly have had their traction reduced.

    Hire more or better testers guys. These things are solvable.
    Post edited by evilprofesseur on
  • BoxtaSBoxtaS Member Posts: 605 ✭✭✭✭
    I’m just a bit shocked they went from 30-60% to 8-18% for the hills. Surely a slight reduction would have made more sense than a massive change. 
  • TD01055TD01055 Member Posts: 1,354 ✭✭✭✭✭
    TD01055 said:
    Honestly you should kinda go back to the hillclimb before update 7.0. It worked pretty well (not perfectly) back then 
    I don’t remember anyone really complaining about the hills in pl7. It’s good that some cars don’t make it up and the traction changes to 4wd perf and off road tyres should have solved the one or 2 funny cases

    Even 2wd-off road cars getting stuck on tough twisty road was fine with me, it meant more diversity and not having hands full of datsuns.
    There was a lot of complaining, everything started with the dakar being useless at hill climbs. this uncovered an issue after an issue.

    @Hutch_Tim
    Hutch_Tim said:

    What are we doing about it

    We can (and do) batch test all results on a given challenge and see how things altered overall - but this won’t tell you if the wrong cars are winning. We can (and do) make hundreds of spot-checks of individual match-ups to see if the results are expected, and sometimes identify issues, but we won’t catch everything. But as soon as we put the update live, within 24 hours literally millions of match-ups have been effectively tested by humans, who can judge if the result is correct far better than any algorithm can, and cover significantly more cases than our spot-checks.

    Come on guys, this is not THAT difficult to test

    I assume that running every car through every hill climb can be automated easily - seeing as you do automated testing for rqs. Make the simulation return the time along with the information about the car used. plot chosen datasets (for instance everything in rare, then everything doing 0-60 in less than 6s etc) on a graph using different colors for key characteristics (tires, drivetrain) and compare the points to baseline climbers in each category (GE280 in rares for instance). Bam, you've got a way to see odd results instantly without tedious comparisons between each car or offloading testing to us.

    Not to mention some automated tests for things that should never happen and should automatically flag a build as useless - like a low car on slicks making it up the hill. 

    please do take note that ride height doesn't matter at all anymore in hillclimbs. You've also mentioned that the hills are now steeper than they used to be a while before but if that was the case performance tires cars should not be faster up the hill now - especially since they ostensibly have had their traction reduced.

    Hire more or better testers guys. These things are solvable.
    Agrees things got messy, but the Dakar was fixed in PL7.0 was it not? It’s certainly the best hillclimber in PL7. The only issue left with hillclimb was top-4wd performance cars winning over off roaders. Traction reduction alone should have fixed it.

    unless I’m forgetting something, it’s hard to see the bugs from the trees sometimes 
  • 43MK443MK4 Member Posts: 1,289 ✭✭✭✭✭
    @Hutch_Tim improvements made to Pagani 760rs made in 7.5 (0-60 and top speed) indicate that it should be better than 7.0 version in ALL races. When you release next patch fixing it’s current broken performance, please make sure that it will beat 7.0 version in ALL races.
  • Hutch_TimHutch_Tim Administrator, Hutch Staff Posts: 396 admin
    Honestly you should kinda go back to the hillclimb before update 7.0. It worked pretty well (not perfectly) back then 
    I don't agree, everyone had just got used to how it worked! Heavier cars got up it faster. Upgrading weight made you slower. 4WD Performance cars could outperform off-road cars, and apparently people were of the opinion that their "raw power" shouldn't allow them to do that.

    TD01055 said:
    Honestly you should kinda go back to the hillclimb before update 7.0. It worked pretty well (not perfectly) back then 
    I don’t remember anyone really complaining about the hills in pl7. It’s good that some cars don’t make it up and the traction changes to 4wd perf and off road tyres should have solved the one or 2 funny cases

    Even 2wd-off road cars getting stuck on tough twisty road was fine with me, it meant more diversity and not having hands full of datsuns.
    This is an interesting point, and is exactly why we want to consult on it this time. In many posts, it's hard to tell the difference between surprise ("I didn't expect that") and disappointment ("that shouldn't happen"). But I do think it was too steep there - roughly twice as steep as the steepest road in real life, steep enough that off-road cars you might expect to get up a dirt hill (like the RQ12 1970 Range Rover Mk1) would just DNF on it. 

    Thanks @Hutch_Tim for playing the messenger in this one.

    Will hutch take urgent action, meaning PL7.6 is planned or will the fixing been done with the regular PL8.0?
    Good question. I can't confirm 100% until it's done (we may find other complicated knock-on bugs, critical team members might fall ill etc), but we're at least planning on a PL7.6 to address this more quickly.


    I assume that running every car through every hill climb can be automated easily - seeing as you do automated testing for rqs. Make the simulation return the time along with the information about the car used. plot chosen datasets (for instance everything in rare, then everything doing 0-60 in less than 6s etc) on a graph using different colors for key characteristics (tires, drivetrain) and compare the points to baseline climbers in each category (GE280 in rares for instance). Bam, you've got a way to see odd results instantly without tedious comparisons between each car or offloading testing to us.

    Not to mention some automated tests for things that should never happen and should automatically flag a build as useless - like a low car on slicks making it up the hill. 

    please do take note that ride height doesn't matter at all anymore in hillclimbs. You've also mentioned that the hills are now steeper than they used to be a while before but if that was the case performance tires cars should not be faster up the hill now - especially since they ostensibly have had their traction reduced.

    Hire more or better testers guys. These things are solvable.
    It's true I indicated that hills are steeper in PL7.5 than before PL7.0; having looked into this more, I now think it was a mistake to say that. Before PL7.0 hills didn't have much bearing on real physics, given that heavier cars could get up them faster. There isn't a meaningful way to say how steep they were in that case!

    You're correct that it's pretty easy to evaluate relative performance on hill climbs. The wider issue though is not that though - it's about knowing what to test. Each time we accidentally produce physics bugs, we then build new tools or implement new processes to catch that issue if it comes up again. So generally each time a new issue happens, it's something we weren't testing for. That's why we now plan to check just what players would test when we're planning to make a change. Testing the right thing would catch almost all the issues we'd had, and I expect the vast majority of potential future issues. In the case of hills getting a lot lower, I expect players would have suggested testing performance tire cars vs off-road tires just to check the balance hadn't tipped too far in their favour, and it would then indeed have been easy for us to see that.

    I'd put automatic testing for "things that should never happen" in the same category: we could build out a list of such cases, but I'm certain such a list would be incomplete, and each new issue would again just add something to the list of "make sure that doesn't happen again" without preventing new issues.
  • MettitiMettiti Member Posts: 1,766 ✭✭✭✭✭
    I like how you have your excuses ready 😂😂😂
  • evilprofesseurevilprofesseur Member Posts: 336 ✭✭✭
    Hutch_Tim said:

    I assume that running every car through every hill climb can be automated easily - seeing as you do automated testing for rqs. Make the simulation return the time along with the information about the car used. plot chosen datasets (for instance everything in rare, then everything doing 0-60 in less than 6s etc) on a graph using different colors for key characteristics (tires, drivetrain) and compare the points to baseline climbers in each category (GE280 in rares for instance). Bam, you've got a way to see odd results instantly without tedious comparisons between each car or offloading testing to us.

    Not to mention some automated tests for things that should never happen and should automatically flag a build as useless - like a low car on slicks making it up the hill. 

    please do take note that ride height doesn't matter at all anymore in hillclimbs. You've also mentioned that the hills are now steeper than they used to be a while before but if that was the case performance tires cars should not be faster up the hill now - especially since they ostensibly have had their traction reduced.

    Hire more or better testers guys. These things are solvable.
    It's true I indicated that hills are steeper in PL7.5 than before PL7.0; having looked into this more, I now think it was a mistake to say that. Before PL7.0 hills didn't have much bearing on real physics, given that heavier cars could get up them faster. There isn't a meaningful way to say how steep they were in that case!

    You're correct that it's pretty easy to evaluate relative performance on hill climbs. The wider issue though is not that though - it's about knowing what to test. Each time we accidentally produce physics bugs, we then build new tools or implement new processes to catch that issue if it comes up again. So generally each time a new issue happens, it's something we weren't testing for. That's why we now plan to check just what players would test when we're planning to make a change. Testing the right thing would catch almost all the issues we'd had, and I expect the vast majority of potential future issues. In the case of hills getting a lot lower, I expect players would have suggested testing performance tire cars vs off-road tires just to check the balance hadn't tipped too far in their favour, and it would then indeed have been easy for us to see that.

    I'd put automatic testing for "things that should never happen" in the same category: we could build out a list of such cases, but I'm certain such a list would be incomplete, and each new issue would again just add something to the list of "make sure that doesn't happen again" without preventing new issues.
    As for automatic tests - sure they won't map a complete space of possible faults, they never do. But with a number of these, when adding new ones as every update uncovers new problems you'd be able to prevent old issues from popping up again. The fact that it won't catch everything doesn't mean that it's not worth doing, especially since most of these would be trivial to write. Think of them as regression testing.

    Sure that the graph solution I presented is won't predict new issues. It can be easily adapted for other tracks and completely automated to check if slaloms, city streets etc don't yield weird results. Mark current performance on each track for each car, run them again after physics changes, see if there are very significant changes for some cars and investigate if that's intended. It would simulate a lot of player input very well if it used the cars that we know are very good on the tracks as baselines. I can easily see a system that wouldn't take very long to write and would be able to find unusual changes on its own so that devs could easily check how new physics influence performance. And yes - this won't find every fault. But isn't cutting most of them down worth it? 

    I can't pretend to know about all of the complexities you face in the development of this particular game but I can tell you that the effects of such development would not fly in my team. I don't know whether it's your policy on 'known shippables' or you genuinely don't find these issues before releasing but there are way too many of them and there must be a way to prevent a significant number of them. 
  • MettitiMettiti Member Posts: 1,766 ✭✭✭✭✭
    sinner said:
    Ok the result is debatable, even though i feel like I should of lost this. But the reason given for the levante loosing is not. ABS on half mile drag. 
    here's my theory
    when a car loses an event to another car, Race info doesn't display WHY the car lost.. it just shows WHATEVER the losing car doesn't have....ELSE print ("better midrange acceleration" OR "better blueprint for surface conditions")
    physics engine my ****
    prove me wrong 
  • Parimal_2000Parimal_2000 Member Posts: 700 ✭✭✭✭
    Mettiti said:
    sinner said:
    Ok the result is debatable, even though i feel like I should of lost this. But the reason given for the levante loosing is not. ABS on half mile drag. 
    here's my theory
    when a car loses an event to another car, Race info doesn't display WHY the car lost.. it just shows WHATEVER the losing car doesn't have....ELSE print ("better midrange acceleration" OR "better blueprint for surface conditions")
    physics engine my ****
    prove me wrong 
    Yeah I agree because I had the exact thought in my mind
  • OzzmanOzzman Member Posts: 376 ✭✭✭
    Just wondering @Hutch_Tim, do you guys re-run the RQ resolver every time a new patch comes out? Especially when you're modifying the physics engine or car stats. Obviously this means that some results change, and that should/could cause changes in RQs as well. Take the Dodge Stealth and the Chevy Lumina for instance, they seem like strong Rare cars now, yet they are still Uncommon RQ9 and 10 respectively.
  • greddygreddy Member Posts: 606 ✭✭✭✭
    I think they are keeping any rq corrections for 8.0 with new cars coming. 
  • mauro07mauro07 Member Posts: 1,580 ✭✭✭✭
    @Hutch_Tim ; i think the first problem is to explain what is Hillclimb... i used italian and if you want you can traslate.
    Hillclimb: che cos'è? è una corsa in salita, senza curve, con dossi; si avvicina molto alle drag race.
    Hillclimb sull'asfalto: penso che qui sia importante solo la potenza dell'auto e non l'altezza da terra, la trazione o le goome
    Hillclimb dirty: qui nasce il problema, auto basse (low ground clereance) non dovrebbero finire un corsa in salita in queste condizioni per via dell'altezza (es. Bugatti dovrebbero avere una DNF); qui l'importanza dovrebbe essere data all'altezza dell'auto (ground clereance), dalla trazione e dalle gomme; se le performance sono per l'asfalto, le all terrain/offroad/standard dovrebbero essere più performanti; le 4wd dovrebbero avere un bonus rispetto alla trazione posteriore o anteriore e l'importanza dell'altezza dal terreno dato che ci possono essere buche che rallentano le auto!!!
    Dopo aver aggiustato queste 2 tipologie diverse di Hillclimb, le altre vengono a cascata; sul bagnato, sulla sabbia, sulla neve, sulla ghiaia dove trazione e altezza dovranno avere sempre lo stesso tipo di vantaggio e le gomme influiranno in base al terreno.
    This is what i think about hillclimb, i know, in english it will be better but with google translate or my english i can't explain my thought
  • lemmings99olemmings99o Member Posts: 762 ✭✭✭✭
    edited November 22
    Just one thought that i havn't checked yet, since altering the 0-60 and top speed on the pagani killed its mra, has anyone checked the dodge stealth and the chevy lumina to see if they've also suffered with bad mra after the 0-60 was improved?
  • erjuitoerjuito Member Posts: 60
    Just one thought that i havn't lochecked yet, since altering the 0-60 and top speed on the pagani killed its mra, has anyone checked the dodge stealth and the chevy lumina to see if they've also suffered with bad mra after the 0-60 was improved?

    Funcionan como si fueran rq13-14
  • Parimal_2000Parimal_2000 Member Posts: 700 ✭✭✭✭
    @Hutch_Tim the ads don't work all 3 times. I have to restart the game each time to watch 1 ad.
  • Olli8888Olli8888 Member Posts: 568 ✭✭✭
    All 4WD cars are now to high in RQ.
  • SubaruLover04SubaruLover04 Member Posts: 326 ✭✭✭✭
    What? I'm no expert but i think i should've won.
  • bobdylanbobdylan Member Posts: 2,813 ✭✭✭✭✭
    In the mean time .... free plastics
  • SaNeRSaNeR Member Posts: 3
    edited November 27
    Give him good mid-range acceleration.He very bad.
  • evilprofesseurevilprofesseur Member Posts: 336 ✭✭✭
    SaNeR said:
    Give him good mid-range acceleration.He very bad.
    They're actually about to give it low ride height  :#
    down to ultras with it then. will have to fuse mine before that happens
«1
Sign In or Register to comment.