8000 Choosing an apt threshold for Visual Regression tests · Issue #143 · freshworks/nucleus · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Choosing an apt threshold for Visual Regression tests #143
Open
@sharath-sriram

Description

@sharath-sriram

Current Scenario

All existing components have a visual regression threshold of 0.1. This means that Visual Regression will not throw errors until the difference in the images crosses 0.1. While this works fine enough at the moment, there are times when the threshold fails to identify certain breaking changes for a few components. At such times, it is left to the developer committing the changes to responsibly identify the scope of error and reduce threshold to create new reference images and revert back to the old threshold.

What needs to be done ?

Given how each component has its own set of renders for which the visual regression tests are run, it will make sense for the developer to identify what might be an apt threshold for each of the components and set the same, ensuring that it doesn't fail on CI. It is definitely not a necessity that we should maintain a uniform threshold (open to arguments on this) for all components.

For more understanding on how the mismatch threshold is calculated, visit:
https://github.com/rsmbl/Resemble.js/blob/af57cb2f4edfbe718d24b350b2be1d956b764298/resemble.js#L495

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0