You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR refactors the usage of the softmax function across various modules within the cleanlab package. A dedicated softmax function is introduced to enhance code reusability and improve numerical stability, ensuring the codebase remains robust and maintainable.
Changes
Introduce a softmax utility function: Located in cleanlab/internal/numerics.py, this function includes options for softmax temperature, selection of axis, and numeric stability shift.
Refactored Existing Code: Replaced all explicit implementations of softmax with calls to the new softmax utility function across multiple modules (multiannotator_utils, multilabel_scorer, object_detection_utils, and token_classification/rank).
(secondary) Improved find_best_temp_scaler logic: Refactored the temperature scaling in the multiannotator_utils module to use the new softmax function. This also led to the introduction of the helper function _set_fine_search_range to better organize and segregate logic.
The PR assumes that the current testing suite covers these modules and that any potential deviations would be flagged.
It does not treat related implementations like "softmin", etc.
Usage
The softmax function can be used for both 1D and 2D arrays.
Here's a quick demonstration:
fromcleanlab.internal.numericsimportsoftmaximportnumpyasnp# Sample dataarray_1d=np.array([2.0, 1.0, 0.1])
array_2d=np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
array_2d_large_values=array_2d+1e10# 1D Array with temperature and shiftsoftmax_1d_temp=softmax(array_1d, temperature=2.0)
softmax_1d_shift=softmax(array_1d, shift=False)
print("Softmax for 1D array with temperature:", softmax_1d_temp)
print("Softmax for 1D array without shift:", softmax_1d_shift, "\n")
# 2D Array with temperature and shiftsoftmax_2d_temp=softmax(array_2d, temperature=2.0, axis=1
8000
)
softmax_2d_shift=softmax(array_2d, shift=False, axis=1)
print("Softmax for 2D array with temperature:", softmax_2d_temp)
print("Softmax for 2D array without shift:", softmax_2d_shift, "\n")
# 2D Array with large valuessoftmax_2d_large_values_temp=softmax(array_2d_large_values, temperature=2.0, axis=1)
softmax_2d_large_values_shift=softmax(array_2d_large_values, shift=False, axis=1)
print("Softmax for 2D array with large values and temperature:", softmax_2d_large_values_temp)
print("Softmax for 2D array with large values without shift:", softmax_2d_large_values_shift)
Outputs:
Softmax for 1D array with temperature: [0.50168776 0.30428901 0.19402324]
Softmax for 1D array without shift: [0.65900114 0.24243297 0.09856589]
Softmax for 2D array with temperature: [[0.18632372 0.30719589 0.50648039]
[0.18632372 0.30719589 0.50648039]]
Softmax for 2D array without shift: [[0.09003057 0.24472847 0.66524096]
[0.09003057 0.24472847 0.66524096]]
/workspaces/cleanlab/cleanlab/internal/numerics.py:36: RuntimeWarning: overflow encountered in exp
exp_x = np.exp(x)
/workspaces/cleanlab/cleanlab/internal/numerics.py:37: RuntimeWarning: invalid value encountered in divide
return exp_x / np.sum(exp_x, axis=axis, keepdims=True)
Softmax for 2D array with large values and temperature: [[0.18632372 0.30719589 0.50648039]
[0.18632372 0.30719589 0.50648039]]
Softmax for 2D array with large values without shift: [[nan nan nan]
[nan nan nan]]
Could you once run each of the main user facing methods (ie find_label_issues analog from each tutorial notebook) and verify the outputs have not changed before vs after this PR?
I'm not 100% confident any introduced mathematical error would be caught without this manual check
- Update type hint for `min_entropy_ind` from built-in `int` to `np.intp`.
- This refinement addresses a type compatibility warning.
- `np.intp` is the integer type used by numpy for indexing and can differ in size from the built-in Python `int` depending on the platform (32-bit vs 64-bit).
- Mypy highlighted this type hint discrepancy.
Could you once run each of the main user facing methods (ie find_label_issues analog from each tutorial notebook) and verify the outputs have not changed before vs after this PR?
I'm not 100% confident any introduced mathematical error would be caught without this manual check
Verified that the outputs of the corresponding methods are not affected.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR refactors the usage of the softmax function across various modules within the cleanlab package. A dedicated softmax function is introduced to enhance code reusability and improve numerical stability, ensuring the codebase remains robust and maintainable.
Changes
softmaxutility function: Located incleanlab/internal/numerics.py, this function includes options for softmax temperature, selection of axis, and numeric stability shift.softmaxutility function across multiple modules (multiannotator_utils, multilabel_scorer, object_detection_utils, and token_classification/rank).find_best_temp_scalerlogic: Refactored the temperature scaling in the multiannotator_utils module to use the new softmax function. This also led to the introduction of the helper function_set_fine_search_rangeto better organize and segregate logic.The PR assumes that the current testing suite covers these modules and that any potential deviations would be flagged.
It does not treat related implementations like "softmin", etc.
Usage
The
softmaxfunction can be used for both 1D and 2D arrays.Here's a quick demonstration:
Outputs: