Improve sorting of data in reports #817

Spacetown · 2023-08-03T20:50:05Z

Sort file names alpha numerical (file_10.c comes after file_0.c).
Always sort at the end by filename if line or branch coverage is identical for a file.
Add --sort-branches to sort by branches instead of lines, this is the default if --branch is used.
Add --sort-decreasing to start with the files with lower coverage values.

Closes #810

The option --show-branch now sets the new option --sort-branches.

If sorting by line or branch coverage same values are additional sorted by filename.

codecov · 2023-08-03T21:30:40Z

Codecov Report

Patch coverage: 97.36% and project coverage change: +0.02% 🎉

Comparison is base (4d2a8d3) 95.61% compared to head (63f335e) 95.63%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #817      +/-   ##
==========================================
+ Coverage   95.61%   95.63%   +0.02%     
==========================================
  Files          44       44              
  Lines        4306     4328      +22     
  Branches      841      844       +3     
==========================================
+ Hits         4117     4139      +22     
  Misses        111      111              
  Partials       78       78

Files Changed	Coverage Δ
gcovr/configuration.py	`99.50% <ø> (ø)`
gcovr/formats/base.py	`77.77% <ø> (ø)`
gcovr/formats/csv/write.py	`100.00% <ø> (ø)`
gcovr/formats/html/write.py	`95.60% <ø> (ø)`
gcovr/formats/json/write.py	`96.73% <ø> (ø)`
gcovr/formats/txt/__init__.py	`100.00% <ø> (ø)`
gcovr/coverage.py	`97.21% <93.33%> (+0.03%)`	⬆️
gcovr/__main__.py	`96.41% <100.00%> (+0.11%)`	⬆️
gcovr/formats/txt/write.py	`97.80% <100.00%> (ø)`
gcovr/tests/test_args.py	`99.16% <100.00%> (+0.02%)`	⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…_reports

ligurio · 2023-08-09T12:24:59Z

Patches version: 697eead, branch: Spacetown:change_sorting_in_reports

With --sort-decreasing sorting works in unexpected order:

With --sort-decreasing and --sort-percentage:

Spacetown · 2023-08-10T18:37:22Z

With --sort-decreasing sorting works in unexpected order:

But this is expected because the file name is used as default sorting column.

With --sort-decreasing and --sort-percentage:

Is it possible that you also used --show-branches or --sort-branches?

I think renaming the option to --sort-reverse will be better because the uncovered lines are sorted increasing by default and the percentage is sorted for historical reason decreasing.

…anch coverage

Spacetown · 2023-08-11T21:15:41Z

@ligurio What do you think? Can this be merged or is a change needed?

ligurio · 2023-08-13T08:56:07Z

This doesn't work for me at all.

--sort-uncovered:

Other used options:

        # See https://gcovr.com/en/stable/guide/configuration.html                 
        --root ${PROJECT_SOURCE_DIR}                                               
        --object-directory ${PROJECT_BINARY_DIR}                                   
        --filter ${PROJECT_SOURCE_DIR}/src                                         
        # Exclude DynASM files, that contain a low-level VM code for CPUs.         
        --exclude ".*\.dasc"                                                       
        # Exclude buildvm source code, it's the project's infrastructure.          
        --exclude ".*/host/"                                                       
        --print-summary                                                            
        --output ${COVERAGE_HTML_REPORT}                                           
        --cobertura ${COVERAGE_XML_REPORT}                                         
        --html                                                                     
        --html-title "Tarantool LuaJIT Code Coverage Report"                       
        --html-details                                                             
        --branches                                                                 
        --decisions                                                                
        -j ${CMAKE_BUILD_PARALLEL_LEVEL}

Let me remind a goal in the original issue: sort uncovered lines (or branches) by descending order. I have tried all options in different combinations and all of them are not suitable for me and doesn't help to reach a goal.

You decided to introduce new options and this makes usage more complicated for users. Take a look a usage with sorting options, it is not trivial to get all of them, some of them are mutual exclusive:

  --sort-branches       Sort entries by branches instead of lines. Can only be
                        used together with --sort-uncovered or --sort-percent
                        is used. Config key(s): sort-branches.
  -u, --sort-uncovered  Sort entries by number of uncovered lines (if the
                        option --sort-branches is given the uncovered branches
                        are used). The default order is increasing and can be
                        changed by --sort-reverse. The secondary sort key (if
                        values are identical) is always the ascending
                        filename. For CSV, HTML, JSON and text report. Config
                        key(s): sort-uncovered.
  -p, --sort-percentage
                        Sort entries by percentage of uncovered lines (if the
                        option --sort-branches is given the percentage of
                        uncovered branches are used). The default order is
                        decreasing and can be changed by --sort-reverse.The
                        secondary sort key (if values are identical) is always
                        the ascending filename. For CSV, HTML, JSON and text
                        report. Config key(s): sort-percentage.
  --sort-reverse        Sort entries in reverse order. For CSV, HTML, JSON and
                        text report. Config key(s): sort-reverse.
  --sort-casefold       Sort file names with case folding. For CSV, HTML, JSON
                        and text report. Config key(s): sort-casefold.

I suggest removing options for separate sorting by branches/lines/functions etc. and left only option sort-by-ascending that will sort by ascending all columns and enable sorting by descending by default.

Spacetown · 2023-08-14T19:16:58Z

In the command line you've the option --branches which switches from sorting by line coverage to branch coverage. This behavior only make sense for the text format since the CSV, JSON and HTML always contain branch information and should use the new option --sort-branches instead.
The option --sort-casefoldcan be the default, removing the option again.

The default sort order for sort-uncovered and --sort-precentage wasn't changed only a option for changing the direction was added.

Both of your screenshots have the same option mentioned and since you'r having the option --branches the sorting is done by branches instead of lines.

If you use --sort-uncovered, your first screenshot is correct since the uncovered branches are sorted from 974-342=632 to 110-74=36.

The option is only used for text report. All other formats need to use --sort-branches instead of --branches to sort output by branches.

…_reports

Default is sorting files with casefold.

Spacetown · 2023-08-15T20:48:31Z

I suggest removing options for separate sorting by branches/lines/functions etc. and left only option sort-by-ascending that will sort by ascending all columns and enable sorting by descending by default.

But which column shall than be the sorted one?

We need following options:

One to select between default, line or branch coverage.
One to select the absolute uncovered stuff or the percentage values.
One to change between default order or reversed order. Here we have the problem that for the absolute uncovered values ascending is the default order and for the percentage descending is the default.

…_reports

Spacetown · 2023-09-06T19:53:08Z

@ligurio Any notes from you?

…_reports

- Sort file names alpha numerical and with casefold (see https://docs.python.org/3.11/library/stdtypes.html?highlight=str%20casefold#str.casefold) (`file_10.c` comes after `file_0.c`). - Always sort at the end by filename if line or branch coverage is identical for a file. - Add option --sort-branches to sort by branches instead of lines, this is the default if option --txt-branches is used. - Add option --sort-reverse to reverse the sort order.

Spacetown added 7 commits August 3, 2023 22:35

Add error if multiple sort options are used

8d44cbf

Rename sort functions to have a common prefix

25d182b

Update documentation of sort options

b63abf7

Change sort order to alpha numerical

ca8a22f

Add option to sort decreasing and by branch

e04ce67

The option --show-branch now sets the new option --sort-branches.

Allow to change HTML report to be sorted by branch

487a5ed

Always use file name as last level of sort

f2d2b5b

If sorting by line or branch coverage same values are additional sorted by filename.

Spacetown added Type: Enhancement Format: Text Format: HTML Format: JSON Format: CSV labels Aug 3, 2023

Spacetown added this to the Upcoming release milestone Aug 3, 2023

Update CHANGELOG.md

1454334

Spacetown force-pushed the change_sorting_in_reports branch from 38db0ea to 1454334 Compare August 3, 2023 21:02

Fix flake8 error

17a7ebd

Spacetown force-pushed the change_sorting_in_reports branch from 5ca93c3 to 17a7ebd Compare August 3, 2023 21:13

Spacetown added 2 commits August 3, 2023 23:25

Update reference data because of sort level by file name

df8929e

Fix CHANGELOG

321a67d

Merge remote-tracking branch 'upstream/master' into change_sorting_in…

6361802

…_reports

Spacetown marked this pull request as ready for review August 4, 2023 19:14

Spacetown added 3 commits August 4, 2023 21:24

Merge remote-tracking branch 'upstream/master' into change_sorting_in…

9f18d2c

…_reports

Add option to sort with casefold

b2fc355

Extend tests

e016df6

Spacetown force-pushed the change_sorting_in_reports branch from 3ee6906 to a8e6336 Compare August 7, 2023 04:26

Add test for --sort-casefold

697eead

Spacetown force-pushed the change_sorting_in_reports branch from a8e6336 to 697eead Compare August 7, 2023 19:31

Spacetown force-pushed the change_sorting_in_reports branch from 123d53c to 96b73a0 Compare August 10, 2023 19:51

Spacetown added 3 commits August 10, 2023 22:04

Rename --sort-decreasing to --sort-reverse

c88fdb3

Fix CI error

aa46756

Ignore --sort-reverse for file name if primary sort key is line or br…

c9a360e

…anch coverage

Spacetown force-pushed the change_sorting_in_reports branch from 96b73a0 to c9a360e Compare August 10, 2023 20:04

Remove not needed tests

0166319

Spacetown force-pushed the change_sorting_in_reports branch 2 times, most recently from 4fc0444 to d43a3a7 Compare August 11, 2023 20:45

Use casefold after modifying the key

d299750

Spacetown force-pushed the change_sorting_in_reports branch from d43a3a7 to d299750 Compare August 11, 2023 20:55

Spacetown added 4 commits August 14, 2023 21:23

Change primary option --branch to --txt-branch

9ec0161

The option is only used for text report. All other formats need to use --sort-branches instead of --branches to sort output by branches.

Sort case insensitive by default

3501312

Merge remote-tracking branch 'upstream/master' into change_sorting_in…

f4addc4

…_reports

Update reference data

d6e67a9

Default is sorting files with casefold.

Spacetown force-pushed the change_sorting_in_reports branch from 29445c9 to d6e67a9 Compare August 15, 2023 20:31

Merge remote-tracking branch 'upstream/master' into change_sorting_in…

273b39f

…_reports

Spacetown added 2 commits September 14, 2023 21:13

Change sort order to be always increasing

07502fe

Merge remote-tracking branch 'upstream/master' into change_sorting_in…

63f335e

…_reports

Spacetown merged commit 0418545 into gcovr:master Sep 15, 2023

Spacetown deleted the change_sorting_in_reports branch September 15, 2023 19:50

Spacetown removed this from the Upcoming release milestone Jan 26, 2024

Spacetown mentioned this pull request Apr 20, 2024

Sort files on actual uncovered percentage #918

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve sorting of data in reports #817

Improve sorting of data in reports #817

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Improve sorting of data in reports #817

Improve sorting of data in reports #817

Uh oh!

Conversation

Uh oh!

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!