Aggregate stat service #9757

alanhartless · 2021-03-08T04:19:10Z

Q	A
Branch?	"features"
Bug fix?	no
New feature?	yes
Deprecations?	no
BC breaks?	no
Automated tests included?	yes
Related user documentation PR URL	na
Related developer documentation PR URL
Issue(s) addressed

This adds the service to use a new event to collect stats to allow 3rd party to manage and inject stats. The email graph's TEMPLATE graph has been rewritten to use it. Note that a lot of the statistic collection for the email was refactored into individual classes an are not necessarily part of the example for how to use the service.

(Note this PR is a blocker to Acquia contributing a number of PRs as code changed depends on this code).

https://github.com/mautic/mautic/pull/9757/files#diff-a9a319bd79a694d0b73f80b6ceb80b383601bd77b0d943890d35eb5275f1d6b8R65 demonstrates how to dispatch a request to collect a specific set of stats (by hour) and formats it for whatever its purpose is. In this case, it's the data for chart.js.

https://github.com/mautic/mautic/pull/9757/files#diff-20b8dee755b891bca4995d9bfe8f2d065b0ec8577261fda99bc1fec627eea70dR44 listens to the event dispatched by the collector service.

https://github.com/mautic/mautic/pull/9757/files#diff-a9a319bd79a694d0b73f80b6ceb80b383601bd77b0d943890d35eb5275f1d6b8R145 injects the stats into the StatCollection.

To test this PR for the email graph,

create a template email.
create a campaign to send that email to some contacts.
Open some of the emails, click on the links in some of the emails, and unsubscribe from one of the emails.
View the email details page to see if the stats are legit.

codecov · 2021-03-08T05:12:52Z

Codecov Report

Merging #9757 (85f8624) into features (07cd6b4) will increase coverage by 0.02%.
The diff coverage is 38.03%.

@@              Coverage Diff               @@
##             features    #9757      +/-   ##
==============================================
+ Coverage       39.67%   39.70%   +0.02%     
- Complexity      33978    34163     +185     
==============================================
  Files            1989     2016      +27     
  Lines          105731   106275     +544     
==============================================
+ Hits            41951    42192     +241     
- Misses          63780    64083     +303

Impacted Files	Coverage Δ	Complexity Δ
.../bundles/CoreBundle/Helper/Chart/AbstractChart.php	`72.72% <ø> (-2.86%)`	`23.00 <0.00> (-11.00)`
...ailBundle/Controller/EmailGraphStatsController.php	`0.00% <ø> (ø)`	`8.00 <0.00> (ø)`
...mailBundle/Stats/FetchOptions/EmailStatOptions.php	`0.00% <0.00%> (ø)`	`14.00 <14.00> (?)`
...p/bundles/EmailBundle/Stats/Helper/FilterTrait.php	`0.00% <0.00%> (ø)`	`12.00 <12.00> (?)`
...bundles/StatsBundle/Event/Options/FetchOptions.php	`0.00% <0.00%> (ø)`	`5.00 <5.00> (?)`
app/bundles/EmailBundle/Model/EmailModel.php	`32.87% <3.03%> (+1.79%)`	`319.00 <18.00> (-24.00)`	⬆️
...bundles/EmailBundle/Stats/Helper/ClickedHelper.php	`9.52% <9.52%> (ø)`	`4.00 <4.00> (?)`
...undles/EmailBundle/Stats/Helper/AbstractHelper.php	`10.20% <10.20%> (ø)`	`16.00 <16.00> (?)`
...bundles/EmailBundle/Stats/Helper/BouncedHelper.php	`11.76% <11.76%> (ø)`	`3.00 <3.00> (?)`
...es/EmailBundle/Stats/Helper/UnsubscribedHelper.php	`11.76% <11.76%> (ø)`	`3.00 <3.00> (?)`
... and 48 more

kuzmany · 2021-03-08T07:13:44Z

@alanhartless can you point the lines with dispatcher again? Now does not show the right lines

alanhartless · 2021-03-08T15:28:14Z

@alanhartless can you point the lines with dispatcher again? Now does not show the right lines

@kuzmany updated

kuzmany

I've checked the code and looks very nice. I also made some basic tests on emails stats and works properly.
I'm exited to see it in core.
Unit test included.
Count Acquia use it in production.
Then we can merge it :

…r methods for now

…o 1 second

…me result.

…ek format methods

update phpdoc year Co-Authored-By: galvani <galvani78@gmail.com>

Co-Authored-By: galvani <galvani78@gmail.com>

Deleting file in "unit" to be re-added under "Unit" (aggravating case insensitive filesystem)

RCheesley

Based on the +1 from @kuzmany and that Acquia has this in production, LGTM! 🚀

cla-bot bot added the cla-signed The PR contributors have signed the contributors agreement label Mar 8, 2021

RCheesley added feature A new feature for inclusion in minor or major releases reports Anything related to reports T3 Hard difficulty to fix (issue) or test (PR) labels Mar 8, 2021

RCheesley added this to the Mautic 4.0 milestone Mar 8, 2021

RCheesley changed the title ~~Aggretate stat service~~ Aggregate stat service Mar 8, 2021

kuzmany previously approved these changes Mar 9, 2021

View reviewed changes

RCheesley added the has-conflicts Pull requests that cannot be merged until conflicts have been resolved label Mar 28, 2021

alanhartless added 19 commits March 28, 2021 11:09

StatBundle for stat collection

4fb4927

Stat DAO that calculate

36920f2

Dispatch event to collect the stats

7066bae

Created an object to store stats

7d41234

Calculator for summing and averaging stats

d392d69

Fixed stat objects where getStat was not useful.

36b3bb5

Add StatsDAO test

117d9ba

Added proxy to get Calculator

3e3b02c

Collector test

4f67af5

Format key once so it doesn't have to be done everywhere else

1961962

Fixed average methods

334036f

Add means to pass options to listeners

ef1a4a9

removed get average in favor of get count

1ec48c0

Fixed namespace

42bdc3c

Add config file

247342c

Resolve conflicts with c6acef3d10e50ab0063816d161dbcc2123b40269

0826ad7

Resolve conflicts with 36df6286210943245b40e15f94b94eb945b854e9

d8e0121

Fixed filling in missing days and got rid of unused average calculato…

a90cd12

…r methods for now

10000 Resolve conflicts with 9d5024ba335b8145b2014fef2eb7d34ef3356323

64fb202

alanhartless and others added 23 commits March 28, 2021 11:13

Fixed bad merge

32ef139

Use generated column if available to speed up query from 15 minutes t…

0f9d57d

…o 1 second

Use 1 method to generate units from date range. It must return the sa…

db5d6ea

…me result.

CS fix

cd5cc19

DateRangeUnitTrait fully tested

95f33ec

add week methods to stats, fix addStat to use 24hrs format, create we…

5b93308

…ek format methods

fix code style

90c95c9

revert stupidity

6bb31c9

Update app/bundles/StatsBundle/Aggregate/Collection/Stats/WeekStat.php

2b07ab7

update phpdoc year Co-Authored-By: galvani <galvani78@gmail.com>

Update app/bundles/StatsBundle/Aggregate/Helper/CalculatorHelper.php

170a172

Co-Authored-By: galvani <galvani78@gmail.com>

Update app/bundles/StatsBundle/Aggregate/Helper/CalculatorHelper.php

b76f198

Co-Authored-By: galvani <galvani78@gmail.com>

Update app/bundles/StatsBundle/Aggregate/Helper/CalculatorHelper.php

6e000da

Co-Authored-By: galvani <galvani78@gmail.com>

make correct use of week unit in queries

581c3ef

remove comment

e1204c2

simplify week data calculation

837b2b2

Fixed tests

d06eabc

Delete DateRangeUnitTraitTest.php

628d3ee

Deleting file in "unit" to be re-added under "Unit" (aggravating case insensitive filesystem)

Added back DateRangeUnitTraitTest

316d212

Fixed namespace for DateRangeUnitTraitTest

7c4e38d

CS fixes

ef62411

Rector fixes

dd132be

PHPStan fixes from Rector fixes

6DB6

bb17f68

More PHPStan fixes from Rector fixes

fdac857

alanhartless dismissed kuzmany’s stale review via fdac857 March 28, 2021 16:15

alanhartless force-pushed the feature-aggregate-stat-service branch from 3f36d4e to fdac857 Compare March 28, 2021 16:15

RCheesley approved these changes Mar 28, 2021

View reviewed changes

RCheesley added the ready-to-commit PR's with 2 successful tests, 1 approval, automated tests and docs and is ready to be merged label Mar 28, 2021

Merge branch 'features' into feature-aggregate-stat-service

85f8624

RCheesley merged commit 489b00d into mautic:features Mar 29, 2021

kuzmany mentioned this pull request Sep 23, 2021

Dashboard Fatal Error after Upgrade from 3.3.4 to 4.0.0 #10463

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Aggregate stat service #9757

Aggregate stat service #9757

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Aggregate stat service #9757

Aggregate stat service #9757

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!