8000 Dispatcher updates subscriber statuses by vaikas · Pull Request #4435 · knative/eventing · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Dispatcher updates subscriber statuses #4435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Oct 30, 2020

Conversation

vaikas
Copy link
Contributor
@vaikas vaikas commented Oct 29, 2020

Signed-off-by: Ville Aikas vaikas@vmware.com

Fixes #

Proposed Changes

  • Do not update subscriber statuses from the control plane, do it from the dispatcher by using patch to only modify the subscriber statuses

Release Note

- 🐛 Fix bug
Only update the subscriber statuses in IMC after they have been added to handlers. Reduces failures where the
subscribers have been marked before the dataplane has been actually configured.

Docs

Signed-off-by: Ville Aikas <vaikas@vmware.com>
@knative-prow-robot knative-prow-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 29, 2020
@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vaikas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow-robot knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 29, 2020
@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Oct 29, 2020
Copy link
Contributor
@antoineco antoineco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea! A few early comments.

Ready: corev1.ConditionTrue,
})
}
patch, err := duck.CreateMergePatch(imc, after)
Copy link
Contributor
@antoineco antoineco Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the merge patch correctly cause a removal of deleted subscribers from the conditions slice? (I think it depends on how the merge behaviour is configured in the CRD?)

8000
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course I need to write tests for it and so forth, but yes it should because when we create the patch, we start from scratch and only add the current subscribers. So, we spin through the spec.subscribers, and if one is removed it won't get added to this list and hence it will be dropped. But "should" just work.

Copy link
Contributor
@antoineco antoineco Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm asking because I've seen weird behaviors with slices when the patchMergeKey isn't set. Old entries can remain in the existing list, even when removed from the "desired" list. That's also how you end up with 2 containers in a Deployment after apply-ing a manifest where containers[0].name has changed.

ref. https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/#notes-on-the-strategic-merge-patch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, we're not doing a strategic merge patch, just a merge patch, but I'll keep this in mind. I need to see if the strategic merge does something different, great point 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol @antoineco you were so right :) And I'm laughing because I now remember in the early days of duck typing we had to switch to json patch just for this reason :) I actually had to upstream a patch to get jsonpatch fakes working lol. geez... Switching to JSON patch :)

@vaikas
Copy link
Contributor Author
vaikas commented Oct 29, 2020

As I mentioned, I was expecting some UT to fail, but it's nice to see all the e2e tests passsing, here and in kind. 👍

@slinkydeveloper
Copy link
Contributor

I like the overrall direction of this patch, it should simplify patching kafkachannel too @devguyio

8000
Signed-off-by: Ville Aikas <vaikas@vmware.com>
Signed-off-by: Ville Aikas <vaikas@vmware.com>
@knative-prow-robot knative-prow-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 30, 2020
@vaikas vaikas changed the title [WIP] dispatcher updates subscriber statuses Dispatcher updates subscriber statuses Oct 30, 2020
@knative-prow-robot knative-prow-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 30, 2020
Signed-off-by: Ville Aikas <vaikas@vmware.com>
Copy link
Contributor
@antoineco antoineco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more questions/comments

@@ -21,6 +21,10 @@ import (
"net/http"
"testing"

"k8s.io/apimachinery/pkg/types"

eventingclient "knative.dev/eventing/pkg/client/injection/client"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be?:

Suggested change
eventingclient "knative.dev/eventing/pkg/client/injection/client"
fakeeventingclient "knative.dev/eventing/pkg/client/injection/client/fake"

Most ctors request fakes directly instead of using that indirect injection if I'm not mistaken. e.g.

fakeeventingclient.Get(ctx), listers.GetApiServerSourceLister(),

fakeeventingclient.Get(ctx), listers.GetInMemoryChannelLister(),

etc.

Signed-off-by: Ville Aikas <vaikas@vmware.com>
Signed-off-by: Ville Aikas <vaikas@vmware.com>
@vaikas
Copy link
Contributor Author
vaikas commented Oct 30, 2020

Thanks @antoineco I think I addressed all your feedback, PTAL.

Signed-off-by: Ville Aikas <vaikas@vmware.com>
@codecov
Copy link
codecov bot commented Oct 30, 2020

Codecov Report

Merging #4435 into master will decrease coverage by 0.01%.
The diff coverage is 81.81%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4435      +/-   ##
==========================================
- Coverage   81.08%   81.06%   -0.02%     
==========================================
  Files         281      281              
  Lines        7966     7981      +15     
==========================================
+ Hits         6459     6470      +11     
- Misses       1120     1122       +2     
- Partials      387      389       +2     
Impacted Files Coverage Δ
...iler/inmemorychannel/controller/inmemorychannel.go 68.49% <ø> (-1.25%) ⬇️
...iler/inmemorychannel/dispatcher/inmemorychannel.go 88.52% <80.95%> (-4.16%) ⬇️
...econciler/inmemorychannel/dispatcher/controller.go 78.26% <100.00%> (+0.48%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af98d55...a2d0621. Read the comment docs.

Signed-off-by: Ville Aikas <vaikas@vmware.com>
@knative-metrics-robot
Copy link

The following is the coverage report on the affected files.
Say /test pull-knative-eventing-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/inmemorychannel/controller/inmemorychannel.go 76.1% 75.5% -0.5
pkg/reconciler/inmemorychannel/dispatcher/inmemorychannel.go 93.8% 91.8% -1.9

Copy link
Contributor
@antoineco antoineco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, I'm just curious about that specific import in the tests.

Sorry was looking at an old diff.

@@ -408,6 +495,7 @@ func TestReconciler_ReconcileKind(t *testing.T) {
},
}
for n, tc := range testCases {
ctx, fakeEventingClient := fakeeventingclient.With(context.Background(), tc.imc)
Copy link
Contributor
@antoineco antoineco Oct 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this already be injected in all tests by MakeFactory, due to the "magic" fakeeventingclient "knative.dev/eventing/pkg/client/injection/client/fake" import?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh no, ReconcileKind is called manually in those tests.

/lgtm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I broke it into two tests, because what I really wanted to do was to see how the resources are handled (using the magic factory) and how the channels are configured by manually calling it. Our test infra could make this easier :)

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 30, 2020
@vaikas
Copy link
Contributor Author
vaikas commented Oct 30, 2020

not really sure what failed in the unit tests:

PASS
ok  	knative.dev/eventing/pkg/reconciler/apiserversource/resources	0.293s
/root/.gvm/gos/go1.14.10/pkg/tool/linux_amd64/link: signal: killed

@vaikas
Copy link
Contributor Author
vaikas commented Oct 30, 2020

/test pull-knative-eventing-unit-tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0