8000 Rework search indexing / crawling by Toflar · Pull Request #981 · contao/contao · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Rework search indexing / crawling #981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

Toflar
Copy link
Member
@Toflar Toflar commented Nov 14, 2019

Implements our https://github.com/terminal42/escargot library.

Escargot can crawl all Contao pages like any other search engine does.
Processing the results is achieved by subscribers and we are now not limited to the search indexer anymore. We may add additional subscribers for features such as checking broken links or accessibility checks (missing alt attributes on images etc.).
However, replacing the existing search logic and providing the framework for additional subscribers is what this PR is all about.

In addtion, everything that can be done in the back end is now also possible using the new contao:crawl command.

I will need styling support here.

Let's bring Contao to the next level :)

leofeyer pushed a commit that referenced this pull request Dec 4, 2019
Description
-----------

This is a sub PR of #981 only containing the CLI variant. It will allow us to focus more on the technical implementation first before we have all UI/CSS changes.

Commits
-------

7bcce12 Implemented contao:crawl command
c305881 Removed left-over controller
35e68cc Added unit tests for the Escargot factory
9a2f0ea CS
772ed1f Added tests for the subscriber result and fixed issues found along the way
0c8d30b Added unit tests for the searchindexsubscriber
ceff452 CS
e52c855 Added unit test for the EscargotSubscriberPass
749e00b Added unit tests for the extension as well as configuration
10f18aa CS
281e43b Adjusted to latest Escargot changes
452dfcc Added more command options and unit tests for the command
7d22b89 Merge branch 'master' into feature/crawl-command
072be02 CS
b990477 Catch the new indexer exceptions in SearchIndexListener
06b6d9a Updated the deps
7d2df71 Fix the coding style
63b0fdf CS
369ad57 Merge branch 'master' into feature/crawl-command
2d95608 Add unit tests for the service definitions
e714b07 Correctly name the interface
leofeyer pushed a commit to contao/core-bundle that referenced this pull request Dec 4, 2019
Description
-----------

This is a sub PR of contao/contao#981 only containing the CLI variant. It will allow us to focus more on the technical implementation first before we have all UI/CSS changes.

Commits
-------

7bcce121 Implemented contao:crawl command
c305881f Removed left-over controller
35e68cc2 Added unit tests for the Escargot factory
9a2f0ea5 CS
772ed1fb Added tests for the subscriber result and fixed issues found along the way
0c8d30b0 Added unit tests for the searchindexsubscriber
ceff4529 CS
e52c8550 Added unit test for the EscargotSubscriberPass
749e00b9 Added unit tests for the extension as well as configuration
10f18aa5 CS
281e43ba Adjusted to latest Escargot changes
452dfcc1 Added more command options and unit tests for the command
7d22b89c Merge branch 'master' into feature/crawl-command
072be02f CS
b9904779 Catch the new indexer exceptions in SearchIndexListener
06b6d9a4 Updated the deps
7d2df716 Fix the coding style
63b0fdfc CS
369ad573 Merge branch 'master' into feature/crawl-command
2d956081 Add unit tests for the service definitions
e714b074 Correctly name the interface
@Toflar
Copy link
Member Author
Toflar commented Dec 4, 2019

Closing this now as all the work was split into multiple PR's.
Foundation was merged in #985 and the back end implementation is continued in #1057

@Toflar Toflar closed this Dec 4, 2019
@Toflar Toflar deleted the feature/rework-search-indexing branch December 4, 2019 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant
0