8000 Release 0.2 to mangaki.fr by RaitoBezarius · Pull Request #381 · mangaki/mangaki · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Release 0.2 to mangaki.fr #381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 75 commits into from
Jul 2, 2017
Merged

Release 0.2 to mangaki.fr #381

merged 75 commits into from
Jul 2, 2017

Conversation

RaitoBezarius
Copy link
Member

No description provided.

RaitoBezarius and others added 30 commits February 20, 2017 19:59
* Unify KNN

* Add __init__.py in test folder

* Allow ANY algorithm in production

* Take reviews into account

* Last changes
* Factorize algorithms, create Dataset class and fit_algo management command

* Fix bug

* Remove useless code
* Add new class RecommendationAlgorithm and refactor algorithms

* Fix typo, BTW this PR fixes #267

* Add comment

* Cleanup

* nb_users and nb_works are instance attributes
* Remove first_name and last_name from Artist

* Remove from model as well
This is a first step towards the resolution of #59. It allows
anonymous users to rate works, and keeps the state of the rated works
across consecutive page loads in Django's session.

For now, the usage of this information is limited: although anonymous
user can rate works, they cannot get recommendations from those
ratings, transfer them to a new or existing account, etc. Support for
these will come in subsequent patches.
* Since TensorFlow 1.0, MangakiWALS models cannot be saved anymore

* Fix pickle path everywhere

* Add minimal test for recommendations

* Fix unused parameter
This is a second step towards the resolution of #59. It allows
anonymous users to get recommendations (using the KNN algorithm only,
since we do not have trained models for them).

This is implemented by making the various algorithms take directly a
list or (user_id, work_id, choice) triplets instead of building those
from a queryset, as well as some glue to add the relevant triplets
from anonymous users at the interface between the database and the
algorithms themselves.
The seed_data.json was not updated after first_name and last_name were
removed from Artist. This fixes that.
* Simplify circle config

* Fix matplotlib.pyplot

* Speculative improvements

* relative paths...

* Move coverarc to mangaki

* Try enabling test reports

* Test reports, take 2

* Parallel run is slower
This provides little information (especially considering how cryptic what it
does is), and the way it is computed gets in the way whenever we want to change
things relative to ratings.

Note that it would be easy to add this again if the needs arises as even though
the value is dynamically updated in various places, it is easy to re-compute it
from the sets of Suggestions and Recommendations the user submitted.
We are not using it anymore. Fixes #264.
Allow users without an existing account to import anonymous ratings upon
account creation.

The backend logic behind this is relatively simple and simply requires
overriding django-allauth's signup view to move the anonymous ratings from the
session to the database, associating them to the freshly created user.

On the frontend side, this adds:
 - On the signup page, if the user has made any anonymous ratings, we add a
   checkbox allowing them to import their ratings (active by default), as well
   as a condensed view of their existing ratings. This leverages the
   works_no_poster.html view and allows the user to change their anonymous
   ratings directly on the signup page if they wish to do so.

 - On the login page, if the user has made any anonymous ratings, we add a
   warning message telling them that they will lose their anonymous ratings
   upon login (even though technically, they lose it upon logging out) and that
   they should create a new account if they want to keep those ratings. In the
   future, this should be replaced with a mechanism to merge existing anonymous
   ratings with the user's existing ratings.

As a byproduct, this patch includes a slight re-design of the login and signup
pages, and changes the library we use for displaying bootstrap forms from
django-bootstrap-form to django-bootstrap3. django-bootstrap3 provides better
hooks to customize forms and made said re-design much easier than it would have
been with django-bootstrap-form (in addition, it is
[faster](zostera/django-bootstrap3#160))
* tests: Enable AniDB testing through mocking using responses library

→ Minor fixes in `anidb.py` file (default arguments)
→ Add fixture directory and test fixture directory in settings.
→ Use test fixture directory to feed mocking (fresh fixture from AniDB!)
→ Test `AniDB.search`.
→ Test `AniDB.get_dict`.
→ Add AniDB constants inside the class for convenience purpose.
→ Reformatting and imports optimization.
* WorkList: use a property for `is_dpp`

* management command: fix analytics

* knn: fix constructor and reformat according to PEP8.

* chrono: remove unused keyword argument.

* imports: clean unused imports.

* knn: remove the ugly [\n and put a comment on its own line

* knn: moar formatting implementation

* management command: remove analytics
…287)

* anonymous-ratings: Review page, clear all ratings, remainder banner

- Add a remainder banner of anonymous ratings.
- Add a review page by reusing `get_profile` view.
- Add a way to clear all ratings (ugly link)

* chore: reformat for better readability
* admin: reformat code and remove useless classes

* admin: improve code style
* Update about and events

* Add videos

* Fix Soubi and Elarnon

* Fix OpenGraph and Twittercard

* Okay, it was a bad idea

* Remove TensorFlow logging at once

* Let's forget about removing logging
* Since TensorFlow 1.0, MangakiWALS models cannot be saved anymore

* Fix pickle path everywhere

* Add minimal test for recommendations

* Fix unused parameter

* Add notebook for benchmarks for cold-start

* Fix KNN

* Add LinearRegression for cold-start

* Add MangakiProxyDPP

* Cleanup compare management command

* Edit experiment

* Add computation of delta and update for ALS

* Add BGS from Anava et al.'s WWW 2015 paper

* Fix BGS, withdraw paper from RecSys

* Fix NMF

* Add Autoencoder with TensorFlow

* Now users receive SVD if it exists, otherwise KNN

* Order whole rating list by any algorithm

* Finally manage those damn logs

* Add logging and fix paths

* Fix DPP and clean a lot of code

* Remove useless print

* Oops, forgot one file

* Fix problems linked to merge
* Add an admin interface for merging works

* Factor code

* Add minor changes

* Fix bug, and Language to admin

* Take Raito changes into account

* Fix bugs and display number of ratings of each work
* mal-import: refactor the whole code into something more maintainable

- Fix doctor's fields and use MALEntry.
- MAL:
        - Refactor the whole module using a requests.Session and
          instantiating a MALClient which handles the API operations.
        - Rewrite the DB interaction to be more efficient and fix it in
          some edge cases.

* add a default user-agent

* mangaki: fix when MAL import is not available

* mal: add some tests and refactor duplicated / unused code

* mal: rewrite the doctests with None

* mal: add missing fixtures
* mal-import: refactor the whole code into something more maintainable

- Fix doctor's fields and use MALEntry.
- MAL:
        - Refactor the whole module using a requests.Session and
          instantiating a MALClient which handles the API operations.
        - Rewrite the DB interaction to be more efficient and fix it in
          some edge cases.

* add a default user-agent

* mangaki: fix when MAL import is not available

* mal: add some tests and refactor duplicated / unused code

* mal: rewrite the doctests with None

* mal: add missing fixtures

* MAL: add synonyms and prepare for genres / types

→ Make language a nullable field for WorkTitle.
→ Added the related migration.
RaitoBezarius and others added 21 commits June 22, 2017 00:09
* recommendations: load all ratings if only and only we will fit the algorithm (SVD, ALS, WALS, …)

* Proper handling of algo_name
…e from admin) (#307)

* Add WorkCluster to admin

* Add migration

* Allow call to merge from WorkCluster admin

* Add test for merge action

* Add all_objects manager and format_html

* Add migration file for manager

* Butchering migrations

* Make a better test for merging, in order to reduce the number of queries

* Cover all merge functions with tests, reduce the number of queries which was tremendous

* Resolve multiplication of migration leaves

* Cut line
* Kill /users route

* Remove user_list template
…r) (#357)

* Add get_entry_from_work and fix get_potential_posters for admin

* Address PR comments

* Thanks Django ; I thought I didn't know how to compute in Boole's algebra.

* Remove moar useless comments
We add a data migration to support many AniDB and MAL languages, fixing the previous inability of administrators to edit languages of a WorkTitle.

Then, we make MAL import and AniDB API aware of these models so that they can attach more data to our works when operating (e.g. importing, adding new works in the DB).

Finally, we add a new administrator action to refresh WorkTitle of a work linked to AniDB, also its `ext_synopsis` which will be useful for l10n of Mangaki I suppose.
This new rewrite gives a huge boost of performance for profiles with a lot of works (1k+).

Also, it improves the privacy of profiles by not displaying information about the count of animes rated, and so on.

Finally, it is paginated and a bit more mobile-friendly that the previous version.
Now Mangaki supports Sentry integration, especially for beta.mangaki.fr.

It'll make it easier to track unexpected exceptions on the back-end.
Fixes Sentry reporting of views exception.
Initial i18n of "about us" pages.
@RaitoBezarius RaitoBezarius requested a review from jilljenn July 2, 2017 18:56
@codecov
Copy link
codecov bot commented Jul 2, 2017

Codecov Report

Merging #381 into stable will increase coverage by 7.01%.
The diff coverage is 61.83%.

Impacted file tree graph

@@            Coverage Diff             @@
##           stable     #381      +/-   ##
==========================================
+ Coverage   48.48%   55.49%   +7.01%     
==========================================
  Files          17       72      +55     
  Lines        1054     4173    +3119     
==========================================
+ Hits          511     2316    +1805     
- Misses        543     1857    +1314
Impacted Files Coverage Δ
mangaki/mangaki/urls.py 100% <ø> (ø)
mangaki/mangaki/utils/nmf.py 44.44% <ø> (ø)
mangaki/mangaki/templates/mangaki/ratings.html 100% <100%> (ø)
mangaki/mangaki/templates/mangaki/work_detail.html 54.89% <100%> (ø)
...i/templates/admin/merge_selected_confirmation.html 100% <100%> (ø)
mangaki/mangaki/utils/tokens.py 100% <100%> (ø)
mangaki/mangaki/choices.py 100% <100%> (ø) ⬆️
mangaki/mangaki/templatetags/anonymous_ratings.py 100% <100%> (ø)
mangaki/mangaki/templates/research.html 100% <100%> (ø)
mangaki/mangaki/utils/db.py 94.73% <100%> (+78.94%) ⬆️
... and 99 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b5e21d...9d14177. Read the comment docs.

@jilljenn
Copy link
Member
jilljenn commented Jul 2, 2017

Bravo pour le +7.01% coverage!

@jilljenn jilljenn merged commit 89d005d into stable Jul 2, 2017
jilljenn added a commit that referenced this pull request Jul 6, 2017
Elarnon pushed a commit that referenced this pull request Jul 6, 2017
Elarnon added a commit that referenced this pull request Jul 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0