8000 Reimplement safe_url_string based on the URL living standard · Issue #204 · scrapy/w3lib · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Reimplement safe_url_string based on the URL living standard #204
Open
@Gallaecio

Description

@Gallaecio

#201 implemented an alternative to safe_url_string based on the URL living standard, while maintaining support for older standards still used in the wild, for maximum safety of the resulting URL.

However, when we compared the performance of the new implementation with the old one, the new implementation was up to 200 times slower, and after some effort I only managed to reduce the gap to 40. 40 times slower is still unacceptable.

This task is about resuming the work started on #201, closing the performance gap by achieving a performance very similar or better than that of the current safe_url_string implementation.

How to do that? At the moment we believe Cython is the way to go, i.e. rework the new implementation as Cython code, which hopefully will solve the performance issues.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0