BLOSC_MAX_BUFFERSIZE disappeared #47
Comments
|
Yes, that's correct. Probably I should keep maintaining the old symbols for backward compatibility. Only BLOSC_MAX_BUFFERSIZE is causing issues? |
|
So far that's all I've run into. https://travis-ci.org/dask/dask/jobs/119893085 |
|
I have had a look at bloscpack, and it seems like it is pinned at python-blosc 1.2.7: https://github.com/Blosc/bloscpack/blob/master/requirements.txt#L1 so it should not be using python-blosc 1.3.0. But anyway, I would be open to put BLOSC_MAX_BUFFERSIZE back into 1.3 series. Thinking aloud, this should be a good moment to produce a new bloscpack based on python-blosc 1.3.0 that would implement support for the new BITSHUFFLE filter: http://python-blosc.blosc.org/tutorial.html#using-different-filters |
|
So, the last time I checked, Bloscpack was pretty incompatible with anything except python-blosc #45 This is why I decided to pin it to that specific version. Anyway, I suggest using Blosc 1.2.7 explicitly in the build-jobs, if you need to keep those going. As for an upgrade, I'm afraid I won't have time to take care of this within any reasonable time-frame. In fact, I think I'd favor deprecating and orphaning the package in favor of things like: https://github.com/wesm/feather which is likely to be a much more maintained/developed solution. Of course you are free to do as you like with the source-code and should you decided to continue development, I can of course give you access to the PyPi project. |
|
I'd be happy to adopt bloscpack and make a quick release supporting the new features introduced in python-blosc 1.3.0. So @esc please add me to the maintainers in PyPI. Thanks! |
|
I am not convinced that it will be "quick"... Anyway, I just checked and you already had the owner role for the project on PyPi so feel free to release as you please, there are notes about doing a release here: https://github.com/Blosc/bloscpack#maintainers-notes-on-cutting-a-release. Furthermore, you probably want to start here: https://github.com/Blosc/bloscpack/blob/master/bloscpack/args.py specifically: https://github.com/Blosc/bloscpack/blob/master/bloscpack/args.py#L296 Also note, that I'll be silently watching. 🙈 🙉 🙊 . |
|
I have spent quite a bit of quality time on this but I am getting sporadic segfaults with bloscpack command line tests (via cram). More investigation reveals that python-blosc (1.3.0) also gets some segfaults. I have been able to consistently reproduce these segfaults only on Ubuntu 15.10 either with gcc 4.9.3 or gcc 5.2.1. On my MacOSX box I am not getting any issue (clang 3.7), so I tried with clang 3.6 and 3.7 on Ubuntu 15.10, and everything is fine there too, so it seems something triggered only by gcc. Valgrind does not offer a clear sign on what's going on with the gcc compilers, so I am a bit at a loss here. Help from someone offering more insights would be highly appreciated. |
|
Any hunches or example code that can be used to reproduce? |
|
Unfortunately very little. Yesterday I tried a lot of things like enabling/disabling different compressors, re-implemented the compression function using several approaches, looking into the refcounts and so on and so forth with no evidence of a nice hint. Anyway, I filed an issue in python-blosc about this: Blosc/python-blosc#110. Also, yesterday I fixed several things in python-blosc, and in particular the disappeared symbols are back again: Blosc/python-blosc@a40bb58. So I think this ticket can be closed now. |
It appears that
blosc.BLOSC_MAX_BUFFERSIZEhas turned intoblosc.MAX_BUFFERSIZE, which I believe causes issues withinbloscpack.The text was updated successfully, but these errors were encountered: