[bug] Conan search returns timeout #950

uilianries · 2019-06-16T13:49:06Z

To help us debug your issue please explain:

Hi!

As Conan Center is growing, the time requested to list all recipes is no longer enough:

conan search -r conan-center
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0). [Remote: conan-center]

As you can see, 60 seconds is not enough to retrieve all information.

However, if I update the timeout request by CONAN_TIMEOUT_REQUEST, I can receive the answer:

CONAN_REQUEST_TIMEOUT=360 conan search -r conan-center
Existing package recipes:

7z_installer/1.0@conan/stable
Args/5.0.1.4@igormironchik/stable
...

Usingtime it took 1m39.

As Conan Center is the official repository, I think search should work without timeouts. So Or Bintray is slow, Or Conan needs to update its timeout limit.

Conan Version: 1.16.1

I've read the CONTRIBUTING guide.
I've specified the Conan version, operating system version and any tool that can be relevant.
I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

The text was updated successfully, but these errors were encountered:

lasote · 2019-06-17T06:11:18Z

Reported to bintray. I don't think that incrementing the default timeout on the client is a sane solution, a list of recipes should be retrieved in milliseconds.

uilianries · 2019-06-17T10:56:12Z

Thanks!

uilianries · 2019-07-19T22:53:05Z

UPDATE:

Running Conan 1.17.0:

real 1m47,145s
user 0m0,861s
sys 0m0,139s

Minimonium · 2020-02-21T19:56:00Z

Reported by the users again. 😅

uilianries · 2020-02-21T20:55:25Z

Do you have a reference to the comment? just in case ...

uilianries · 2020-02-21T21:12:13Z

Anyway, I can confirm such behavior

$ conan --version
Conan version 1.22.2
$ time conan search -r conan-center
DEBUG :conan_api.py   [163]: INIT: Using config '/home/uilian/.conan/conan.conf' [2020-02-21 18:05:31,835]
DEBUG :tracer.py      [156]: CONAN_API: search_recipes(remote_name=conan-center,case_sensitive=False) [2020-02-21 18:05:31,836]
DEBUG :rest_client_common.py[156]: REST: ping: https://conan.bintray.com/v1/ping [2020-02-21 18:05:31,838]
DEBUG :rest_client.py [58]: REST: Cached capabilities for the remote: ['complex_search', 'checksum_deploy', 'revisions'] [2020-02-21 18:05:32,749]
DEBUG :rest_client_common.py[183]: REST: get: https://conan.bintray.com/v2/conans/search [2020-02-21 18:05:32,749]
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v2/conans/search (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

real	4m3,414s
user	0m0,660s
sys	0m0,083s

Minimonium · 2020-02-21T21:16:55Z

It was from the #include Discord server.
They were trying to use the

conan search liii* -r=conan-center

And getting timeout errors (the default is 60 seconds after all)

uilianries · 2020-02-21T21:20:06Z

Discord? That's cool, thanks for sharing! haha

memsharded · 2020-02-24T08:57:10Z

Hi all,

yes, we are aware of this limitation. It is in our todo list, but as it is a complex thing that requires changes in the backend it will take some time. Thanks very much for your patience.

memsharded · 2020-02-24T08:58:09Z

Moved to conan-center-index, as this is mostly about the ConanCenter, not the client side.

kaihowl · 2020-06-02T09:47:38Z

Probably observing the problem again right now. Details: https://twitter.com/kaihowl/status/1267750305443262465

hewigovens · 2020-06-02T22:54:20Z

I just installed and tried searching boost, it took 2min50s to get the results, Is someone working on this?

$ time sudo conan search "boost" --remote=conan-center
Existing package recipes:

boost/1.64.0@conan/stable
boost/1.65.1@conan/stable
boost/1.66.0@conan/stable
boost/1.67.0@conan/stable
boost/1.68.0@conan/stable
boost/1.69.0
boost/1.69.0@conan/stable
boost/1.70.0
boost/1.70.0@conan/stable
boost/1.71.0
boost/1.71.0@conan/stable
boost/1.72.0
boost/1.73.0

real	2m50.146s
user	0m0.530s
sys	0m0.066s

BTW: we manage our c++ project internal dependencies (boost/json/gtest/protobuf) in a script, trying to find a usable package manager

uilianries · 2020-06-03T11:55:43Z

I can say https://conan.io/center is much much faster than the command line when looking for patterns.

kaihowl · 2020-06-10T09:29:39Z

@uilianries I observe this timeout during the resolution of version ranges as part of conan create. See https://gitlab.com/ptr-project/catch2_extended/-/jobs/589079001 for example.

 raise ConanConnectionError(("%s\n\nUnable to connect to %s=%s\n" +
 conans.errors.ConanConnectionError: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost&ignorecase=False (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

The only way to remedy this seems increasing the timeout, pinning the version, or using lockfiles. I don't have any good logical reason for pinning or lockfiles as far as I can see. Increasing the timeout seems ok. Yet, the performance is surprisingly slow for a simple query. Is this something that should be reported here or is there a better place (or even an existing ticket) for this?

uilianries · 2020-06-10T14:40:13Z

@kaihowl Bintray reported an internal problem hours ago: https://status.bintray.com/

Please, try again, it should be solved now.

kaihowl · 2020-06-10T14:53:25Z

{"_action": "REST_API_CALL", "duration": 117.91095399856567, "headers": {"User-Agent": "Conan/1.25.2 (Python 3.7.7) python-requests/2.22.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": ""}, "met
hod": "GET", "time": 1591800609.898228, "url": "https://conan.bintray.com/v2/conans/search?q=boost&ignorecase=False"}

Still observing the issue. For "smaller" packages like sqlite3 I see the same problem, yet on a smaller scale:

{"_action": "REST_API_CALL", "duration": 20.84811496734619, "headers": {"User-Agent": "Conan/1.25.2 (Python 3.7.7) python-requests/2.22.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": ""}, "method": "GET", "time": 1591800488.0407581, "url": "https://conan.bintray.com/v2/conans/search?q=sqlite3&ignorecase=False"}

I don't think the reported outage correlates with this problem. I continually observe this problem since my original report, @uilianries.

kuklinistvan · 2020-08-05T10:55:00Z

I'm experiencing this too :(

thejohnfreeman · 2020-08-07T22:59:10Z

4 minutes seems like too long.

$ time conan search "boost/*" -r=conan-center
Existing package recipes:

boost/1.64.0@conan/stable
boost/1.65.1@conan/stable
boost/1.66.0@conan/stable
boost/1.67.0@conan/stable
boost/1.68.0@conan/stable
boost/1.69.0
boost/1.69.0@conan/stable
boost/1.70.0
boost/1.70.0@conan/stable
boost/1.71.0
boost/1.71.0@conan/stable
boost/1.72.0
boost/1.73.0

real    3m53.877s
user    0m0.483s
sys     0m0.060s

DominikDeak · 2020-08-24T03:11:35Z

Unfortunately, search performance is becoming a significant bottleneck and often fails in my case. Sample error message:

conan search "boost/*" --remote=all --raw

ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost%2F%2A (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

sonman · 2020-09-02T11:14:54Z

I am using a conan center on premise. Search takes sometimes minutes because local ConanCenter waits for bintray to answer. The conan search has therefore a not so good user experience.

AndWass · 2020-10-15T11:08:40Z

Also hit this issue.

It also happens if I have a range in conanfile.txt:

[requires]
boost/[>=1.73]

$> conan install -if build .
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=16
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost%2F%2A&ignorecase=False (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Output of time conan search boost/* -r all:

ERROR: HTTPSConnectionPool(host='api.bintray.com', port=443): Max retries exceeded with url: /conan/conan/conan-center/v1/conans/search?q=boost%2F%2A (Caused by ReadTimeoutError("HTTPSConnectionPool(host='api.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://api.bintray.com/conan/conan/conan-center
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

real    4m2.708s
user    0m0.015s
sys     0m0.000s

tjwrona · 2021-01-04T15:32:58Z

@AndWass this is also happening for me too. I noticed it when I tried to add a range in a conanfile "requires" statement.

I then tried doing a manual search and it failed. I stumbled upon this ticket and it looks like I am experiencing the same problem as others.

Searching for boost fails:

>conan search -r all boost
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Searching in general fails:

>conan search -r conan-center
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Searching for a specific package version does work (although it is quite slow... took about 10 seconds):

>conan search -r conan-center boost/1.75.0
Existing package recipes:

boost/1.75.0

I am using Conan version 1.29.2.

@memsharded, do you have any updates on the status of this issue? This is causing a big bottleneck in my development workflow.

uilianries · 2021-01-04T20:53:22Z

@tjwrona1992 Indeed the timeout persists. As alternative, you can use https://conan.io/center. The web search is pretty fast.

For the current timeout problem, we don't have a date for a fix, not soon at least. It requires maintenance on server side, not on Conan client.

tjwrona · 2021-01-04T22:19:27Z

@uilianries I appreciate the feedback. Unfortunately the problem I am having is that I want to be able to specify version ranges within my conanfiles for different packages I am consuming. Since these dependencies are downloaded during my CMake run with cmake-conan (https://github.com/conan-io/cmake-conan) i don't think there is any way to avoid this problem beacuse CMake will always be using the command line to fetch the dependencies.

For now I will have to just hard-code all of the version numbers to not be ranges, but I would prefer to be able to make my dependencies more flexible than that.

nminkov · 2021-01-25T10:49:41Z

The problem persists. Any solution insight?

conan search -r all openssl
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=openssl (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

blockspacer · 2021-03-09T07:40:50Z

Looks like conan search not using bloom filter for popular packages, see https://stackoverflow.com/questions/18447874/google-chrome-usage-of-bloom-filter

aleksa2808 · 2021-03-16T02:17:48Z

+1

I was planning to experiment with a VS Code extension for finding/installing packages, but after observing conan-center queries being this slow (timing out on most popular packages for me) I had to hang up my hat for now. ☹️

AJIOB · 2021-05-23T09:23:47Z

I cannot mirror a specific boost version:

Downloading conanmanifest.txt
Downloading conanfile.py
Downloading conan_export.tgz
Downloading conan_sources.tgz
boost/1.72.0: Getting the complete package list from 'boost/1.72.0#0'...
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/boost/1.72.0/_/_/search (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))
Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Do we have any solution?

- in hopes of avoiding CI job failures caused by slow JFrog REST API. - example failure: - job: https://github.com/johnmcfarlane/cnl/runs/4427931463?check_suite_focus=true#step:7:51 - message: ERROR: HTTPSConnectionPool(host='center.conan.io', port=443): Max retries exceeded with url: /v1/conans/search?q=boost%2F%2A&ignorecase=False (Caused by ReadTimeoutError("HTTPSConnectionPool(host='center.conan.io', port=443): Read timed out. (read timeout=60.0)")) - GitHub issue: conan-io/conan-center-index#950 - JFrog issue: https://www.jfrog.com/jira/browse/RTFACT-26310

jgsogo · 2022-09-23T09:02:11Z

This is not an issue anymore with latest changes to Artifactory

memsharded transferred this issue from conan-io/conan Feb 24, 2020

memsharded added the infrastructure Waiting on tools or services belonging to the infra label Feb 24, 2020

uilianries mentioned this issue Sep 30, 2020

[bug] conan search comman stuck/infinity conan-io/conan#7774

Closed

czoido mentioned this issue Jan 5, 2021

Create package with dependency on boost >=1.69.0 fails conan-io/conan#8265

Closed

johnmcfarlane mentioned this issue Dec 6, 2021

Clamp Boost version johnmcfarlane/cnl#960

Merged

lacasseio mentioned this issue Feb 14, 2022

Spike Conan package resolution nokeedev/gradle-native#465

Open

jgsogo closed this as completed Sep 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[bug] Conan search returns timeout #950

[bug] Conan search returns timeout #950

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[bug] Conan search returns timeout #950

[bug] Conan search returns timeout #950

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!