8000 using BigTableClient when multiprocessing causes segfaults · Issue #23 · Unoperate/pytorch-cbt · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
using BigTableClient when multiprocessing causes segfaults #23
Open
@kboroszko

Description

@kboroszko

Here I'm reffering to google-cloud-cpp bigtable API
Because the DataClient holds a reference to [some queue object] within a Table, each table must be destroyed before the fork or before the client is destroyed after the fork. Otherwise, when the destructor is called, this client holds a reference to [some queue object] and tries to reference it causing a segfault.

Because we have no control over when a user might fork the process, we would have to construct a very complicated logic, detecting forks and handling objects we create. To avoid all that ordeal, we opted for a different approach: creating the connection every time we need to interact with BigTable and closing it afterwards. This is slightly inefficient but simple, reliable and robust.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0