8000 [Pytorch] example with actual BigTable access · Issue #2 · Unoperate/pytorch-cbt · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Pytorch] example with actual BigTable access #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

A897 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
kboroszko opened this issue Aug 23, 2021 · 1 comment
Open

[Pytorch] example with actual BigTable access #2

kboroszko opened this issue Aug 23, 2021 · 1 comment

Comments

@kboroszko
Copy link
Collaborator

Create an example method where you supply project-id, table name and columns and get a Pytorch Tensor with the results.
@kboroszko
Copy link
Collaborator Author
kboroszko commented Aug 24, 2021 •
edited
Loading

Decisions, decisions...

  • In terms of code structure, to make the setup as simple as possible, all c++ extensions are in a separate package called pbt_C. We later call them from python code by importing that extra package. Nvidia used a similar approach: LINK
  • Because we want our lib to have no dependencies we compile all of the libraries statically and with fPIC flag
  • We also compile with a flag -D_GLIBCXX_USE_CXX11_ABI=0 every c++ dependency because the libraries need the same version of basic_string as pytorch. For some reason pytorch is compiled with this flag set to 0 which means that we either need to recompile all dependencies the same way or we would need to compile pytorch from source. more details

From pytorch doc:

A small note on compilers: Due to ABI versioning issues, the compiler you use to build your C++ extension must be ABI-compatible with the compiler PyTorch was built with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kboroszko
0