8000 Installer Improvements · Issue #1603 · databrickslabs/lakebridge · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Installer Improvements #1603
Open
Open
@sundarshankar89

Description

@sundarshankar89
          As things stand with this PR, there are three main behaviours that it introduces that need to be addressed. In order of priority:
  • Update to not use the Debian CDN. This is an external service unaffiliated with PyPi and intended for use by the Debian project. We should not be using it.
    (Issue created: [BUG]: Download from PyPI instead of Debian #1600. Needs to be assigned.)
  • Update the installer to: a) not copy the entire lsp/ directory out of the installed artefact, but instead just the config.yml file; b) not try to run install.sh.
  • Tech-debt: Use the venv module to set up the virtual environment rather than shell out, as well use it for all the path manipulation instead of doing it ourselves. Use python3 -m pip to run pip within the virtualenv, rather than trying to figure out the entry-point ourselves. (The path to python3 comes from the venv module.) This will eliminate a lot of platform-specific code.

I don't really mind if the above items are addressed in this PR or if an issue is created to cover it, so long as the issue is assigned so that it's clear who will be doing to work.

Some to this some additional tickets are needed:

  • Tech-debt: When launching the LSP server, just use the venv module to grab the bin/ directory if it exists and put it on the PATH so that the command from config.yml just works. This will eliminate most (all?) of python- and java-specific code, along with the platform-specific code. The PYTHONPATH manipulation might also no longer be necessary.
  • Wheels are architecture-specific, with any being a catch-all. We should support architecture-specific wheels, not just blindly use the any one.
  • Do we need to detect the version of the plugin we're about to install? If we do, we need to document why. (Maybe I missed it somewhere?) If we don't, just use pip within the virtualenv to find the latest version and install it. Probably solves the prior point, and also resolves an issue that many customers will have where they are behind a corporate firewall/proxy or have to use an internal PyPi mirror: pip can be configured to work.
  • Improvement: For maven, prefer mvn if installed and use mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:get to get the latest version. This will also deal with the common problems of firewalls/proxies, because mvn can be configured to work with internal mirrors whereas we can't. (Related issue [BUG]: Don't use search.maven.org when installing morpheus #1602: remove our dependency on the search.maven.org service.)
  • Tech-debt: Tests shouldn't be leaving stuff in the tests/ tree, for example logs and .lst files. The solution is to run things in a temporary directory and clean up. (Updating .gitignore is not the way to solve this problem.)
  • Tech-debt: We should not have wheels and JARs in the source tree. To test the installer, the tests can build a reference plugin and use that. Integration testing against BB and Morpheus is primarily their responsibility. (Once they are released, we can also add downstream CI testing, like with blueprint.)

Originally posted by @asnare in #1537 (review)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    0