8000 fix(docker): missing libc package by FredrikBakken · Pull Request #973 · unitycatalog/unitycatalog · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix(docker): missing libc package #973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(docker): missing libc package #973

wants to merge 1 commit into from

Conversation

FredrikBakken
Copy link
Contributor

PR Checklist

  • A description of the changes is added to the description of this PR.
  • If there is a related issue, make sure it is linked to this PR.
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added or modified a feature, documentation in docs is updated

Description of changes

This PR resolves the missing libc-package when trying to read a table.

Error message previous encountered:

$ bin/uc table read --full_name unity.default.marksheet
Error occurred while executing the command. Failed to read delta table file:///tmp/marksheet_uniformFailed to read delta table
aaaed5aee4e0:~$ bin/uc table read --full_name unity.default.numbers
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.10-55889ad3-380d-4478-ab57-9bca6c00c1bb-libsnappyjava.so: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /tmp/snappy-1.1.10-55889ad3-380d-4478-ab57-9bca6c00c1bb-libsnappyjava.so)
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2394)
        at java.base/java.lang.Runtime.load0(Runtime.java:755)
        at java.base/java.lang.System.load(System.java:1970)
        at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:182)
        at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:157)
        at org.xerial.snappy.Snappy.init(Snappy.java:70)
        at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
        at org.apache.parquet.hadoop.codec.SnappyDecompressor.decompress(SnappyDecompressor.java:62)
        at org.apache.parquet.hadoop.codec.NonBlockedDecompressorStream.read(NonBlockedDecompressorStream.java:51)
        at java.base/java.io.DataInputStream.readFully(DataInputStream.java:201)
        at java.base/java.io.DataInputStream.readFully(DataInputStream.java:172)
        at org.apache.parquet.bytes.BytesInput$StreamBytesInput.toByteArray(BytesInput.java:286)
        at org.apache.parquet.bytes.BytesInput.toByteBuffer(BytesInput.java:237)
        at org.apache.parquet.bytes.BytesInput.toInputStream(BytesInput.java:246)
        at org.apache.parquet.column.impl.ColumnReaderBase.readPageV1(ColumnReaderBase.java:680)
        at org.apache.parquet.column.impl.ColumnReaderBase.access$300(ColumnReaderBase.java:57)
        at org.apache.parquet.column.impl.ColumnReaderBase$3.visit(ColumnReaderBase.java:623)
        at org.apache.parquet.column.impl.ColumnReaderBase$3.visit(ColumnReaderBase.java:620)
        at org.apache.parquet.column.page.DataPageV1.accept(DataPageV1.java:120)
        at org.apache.parquet.column.impl.ColumnReaderBase.readPage(ColumnReaderBase.java:620)
        at org.apache.parquet.column.impl.ColumnReaderBase.checkRead(ColumnReaderBase.java:594)
        at org.apache.parquet.column.impl.ColumnReaderBase.consume(ColumnReaderBase.java:735)
        at org.apache.parquet.column.impl.ColumnReaderImpl.consume(ColumnReaderImpl.java:30)
        at org.apache.parquet.column.impl.ColumnReaderImpl.<init>(ColumnReaderImpl.java:47)
        at org.apache.parquet.column.impl.ColumnReadStoreImpl.getColumnReader(ColumnReadStoreImpl.java:82)
        at org.apache.parquet.io.RecordReaderImplementation.<init>(RecordReaderImplementation.java:271)
        at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:147)
        at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:109)
        at org.apache.parquet.filter2.compat.FilterCompat$NoOpFilter.accept(FilterCompat.java:177)
        at org.apache.parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:109)
        at org.apache.parquet.hadoop.InternalParquetRecordReader.checkRead(InternalParquetRecordReader.java:141)
        at org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:230)
        at org.apache.parquet.hadoop.ParquetRecordReaderWrapper.nextKeyValue(ParquetRecordReaderWrapper.java:28)
        at io.delta.kernel.defaults.internal.parquet.ParquetFileReader$1.hasNext(ParquetFileReader.java:85)
        at io.delta.kernel.defaults.engine.DefaultParquetHandler$1.hasNext(DefaultParquetHandler.java:76)
        at io.delta.kernel.defaults.engine.DefaultParquetHandler$1.hasNext(DefaultParquetHandler.java:87)
        at io.delta.kernel.Scan$1.hasNext(Scan.java:160)
        at io.unitycatalog.cli.delta.DeltaKernelReadUtils.readData(DeltaKernelReadUtils.java:116)
        at io.unitycatalog.cli.delta.DeltaKernelUtils.readDeltaTable(DeltaKernelUtils.java:113)
        at io.unitycatalog.cli.TableCli.readTable(TableCli.java:169)
        at io.unitycatalog.cli.TableCli.handle(TableCli.java:51)
        at io.unitycatalog.cli.UnityCatalogCli.main(UnityCatalogCli.java:136)

After adding the libc package:

$ bin/uc table read --full_name unity.default.marksheet
┌──────────────────────────┬─────────────────────────┬─────────────────────────┐
│id(integer)               │name(string)             │marks(integer)           │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│1                         │nWYHawtqUw               │930                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│2                         │uvOzzthsLV               │166                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│3                         │WIAehuXWkv               │170                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│4                         │wYCSvnJKTo               │709                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│5                         │VsslXsUIDZ               │993                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│6                         │ZLsACYYTFy               │813                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│7                         │BtDDvLeBpK               │52                       │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│8                         │YISVtrPfGr               │8                        │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│9                         │PBPJHDFjjC               │45                       │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│10                        │qbDuUJzJMO               │756                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│11                        │EjqqWoaLJn               │712                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│12                        │jpZLMdKXpn               │847                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│13                        │acpjQXpJCp               │649                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│14                        │nOKqHhRwao               │133                      │
├──────────────────────────┼─────────────────────────┼─────────────────────────┤
│15                        │kxUUZEUoKv               │398                      │
└──────────────────────────┴─────────────────────────┴─────────────────────────┘

@FredrikBakken
fix(docker): missing libc package
4b741ae 
Signed-off-by: Fredrik Bakken <Fredrik.Bakken@norges-bank.no>
@dennyglee
Copy link
Contributor

Thanks for this fix @FredrikBakken - out of curiosity, what was your environment when you ran into the initial issues? cc @roeap

@FredrikBakken
Copy link
Contributor Author

@dennyglee, no worries. My current setup is Windows 11 with WSL using Ubuntu and Docker Engine:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.5 LTS
Release:        22.04
Codename:       jammy

$ docker --version
Docker version 28.1.1, build 4eba377

dennyglee
dennyglee reviewed May 12, 2025
View reviewed changes
Dockerfile
@@ -18,6 +18,9 @@ RUN apk add --no-cache bash && ./build/sbt -info clean package
# Small runtime image
FROM alpine:${ALPINE_VERSION} as runtime

# Resolve missing libc package
RUN apk update && apk add --no-cache libc6-compat
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tnightengale do you have any concerns with updating an older version of libc6-compat? Per the issue original issue, I believe this was an issue due to running the docker on a Windows environment. I don't believe there is an issue with current *nix environments but figured I would check with you on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dennyglee dennyglee dennyglee left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@FredrikBakken @dennyglee
0