10000 Segfault at import · Issue #36 · photoprism/photoprism · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Segfault at import #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gmelchett opened this issue Oct 16, 2018 · 11 comments
Closed

Segfault at import #36

gmelchett opened this issue Oct 16, 2018 · 11 comments
Assignees
@lastzero
Labels
bug Something isn't working released Available in the stable release
Milestone
MVP

Comments

@gmelchett
Copy link

A fresh docker install (commit 6844c5f) crashed
when importing about 3000 photos.

Moving main jpg file "Camera/IMG_20170425_062158425.jpg" to "/Photos/Originals/2017/04/20170425_062158_2343698269B6.jpg"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x52fbd9]

goroutine 1 [running]:
github.com/RobCherry/vibrant.NewPaletteBuilder(0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/!rob!cherry/vibrant@v0.0.0-20160904011657-0680b8cf1c89/palette.go:202 +0x29
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).GetColors(0xc0003d7b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/colors.go:33 +0xcf
github.com/photoprism/photoprism/internal/photoprism.(*Indexer).IndexMediaFile(0xc000410da0, 0xc0003d7b80, 0x1, 0x1)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/indexer.go:92 +0x1e81
github.com/photoprism/photoprism/internal/photoprism.(*Indexer).IndexRelated(0xc000410da0, 0xc0003d7ae0, 0xc0003d7ae0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/indexer.go:231 +0x9b
github.com/photoprism/photoprism/internal/photoprism.(*Importer).ImportPhotosFromDirectory.func1(0xc000027920, 0x30, 0xecac40, 0xc00041f450, 0x0, 0x0, 0x1a, 0xc0007a76c0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/importer.go:104 +0x869
path/filepath.walk(0xc000027920, 0x30, 0xecac40, 0xc00041f450, 0xc00070ca20, 0x0, 0x0)
        /usr/local/go/src/path/filepath/path.go:358 +0x41c
path/filepath.walk(0xc0004203e0, 0x15, 0xecac40, 0xc0003cf930, 0xc00070ca20, 0x0, 0x0)
        /usr/local/go/src/path/filepath/path.go:382 +0x2fe
path/filepath.walk(0xc000026047, 0xe, 0xecac40, 0xc0003cf380, 0xc00070ca20, 0x0, 0x30)
        /usr/local/go/src/path/filepath/path.go:382 +0x2fe
path/filepath.Walk(0xc000026047, 0xe, 0xc00070ca20, 0x30, 0x28)
        /usr/local/go/src/path/filepath/path.go:404 +0x105
github.com/photoprism/photoprism/internal/photoprism.(*Importer).ImportPhotosFromDirectory(0xc00070c9f0, 0xc000026047, 0xe)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/importer.go:38 +0xb5
github.com/photoprism/photoprism/internal/commands.importAction(0xc00046e160, 0xeb9000, 0x33)
        /go/src/github.com/photoprism/photoprism/internal/commands/import.go:35 +0x307
github.com/urfave/cli.HandleAction(0xc87100, 0xdfb058, 0xc00046e160, 0xc0003fc100, 0x0)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0xda5f8c, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0xdaca99, 0xe, 0x0, ...)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/command.go:210 +0x990
github.com/urfave/cli.(*App).Run(0xc0004544e0, 0xc00000c080, 0x2, 0x2, 0x0, 0x0)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:255 +0x687
main.main()
        /go/src/github.com/photoprism/photoprism/cmd/photoprism/photoprism.go:29 +0x2d7
@lastzero
Copy link
Member

Looks like an issue with color indexing, see #7

Guess when github.com/RobCherry/vibrant is gone, the issue will be gone...

@lastzero lastzero added bug Something isn't working help wanted Help with this would be much appreciated! labels Oct 16, 2018
@gmelchett
Copy link
Author

Thanks for having a look! I tried to import only the photo that caused the segfault in a new docker instance, and the crash occurred again.
I don't know if it is of any help, but here is a link to the photo that triggers this bug http://lithops.se/IMG_20170425_062158425.jpg (Original, ~10 MB large)

@u5surf
Copy link
u5surf commented Oct 18, 2018

@lastzero , @gmelchett
I tested @gmelchett 's image on vibrant library.
And I figured out the following issue of go image library that some special jpg cannot decode.
golang/go#10447

reproducing
wget http://lithops.se/IMG_20170425_062158425.jpg -O sample.jpg
and following code runs, then it can be output invalid JPEG format: short Huffman data

package main

import (
  "fmt"
  "image"
  _ "image/jpeg" // "net/http"
  "os"

  "github.com/RobCherry/vibrant"
)

func main() {
  file, _ := os.Open("sample.jpg")
  decodedImage, _, err := image.Decode(file)
  if err != nil {
    fmt.Println(err)
  }
  palette := vibrant.NewPaletteBuilder(decodedImage).Generate()
  fmt.Println(palette)
}

result:

$go run main.go
invalid JPEG format: short Huffman data
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x10c8329]

goroutine 1 [running]:
github.com/RobCherry/vibrant.NewPaletteBuilder(0x0, 0x0, 0x1)
        /Users/yugo-horie/go/src/github.com/RobCherry/vibrant/palette.go:202 +0x29
main.main()
        /Users/yugo-horie/vibrant_test/main.go:26 +0xc7
exit status 2

But also I think it's better to be handling those errors properly.

@lastzero
Copy link
Member

Would https://github.com/disintegration/imaging also be affected? This is what we're using for resizing etc...

@u5surf
Copy link
u5surf commented Oct 18, 2018 •
edited
Loading

I checked github.com/disintegration/imaging is affected because
image.Decode is called by imaging.Open(m.filename, imaging.AutoOrientation(true)) in your code.
i.e.

photoprism/internal/photoprism/thumbnails.go

Line 103 in 6844c5f

img, err := imaging.Open(m.filename, imaging.AutoOrientation(true))

but it maynot cause segfalt if you care failed safe the error that is catches next statements.

@lastzero
Copy link
Member

So ultimately that means we need to replace "image/jpeg" or we can not import certain images as the issue was closed by the golang team. Any opinions / alternatives?

@gmelchett
Copy link
Author

Looking at golang/go/issues/10447 it looks like the cause could be that the image is truncated, and indeed my photo is truncated. In Firefox (62), the bottom lines are white, while in Gimp (2.10) only the down right corner isn't correct.
The photos is taken by a Qualcomm based mobile phone with stock Android. (Moto X Force)

@lastzero
Copy link
Member

So ignoring it would be OK until we find a better solution?

< 8000 circle cx="8" cy="8" r="7" stroke="currentColor" stroke-opacity="0.25" stroke-width="2" vector-effect="non-scaling-stroke" fill="none" />

@gmelchett
Copy link
Author

Yes, it would be OK. However, it would be nice it photoprism didn't crash upon such images. Ignoring them would be prefered, maybe copy/move these faulty images to a special folder with faulty images?

@lastzero
Copy link
Member

Of course it shouldn't crash later, but for now this is what we want. Fail fast. Otherwise, nobody would have complained and it might not have gotten any attention. Now we know there is a problem.

@orsenthil orsenthil mentioned this issue Dec 31, 2018
Closed
@lastzero
Copy link
Member

Think we can close this.

@lastzero lastzero added released Available in the stable release and removed help wanted Help with this would be much appreciated! labels Jan 20, 2020
@lastzero lastzero self-assigned this Jan 20, 2020
@lastzero lastzero added this to the MVP milestone Jan 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lastzero lastzero

Labels
bug Something isn't working released Available in the stable release
Projects
None yet
Milestone
MVP
Development

No branches or pull requests
3 participants
@lastzero @gmelchett @u5surf
0