Add Identify Galleries, much like Identify Scenes #4423

github-user-t · 2024-01-06T02:44:03Z

Added Gallery Identify, which is working the same way as the scene identify.

Most of the Gallery Identify code is an exact copy of the Scene Identify, although serving a different purpose. I have my doubts merging the 2 concerns would be beneficial.

Most of the Gallery Identify new code is in the same identify package. Could be split differently. No preference here.

Please also see PR comments in files.

Open to all suggestions for improvements.

github-user-t · 2024-01-06T02:45:36Z

docker/build/x86_64/Dockerfile

@@ -32,7 +32,12 @@ RUN make flags-release flags-pie stash

 # Final Runnable Image
 FROM alpine:latest
-RUN apk add --no-cache ca-certificates vips-tools ffmpeg
+RUN apk add --no-cache --virtual .build-deps gcc python3-dev musl-dev \


Python was missing from this "quick" image used locally.
Could not test scrapers without it.

Took it from the CI Dockerfile.

github-user-t · 2024-01-06T02:53:26Z

pkg/sqlite/gallery.go

@@ -787,23 +800,58 @@ func (qb *GalleryStore) makeQuery(ctx context.Context, galleryFilter *models.Gal
 	return &query, nil
 }

-func (qb *GalleryStore) Query(ctx context.Context, galleryFilter *models.GalleryFilterType, findFilter *models.FindFilterType) ([]*models.Gallery, int, error) {
-	query, err := qb.makeQuery(ctx, galleryFilter, findFilter)
+func (qb *GalleryStore) Query(ctx context.Context, options models.GalleryQueryOptions) (*models.GalleryQueryResult, error) {


Align on how it work for Scene to be able use the exact same logic.
Multiple changes are just a side of this.

Alternative would've been to create an additional QueryWithOptions method, which would end up of having 2 ways to do the same thing.

github-user-t · 2024-01-06T02:55:21Z

internal/api/resolver_query_find_gallery.go

@@ -25,15 +25,31 @@ func (r *queryResolver) FindGallery(ctx context.Context, id string) (ret *models

 func (r *queryResolver) FindGalleries(ctx context.Context, galleryFilter *models.GalleryFilterType, filter *models.FindFilterType) (ret *FindGalleriesResultType, err error) {
 	if err := r.withReadTxn(ctx, func(ctx context.Context) error {
-		galleries, total, err := r.repository.Gallery.Query(ctx, galleryFilter, filter)
+		var galleries []*models.Gallery


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T02:57:19Z

internal/autotag/performer_test.go

@@ -275,7 +276,8 @@ func testPerformerGalleries(t *testing.T, performerName, expectedRegex string) {
 		Direction: &direction,
 	}

-	db.Gallery.On("Query", mock.Anything, expectedGalleryFilter, expectedFindFilter).Return(galleries, len(galleries), nil).Once()
+	db.Gallery.On("Query", mock.Anything, gallery.QueryOptions(expectedGalleryFilter, expectedFindFilter, false)).


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T02:57:51Z

internal/autotag/studio_test.go

@@ -1,6 +1,7 @@
 package autotag


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T02:58:09Z

internal/autotag/tag_test.go

@@ -1,6 +1,7 @@
 package autotag


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T02:59:06Z

internal/identify/identify.go

@@ -15,18 +15,10 @@ import (
 	"github.com/stashapp/stash/pkg/utils"


code moved to identify_common.go

github-user-t · 2024-01-06T03:00:07Z

internal/manager/task_autotag.go

@@ -516,12 +517,19 @@ func (t *autoTagFilesTask) getCount(ctx context.Context) (int, error) {

 	imageCount := imageResults.Count

-	_, galleryCount, err := r.Gallery.Query(ctx, t.makeGalleryFilter(), findFilter)
+	galleryCount, err := r.Gallery.Query(ctx, models.GalleryQueryOptions{


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T03:00:16Z

internal/manager/task_clean.go

@@ -64,7 +65,7 @@ func (j *cleanJob) cleanEmptyGalleries(ctx context.Context) {
 	if err := r.WithTxn(ctx, func(ctx context.Context) error {
 		found := true
 		for found {
-			emptyGalleries, _, err := r.Gallery.Query(ctx, &models.GalleryFilterType{
+			emptyGalleries, err := gallery.Query(ctx, r.Gallery, &models.GalleryFilterType{


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T03:05:46Z

pkg/gallery/query.go


 	"github.com/stashapp/stash/pkg/models"
 )

+// QueryOptions returns a GalleryQueryOptions populated with the provided filters.


Related to this method definition change to accommodate the same structure for Identify Galleries as of Identify Scenes: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818

github-user-t · 2024-01-06T03:07:11Z

pkg/gallery/update.go


 	"github.com/stashapp/stash/pkg/models"
 )

+var ErrEmptyUpdater = errors.New("no fields have been set")


Only related to this method definition change: https://github.com/stashapp/stash/pull/4423/files?diff=split&w=1#r1443579818
No change in behavior expected.

github-user-t · 2024-01-06T03:12:53Z

pkg/models/model_scraped_item.go

+	if !excluded["image"] {
+		if len(p.Images) > 0 {
+			img, err = utils.ProcessImageInput(ctx, p.Images[0])
+		} else if p.Image != nil && len(*p.Image) > 0 {


Now downloading image from URL received in ScrapedPerformer.Image if there's nothing in ScrapedPerformer.Images

github-user-t · 2024-01-06T03:14:14Z

pkg/plugin/plugins.go

@@ -292,6 +292,15 @@ func (c Cache) RegisterPostHooks(ctx context.Context, id int, hookType HookTrigg
 	})
 }

+func (c Cache) ExecuteGalleryUpdatePostHooks(ctx context.Context, input models.GalleryUpdateInput, inputFields []string) {


Not sure about that one.
I duplicated the Scene logic.
Everything seems to be there to make it work.

github-user-t · 2024-01-06T04:17:46Z

internal/manager/task_identify.go

+			j.identifyGallery(ctx, gallery, sources)
+			if countResult.Count > 1 {
+				// Avoid flooding site
+				time.Sleep(time.Duration(3+rand.Intn(3)) * time.Second)


I haven't seen an existing mechanism for this kind of behavior. Is there?

add identify gallery

4e25ccb

github-user-t commented Jan 6, 2024

View reviewed changes

fix copy leftover

6bcb554

github-user-t commented Jan 6, 2024

View reviewed changes

github-user-t marked this pull request as ready for review January 6, 2024 03:33

github-user-t commented Jan 6, 2024

View reviewed changes

github-user-t added 2 commits January 11, 2024 19:32

forgot this

805c2c1

Merge branch 'stashapp:develop' into identify-gallery

7ea888b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Identify Galleries, much like Identify Scenes #4423

Add Identify Galleries, much like Identify Scenes #4423

github-user-t commented Jan 6, 2024 •

edited

github-user-t Jan 6, 2024 •

edited

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

github-user-t Jan 6, 2024

		@@ -15,18 +15,10 @@ import (
		"github.com/stashapp/stash/pkg/utils"

Add Identify Galleries, much like Identify Scenes #4423

Are you sure you want to change the base?

Add Identify Galleries, much like Identify Scenes #4423

Conversation

github-user-t commented Jan 6, 2024 • edited

github-user-t Jan 6, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-user-t commented Jan 6, 2024 •

edited

github-user-t Jan 6, 2024 •

edited