Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Seamless texture SD upscale #3590

Open
1 task done
Aza-zeal opened this issue Oct 24, 2022 · 10 comments
Open
1 task done

[Feature Request]: Seamless texture SD upscale #3590

Aza-zeal opened this issue Oct 24, 2022 · 10 comments
Labels
enhancement New feature or request

Comments

@Aza-zeal
Copy link

Is there an existing issue for this?

  • I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

I wish to be able to use SD upscaling on seamless textures. Upscaling with the SD upscale script should maintain the original color and composition at the edges of the image when "Tiling" is ticked.

Proposed workflow

  1. Upload a seamless texture to Sd upscale
  2. Tick "tiling" field and generate
  3. Results in an SD upscaled image that is still seamless at the edges (tileable)

Additional information

I wish to omit arduous, repetitive processes like fixing seams from the process of texture creation.

@rasamaya
Copy link

I always use upscale, after highres fix. This tile feature would be usefull. Good idea.

@jfjensen
Copy link

This is exactly what I'm looking for too.

@erdostamasa
Copy link

This would be the exact thing I need. Right now I can only use the tiling images in their original 512 * 512 resolution because upscaling introduces edge artifacts.

@erdostamasa
Copy link

Does anybody have an idea how could this be achieved?

@jfjensen
Copy link

jfjensen commented Dec 3, 2022

I've found a work-around, that may even be used as a solution, if someone is willing to code it.

These are the steps:

  1. generate your texture with the tileable setting turned on (result: 512x512 image)
  2. tile the resulting image 2 by 2; meaning 2 tiles in X and Y direction = 4 tiles in total (result: 1024x1024 image)
  3. upscale the 2-by-2 tiled image as much as you like (result: for example 4096x4096 for 4x upscaling)
  4. crop the center part of the upscaled image (result in this case an 2048x2048 image)
  5. check that the center crop is seamlessly tileable - which it usually is...

Like this you get a tileable version of the original image that is upscaled 4x. Do note that the resulting image is offset by width/2 and height/2. With GIMP or Photoshop you can re-offset it the image to get the original one, if you need this.

@rexelbartolome
Copy link

Ohh interesting, my workaround so far is

  1. generate a tiling texture, let's call it the A-tile
  2. go to Krita/GIMP/PS, offset the A-tile, now let's call it B-tile
  3. upscale A-tile
  4. upscale B-tile
  5. go to Krita/GIMP/PS, open upscaled A-tile
  6. in the same document, open upscaled B-tile and offset it
  7. erase the center of B-tile (with a soft brush ideally) to your liking, since the edges are what we mainly need

but @jfjensen 's seem much simpler :)

If one wants more control, then ideally you'd be doing this process inside the painting program that's using a plugin to connect to SD. Krita, GIMP, and Photoshop I think all have their own plugins, better masking and inpainting experience overall.

@rexelbartolome
Copy link

Hello again, with the help of ChatGPT I've managed to make a script that simplifies and modifies some of the steps in @jfjensen 's workflow.

First we need ofc a tileable image
image

then we use this tile and crop script: this will basically extend the image by a half tile

from PIL import Image

# Path of where the source image is (use double backslash)
inpath = "image.png"
# Path of where to save the image
saveloc = "1-5x1-5.png"

# Open the image
image = Image.open(inpath)

# Get the width and height of the image
width, height = image.size

# Calculate the size of the tiled image
tiled_width = width * 3
tiled_height = height * 3

# Create a new image with the calculated size
tiled_image = Image.new(image.mode, (tiled_width, tiled_height))

# Paste the image onto the tiled image in a 3x3 grid
for i in range(3):
    for j in range(3):
        tiled_image.paste(image, (i * width, j * height))

# Calculate the center point of the tiled image
center_x = tiled_width / 2
center_y = tiled_height / 2

# Calculate the size of the cropped image
cropped_width = tiled_width / 1.5
cropped_height = tiled_height / 1.5

# Crop the tiled image using the center point as the anchor
cropped_image = tiled_image.crop((center_x - cropped_width / 2, center_y - cropped_height / 2,
                                  center_x + cropped_width / 2, center_y + cropped_height / 2))

# Save the tiled image
cropped_image.save(saveloc)

resulting image:
1-5x1-5

Then, the output image should be the one that goes into the upscaler.

00077

After getting upscaled, you just need to use this next script to crop it back to a 1x1 tile.

from PIL import Image

# Path of where the source image is (use double backslash)
inpath = "upscaled.png"
# Path of where to save the image
saveloc = "1x1tile.png"

# Open the image
image = Image.open(inpath)

# Get the width and height of the image
width, height = image.size

# Calculate the center point of the image
center_x = width / 2
center_y = height / 2

# Calculate the size of the cropped image
cropped_width = width / 2
cropped_height = height / 2

# Crop the image using the center point as the anchor
cropped_image = image.crop((center_x - cropped_width / 2, center_y - cropped_height / 2,
                            center_x + cropped_width / 2, center_y + cropped_height / 2))

# Save the cropped image
cropped_image.save(saveloc)

1x1tile

Here's a closeup on the (barely noticeable) seams

krita_rJSaxRieDF.mp4

Now the only thing left is I guess implementing it in a way that talks to A1111 webui which is way beyond my knowledge...

There's also no scripting section in the Extras/upscalers tab so the script should be either an extension in it's own separate tab that's basically a copy of Extras but with the extra buttons that applies the 2 scripts or maybe just a script in general but it only works in img2img under the scripts section(?)

So if any of you knows how this can be implemented, please feel free to pick up from here :D

@rexelbartolome
Copy link

If you guys want a more automated way and are okay with a bit of a learning curve to use a node system, then this will likely be the best solution https://github.com/chaiNNer-org

AFAIK it can do what my script just did, tile a texture, upscale, then crop afterwards. Can also do them in batches and you can integrate multiple upscalers to use it with instead of A1111's two upscaling layer system

@mezotaken mezotaken added the enhancement New feature or request label Jan 12, 2023
@MultiUpGame
Copy link

MultiUpGame commented Jan 17, 2023

Hello. Can't make a video? I don't understand where to write the code. Or what did you do. Thanks

@rexelbartolome
Copy link

For @MultiUpGame and anyone still wondering how to do it I have made a video about it https://youtu.be/sUyWWjFlszY

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

7 participants