Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thumbnail wazero integrate #1474

Open
wants to merge 35 commits into
base: sprint-1.15
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
58fa5e0
thumbnail gen through wazero
storybehind Apr 25, 2024
7a5287b
update .gitignore
storybehind Apr 25, 2024
a90c4d4
Merge branch 'sprint-1.14' into thumbnail-wazero-integrate
storybehind Apr 25, 2024
fbf3407
add wasm file
storybehind Apr 25, 2024
dbec4a9
refactor and resolve errors
storybehind Apr 28, 2024
a522d30
Merge branch 'sprint-1.14' into thumbnail-wazero-integrate
storybehind Apr 28, 2024
1e9e704
support go native decoder and add tests for individual converters
storybehind May 2, 2024
babe874
Merge branch 'sprint-1.14' into thumbnail-wazero-integrate
storybehind May 2, 2024
cd918ab
revert changes to go version
storybehind May 2, 2024
bf008ae
Merge branch 'thumbnail-wazero-integrate' of https://github.com/0chai…
storybehind May 2, 2024
078f90f
add benchmark test
storybehind May 6, 2024
9092d33
benchmark test
storybehind May 14, 2024
c0dec98
Merge branch 'sprint-1.14' into thumbnail-wazero-integrate
storybehind May 14, 2024
7e69fa3
provide js binding and use image-rs convert
storybehind May 17, 2024
1a7298e
fix golint and dependencies
storybehind May 17, 2024
68ce31d
include pkg dir
storybehind May 17, 2024
e2a15ef
remove unwanted files
storybehind May 17, 2024
ffdd079
remove pkg dir
storybehind May 17, 2024
f265c15
remove pkg dir
storybehind May 17, 2024
bb3dc8c
installing rustc while for wasm binary
shahnawaz-creator May 18, 2024
ae86d3b
executing build.sh script
shahnawaz-creator May 20, 2024
02bb0b4
added auto install
shahnawaz-creator May 20, 2024
2cb904f
executing build.sh script
shahnawaz-creator May 20, 2024
24d5a53
debugging cargo
shahnawaz-creator May 20, 2024
87fe489
debugging cargo
shahnawaz-creator May 20, 2024
5882831
updated cargo binary path
shahnawaz-creator May 20, 2024
0e60c9f
Merge pull request #1495 from 0chain/install/rustc
storybehind May 21, 2024
5a2842b
update build-wasm run
storybehind May 21, 2024
d6ae6c4
provide full path to wasm-pack
storybehind May 21, 2024
412f3fe
fix errors
storybehind May 21, 2024
9d17957
set env variable
storybehind May 21, 2024
11c2269
added build.sh into iod, android, macos (#1503)
shahnawaz-creator May 23, 2024
9fb460f
upload pkg dir as an artifact
shahnawaz-creator May 23, 2024
089b103
rename image files
storybehind May 25, 2024
79fb902
Merge branch 'thumbnail-wazero-integrate' of https://github.com/0chai…
storybehind May 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
91 changes: 89 additions & 2 deletions .github/workflows/build-sdks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,24 @@ jobs:
with:
go-version: 1.21.5

- name: Install dependencies
run: |
# curl https://sh.rustup.rs -sSf | sh -s -- -y
ls -lha $HOME/.cargo/bin
# export PATH=$PATH:/bin/
. "$HOME/.cargo/env"
cargo install cargo-wasi
cargo wasi --version
cargo install wasm-pack
wasm-pack --version

- name: Executing core/imageutil/image_rs/build.sh script
run: |
. "$HOME/.cargo/env"
cd core/imageutil/image_rs
bash build.sh
cd ../../../

- name: Clean build
run: make clean-mobilesdk

Expand Down Expand Up @@ -109,6 +127,21 @@ jobs:
#echo "deb http://mirror.pit.teraswitch.com/ubuntu/ $codename main" >> ./gitaction.list
#sudo mv -f ./gitaction.list /etc/apt/sources.list.d/
sudo apt-get update -y && sudo apt-get install build-essential wget unzip -y
curl https://sh.rustup.rs -sSf | sh -s -- -y
ls -lha $HOME/.cargo/bin
# export PATH=$PATH:/bin/
. "$HOME/.cargo/env"
cargo install cargo-wasi
cargo wasi --version
cargo install wasm-pack
wasm-pack --version

- name: Executing core/imageutil/image_rs/build.sh script
run: |
. "$HOME/.cargo/env"
cd core/imageutil/image_rs
bash build.sh
cd ../../../

- name: Install Java
uses: actions/setup-java@v3
Expand Down Expand Up @@ -204,6 +237,24 @@ jobs:
with:
go-version: 1.21.5

- name: Install dependencies
run: |
# curl https://sh.rustup.rs -sSf | sh -s -- -y
ls -lha $HOME/.cargo/bin
# export PATH=$PATH:/bin/
. "$HOME/.cargo/env"
cargo install cargo-wasi
cargo wasi --version
cargo install wasm-pack
wasm-pack --version

- name: Executing core/imageutil/image_rs/build.sh script
run: |
. "$HOME/.cargo/env"
cd core/imageutil/image_rs
bash build.sh
cd ../../../

- name: Clean build
run: make clean-mobilesdk

Expand Down Expand Up @@ -284,6 +335,21 @@ jobs:
#echo "deb http://mirror.pit.teraswitch.com/ubuntu/ $codename main" >> ./gitaction.list
#sudo mv -f ./gitaction.list /etc/apt/sources.list.d/
sudo apt-get update -y && sudo apt-get install build-essential wget -y
curl https://sh.rustup.rs -sSf | sh -s -- -y
ls -lha $HOME/.cargo/bin
# export PATH=$PATH:/bin/
. "$HOME/.cargo/env"
cargo install cargo-wasi
cargo wasi --version
cargo install wasm-pack
wasm-pack --version

- name: Executing core/imageutil/image_rs/build.sh script
run: |
. "$HOME/.cargo/env"
cd core/imageutil/image_rs
bash build.sh
cd ../../../

- name: Build windows
run: |
Expand Down Expand Up @@ -347,11 +413,32 @@ jobs:
run: |
sudo apt-get update
sudo apt-get -y install build-essential nghttp2 libnghttp2-dev libssl-dev wget
curl https://sh.rustup.rs -sSf | sh -s -- -y
ls -lha $HOME/.cargo/bin
# export PATH=$PATH:/bin/
. "$HOME/.cargo/env"
cargo install cargo-wasi
cargo wasi --version
cargo install wasm-pack
wasm-pack --version

- name: Executing core/imageutil/image_rs/build.sh script
run: |
. "$HOME/.cargo/env"
cd core/imageutil/image_rs
bash build.sh
cd ../../../

- name: 'Upload pkg as an Artifact'
uses: actions/upload-artifact@v3
with:
name: pkg
path: core/imageutil/image_rs/pkg

- name: Build
- name: Build wasm binary
run: docker run --rm -v $PWD:/gosdk -w /gosdk golang:1.21.5 make wasm-build

- name: 'Upload Artifact'
- name: 'Upload zcn.wasm as an Artifact'
uses: actions/upload-artifact@v3
with:
name: zcn.wasm
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ out
vendor
.DS_Store
_sdkver/*.wasm
*.wasm
*.log
zcnbridge/example/logs/
*.log
Expand All @@ -33,3 +32,6 @@ wasmsdk/demo/demo
sdkversion.go
gosdk.code-workspace
sdkversion

# generated images
core/imageutil/demo/images/
Binary file added core/imageutil/benchmark_data/sample.jfif
Binary file not shown.
Binary file added core/imageutil/benchmark_data/sample.webp
Binary file not shown.
Binary file not shown.
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23,270 changes: 23,270 additions & 0 deletions core/imageutil/benchmark_data/sample_1280x853.pnm

Large diffs are not rendered by default.

Binary file not shown.
Binary file not shown.
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5,146 changes: 5,146 additions & 0 deletions core/imageutil/benchmark_data/sample_640x426.pnm

Large diffs are not rendered by default.

Binary file not shown.
144 changes: 144 additions & 0 deletions core/imageutil/demo/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
package main

import (
_ "embed"
"fmt"
"io"
"io/fs"
"log"
"net/http"
"os"
"path/filepath"

"github.com/0chain/gosdk/core/imageutil"
)

type img struct {
// location to download image
location string
// location type can be either remote or local
locationType string
// expected width of thumbnail
width int
// expected height of thumbnail
height int
}

var (
permissions = 0644
images = []img{
{
location: filepath.Join("..", "test_data", "input.bmp"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.dds"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.exr"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.gif"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.hdr"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.ico"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.jfif"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.jpe"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.jpeg"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.jpg"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.jps"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.png"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.pnm"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.svg"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.tga"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.tiff"), locationType: "local", width: 500, height: 500,
},
{
location: filepath.Join("..", "test_data", "input.webp"), locationType: "local", width: 500, height: 500,
},
}
)

func main() {

folderPath := "images"
if _, err := os.Stat(folderPath); err == nil {
// folderPath exists
err = os.RemoveAll(folderPath)
if err != nil {
log.Panicf("err deleting images folder: %v", err)
}
}
err := os.MkdirAll(folderPath, fs.FileMode(permissions))
if err != nil {
log.Panicf("err creating images folder: %v", err)
}

for i, input := range images {
log.Printf("%d) Location: %s", i, input.location)
var b []byte
switch input.locationType {
case "remote":
log.Printf("downloading image...")
resp, err := http.Get(input.location)
if err != nil {
log.Printf("err downloading url: %v", err)
continue
}
defer resp.Body.Close()
b, err = io.ReadAll(resp.Body)
if err != nil {
log.Printf("err reading response body: %v", err)
continue
}
case "local":
log.Printf("reading file...")
bytes, err := os.ReadFile(input.location)
if err != nil {
log.Printf("err reading file: %v", err)
continue
}
b = bytes
default:
log.Printf("unsupported location type: %v", input.locationType)
}

log.Printf("creating Thumbnail...")
res, err := imageutil.Thumbnail(b, input.width, input.height)
if err != nil {
log.Printf("err creating thumbnail: %v", err)
continue
}

log.Printf("creating outfile file")
opath := filepath.Join(folderPath, fmt.Sprintf("output%d.jpeg", i))
err = os.WriteFile(opath, res, fs.FileMode(permissions))
if err != nil {
log.Printf("err creating outfile file: %v", err)
}
log.Printf("%s success!", input.location)
}
log.Println("Done!")
}
2 changes: 2 additions & 0 deletions core/imageutil/image_rs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
target/
Cargo.lock
21 changes: 21 additions & 0 deletions core/imageutil/image_rs/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[package]
name = "image_rs"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
crate-type = ["cdylib"]
name = "image_rs"
path = "image.rs"

[dependencies]
image = { git = "https://github.com/image-rs/image", version = "0.25.1" }
wasm-bindgen = "0.2"
js-sys = "0.3.69"

[profile.release]
opt-level = "z"
lto = true
codegen-units = 1
9 changes: 9 additions & 0 deletions core/imageutil/image_rs/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh
set -e

cargo wasi build --release
cp target/wasm32-wasi/release/image_rs.wasm .

wasm-pack build --target web

echo "Done!"
Binary file added core/imageutil/image_rs/file-icon.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.