Skip to content

Commit

Permalink
Add scripts/benchmark and cleanup
Browse files Browse the repository at this point in the history
- Move image build into scripts/benchmark
  for improved visibility..makes Makefile
  only responsible for gathering necessary
  external artifacts
- Use WORKDIR in Dockerfile to clarify where
  commands are being run

TODO: Fixup commented out new git code in build.sbt
  • Loading branch information
CloudNiner committed Sep 26, 2019
1 parent 3d1d5ea commit 8dcf15d
Show file tree
Hide file tree
Showing 5 changed files with 54 additions and 32 deletions.
14 changes: 11 additions & 3 deletions Dockerfile.benchmark
Expand Up @@ -6,21 +6,29 @@ RUN add-apt-repository -y ppa:openjdk-r/ppa
RUN apt-get update
RUN apt-get install -y openjdk-8-jdk
RUN apt-get install -y wget
RUN apt-get install -y git
RUN apt-get clean

ADD archives/geotrellis-contrib.tar /
ADD archives/imageio-ext-1.1.24-jars.zip /tmp
RUN (cd /geotrellis-contrib/lib ; unzip /tmp/imageio-ext-1.1.24-jars.zip)

WORKDIR /geotrellis-contrib/lib
RUN unzip /tmp/imageio-ext-1.1.24-jars.zip

WORKDIR /geotrellis-contrib
ADD archives/gdal192-Ubuntu12-gcc4.6.3-x86_64.tar.gz /geotrellis-contrib/benchmark/gdal/native/
ADD archives/gdal-data.zip /tmp
RUN (cd /geotrellis-contrib/benchmark/gdal ; unzip /tmp/gdal-data.zip)

WORKDIR /geotrellis-contrib/benchmark/gdal
RUN unzip /tmp/gdal-data.zip
ADD archives/libgdal-java_1.10.1+dfsg-5ubuntu1_amd64.deb /tmp/
RUN (dpkg -x /tmp/libgdal-java_1.10.1+dfsg-5ubuntu1_amd64.deb /tmp/moop ; \
cp -f /tmp/moop/usr/lib/jni/*.so /geotrellis-contrib/benchmark/gdal/native/ ; \
cp -f /tmp/moop/usr/share/java/gdal.jar /geotrellis-contrib/lib/)

RUN update-ca-certificates -f
RUN (cd /geotrellis-contrib ; ./sbt "project benchmark" compile)
WORKDIR /geotrellis-contrib
RUN ./sbt "project benchmark" compile

ADD benchmark/src/main/resources/LC08_L1GT_001003_20170921_20170921_01_RT_B1.TIF /geotrellis-contrib/benchmark/src/main/resources/

Expand Down
1 change: 0 additions & 1 deletion Makefile.benchmark
Expand Up @@ -7,7 +7,6 @@ all: archives/geotrellis-contrib.tar \
archives/gdal-data.zip \
archives/libgdal-java_1.10.1+dfsg-5ubuntu1_amd64.deb \
benchmark/src/main/resources/LC08_L1GT_001003_20170921_20170921_01_RT_B1.TIF
docker build -t benchmark -f Dockerfile.benchmark .

archives/geotrellis-contrib.tar:
git archive --format=tar --prefix=geotrellis-contrib/ -o $@ HEAD
Expand Down
31 changes: 10 additions & 21 deletions benchmark/README.md
@@ -1,49 +1,38 @@
# With Docker #
# With Docker

## Build ##
In the root of the repository, run `./scripts/benchmark`

In the root of the repository, type `make -f Makefile.benchmark`.
# Without Docker

## Run ##
## Get Dependencies

With the Docker image built, type `docker run -it --rm benchmark`.
Inside of the docker container, type
```bash
cd /geotrellis-contrib
./sbt "project benchmark" "jmh:run"
```

# Without Docker #

## Get Dependencies ##

### Dependency Jars ###
### Dependency Jars

Pre-built ImageIO jars can be found [here](https://demo.geo-solutions.it/share/github/imageio-ext/releases/1.1.X/1.1.24/).
The jars are in [this](https://demo.geo-solutions.it/share/github/imageio-ext/releases/1.1.X/1.1.24/imageio-ext-1.1.24-jars.zip) archive.

### Native Dependencies ###
### Native Dependencies

Pre-built native dependencies can be found [here](https://demo.geo-solutions.it/share/github/imageio-ext/releases/1.1.X/1.1.24/native/gdal/).
For Ubuntu 12.04 and compatible systems, [this tarball](https://demo.geo-solutions.it/share/github/imageio-ext/releases/1.1.X/1.1.24/native/gdal/linux/gdal192-Ubuntu12-gcc4.6.3-x86_64.tar.gz) is known to work.
GDAL also needs certain data to operate, those files can be found [here](https://demo.geo-solutions.it/share/github/imageio-ext/releases/1.1.X/1.1.24/native/gdal/gdal-data.zip).

### GDAL Java Bindings ###
### GDAL Java Bindings

Java bindings are also needed.
For Ubuntu 12.04 and compatible systems, [this package](https://packages.ubuntu.com/trusty/amd64/libgdal-java/download) contains the needed files.

## Get Data ##
## Get Data

The test data can be found [here](https://landsatonaws.com/L8/001/003/LC08_L1GT_001003_20170921_20170921_01_RT).

## Run ##
## Run

```bash
GDAL_DATA=$(pwd)/benchmark/gdal/gdal-data LD_LIBRARY_PATH=$(pwd)/benchmark/gdal/native:$LD_LIBRARY_PATH ./sbt "project benchmark" "jmh:run"
```

# Using JMH #
# Using JMH

We use JMH to wire up all benchmarks. Understanding the semantics of the JMH
annotations takes some time. For clues, take a look at this
Expand Down
15 changes: 8 additions & 7 deletions build.sbt
Expand Up @@ -10,13 +10,14 @@ lazy val commonSettings = Seq(
// We are overriding the default behavior of sbt-git which, by default,
// only appends the `-SNAPSHOT` suffix if there are uncommitted
// changes in the workspace.
version := {
// Avoid Cyclic reference involving error
if (git.gitCurrentTags.value.isEmpty || git.gitUncommittedChanges.value)
git.gitDescribedVersion.value.get + "-SNAPSHOT"
else
git.gitDescribedVersion.value.get
},
// version := {
// // Avoid Cyclic reference involving error
// if (git.gitCurrentTags.value.isEmpty || git.gitUncommittedChanges.value)
// git.gitDescribedVersion.value.get + "-SNAPSHOT"
// else
// git.gitDescribedVersion.value.get
// },
version := "0.0.1",
homepage := Some(url(Info.url)),
scmInfo := Some(ScmInfo(
url("https://github.com/geotrellis/geotrellis-contrib"), "scm:git:git@github.com:geotrellis/geotrellis-contrib.git"
Expand Down
25 changes: 25 additions & 0 deletions scripts/benchmark
@@ -0,0 +1,25 @@
#!/bin/bash

set -e

if [[ -n "${GEOTRELLIS_CONTRIB_DEBUG}" ]]; then
set -x
fi

function usage() {
echo -n \
"Usage: $(basename "$0")
Run GeoTrellis Contrib benchmarks
"
}

if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
if [[ "${1:-}" == "--help" ]]; then
usage
else
echo "Executing GeoTrellis Contrib benchmarks"
make -f Makefile.benchmark
docker build -t benchmark -f Dockerfile.benchmark .
docker run -it --rm benchmark ./sbt "project benchmark" "jmh:run"
fi
fi

0 comments on commit 8dcf15d

Please sign in to comment.