New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: startup API and APP with dsp-tools (DEV-126) #246
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,6 +3,8 @@ | |
""" | ||
import argparse | ||
import datetime | ||
import os | ||
import subprocess | ||
import sys | ||
from importlib.metadata import version | ||
|
||
|
@@ -14,8 +16,8 @@ | |
from knora.dsplib.utils.onto_create_ontology import create_project | ||
from knora.dsplib.utils.onto_get import get_ontology | ||
from knora.dsplib.utils.onto_validate import validate_project | ||
from knora.dsplib.utils.xml_upload import xml_upload | ||
from knora.dsplib.utils.shared import validate_xml_against_schema | ||
from knora.dsplib.utils.xml_upload import xml_upload | ||
from knora.excel2xml import excel2xml | ||
|
||
|
||
|
@@ -41,6 +43,7 @@ def program(user_args: list[str]) -> None: | |
default_pw = 'test' | ||
dsp_tools_version = version('dsp-tools') | ||
now = datetime.datetime.now() | ||
current_dir = os.path.dirname(os.path.realpath(__file__)) | ||
|
||
# parse the arguments of the command line | ||
parser = argparse.ArgumentParser(description=f'dsp-tools (Version {dsp_tools_version}) DaSCH Service Platform data ' | ||
|
@@ -135,6 +138,20 @@ def program(user_args: list[str]) -> None: | |
parser_excel2xml.add_argument('shortcode', help='Shortcode of the project that this data belongs to') | ||
parser_excel2xml.add_argument('default_ontology', help='Name of the ontology that this data belongs to') | ||
|
||
# startup DSP-API | ||
parser_stackup = subparsers.add_parser('start-api', help='Startup a local instance of DSP-API') | ||
parser_stackup.set_defaults(action='start-api') | ||
|
||
# shutdown DSP-API | ||
parser_stackdown = subparsers.add_parser('stop-api', help='Shut down the local instance of DSP-API, delete ' | ||
'volumes, clean SIPI folders') | ||
parser_stackdown.set_defaults(action='stop-api') | ||
|
||
# startup DSP-APP | ||
parser_dsp_app = subparsers.add_parser('start-app', help='Startup a local instance of DSP-APP') | ||
parser_dsp_app.set_defaults(action='start-app') | ||
|
||
|
||
|
||
# call the requested action | ||
args = parser.parse_args(user_args) | ||
|
@@ -204,6 +221,12 @@ def program(user_args: list[str]) -> None: | |
excel2xml(datafile=args.datafile, | ||
shortcode=args.shortcode, | ||
default_ontology=args.default_ontology) | ||
elif args.action == 'start-api' and not sys.platform.startswith('win'): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is probably a better way to check the OS, I guess? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not sure how that would behave with WSL... but probably you shouldn't worry about that too much. So this looks good to me |
||
output = subprocess.run(['/bin/bash', os.path.join(current_dir, 'dsplib/utils/start-api.sh')]) | ||
elif args.action == 'stop-api' and not sys.platform.startswith('win'): | ||
output = subprocess.run(['/bin/bash', os.path.join(current_dir, 'dsplib/utils/stop-api.sh')]) | ||
elif args.action == 'start-app' and not sys.platform.startswith('win'): | ||
output = subprocess.run(['/bin/bash', os.path.join(current_dir, 'dsplib/utils/start-app.sh')]) | ||
|
||
|
||
def main() -> None: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
#! /bin/bash | ||
# make this file executable with chmod u+x (filename).sh | ||
set -u # exit if an uninitialised variable is used (https://www.davidpashley.com/articles/writing-robust-shell-scripts/) | ||
set -e # exit if any statement returns a non-true return value (https://www.davidpashley.com/articles/writing-robust-shell-scripts/) | ||
|
||
# only allow to run this command if Docker is running | ||
[[ $(docker stats --no-stream 2>/dev/null ) == "" ]] && printf "\e[31mERROR: Please start Docker before running DSP-API.\e[0m\n" && return | ||
|
||
# check dependencies | ||
echo "check for outdated dependencies..." | ||
[[ "$(brew outdated)" != "" ]] && printf "\e[31mWARNING: Some of your Homebrew formulas/casks are outdated. Please execute \"brew upgrade\"\e[0m\n" | ||
[[ "$(java --version)" =~ .*[Tt]emurin-17.* ]] || printf "\e[31mWARNING: Your JDK seems to be outdated. Please install JDK 17 Temurin.\e[0m\n" | ||
if echo -e "GET http://google.com HTTP/1.0\n\n" | nc google.com 80 -w 10 > /dev/null 2>&1; then | ||
# pip should only be called if there is an internet connection | ||
[[ "$(pip list --outdated)" =~ .*dsp-tools.* ]] && printf "\e[31mWARNING: Your version of dsp-tools is outdated. Please update it with \"pip install --upgrade dsp-tools\"\e[0m\n" | ||
[[ "$(pip list --outdated)" != "" ]] && printf "\e[31mWARNING: Some of your pip packages are outdated. List them with \"pip list --outdated\" and consider updating them with \"pip install --upgrade (package)\"\e[0m\n" | ||
fi | ||
|
||
logfile="../dsp-api-stackup.log" | ||
|
||
cd ~ | ||
mkdir -p .dsp-tools | ||
cd .dsp-tools | ||
if [[ ! -d dsp-api ]]; then | ||
echo "git clone https://github.com/dasch-swiss/dsp-api.git" 2>&1 | tee -a "$logfile" | ||
git clone https://github.com/dasch-swiss/dsp-api.git >>"$logfile" 2>&1 | ||
fi | ||
cd dsp-api | ||
rm -f "$logfile" | ||
echo "make stack-down-delete-volumes..." 2>&1 | tee -a "$logfile" | ||
jnussbaum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
make stack-down-delete-volumes >>"$logfile" 2>&1 | ||
if echo -e "GET http://google.com HTTP/1.0\n\n" | nc google.com 80 -w 10 > /dev/null 2>&1; then | ||
# only pull if there is an internet connection | ||
echo "git pull ..." 2>&1 | tee -a "$logfile" | ||
git pull >>"$logfile" 2>&1 | ||
fi | ||
echo "make init-db-test..." 2>&1 | tee -a "$logfile" | ||
make init-db-test >>"$logfile" 2>&1 | ||
echo "make stack-up..." 2>&1 | tee -a "$logfile" | ||
make stack-up >>"$logfile" 2>&1 | ||
echo "DSP-API is up and running." # due to "set -e", this will only be printed if everything went well |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
#! /bin/bash | ||
# make this file executable with chmod u+x (filename).sh | ||
set -u # exit if an uninitialised variable is used (https://www.davidpashley.com/articles/writing-robust-shell-scripts/) | ||
set -e # exit if any statement returns a non-true return value (https://www.davidpashley.com/articles/writing-robust-shell-scripts/) | ||
|
||
[[ "$(npm -g outdated)" =~ .*@angular/cli.* ]] || printf "\e[31mWARNING: Your Angular seems to be outdated. Please update it with \"npm update -g @angular/cli\"\e[0m\n" | ||
|
||
logfile="../dsp-app-startup.log" | ||
|
||
cd ~ | ||
mkdir -p .dsp-tools | ||
cd .dsp-tools | ||
if [[ ! -d dsp-app ]]; then | ||
echo "git clone https://github.com/dasch-swiss/dsp-app.git" 2>&1 | tee -a "$logfile" | ||
git clone https://github.com/dasch-swiss/dsp-app.git >>"$logfile" 2>&1 | ||
fi | ||
cd dsp-app | ||
rm -f "$logfile" | ||
|
||
if echo -e "GET http://google.com HTTP/1.0\n\n" | nc google.com 80 -w 10 > /dev/null 2>&1; then | ||
# only pull if there is an internet connection | ||
echo "git pull ..." 2>&1 | tee -a "$logfile" | ||
git checkout HEAD package-lock.json >>"$logfile" 2>&1 | ||
git pull >>"$logfile" 2>&1 | ||
fi | ||
echo "npm i --legacy-peer-deps ..." 2>&1 | tee -a "$logfile" | ||
npm i --legacy-peer-deps >>"$logfile" 2>&1 | ||
echo "ng s ..." 2>&1 | tee -a "$logfile" | ||
npm run ng s 2>&1 | tee -a "$logfile" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
#! /bin/bash | ||
# make this file executable with chmod u+x (filename).sh | ||
set -u # exit if an uninitialised variable is used (https://www.davidpashley.com/articles/writing-robust-shell-scripts/) | ||
set -e # exit if any statement returns a non-true return value (https://www.davidpashley.com/articles/writing-robust-shell-scripts/) | ||
|
||
[[ $(docker stats --no-stream 2>/dev/null ) == "" ]] && printf "\e[31mERROR: Docker is not running, so there is no DSP-API to shut down.\e[0m\n" && return | ||
|
||
logfile="../dsp-api-stackdown.log" | ||
|
||
cd ~ | ||
[[ ! -d .dsp-tools/dsp-api ]] && return | ||
cd .dsp-tools/dsp-api | ||
rm -f "$logfile" | ||
echo "make stack-down-delete-volumes ..." 2>&1 | tee -a "$logfile" | ||
make stack-down-delete-volumes >>"$logfile" 2>&1 | ||
echo "make clean-sipi-tmp ..." 2>&1 | tee -a "$logfile" | ||
make clean-sipi-tmp >>"$logfile" 2>&1 | ||
echo "make clean-sipi-projects ..." 2>&1 | tee -a "$logfile" | ||
make clean-sipi-projects >>"$logfile" 2>&1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
deleting docker volumes through such a command can potentially be risky. You could consider having that as a separate command/flag
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand. I always use
make stack-down-delete-volumes
to shut down the API. Is that not good?By the way, thanks for pointing out that I mistakenly echoed
make init-db-test-minimal
. The command that I execute is without the "minimal".There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not necessarily "not good" - but if you invest time to set up specific test data, then that will be gone if you delete the volumes (and you need to re-initialize the DB). So e.g. if you modify your datamodel in the App, and then you want to export it to json again using dsp-tools, delete-volumes would not be what you want.