New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
V1 for Automation Website translations Issues Generation Ref: Issue 325 #407
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Dianmz <dcmc2297@gmail.com>
Signed-off-by: Dianmz <dcmc2297@gmail.com>
Signed-off-by: Dianmz <dcmc2297@gmail.com>
Signed-off-by: Dianmz <dcmc2297@gmail.com>
Signed-off-by: Dianmz <dcmc2297@gmail.com>
Signed-off-by: Dianmz <dcmc2297@gmail.com>
Merge workflow
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com> feat: added branches main to check outdated content workflow
Signed-off-by: leonardpahlke <leonard.pahlke@googlemail.com>
Signed-off-by: leonardpahlke <leonard.pahlke@googlemail.com>
Also includes translation of search.md file Signed-off-by: thelooter <evekolb2204@gmail.com>
Signed-off-by: Kristina Devochko <guidemetothemoon@gmail.com>
Signed-off-by: leonardpahlke <leonard.pahlke@googlemail.com>
…cncf#376) * Add German translation for _index.md Signed-off-by: Rishikesh <rishikeshjagadale19@gmail.com>
Signed-off-by: Kristina Devochko <kristina.devochko@tietoevry.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com> feat: updated check outdated content
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com> feat: added v2 for create issue feat: changes on workflow
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com> feat: change on es
…nguages (#40) * change file * adding base * changes Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: Adding Assigneeses Signed-off-by: Dianmz <dcmc2297@gmail.com> * labels * test * test2 * test3 * change to en * fix: comment step Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: testing various languages Signed-off-by: Dianmz <dcmc2297@gmail.com> * test: comparing languages Signed-off-by: Dianmz <dcmc2297@gmail.com> * test: testing changes Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: creating issue for zh language Signed-off-by: Dianmz <dcmc2297@gmail.com> * test: testing all languages Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: testing a new step Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: testinbg more languages Signed-off-by: Dianmz <dcmc2297@gmail.com> * fix: iteration Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: changes Signed-off-by: Dianmz <dcmc2297@gmail.com> * fix: iteration Signed-off-by: Dianmz <dcmc2297@gmail.com> * text: changes Signed-off-by: Dianmz <dcmc2297@gmail.com> * test: new call Signed-off-by: Dianmz <dcmc2297@gmail.com> * fix: calling a new yml Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: adding create issue file Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: calling the file Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat: creating issues Signed-off-by: Dianmz <dcmc2297@gmail.com> * fix:CURL Signed-off-by: Dianmz <dcmc2297@gmail.com> * fix:adding credentials Signed-off-by: Dianmz <dcmc2297@gmail.com> * feat:testing Signed-off-by: Dianmz <dcmc2297@gmail.com> * Update check-outdated-content.yml Signed-off-by: Diana Menendez <61333059+Dianmz@users.noreply.github.com> * feat:final details per language Signed-off-by: Dianmz <dcmc2297@gmail.com> --------- Signed-off-by: Dianmz <dcmc2297@gmail.com> Signed-off-by: Diana Menendez <61333059+Dianmz@users.noreply.github.com>
…hanges Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
✅ Deploy Preview for tag-env-sustainability ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
Signed-off-by: Sergio Méndez <sergioarm.gpl@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much! I added a couple of comments.
cc @nate-double-u you may be interested in this work too. It about automatically detecting if translations to the TAG website need to happen & opens issues for translators to do translations.
--- | ||
name: Check outdated content |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for all the work! Awesome that it works! Three comments from my side:
- We need documentation
- We may need to revise some parts of the script
Documentation
Could we add some documentation about this. We do not have any development guide for the website so lets create a new document WEBSITE_DEVELOPMENT.md which has a section about how we do translations. The document can be extended with additional content (website dev related stuff).
About the translations, it would be good to explain:
- How does it work / What happens if you make a change to the english version of the website
- maintenance steps
2.1. Adding a new translation. (for example, French - which steps do we need to take to support translations)
2.2. Which people are notified to facilitate translations and how to edit this information - Explain what happens if the workflow is not followed; like:
3.1. Changes are made to the Spanish side initially (our workflow does not work, right - this is ok! But it would be good to write a sentence about that)
3.2. The Translation issue is closed, and no translation was made. In this case, the translation will stay out of sync, right. (this is fine too 😄 - but would be good to document it)
3.3. ... - How to work with the pipeline - how to test it if you make edits (step 1, step 2)
Adding this would be fantastic! (feel free to add to this if you feel it is relevant information)
Script / Code
For my taste, the script is too long to be inline. It's not just a few commands which we can understand quickly, it's a bit more complicated; so we need to move it. We can stick to shell scripts.
example:
- name: Run script file
run: |
chmod +x ./public/scripts/test.sh
./public/scripts/test.sh
shell: bash
After these changes were made, I will have a closer look at the logic. I added a few more comments to the code already.
🙌
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks true, I will add this to the contributors document.
- name: Detecting pending translations | ||
shell: bash | ||
run: | | ||
##### DEBUG section, this will be removed later ########### |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment says "remove later". It would be good to keep this logging functionality and enable these logs over a --debug
flag.
Example code by an LLM (needs to be tested)
# Default debug level
DEBUG_LEVEL="info"
# Parse command-line options
while [[ "$1" != "" ]]; do
case $1 in
--debug)
DEBUG_LEVEL="debug"
;;
*)
# Unknown option
echo "Unknown option: $1"
exit 1
;;
esac
shift
done
# Log function
_log() {
local message="$1"
local level="$2"
# Determine if the message should be logged based on DEBUG_LEVEL
case "$DEBUG_LEVEL" in
debug)
# Log all messages
echo "$level: $message"
;;
info)
# Log only info and error messages
if [[ "$level" == "info" || "$level" == "error" ]]; then
echo "$level: $message"
fi
;;
error)
# Log only error messages
if [[ "$level" == "error" ]]; then
echo "$level: $message"
fi
;;
esac
}
# Example usage:
_log "This is an info message" "info"
_log "This is a debug message" "debug"
_log "This is an error message" "error"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, it would be spectacular if there is an --dry-run
option. So we can see which issues would be opened without opening any. (but we dont need to add this - would be great tho 😄, very helpful debugging, i suppose). Example execution: translation.sh <ORG> <REPO> <PR NUMBER> --dry-run --debug
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, We were thinking about to have a script instead of put this in the pipeline, is a good idea.
|
||
OUTPUT_DIR="./outdated" | ||
CONTENT_DIR="website/content" | ||
languages=("es" "zh") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering how we make sure that we have minimal maintenance, adding new translations in the future. We could for example look at the repository and check the folder names https://github.com/cncf/tag-env-sustainability/tree/main/website/content - we have a folder en
& zh
& es
&de
, we establish a base with en
(we dont need to open a PR for en since its already merged). This could eliminate referencing languages in the script.
Assigning people over groups could work this way too. We assign tag-env-translators-<LANGUAGE REFERENCE>
so for example tag-env-translators-es
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we could work on it.
for lang in ${languages[@]}; do | ||
touch $lang.txt | ||
done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mind explaining this code? We create a txt file per translation language. So we end up with zh.txt, es.txt ... why do we need these files? Probably we add this info to the issue descriptions, right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure we have to document that part.
compare () { | ||
FILE_PATH=$1 | ||
# Actually compare between the old and latest English content and log diff in the file | ||
if [[ -f "${FILE_PATH}" ]]; then | ||
# File exists | ||
# Check changes | ||
git diff ${OLD_BRANCH}..${LATEST_BRANCH} -- ${FILE_PATH} > temp.diff | ||
if [[ -s "temp.diff" ]]; then | ||
echo "(DEBUG) ${FILE_PATH} is outdated." | ||
mkdir -p ${OUTPUT_DIR}/${FILE_PATH%/*} | ||
mv temp.diff ${OUTPUT_DIR}/${FILE_PATH} | ||
else | ||
echo "check if ${FILE_PATH} exist in other languages" | ||
for lang in ${languages[@]}; do | ||
NEW_FILE_PATH=$(echo "${FILE_PATH}" | sed -e "s/${base_lang}\//${lang}\//g") | ||
if [[ ! -e "${NEW_FILE_PATH}" ]]; then | ||
echo "The file ${FILE_PATH} needs to be translated to $lang" | ||
echo ${NEW_FILE_PATH} >> $lang.txt | ||
fi | ||
done | ||
fi | ||
else | ||
echo "(DEBUG) ${FILE_PATH} does not exist." | ||
fi | ||
} | ||
|
||
find $base_lang -iname "*.md" > files.txt | ||
sort files.txt > files_temp.txt;mv files_temp.txt files.txt | ||
INFILE=files.txt | ||
|
||
while IFS= read -r line | ||
do | ||
compare "$line" | ||
done < "$INFILE" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part makes me think if we may want to move the logic to smth like Go or Python, which handles more complex logic better IMO. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that it could be less lines with bash, but let me document the script.
It looks like there was some commit mess-up, since there are some commits in here, that are already in master |
Hi Dear TAG Env Team, this is the final the work that we are doing with @Dianmz.
This includes a workflow called Check outdated content, that generates new Issues for Languages for ES & ZH languages at the moment. Currently the issues are assigned to Diana and I, but should be modified to the new groups that you suggested or new ones.
The issues will look like this:
And inside the issue will look like this:
cc: @leonardpahlke @guidemetothemoon