Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

working with aws MWAA #58

Open
HugoCornu opened this issue Aug 26, 2022 · 3 comments
Open

working with aws MWAA #58

HugoCornu opened this issue Aug 26, 2022 · 3 comments

Comments

@HugoCornu
Copy link

Hi,

I would like to solve
https://stackoverflow.com/questions/72726969/how-can-we-use-airflow-code-editor-or-simple-dag-editor-on-aws-airflow-mwaa/73454971#73454971

I follow every Andrea instruction but it doesn't work.
I will post more info here to help

Our configuration :

MWAA airflow version is 2.2.2

To setup airflow-code-editor :
I added the code_editor.mount envariable in MWAA (name=dags,path=s3://our_bucket)
I added this to my requirements

# plugins
black  # for airflow-code-editor
fs-s3fs # for airflow-code-editor
airflow-code-editor>=7.0.0

MWAA has been tested with s3:* on * permissions

What works:

airflow-code-editor is accessible
in the "files" folder we fine our dags
in the "dags" folder we fine our dags
reading dags

What doesn't work:

upload does nothing
+new open an editor, the button save make the following error
Error saving dag/new file.txt: operation failed, [Errno 30] Read-only file system: b'/usr/local/airflow/dags/'
opening a dags and clicking save
Error saving tutorial_dag_from_airflow.py: operation failed, [Errno 30] Read-only file system: b'/usr/local/airflow/dags/tutorial_dag_from_airflow.py'

What I can do to help

I would take me a bit of time to anonymize but I could provide the terraform code to reproduce
I also can write PR with a bit of guidance for where to search the bug

Thanks
Regards
Hugo

@andreax79
Copy link
Owner

Hi Hugo,
I created a similar example on AWS and I can save/upload the files. I'm trying to find the differences.
Could you try click on the "dags" and try to create a new file.
When you save the file, the name will be something like "~dags/dag/new file.txt".
Thanks,
Andrea

@HugoCornu
Copy link
Author

HugoCornu commented Aug 29, 2022

Hi
Good news
I create a new file and this time is worked. But the filegot created outside of the dags/folder
so I clicked "saved as" and entered the path "~dags/dag/new file.txt"
that work and the "dags" node in airflow-code-editor somehow refreshed and now it is show the file in my bucket (all of them, not just the dags prefix). Now editing work for all files which was the goal.

There are still some issues like the "delete" not working but airflow-code-editor is operational and thats great !

Now I got to learn how to bind it with git but that is another story

Thanks

@HugoCornu
Copy link
Author

HugoCornu commented Aug 29, 2022

I changed code_editor.mount to name=s3,path=s3://our_bucket/dags
And I got the same issue as before
I create a new file and it started working again. Like creating that new file is setting a variable up or something

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants