Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support for recent rebar3 version #84

Open
tothlac opened this issue Feb 14, 2022 · 5 comments · May be fixed by #89
Open

support for recent rebar3 version #84

tothlac opened this issue Feb 14, 2022 · 5 comments · May be fixed by #89

Comments

@tothlac
Copy link

tothlac commented Feb 14, 2022

I was using a very old rebar3 version (3.13.0).
During the recent days I wanted to try out 3.18.0.

On this repository when I run rebar3 eunit katt_blueprint.erl is generated, but it can't generate katt_blueprint.beam. The reason is simple as I see. Earlier you had this commit which changed pre_hooks to post_hooks in rebar.config:

2bc3255

The problem is that with the current version of rebar3 priv/compile-parser is executed after katt is compiled, so it generates katt_blueprint.erl but compile won't run twice, so rebar3 won't generate katt_blueprint.beam into the ebin directory.

I made some modifications on katt for this repository, and now it is called twice (from pre_hooks and from post_hooks).

I see the problem why it can't work with pre_hooks is that neotoma is compiled after the pre_hooks of katt is called so neotoma:file("priv/katt_blueprint.peg", [{output, "src/"}]) will fail since neotoma.beam had not been generated.

===> Fetching unicode_util_compat v0.7.0
Compile katt_blueprint
Could not compile katt_bluepring.peg. Reason: undef
===> Analyzing applications...
...
===> Compiling neotoma
===> Compiling jsx
...
===> Compiling katt
Compile katt_blueprint
===> Analyzing applications

The first

Compile katt_blueprint
Could not compile katt_bluepring.peg. Reason: undef

is printed when compile-parser is called from the pre_hooks and as you can see neotoma:file/2 is undef, the second Compile katt_blueprint is successful but compile has been already called, so katt_blueprint.beam won't be generated.

I've pushed the code here: https://github.com/tothlac/try_katt
@tothlac tothlac mentioned this issue Feb 14, 2022
Closed
@tothlac
Copy link
Author

tothlac commented Feb 15, 2022

The problem is definitely with the current compile order in rebar3. Now the pre_hooks are called before the dependencies of the project are compiled which is wrong.

Fred Herbert mentioned me this neotoma plugin:

https://github.com/tsloughter/rebar3_neotoma_plugin

I've tried and it actually works, the only problem is that it does not support an src_dir parameter. This is a problem because katt_blueprint.peg is stored in the priv folder, and the plugin wants to compile .peg files stored only in the src directory.

Either katt_blueprint.peg should be moved into ./src or the plugin should be slightly modified. Can you see any other options?

@tothlac
Copy link
Author

tothlac commented Feb 15, 2022

So I have just tried it on our projects with a simple modification in katt's code. I've moved prv/katt_blueprint.peg => src/katt_blueprint.peg and added

{plugins, [
           rebar3_neotoma_plugin
          ]}.

{provider_hooks, [
                 {pre, [
                        {compile, {neotoma, compile}}
                       ]}
                 ]}.

to rebar.config and commented out

{post_hooks, [{compile, "./priv/compile-parser"}]}.

Actually now there is no need for priv/compile-parser anymore. Should I create a pull request?

@brucify
Copy link
Contributor

brucify commented Feb 16, 2022 •
edited

@tothlac I tried your https://github.com/tothlac/try_katt with a branch of katt using the rebar3_neotoma_plugin plugin:

               {katt,
                   {git,
                       "git@github.com:brucify/katt.git",
                       {branch,"use-rebar3-neotoma-plugin"}}}

With katt rebar.config:

{plugins, [rebar3_neotoma_plugin]}.
{provider_hooks, [{pre, [{compile, {neotoma, compile}}]}]}.

It seems that katt_blueprint.erl is still not generated:

✗ rebar3 eunit
...
===> Performing EUnit tests...
F
Failures:

  1) try_katt_test:katt_test/0: module 'try_katt_test'
     Failure/Error: {error,undef,
                        [{katt_blueprint,module_info,[],[]},
                         {try_katt_test,katt_test,0,
                             [{file,
                                  "/Users/bruce/git/try_katt/test/try_katt_test.erl"},
                              {line,6}]},
                         {try_katt_test,katt_test,0,[]}]}
     Output:

Finished in 0.063 seconds
1 tests, 1 failures

✗ ls -l _build/test/lib/katt/src
total 240
-rw-r--r--  1 bruce  staff   1140 Feb 16 17:47 katt.app.src
-rw-r--r--  1 bruce  staff  16572 Feb 16 17:47 katt.erl
-rw-r--r--  1 bruce  staff   4006 Feb 16 17:47 katt.hrl
-rw-r--r--  1 bruce  staff   2174 Feb 16 17:47 katt_blueprint_parse.erl
-rw-r--r--  1 bruce  staff  12642 Feb 16 17:47 katt_callbacks.erl
-rw-r--r--  1 bruce  staff   7407 Feb 16 17:47 katt_callbacks_json.erl
-rw-r--r--  1 bruce  staff   6989 Feb 16 17:47 katt_callbacks_wfu.erl
-rw-r--r--  1 bruce  staff   6140 Feb 16 17:47 katt_cli.erl
-rw-r--r--  1 bruce  staff   6766 Feb 16 17:47 katt_har_cli.erl
-rw-r--r--  1 bruce  staff   2341 Feb 16 17:47 katt_request.erl
-rw-r--r--  1 bruce  staff   2126 Feb 16 17:47 katt_response.erl
-rw-r--r--  1 bruce  staff  20030 Feb 16 17:47 katt_util.erl
-rw-r--r--  1 bruce  staff  11000 Feb 16 17:47 katt_validate_type.erl
✗ rebar3 version
rebar 3.18.0 on Erlang/OTP 22 Erts 10.6.1

However if I go into _build/test/lib/katt, change the rebar.config and compile, it correctly generates the katt_blueprint.erl file.

Did you get it to work?

Edit:
It seems I forgot to move priv/katt_blueprint.peg to src/katt_blueprint.peg. Everything works after I did that.

===> Performing EUnit tests...
.
Finished in 0.105 seconds
1 tests, 0 failures

✗ ls -l _build/test/lib/katt/src
total 320
...
-rw-r--r--  1 bruce  staff  26348 Feb 16 18:39 katt_blueprint.erl
-rw-r--r--  1 bruce  staff   8525 Feb 16 18:39 katt_blueprint.peg
...
✗ ls -l _build/test/lib/katt/ebin
total 328
...
-rw-r--r--  1 bruce  staff  46032 Feb 16 18:39 katt_blueprint.beam
...

This is probably because rebar3_neotoma_plugin only looks for and outputs .peg files in the src directory:

         SourceDir = filename:join(rebar_app_info:dir(AppInfo), "src"),
         CompileFun = fun(Source, Target, _Config) ->
                              OutDir = filename:dirname(Target),
                              neotoma:file(Source, [{output, OutDir}])
                      end,

         rebar_base_compiler:run(Opts, [], SourceDir, ".peg", SourceDir, ".erl", CompileFun, [{check_last_mod, true}])

@andreineculau
Copy link
Member

bump
Sorry for the silence. Before I jump into this, I wanted to check if you have reached a solution @brucify @tothlac .
Thanks for reporting!

@brucify
Copy link
Contributor

brucify commented Jun 23, 2022

Sorry for a late response. I have a commit here that does what I mentioned above:

  • uses rebar3 plugin rebar3_neotoma_plugin
  • removes priv/compile-parser
  • moves priv/katt_blueprint.peg to src/katt_blueprint.peg

I can create a PR for this.

@brucify brucify linked a pull request Jun 23, 2022 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use rebar3_neotoma_plugin brucify/katt
3 participants
@andreineculau @brucify @tothlac