New Feature: Use abapGit as a small ABAP SDK (Objects API)

[fmabap]

Hello,
I want to enhance abapGit by an API that allows to serialize, deserialize and delete ABAP objects without a git repository.
So that you can use abapGit like a small ABAP SDK.

I want to use it for example to generate new ABAP Objects with scaffolding tools like Yeoman or for simple things like uploading new data elements.

I know that SAP provides an SDK for the ABAP Development Tools to do such stuff. But it supports not all kind of objects, and it is also not possible to maintain the objects in several languages with it.

I want to enhance therefore the class ZCL_ABAPGIT_OBJECTS with new methods:
The new method SERIALIZE_OBJECTS shall import a table of OBJ_TYPE and OBJ_NAME and return a table of the existing type ZCL_ABAPGIT_OBJECTS=>TY_SERIALIZATION. It will be implemented like the method ZCL_ABAPGIT_ZIP=>EXPORT_OBJECT.
The new method DESERIALIZE_OBJECTS shall import a table of the existing type ZCL_ABAPGIT_OBJECTS=>TY_SERIALIZATION (like the result of SERIALIZE_OBJECTS). It will be implemented like the method ZCL_ABAPGIT_OBJECTS=>DESERIALIZE, but without IO_REPO. It will reuse the private methods of the class ZCL_ABAPGIT_OBJECTS, were possible.
The new method DELETE_OBJECTS shall import a table of OBJ_TYPE and OBJ_NAME and Transport Request. It will delete the objects from the system. It will be implemented like the method ZCL_ABAPGIT_OBJECTS=>DELETE.

I want to create additional in a separate git repository a web service that calls these new methods. I assume that a separate git repository is required for it, because you cannot generate the web service into the standalone program.
The web service will be an implementation of the BADI BADI_ADT_REST_RFC_APPLICATION. It will contain POST methods to call the API methods for several objects.

Can I start with the implementation of it?
Will you create the new repository for the web service, so that I can fork it?

Best regards
Frank

[larshp]

SERIALIZE_OBJECTS suggest using existing method SERIALIZE, it looks identical. DESERIALIZE_OBJECTS use DESERIALIZE, we are in progress of decoupling the REPO, DELETE_OBJECTS use DELETE. Basically I dont think it needs additional methods to support the use-case.

[mbtools]

Hi Frank,

this goes in a direction that's also of general interest (mine, too):

Clear separation of UI, repo, and object layers in the abapGit code

The question is how... There are other things to consider like AG v2, LXE, wishes to parallelize more, etc. This is a bigger change so please don't jump into coding. Overall, our strategy is to do smaller changes that are easy to follow and merge leading to longer term goals.

I have thought about this for a while already. I'm out next week but let's discuss after. I will set something up via slack.

[fmabap]

Hi Marc, hi Lars,

I'm currently on vacation and so I have some time for this stuff.

I already started with some prototyping (can be thrown away or enhanced).

I implemented already the serialization functionality. It is like the ZCL_ABAPGIT_ZIP=>EXPORT_OBJECT method. But it is designed for multiple objects (currently without parallelization) and it uses TADIR-MASTERLANG instead of SY-LANGU as main language for io_i18n_params (this looks more correct for me). It sets also the TADIR fields like the DEVCLASS in the returning table. The existing SERIALIZE method does this not. The existing SERIALIZE method imports also the parameter IS_ITEM (ZIF_ABAPGIT_DEFINITIONS=>TY_ITEM). This structure has additional fields like the DEVCLASS and other TADIR fields. But it looks like that they are not required to serialize objects. This is not such a nice interface for an API. So, I created the new types ZIF_ABAPGIT_DEFINITIONS=>TY_OBJ and TY_OBJ_TT which contains only the relevant fields OBJ_TYPE and OBJ_NAME.
I did not change the existing SERIALIZE method to avoid breaking something.

I enhanced also the zcl_abapgit_tadir to read multiple entries.
The changes can be found here ( I created no pull request so far ).

If you like, then we can discuss this also on Slack in detail.

Best regards
Frank

[larshp]

What is the background for going for a direct approach? It bypasses transparency, separation, testing, review, static analysis, testing, which are all important?

But anyhow, I think still the existing methods can use some refactoring, to more clearly separate the repo.

[fmabap]

My purpose is to generate ABAP Objects in the system with external tools. For example a TS tool that generates ABAP Objects based on templates (Yeoman). I don't want to handle with these kinds of tools a whole git repository. I will add the new objects afterwards to the git repository (Stage by Transport) to have the transparency.

Another user of the new web services could be also the ABAP remote filesystem in VS Code to support more ABAP Objects than the SAP Standard ADT SDK (including translation).

Regarding the refactoring of the existing methods: I think we should create the new methods to handle the objects and refactor afterwards the existing repo methods to use the new methods. So we can test and enhance (parallelization for example) the new functionality first in a production envoirment without breaking existing functionality.

[larshp]

TS Tool: Personally I'd like to generate => commit to git => static analysis => unit testing => code review => merge => then pull to system. That way errors or anything can be discovered before hitting central(?) development. Note: this also makes it easier to test the TS Tool locally/standalone

Adding new methods: I'd prefer not to add methods which are not statically referenced in abapGit, these are just candidates to be deleted. Its okay to refactor, there is static analysis + unit tests + reviews before anything is merged.

[fmabap]

I understand and like your approach, but we use in our project abaGit only as a documentation system to add the ABAP changes to a JIRA Issue.
By the way, will the ABAP Lint transpiler also support AMDPs in future?

Regarding Refactoring: I will see what I can do. I will first finalize and test the new deserialization method with multiple packages and then I will see how I can refactor the existing coding...

[fmabap]

I have an issue with the package structure. I don't have a starting package to create a folder structure.

One approach could be not to serialize / deserialize package objects and to check during deserialization that all packages exists.

Another approach could be to determine the whole package hierarchie of the selected objects and to use it as path, but serialize only the selected package objects.

Do you have any preferences or better ideas?

By the way regarding refactoring:
There is the class ZCL_ABAPGIT_SERIALIZE to serialize a repo. It uses the method zcl_abapgit_objects=>serialize to serialize one object without specificas of a repo. Also the path logic is in the ZCL_ABAPGIT_SERIALIZE class (should be renamed to ZCL_ABAPGIT_SERIALIZE_REPO) .
So I think, may we should create a new class ZCL_ABAPGIT_DESERIALIZE_REPO to handle the repo logic for deserialize and refactor the class zcl_abapgit_objects to prepare the deserialization of one object without the repo specific things.

The part to execute the deserialization steps of multiple objects should be in a separate method of the class zcl_abapgit_objects.

The specific logic to serialize / deserialize only objects can then also be separated in new classes (ZCL_ABAPGIT_SERIALIZE_OBJS, ZCL_ABAPGIT_DESERIALIZE_OBJS)

What do you think about it?

[mbtools]

I think right direction but such big changes can't be done via texting here. There are over two dozen open refactoring issues, many related. Peacemeal PRs might make it worse. That's why I suggested a meeting. Others like Christian and Alexander might have input, too. Once we agree, then code.

[fmabap]

Ok, can you then please schedule a meeting?

[fmabap]

I created a new version of it. I refactored the repo deserialization, so that it is possible to use most of these methods also without a repo (it is a jungle of coding ;-) ).

I also created the new class ZCL_ABAPGIT_OBJECTS_API. It contains the APIs for serialization and deserialization / delete only objects. It has also methods to use it as a Web API (Base64 encoded data instead of XSTRING).

For this is still a top package required, else we would also need a new folder logic.

Maybe we can take this new version as basic for our meeting on monday.

The following methods are now obsolete. I will delete them later (after the discussion):

• ZCL_ABAPGIT_OBJECTS=>DESERIALIZE
• ZCL_ABAPGIT_OBJECTS=>COMPARE_REMOTE_TO_LOCAL

[sandraros]

First of all, sorry because I'm posting here without reading the discussion. I just read the question. For information, I'm successfully using abapGit to generate objects by program in this project: https://github.com/sandraros/shrinker.
I'm running the abapGit classes in dialog, so I don't know if it works in background (abapGit displays popups and so on, so might not work in background).

You may find the relevant code by starting looking at these lines: https://github.com/sandraros/shrinker/blob/33e0687b1375d7fa286a2f6209a5992d9a8c95db/src/zcl_shrinker_connect_abapgit.clas.abap#L76-L153