Generate Static Metamodel Code with eGenericSuperTypes

[CrazyAutomotiveHacker]

I want to generate a static metamodel in python from an .ecore file. But when I try to generate the python code with the pyecoregenerator, I get the following error message:

Traceback (most recent call last):
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\Scripts\pyecoregen-script.py", line 11, in
load_entry_point('pyecoregen==0.4.3', 'console_scripts', 'pyecoregen')()
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecoregen-0.4.3-py3.5.egg\pyecoregen\cli.py", line 15, in main
generate_from_cli(sys.argv[1:]) # nocover
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecoregen-0.4.3-py3.5.egg\pyecoregen\cli.py", line 57, in generate_from_cli
model = load_model(parsed_args.ecore_model)
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecoregen-0.4.3-py3.5.egg\pyecoregen\cli.py", line 88, in load_model
resource = rset.get_resource(uri_implementation(ecore_model_path))
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecore-0.9.0-py3.5.egg\pyecore\resources\resource.py", line 86, in get_resource
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecore-0.9.0-py3.5.egg\pyecore\resources\xmi.py", line 60, in load
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecore-0.9.0-py3.5.egg\pyecore\resources\xmi.py", line 169, in _decode_eobject
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecore-0.9.0-py3.5.egg\pyecore\resources\xmi.py", line 148, in _decode_eobject
File "C:\Users\RBO7SI\AppData\Local\Programs\Python\Python35\lib\site-packages\pyecore-0.9.0-py3.5.egg\pyecore\resources\xmi.py", line 183, in _decode_node
ValueError: Feature "eGenericSuperTypes" is unknown for EClass, line 24

Are there any options to solve this issue? Is it possible to extend / adapt the generator to work with 'eGenericSuperTypes'?

Here is my .ecore file (testecore.ecore):

<?xml version="1.0" encoding="UTF-8"?> <ecore:EPackage xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ecore="http://www.eclipse.org/emf/2002/Ecore" name="testecore" nsURI="http://www.example.org/testecore" nsPrefix="testecore"> <eClassifiers xsi:type="ecore:EClass" name="Owner"> <eStructuralFeatures xsi:type="ecore:EReference" name="buildings" upperBound="-1" eType="#//Building"/> <eStructuralFeatures xsi:type="ecore:EAttribute" name="name" eType="ecore:EDataType http://www.eclipse.org/emf/2002/Ecore#//EString"/> </eClassifiers> <eClassifiers xsi:type="ecore:EClass" name="Building" abstract="true"> <eTypeParameters name="T"/> <eStructuralFeatures xsi:type="ecore:EReference" name="test" upperBound="-1" containment="true"> <eGenericType eTypeParameter="#//Building/T"/> </eStructuralFeatures> </eClassifiers> <eClassifiers xsi:type="ecore:EClass" name="House" eSuperTypes="#//Building"> <eStructuralFeatures xsi:type="ecore:EAttribute" name="rooms" eType="ecore:EDataType http://www.eclipse.org/emf/2002/Ecore#//EInt"/> </eClassifiers> <eClassifiers xsi:type="ecore:EClass" name="Test"> <eTypeParameters name="T"/> <eStructuralFeatures xsi:type="ecore:EReference" name="hallo" upperBound="-1" containment="true"> <eGenericType eTypeParameter="#//Test/T"/> </eStructuralFeatures> <eGenericSuperTypes eClassifier="#//Building"> <eTypeArguments eTypeParameter="#//Test/T"/> </eGenericSuperTypes> </eClassifiers> </ecore:EPackage>

testecore.zip

[aranega]

@CrazyAutomotiveHacker Sorry about the late answer. The issue you have here comes from the implementation of the Ecore metamodel which misses some parts which I will add for the next release. To be able to open it and generates code for it, with the current PyEcore version, you need to patch it by hand. The code would be this (for the patching + the code generation)

from pyecore.resources import ResourceSet
from pyecore.ecore import EClass, EGenericType, ETypeParameter, EReference, \
                          EClassifier, ETypeParameter
import pyecore.ecore as Ecore
from pyecoregen.ecore import EcoreGenerator

# Ecore metamodel patch (will be added in PyEcore for the next release)
eGenericSuperTypes = EReference('eGenericSuperTypes', EGenericType,
                                containment=True, upper=-1)
EClass.eClass.eStructuralFeatures.append(eGenericSuperTypes)
EClass.eGenericSuperTypes = eGenericSuperTypes

eClassifier = EReference('eClassifier', EClassifier)
EGenericType.eClass.eStructuralFeatures.append(eClassifier)
EGenericType.eClassifier = eClassifier

eTypeArguments = EReference('eTypeArguments', EGenericType, containment=True,
                            upper=-1)
EGenericType.eClass.eStructuralFeatures.append(eTypeArguments)
EGenericType.eTypeArguments = eTypeArguments

eTypeParameter = EReference('eTypeParameter', ETypeParameter,
                            eOpposite=ETypeParameter.eGenericType)
EGenericType.eClass.eStructuralFeatures.append(eTypeParameter)
EGenericType.eTypeParameter = eTypeParameter

# We open the metamodel
rset = ResourceSet()
mm_root = rset.get_resource('testecore.ecore').contents[0]

# We generate the code using the EcoreGenerator
EcoreGenerator().generate(mm_root, outfolder='.')

However, please note that EGenerics are not yet supported by PyEcore, neither in the implementation (an eType is currently required for dynamic metamodels) nor in pyecoregen.. This results to an incomplete code generation which should be pateched by hand again. In the example you provided, I had to change the generated code from __init__.py in order to include types for the test and hallo references. I assumed here that the type could only be EObject, but it could be modified:

# ... generated code here
from pyecore.ecore import EObject

Building.test.eType = EObject  
Test.hallo.eType = EObject
# ... generated code here

I will try to find a way of dealing with EGeneric, at least in the code generation, and at the moment, I will add this points as "on the roadmap" in the README.rst, sorry about that, it should have been there...

[CrazyAutomotiveHacker]

Hello, thanks a lot for the quick response!
The code generation is working now with the development branch of pyecore.

But now I have the following problem with the example:
genericTest.zip

I created the following 3 classes:
Container, with a place holder parameter T (type EObject)
Bar, with Composition(myList) from Bar to Container, eType of test should be EBigInteger
Foo, with Composition(myList) from Foo to Container, eType of test should be EString

If I try to instantiate these classes and set the parameter test (either as String or as BigInteger) in the following way:

myContainerBar = Container(test=[3])
myContainerFoo = Container(test="ThisDoesNotWork")
myBar.myList.append(myContainerBar)
myFoo.myList.append(myContainerFoo)

I get the error:

pyecore.valuecontainer.BadValueError:
Expected type <class 'pyecore.ecore.EObject'>, but got type int with value x instead

Do you maybe have an idea for an workaround to handle these kind of problems, especially for larger models?
Thank you very much!

[aranega]

Hi @CrazyAutomotiveHacker .

Unfortunately, I think there is no easy solution for you issue (beside modifying the metamodel). The use of EObject as explicit type for eType is the issue here. From what I read in your metamodel, I assumed that each element in the collection should be an instance of EObject at some point (as it uses an EReference in the metamodel). I'm working on this, I started adding missing code for generic, but it still requires more work and a modification of pyecoregen.

Currently, for the case of Container, what you can do is to modify the test from an EReference to an EAttribute like this:

# before
class Container(EObject, metaclass=MetaEClass):
    test = EReference(unique=True, derived=False, ordered=True, containment=True, upper=-1)

# after
class Container(EObject, metaclass=MetaEClass):
    test = EAttribute(eType=Ecore.ENativeType, unique=True, derived=False,
                      ordered=True, upper=-1)

This should work, but I'm aware that it is not exactly what you expressed in your metamodel... I'm really sorry for the inconvenience...

[CrazyAutomotiveHacker]

Hello, thank you very much for the support. The workaround works for now.
But in future project this is not very applicable. Is there a rough estimation(weeks, months, years) when it might be possible to work with eGenerics in PyEcore?
Thanks a lot!

[aranega]

Hi @CrazyAutomotiveHacker ,

I will definitely work on it, I really want to be able to express the same kind of semantic in Python. Actually, I have a genral idea, but I still don't exactly know how I can implement it in a proper/clean way (that will perhaps uses annotations for the generated code, but I will try to avoid if I can). I really need also to check the original EMF Java implementation so PyEcore will have the same behavior.

In the development branch, I started to brainstorm on the problem, but I think the few I did was only a small step in the right direction.

Regarding a roadmap, perhaps 1 or 2 months, I'm really under the water at work and this is really a not easy task unless I get the magical idea :) . I'm really sorry about the delay... I would really love to work full time on PyEcore ;)

[tobias-huermann]

Exactly the Feature i am currently missing. I am working in the automotive domain and trying to read, create and update Amalthea files with python/ pyecore. See https://Eclipse.org/app4mc

[aranega]

@tobias-huermann @CrazyAutomotiveHacker I will try to work on it during christmas holidays! I'm so sorry about the delay. The struggle here is clearly time (and a little bit of manpower), I have some solutions, but they are for the static part only, dynamic part needs more work to ensure that performances are kind of equivalent.

[aranega]

So, I had time to work a little bit on them, I have a first working version in PyEcore for dynamic metamodels (in the develop branch). The only thing that misses for static metamodels is the code for the generation. However, this version does not catch the full semantic of the "generics", only a subpart of it. Nevertheless, it should not be en issue to open models and metamodels that uses them. I will continue on the code generation that uses this solution very soon!

@tobias-huermann I looked to Amalthea ecore metamodel, but I didn't found it. Do you know if there is one somewhere? The only things I found are .xcore file and .xsd files.

[tobias-huermann]

amalthea.zip

@aranega Sorry i missed to answer. Find the Amalthea ecore attached. I used Eclipse to convert xcore format to ecore. The attached demo_car example model is chipped with APP4MC and covers a lot of the possible entities of Amalthea. The Runnable elements do store for example Weibull Deviation elements which are causing problems when accessing / creating models with PyEcore. We are not able to handle entities/ attributes related to "Deviation" meta-model element. See 3.2.9 Deviation here for Meta-model description.
By the way: Great project thanks for your excellent work :)

[aranega]

@tobias-huermann Thanks a lot for the test case! I will try it asap with the new pyecoregen version I have that partially deals with generics. From what I saw quickly in the .ecore, there will be only a small setup to generate the code (using the URI mapping).
I try that very soon! Thanks for your patience!

[alessiostalla]

It looks like pyecoregen doesn't yet support generic types. Can we do anything to help with this? I could work on a PR if I can get some guidance on how to proceed.

[aranega]

Hi @alessiostalla
Thanks for the message! It definitely reminds me that I put that a little bit out of my head... Indeed, the support for generic improved a little bit in PyEcore, but I didn't have time to go way further in the modification of pyecogen regarding to it, or testing.
I had started a branch on pyecoregen to deal with that: https://github.com/pyecore/pyecoregen/tree/feature/generics
From what I remember, I don't think all of generics are tackled inside, but a subset. I would need to check it to see what's missing.
I'll try to revive this asap, but in the meantime, you can take a look to the first modifications I made and see if it fits your needs (let's cross fingers 🤞) pyecore/pyecoregen@master...feature/generics

EDIT> Of course, if you have ideas how to modify/fix/add things in pyecoregen, please feel free to propose PRs with what works with you, I'll review/help with the code and we can try together to reach a fixed point where what's work for you is also tackling most use cases (hopefully all use cases from the first working version!) 🙂

[alessiostalla]

Looks cool, I'll try it out asap, probably early next week!