DSL evaluation for mindstorms - please take a look

[dzw1999]

Hello, greetings from the software engineering group from Beihang University, China. We have been working on the evaluation of Domain Specific Modeling Languages (DSMLs ), the main purpose is to find out the flaws of the design of the DSML. To test our evaluation framework, we evaluated your software design, mainly based on your ecore and odesign files, and found out that even though the overall quality of your software design is very good, there still exists some minor problems. The details are as follows:

rule-based evaluation

Firstly, we did some rule-based evaluation, mainly focusing on the graphic part.

Evaluation of Semiotic Clarity

We've checked your odesign file according to Goodman's theory of symbols[Goodman N. Languages of Art: An Approach to a Theory of Symbols[M]. Hackett publishing, Indianapolis, 1976.], and there are some minor problems that can decrease the semiotic clarity of your design as follows:

1. overloading

   There are some graphic symbols in your design that is assigned to more than one element.

   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\icons\Begin_48px.png : mindstorms.Main; mindstorms.BlockContainer;
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\icons\End_48px.png : mindstorms.Main; mindstorms.BlockContainer;

2. Excess

   There are some graphic symbols in your design that does not represent any of the elements of your language.

   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\images\TouchSensor.svg
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\icons\Reuse_Procedure_32px.png
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\icons\UltrasonicSensor_48px.png
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\images\Score.svg
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\icons\ColorSensorRed_16px.png
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\images\Arbitrator.svg
   • mindstorms\plugins\fr.obeo.dsl.mindstorms.design\images\mindstorm_red&green_palette.svg
   • and so on... (in all 114 pictures)

3. Deficit Elements

   There are some elements in your design that are not represented by any graphic symbols.

   Certainly, we know that some elements do not need to be assigned symbols to, but we believe it is best to check whether these elements really do not need symbols or they were just overlooked.

   • NamedElement
   • Arbitrator
   • Behavior
   • Flow
   • Sensor
   • Condition
   • AvoidObstacle
   • ReturnBottleToBase
   • ExploreForward
   • TouchSensor
   • UltrasonicSensor
   • ColorSensor
   • Color
   • OperatorKind
   • ConditionContainer

LLMS-based evaluation

Secondly, we did some LLMS-based evaluation, mainly focusing on the model part. Generally speaking, we try to make LLM act as a domain expert to provide multidimensional evaluations of your language design.

Model Completeness

We give your ecore design to LLM and ask the LLM to guess which domain your language is designed for. And then we ask LLM to add possible missing elements to your language (perhaps not taken into account in the first version of the design, but can be considered for inclusion in future versions). The results are as follows:

Based on the provided information, this metamodel appears to be established for the field of robot programming, particularly for education or competitions such as Lego Mindstorms or VEX robotics. It covers the basic operations of robots (forward, backward, rotating, grasping, releasing, etc.), sensing (touch, ultrasound, color recognition), and decision-making (conditions, cycles). Here are some additional key elements and relationships for suggestions:

Element:

-Battery: level// battery level 
-Motor: speed; Direction// Motor, controlling wheels or other components
-Speaker: volume; Sound// Speaker, can be used for sound feedback
-LED: color; Status// LED lights for visual feedback
-Button: isPressed// Physical buttons that can be used as inputs
-GPS: Latitude; Longitude// GPS module for locating outdoor robots
-Gyroscope: x; Y; Z// Gyroscope, used for measuring and maintaining direction
-Accelerometer: x; Y; Z// Accelerometer, measuring acceleration
-Encoder: position// Encoder, used to measure the exact position of wheels or joints

Relationship:

-Motor (1)<- controls ->Block (1)//The motor controls a certain action
-Block (1)<- checks ->Sensor (1)//Check the status of the sensor before and after executing the action
-Procedure (1)<- commands ->Action (n)//The process contains multiple actions
-Main (1)<- uses ->Battery (1)//The main control components use batteries
-Action (1)<- triggers ->Speaker (1)//An action triggers a speaker
-Action (1)<- lights ->LED (1)//An action triggers an LED light
-Behavior (1)<- initiates ->Motor (n)//Behavior triggers motor action
-Behavior (1)<- reads ->GPS (1)//Behavior reads GPS data
-Behavior (1)<- utilities ->Gyroscope (1)//Behavior using gyroscope data
-Behavior (1)<- displays ->Accelerometer (1)//Behavior using accelerometer data
-Block (1)<- getsInputFrom ->Button (1)//A program segment obtains a button input
-Procedure (1)<- corporations ->Encoder (1)//The program uses encoder data

These additional elements and relationships further enrich the meta model, covering more aspects of robot programming, from simple action execution to complex environmental interactions and data processing.

We are not sure if these issues actually constitute a problem, the decision to fix them or not is still up to your team, looing forward to your respond, thanks!