Showing posts with label Eclipse. Show all posts
Showing posts with label Eclipse. Show all posts

Thursday, August 17, 2017

Two new Eclipse projects: Xpect and Xsemantics

Yesterday, two new Eclipse projects had been created: Both project are around for quite some time and they both are based on Eclipse Xtext, the famous framework to create editors with all state-of-the-art features (parser, linker, validators, content assist, etc.) for your own textual DSLs, simply based on a grammar.

Xpect is written by Moritz Eysholdt, who is also a committer of Xtext. It is a unit- and integration-testing framework to be used for Xtext-based languages. Instead of writing fiddly JUnit tests, you can simply write things like

// XPECT errors --> "cannot divide two strings"
"hello" / "world"
assuming your language supports division, strings, and types.

And types is the topic of Xsemantics, written by Lorenzo Bettini, author of the book "Implementing DSLs with Xtext and Xtend". It is a DSL (implemented in Xtext itself) for writing type systems, reduction rules, interpreters and general relation rules for languages implemented in Xtext. So you can write rules like that:

rule subtypeUnion_Left
 G|- UnionTypeExpression U <: TypeRef S
from {
 U.typeRefs.forall[T| G |- T <: S]
}
Well, you probably need to know type theory a little bit to see the beauty in that.

Both of these frameworks are heavily used by Eclipse N4JS: it contains more than 10.000 Xpect tests (e.g., all specification tests) and a Java 8 like type system defined in Xsemantics, and this is also why I act as kind of "assisting project lead" to help the original authors to bring these great tools to Eclipse. The authors and the N4JS team at enfore are now working on bringing the code to Eclipse as soon as possible, to simplify the usage of these frameworks for all Xtext users!

Keep on modelling!

I once said that modelling without graphical editors is like Tour de France without mountains. Well, I have to correct myself: a great test suite and a complicated type system are just as exciting ;-)

Thursday, November 3, 2016

BoF @ ECE2016: Eclipse and AsciiDoc

At NumberFour, we use AsciiDoc for our specifications and documentation. We are even working on producing AsciiDoc API documentation from JSDoc-like comments (similar to JavaDoc) for our JavaScript language N4JS. Since we ran into a couple of problems with AsciiDoctor, I thought that it might be interesting to learn what other people in the Eclipse Community think about AsciiDoc(tor) and if we might be able to "join forces" to overcome certain obstacles. So I organized a "Birds of Feather" about "Documentation with AsciiDoc and Eclipse" at EclipseCon 2016 Europe. We were about 10 participants and this blog post serves more or less as a kind of meeting minutes of this BoF combined with our findings at NumberFour.

AsciiDoc is a lightweight markup language, similar to Markdown. The original AsciiDoc was written in Python, but Asciidoctor, probably the most popular tool nowadays, is written in Ruby with a variant AsciidoctorJ which provides Java support (via JRuby) and Asciidoctor.js for JavaScript (via Opal). From an Asciidoc document (.adoc) you can either create HTML, PDF or even EPub. Unfortunately, the PDF support is lacking some major features (e.g., footnotes are not supported yet), but there is a common workaround to produce DocBook XML files from AsciiDoctor and then use FOP to eventually create a beautiful PDF. Lars Vogel wrote a nice tutorial about AsciiDoc and AsciiDoctor. One of the great things about AsciiDoc is that GitHub provides basic support for AsciiDoc as well!

Writing AsciiDoc

The first question is how to write AsciiDoc. One solution is to use stand-alone editors such as Sublime (there is an AsciiDoc plug-in for syntax highlighting and coce completion which, together with the preview plug-in, is a really great solution).

But usually, you do not want to switch editors when you are already using Eclipse. Fortunately, there's an existing AsciiDoc editor which is part of Mylyn Wiki Text. This editor has some syntax highlighting, an outline view and even a preview.

This editor seems to be a good starting point for further support of AsciiDoc on the Eclipse platform, although it might have some shortcomings with regard to features and design. Regardless, the BoF participants (including Torkild, a Mylyn Docs committer) decided to use the mylyn-docs developers mailing list for further communication about AsciiDoc and Eclipse.

Apparently, having a good editor, possibly even with WYSIWYG or at least WYSIWYM support is one of the most important things required.

Building AsciiDoc

There are several ways of building AsciiDoc. The most simple one is about using the AsciiDoctor tools directly from the command line. There is also a Maven plugin provided by the AsciiDoctor project.

We haven't discussed that topic any further though. It seems as if the existing tools are sufficient even though there are some shortcomings when using custom macros as described below. You may have a look at the N4JS documentation pom.xml to get an idea of how this could look like.

Converting to AsciiDoc

During the BoF, the question came up how to convert existing documentation written in other formats to Asciidoc. One solution for that is Pandoc, which we (at NumberFour) are using to convert our LaTeX specification to Asciidoc. Jeremie Bresson has already blogged about the BoF and even provided a solution for using Mylyn Wikitext to convert Eclipse MediaWiki to Asciidoc. Thank you very much, Jeremie!

Using AsciiDoc as a Single Source

I already mentioned above that you can generate HTML and PDF from AsciiDoc. However, HTML is not HTML. We use AsciiDoc to generate

  • the public web site (i.e., gh-pages)
  • the Eclipse help
  • the PDF specification

from a single asciidoc file. To give you an idea, have a look at an adoc source file and the generated web page. If you download the N4JS IDE, you can also have a look at the generated Eclipse help containing the same content. The PDF is not publicly available yet, we are currently working on migrating our language specification from LaTeX to AsciiDoc.

In order to generate Eclipse help, a table of contents file is required. At the moment, we generate them via the geneclipsetoc-maven-plugin Maven plugin, see our pom.xml for our configuration.

You may also use the Mylyn Docs to generate code, which is described in the Mylyn Docs Eclipse help. This also provides support for generating EPub, the first steps towards this support has already been described 2011 in Torkild's blog! It’s now a stable part of Mylyn Docs and has been for a few years. Currently it is in maintenance mode as there are few bugs and there is not much to add – apart for EPUB 3.0 support. You can find an EPUB examples (and presentations about Mylyn Docs in general) on Torkild's GitHub page.

Customizing AsciiDoc

Although AsciiDoc provides a lot of features ("macros" in Asciidoc terminology) for technical documentation, such as source blocks, warning blocks etc., we (at NumberFour) missed some functionality. We required additional support for

  1. definitions -- a typical feature required in specifications
  2. BibTex cites and bibliographies -- we already have a large BibTex database which we want to reuse
  3. todos -- well, also a typical feature required in specifications and documentations
  4. links to source code -- e.g., for adding links to either GitHub, or, in case of Eclipse help, to the files in the workspace
  5. larger documents, e.g., includes similar to LaTeX's chapter folders

One problem we found is that Ruby-based custom macros calling native code cannot be used with AscidoctorJ. E.g., there already exists a custom Ruby macro which provides support for BibTex, but this does not work with AscidoctorJ. We want to write the macros in Java, since we plan to provide some editor support in Eclipse as well -- and then Java would be the better solution. Thus we already wrote our own Java-based, BibTex macro (fixing some smaller AsciidoctorJ problems on the way) and we are also working on solutions for the other topics. If you are interested in that, or if you have written Java-based macros for AsciiDoc as well -- let me know! We haven't open sourced our custom macros yet since they are still under (heavy) development. But we will probably do that once they are in a state to make them public.

Edit: Updated description of EPUB support -- thank you Torkild for the information!

Thursday, July 21, 2016

From Xcore/ecore to OmniGraffle

Some years ago I wrote a small tool for creating OmniGraffle UML diagrams directly from Java source code. Visualizing Java is nice, but since I'm often use ecore/Xcore to define my models, I wanted a tool to also nicely visualize EMF based models.

I have now extended my tool, j2og, to also create UML class diagrams from ecore or Xcore models. Below you see a (manually layouted) version of an automatically generated diagram of the ecore library example.

j2og does not layout the diagram, since OmniGraffle provides some nice layout algorithms anyway. When creating the diagram, you can tweak the output with several settings. For example

  1. show or hide attribute and operation compartments
  2. show context, optionally grayed out -- the context are classifiers defined in an external package
  3. show package names, omit common package prefixes etc.
  4. and more

Note that besides OmniGraffle, you can open the diagrams with other tools (Diagrammix, Lucidchart) as well. See the j2og github page for details. You can install the tool via update site or Eclipse marketplace link.

The following image (click to enlarge) is the result of exporting a large Xcore model defining the AST of N4JS, a statically typed version of JavaScript. I have exported it and applied the hierarchy layout algorithm -- no other manual tweaks were applied. Of course, this diagram is probably too large to be really useable, but it is a great start to document (parts) of the model. Well, in case of an AST you probably prefer using an EBNF grammar ;-)

PS: Of course you could use ecoretools to create an UML diagram. I usually need the diagrams for documentation purposes. In that case, OmniGraffle simply is so much better since it is easier to use and the diagrams look so much nicer, (sorry, ecoretools).

Sunday, August 21, 2011

Java To OmniGraffle

If you ask developers using Mac OS X about their favorite diagramming tool, you will often get the same answer: OmniGraffle. I also like OmniGraffle very much, and I'm still wondering what makes this tool so much better then all the GEF based editors.

When I have to create diagrams for documenting some Java code, I used to manually draw an UML like class diagram with OmniGraffle. This is an error prone process, and a boring one as well. So, I tried to find a better solution. Since I didn't find any existing tool, I wrote a small Eclipse plugin my self. It automatically generates OmniGraffle class diagrams from existing Java code.

Its usage is very simple: Open or create a drawing in OmniGraffle. Then switch back to Eclipse and select "Create OmniGraffle Diagram" from the context menu of a package in the package explorer, as shown in Figure 1. Configure the output, as shown in Figure 2. The plugin will scan the package and add a class diagram of this package to the front most drawing opened with OmniGraffle. Figure 3 shows the result created by the plugin without any manual changes. It is a visualization of the package "ReverseLookup" of GEF3D.
Fig. 1: Context menu entry

Fig. 2: Configuration Dialog
Fig 3: Created class diagram
At this moment, the plugin can only create class diagrams for a single package. Update: The plugin can create class diagrams for selected types, packages, and sub packages. Besides, the context of selected classes, that is types on which the selected types depend on, can be visualized as well. Attributes are drawn as associations if possible, parameterized collection types are recognized and replaced by 0..* associations. You can configure the output with some switches. Besides filtering members based in their scope, I have added some "filters" I often use when I draw diagrams manually:
  • Getter and setters can be omitted
  • Methods implementing or overriding methods of interfaces or classes already shown in the diagram can be omitted as well
  • In order to better see relations between classes, you can force to draw all associations, even if they would be filtered out by the scope filter.
The newly created shapes are initially drawn using OmniGraffle's hierarchical layout algorithm.

Tip: In order to manually change the diagram, you may want to have a look at my collection of UML shapes at Graffletopia.

You can install the plugin via the update site:

http://jevopi.de/updatesite/de.jevopi.JavaToOmniGraffle
This is an beta alpha version, which will expire 2012. If you find this plugin useful, flattr me! Or leave me a comment below. Depending on the feedback, I will continue developing the plugin -- or not ;-)

In the preferences, you can set the default configuration settings and define the name of your OmniGraffle installation (however, the plugin tries to find the latest installed version automatically).

Last but not least: Of course, this plug is only available on Mac OS X, since OmniGraffle is a native Mac application. The communication between Eclipse and OmniGraffle is done via AppleScript, which is very easy thanks to Peter Friese's blog post.

(At Stackoverflow, someone estimated a tool for creating OmniGraffle diagrams from Eclipse UML2 based models would require 18 months development effort. Well, I needed less then 18 hours. But I only convert Java packages to class diagrams... ;-) ).

Update 2011-11-01:
  1. Besides packages, selected types and sub packages can be visualized.
  2. The context of visualized types can be visualized. That is, types on which the selected types depend on, such as super classes, can be rendered additionally to the selected classes. These context types are rendered in gray.
  3. The default package is now handled as well (see comment by mathpup).

Thursday, June 9, 2011

When your MWE2 workflow is not working...

MWE2 is a kind of Ant for model related tasks. Xtext is using MWE2, and I also use it to run my own Xpand generator templates. It's a nice tiny tool, and one of the nice things is that it runs in its own JVM, so you can easily extend MWE2 with new components which resides in your project. When MWE2 is started, the project settings, i.e. the classpath (including all information from the plugin dependencies), are passed to the workflow. Unfortunately, this may produce problems which are really hard to find, mostly because the error message do not really tell you where to find the actual problem. Here is a list of some problems I have run into several times (using Eclipse 3.6 and Xtext/Xpand 1.0.1). Note that all the problems mentioned and fixed below may be caused by other reasons, requiring other fixes.

Problems instantiating module

Error message:
Error message in console:
1    [main] ERROR mf.mwe2.launch.runtime.Mwe2Launcher  - Problems instantiating module ...
...
Caused by: org.eclipse.emf.mwe.core.ConfigurationException: The platformUri location '......' does not exist 
Possible fix:
Fix projectName in MWE2 file.
I run into this problem after renaming a project. Ensure the project name defined in your MWE2 file
var projectName = ".."
matches the actual project name. This line is present in Xtext related MWE2 files.

Couldn't find module with name

Error message:
Error message in console:
ERROR mf.mwe2.launch.runtime.Mwe2Launcher  - Couldn't find module with name ...
Possible fix:
Create missing src-gen folder.
I run into this problem after checking out a project from a code repository. Since we do not add generated code to the repository, the src-gen folder was not added to the repository. Hence, it was not present after having checked out the project. However, it is configured in the build.properties. This seems to lead to a problem when the classpath is computed, so that the src folder is not added to the classpath either. Since the MWE2 file resides in the src folder, is is not found and, consequently, the module is not found. I was able to fix this problem by simply creating the src-gen folder. In order to not cause this problem again, I have added the src-gen folder to the repository and ignore only its content.

Workflow definition is ignored

Error message:
None. However, the selected workflow is completely ignored. It seems as if another workflow is executed.
Possible fix:
Ensure module name of workflow, that is the first line in MWE2 file
module ..
is unique.
This seems to be a typical copy- and paste error. Actually, in deed another workflow is executed. Although one can run an MWE2 file via "Run as../MWE2 workflow", the launcher does not directly call the actual workflow file. Instead, the name of the module is read and then the internal representation of this module is been executed. If you define two of more workflows with the same module name, only one of these modules is actually present (there seems to be some kind of map from module name to module).

Couldn't resolve reference to JvmType 'Workflow'.

Error message:
Error in MWE2 workflow file:
Couldn't resolve reference to JvmType 'Workflow'.
When you try to run the workflow, the following message appears:
Please put bundle 'org.eclipse.mwe2.launch' on your project's classpath.
Possible fix:
Ensure Plug-in Dependencies are correctly added to classpath.
This error can be caused by at least one of the following problems:
  • your project is not an OSGi/Plug-in project. This can be fixed by converting the (Java) project to a Plug-in project.
  • as printed in the dialog, ensure 'org.eclipse.mwe2.launch' to be listed in the plug-in dependencies
While these are obvious reasons for the error, in my case I had configured the project as plug-in project and 'org.eclipse.mwe2.launch' was defined in the dependency list. However, it still didn't worked. This may be caused because I renamed the project, and probably something had gone wrong. I noticed the problem only because the Package Explorer didn't showed the entry "Plug-In Dependencies". I assume there may be some corrupt cache entries. I only was able to fix this problem by creating a new plug-in project and moving the content of the broken project into the new one.

Weird errors when generating the parser etc.

Error message:
When running the Xtext MWE2 workflow to generate the code from your grammar, weird errors occur indicating problems in your grammar.
Possible fix:
Increase memory in MWE2 runtime configuration.
I stumbled over this problem twice, and it is really annoying since it is very hard to find the reason. If the parser generator runs out of memory, it crashes at some arbitrary position and randomly creates error messages. Since it usually happens in the parser generator, the error messages always indicate problems in your grammar, although your grammar may be perfectly ok. So, if you got any grammar problems which are not that obvious real grammar problems, ensure to set VM argument "-Xmx1024m" in the MWE2 runtime configuration.

Disclaimer: This is more a personal note, and some problems may be fixed in the meantime. Feel free to tell me if I got something wrong here :-)

Friday, January 21, 2011

Clickable logging messages

Probably everyone is using logging. The dirty way is to use System.out/err, a better solution is to use the JDK logging or some logging library, such as log4j (or a facade, e.g., slf4j) -- and of course Log4E to generate the logger declaration and other logging code . For rich clients and small tools, I usually use the JDK logging, as I do not have to add another library and the logging is only used for development purposes. For web (or other server) applications, in which logging is an important monitoring tool, other log libraries may be a better choice. Anyway, when using the JDK logging facilities, the log message is written to the Eclipse console view by default. A simple log message usually looks like that:
Jan 14, 2011 4:36:54 PM my.project.MyClass bar
INFO: Demo
This is ok, however I usually do not need the date and time. Also, IMHO two lines for a single message waste too much of my console view space. I have written a small formatter, which produces the following output (scroll to the right to see the whole ouput):
I Demo                     at my.project.MyClass.bar(MyClass.java:88)
That is, the date and time is omitted (as this usually is not needed for development purposes), and instead of printing the full logging level, abbreviations are used (I - info,W - Warning, S - Severe and so on). In case of shorter messages, the location information is printed on the same line with some tabs between message and location for better readability. The best thing about that format is, that the Eclipse console recognizes the location and makes it available as a link, which directly opens the location in the source code where the log message was produced. If you configure the console preferences to use gray for standard error text (as which the log message is interpreted due to the location format), you will something like that:
W Test Warning             at my.project.LoggingTest.main(LoggingTest.java:32)
I Test Info                at my.project.LoggingTest.main(LoggingTest.java:36)
Actually, I'm using this formatter for quite some years now. Today I have created an eclipselabs.org project called delofo (short for Developer Logging Formatter) with this formatter. I've chosen eclipselabs.org because the formatter is optimized for the Eclipse console view, although the project does not provide a plugin and can be used w/o Eclipse. In the project wiki, you will find an installation guide, explaining how to globally install the formatter. By installing the formatter globally, you do not have to modify your projects in anyway.

Friday, March 19, 2010

Quick test: GEF3D and e4

Today I received an email from someone asking whether GEF3D will work with e4. Since GEF3D is "only" an extension of GEF (and GMF), I assumed it will work. In order to be sure, I tried it myself. This is my installation log: Step 1) Downloaded the e4 0.9 release (http://download.eclipse.org/e4/downloads/drops/R-0.9-200907291930/index.html#EclipseE4), in my case the Mac OS X Cocoa version Step 2) Installed the following features via the given update sites: Step 3) Imported GEF3D team project set found at http://eclipse.org/gef3d/resources/GEF3D.psf State: The 3D graph example is working, see screenshot.
GEF3D on e4! (Note the nice vector based fonts :-D Kudos to Kristian!)
Step 4) Now installed the missing features in order to test all examples: Restarted e4 after downloading these bundles. Ups, what's that? My toolbar has gone...? Where is it? However, I doubt GEF3D or GMF to be involved in that bug... Hmm... was there a toolbar before step 4)? Weird. UML2 Tools and Ecore Tools are working, at least the 2D versions. Unfortunately, the 3D versions are not working. There are not much error messages:
java.lang.NullPointerException
at org.eclipse.e4.extensions.LegacyPartRenderer.createEditor(LegacyPartRenderer.java:95)
at org.eclipse.e4.extensions.LegacyPartRenderer.createWidget(LegacyPartRenderer.java:357)
...
I didn't analyzed that error... without a toolbar, it's not very comfortable. Besides, I didn't found the code of LegacyPartRenderer and I didn't find a way to declare an exception breakpoint. Summary: Since the 3D graph example is working, I assume GEF3D and e4 should work in general (e4 does change the workbench, but not SWT or GEF). However, the 3D-fied UML2 and Ecore editors are not working, but I have no idea why they don't. Maybe it's not e4 but a problem with the latest GMF or ecore tools versions, I don't know. Frankly, I have ignored e4 for the time being (simply because I haven't had the time). IMHO you cannot simply install existing 3.x plugins and expect them to work with a 0.9 incubator framework. In so far I'm happy about the 3D graph example running! Besides, I assume my test only tested the legacy layer of e4, and not e4 itself. However, I'm still wondering why the 2D version of the ecore diagram editor is working and why the 3D version isn't. I'd assume there exist documents in the e4 project about how to solve problems like this.

Tuesday, January 19, 2010

Quick'n Dirty Tutorial on Modelling with Eclipse

A friend of mine asked me how to modelling with Eclipse. I gave him a quick tour, demonstrating how to create a model with EMF, how to create an instance of that model, how to query the model with OCL, and how to create a text editor for that model with Xtext. It's pure modelling, i.e. no Java programming at all! I thought that maybe other people would be interested in that topic as well, so I created this little tutorial showing how all these Eclipse modelling "ingredients" can work together.

Preparations

Install Eclipse 3.5 (Galileo) with all the modelling tools, that is the Galileo Eclipse Modeling Tools package. Additionally, we will need the OCL interpreter console, which can be installed using the MDT update site:
http://download.eclipse.org/modeling/mdt/updates/releases/
The OCL console actually is an example, so you will have to install that specific example.

The Runtime Example

Let's assume a small IT company. The employees of that company develop software for other companies. Of course, they use models for that purpose! In order to better organize their work, they want to document who is working on what. That is, they want to assign certain task to developers. A task could be the implementation of a use case or writing a test case for a class. Let's see if we can use models for that...

Company Model with EMF

First of all, we have to create a model describing the company. Let's start from scratch:
  1. create a new project:
    File / New / Project... / Eclipse Modeling Framework /Empty EMF project
  2. create a new ecore diagram (we want to graphically design the model) in the newly created model folder: Right click on model folder and select
    New / Other... / Ecore Tools / Ecore Diagram
  3. enter the Domain file name:: company.ecore and press Finish.
Now, we can "draw" our model using the graphical editor. Create four classes (EClass), add a name attribute (EAttribute) to one of them, create references (EReference) between the classes, and set the properties (in the properties view, this view can be activated by right-clicking in the diagram and select Show Properties View. The result should look like Figure 1.
Figure 1: The domain model of a company
Some remarks:
  • The model is an instance of the ecore model, which is quite similar to class models in UML. All elements in ecore start with an "E", so it is EClass, EAttribute, or EString.
  • The EType of the attribute name is EString.
  • The upper bound of all references is "*" (you can enter "-1", which actually is the same as "*").
  • You will need a class containing all your elements later. In the example, the Company serves as a container. Make sure the Is Containment flag is set for the two references employees and skills (see Fig. 1)
Now that we have created a model of our company, we can create an instance of that model. Later, we will generate an editor, but right now we want to quickly create an instance to get a feeling for our model. An instance of our model actually is an instance of our container element, that is the Company class. There is a generic editor available, which can create an instance of a model (or its elements) directly without the need of code generation. All you need is the ecore model. This is how to activate it:
  1. Close the ecore diagram
  2. Open the ecore model, this time use the Sample Ecore Model Editor. Usually, this editor is used if you double-click the ecore-file, but to be sure use the Open With... context menu entry.
  3. Select the Company class (the class, not the package!), and select Create Dynamic Instance... from its context menu. This is shown in Figure 2.
    Figure 2: Create a dynamic instance.
  4. enter the File name: Company.xmi and press Finish. The instance will be created in the model folder.
  5. edit the model instance using the context menus New Child of the elements in the model.
  6. edit the elements properties in the properties section of the editor, see Figure 3.
    Figure 3: Edit properties with the Generic EMF Form Editor
Now that we have create a model and an instance of that model, we want to "work" with that model instance. E.g., we can use OCL to query the model:
  1. Activate the console view and open the Interactive OCL console with the button on the left of the console, see Figure 4.
    Figure 4: Open OCL interpreter console
  2. Select an element in the editor, This selected element is the context of the OCL query.
  3. Enter your OCL query in the console (in the lower section of the OCL interpreter console). E.g. query the name of the selected element with self.name. The result will be displayed in the upper section of the OCL console as shown in Figure 5. We will demonstrate other queries later on.
    Figure 5: Query the model instance with OCL

Task Model with Xtext

Now that we have a model of our company, we want to assign tasks to the employees. We could create a new ecore model just as demonstrated above, but we want to try something new. So, let's try to not only create a model, but a text editor as well. For that, we will use Xtext. Based on a grammar, Xtext can create an ecore model and a text editor with nice features. The grammar is an annotated EBNF grammar, I do not want to go into the details here (otherwise it wouldn't be a q'n d-tutorial ;-) ). So, we have to create a new project and enter a grammar:
  1. Create a new Xtext project:
    File / New / Project... / Xtext /Xtext Project
  2. SetMain project name, the Language name (to de.feu.Tasks), and the DSL-File extension (to tasks).
  3. Open Tasks.xtext (this is the EBNF-like grammar) and enter your grammar according to Figure 6.
    Figure 6: Xtext grammar defining our task model along with a concrete textual syntax
    Here is the grammer (for copy & paste):
    grammar de.feu.Tasks with org.eclipse.xtext.common.Terminals
    
    import "http://www.eclipse.org/emf/2002/Ecore" as ecore
    import "platform:/resource/de.feu.company/model/company.ecore" as company
    
    generate tasks "http://www.feu.de/Tasks"
    
    Tasks :
     (developers+=Developers)*
     (domainmodels+=DomainModels)*
     (tasks+=Task)*;
     
    Developers :
     'developer' importURI=STRING;
     
    DomainModels :
     'domainmodel' importURI=STRING;
    
    
    Task: 'task' element=[ecore::EObject] 
       'by' developer=[company::Developer] 
       ':' description=STRING;
    
    This grammar defines a container element Tasks. Inside, we can "load" a list of developers and domain models. Eventually, tasks can be defined by assigning an element from a domain model (see element=[ecore::EObject]) to a developer (developer=[company::Developer]) and add a description of that task.
  4. Save the gammar and generate the ecore model along with the text editor using the MWE-workflow. For that, select file GenerateTasks.mwe and run the workflow via its context menu Run As / MWE Workflow
Notes:
  • In Xtext, a model is called a DSL. Experts like to use different names for "model", sometimes it's cooler to call a model "meta-model" (or, even cooler, "meta-meta-model"), sometimes they call it "domain specific language", abbreviated with a nice TLA (three letter acronym): DSL. Of course, there are good (and sometimes not so good) reasons for doing so, but usually it's easier to simply call a model a model (and an "instance of a model" an "instance of a model") .
  • The grammar shown in Figure 6 actually uses a lot of nice Xtext features and you will have to read the Xtext documentation for details. I only want to explain one thing here, which is a little bit advanced. You can import existing models into the grammar (import ...). In the example, we import the ecore model and our previously created company model. We import these models in order to be able to define inter-model references later on. Our Task element will refer to an element, which can be any ecore::EObject, and the developer is to one developer defined in our company model (company::Developer). EMF supports inter-model-references, and Xtext generated editors support that feature as well! The import statements in our grammar only import the models, but later on we want to actually import existing model instances. For that, we need the importURI feature of Xtext to define what instance to import in our actual task model instance.
Although we haven't written a single line of Java code yet, a lot of code has been generated automatically. In order to "activate" that code, which actually defines a new Eclispe plugin, we have to start a new runtime instance of Eclipse, that is we start Eclipse from within Eclipse. In the long run, you will have to get used to Eclipse plugin development anyway... As we have to run a new instance anyway, we can generate an editor for our company model as well. Instead of using a dynamically created instance using the "Generic EMF Form Editor", EMF can generate a Java implementation of the model and an editor as well. We will do that now:
  1. Select the company.ecore file and choose New / Other... / Eclipse Modeling Framework / EMF Generator Model from its context menu. Use the suggested file name, select Ecore Model in the next wizard page. Then, press Load in the next page, and eventually Finish on the last wizard page. This creates a generator model, which basically adds some information to the original ecore model which is necessary in order to actually generate code (e.g., the name of the plugins and so on). We do not want to change anything here, but we still need that model.
  2. In that model, select the very first element (Company) and select Generate All from its context menu (see Figure 7).
    Figure 7 Generate the model and editor code

Work with Multiple Models

Now that we have generate a lot of code, we want to use the newly created tools. Start a new Eclipse runtime instance (e.g. by selecting Run As / Eclipse Application from a project context menu) and do the following:
  1. Create a new project (in the runtime instance):
    File / New / Project... / General /Project
  2. Select the project and choose from its context menu:
    New / Other... / Example EMF Model Creation Wizard / Company Model
    This activates our previously generated editor (and a wizard) for creating a company model instance. We can edit a company instance just as we did it with the generic editor. In the wizard, select the Model Object Company and then create a new company as shown in Figure 8. (Don't forget to save the model ;-) ).
    Figure 8: A company model, edited with the generated editor
  3. Now, create a new file (File / New / File) inside the project and call it project.tasks. The Xtext generated editor is opened automatically and we can now edit the task model instance.
  4. In order to "simulate" a project, we create a simple UML use case model. Simply create a new use case diagram via File / New / Other... / UML 2.1 Diagrams / Use Case Diagram, give it a name (e.g. project_usecase) and draw some use cases, a sample is shown in Figure 9.
    Figure 9: A sample use case diagram
  5. Switch back to project.tasks and "import" our company model and the sample use cases. Add a task using content assist (Ctrl-Space) just as shown in Figure 10. Just play around, add three or four tasks in order to be able to follow the next steps.
    Figure 10: Content assist, demonstrating access to imported models
  6. Just as at the beginning, we want to execute some OCL queries on our model, but this time on our project.task model instance. For OCL, we always need a context, but unfortunately we cannot select a context (or model element) in the text editor. So we have to reopen the project.task with the generic EMF editor: Choose from its context menu Open With / Other... / Generic EMF Form Editor. You will now see the very same model as in the text editor, but this time you see a tree-based version of the model.
  7. Open the OCL console just as above, select an element (here Tasks) and enter a query. For example, we want to know how much tasks are assigned to a specific developer: self.tasks->select(developer.name='Jens')->size(). You can see that in Figure 11
    Figure 11 A little more sensible OCL query

Conclusion

IMHO it is impressing what you can do with all these cool Eclipse modelling tools, without writing a single line of (Java) code. We saw how to
  • create an EMF ecore model with the Ecore Tools diagram editor
  • open the same ecore model with the Generic EMF Form Editor
  • create instances of a model without the need to generate code and without starting a new runtime instance, using the "Create Dynamic Instance" feature (and the generic EMF editor again)
  • generate a text editor and a model without a single line of Java code with Xtext
  • generate a tree based editor and a Java based model implementation with EMF
  • create a UML diagram with the UML Tools diagram editor
  • query your models with OCL from the OCL project
We didn't bothered about how to store our models or model intances (it's all XMI), we were using OMG standards like UML (the EMF based implementation is provided by the UML2 project, EMF's ecore is an EMOF-like model, we were using OCL, other implementations are available, too (e.g., BPMN2, SBVR, or SPEM used by EPF). And if you do not find an Eclipse project, you probably will find a third party project providing an EMF based implementation ;-) With tools like GEF or GMF you can create (or generate) diagram editors for your models, and (well, I couldn't resist) with GEF3D you can even create 3D diagram editors, e.g. for visualizing inter-model connections. And there are many more tools out there, partially for simplifying the use of existing tools, for model transformations and code generation and so on. And of course you can adapt and modify the code generated above to suit your need.

Warning: Thin Ice!

While it is very easy to do impressive things (with the help of a tutorial), it is very hard to get into all these frameworks and tools. There are a lot of traps hidden everywhere! Just two examples from the tutorial:
  • In the Xtext grammar we used here, the ecore model was imported (see Fig. 6). I tried the very same using the UML2 model, and I got a weird error when generating the code (actually I couldn't generate in that case).
  • A NullPointer-Exception is thrown (you can see that in the console of the original Eclipse instance) when opening the project.tasks file with the Generic EMF Form Editor. Fortunately it still is possible to select an element (for the OCL query), but it's a little bit weird.
Well, I probably should file a bug report at least for the first problem... So be warned: Even if the tutorial gives you the impression as if modelling with Eclipse tools is very easy, it sometimes isn't. In general, modifying generated code as well as using tools like GMF is not that simple. But at least there are the newgroups, tutorials, and even books (e.g. the EMF book (2nd edition!) or the GMF book).

Disclaimer

This is a quick and dirty tutorial article. I simply documented an example modelling session by taking some screenshots, and then wrote some explanations. So, there probably are some things I've missed to tell you or errors in my text. Please leave me a comment if you find a bug here ;-) (or if you like the tutorial). May the Model be with you!

Tuesday, November 24, 2009

Retrospect: DemoCamp in Berlin

Yesterday I attended the Eclipse DemoCamp in Berlin. It took place at Fraunhofer Fokus, and (as the last democamps) it was organized by Tom Ritter (thank you, Tom!). This time, there were ten (10!) presentations, seems as if we should organize a DemoDay next time. The camp started with a nice welcome talk by Ralph Müller, encouraging the audience to become Eclipse members (I will talk with my employer about that, promised ;-) ) and visit Eclipse Summit Europe (I won't miss it). Kristian Duske (committer on the GEF3D project) presented a "GEF3D Based Editor for the GMF Mapping Model", which was the subject of his diploma thesis. His editor enables the creation and editing of the GMF mapping model using simple drag-and-drop mouse operations. In contrast to the tree-based GMF editors, all involved models are visualized in a graphical manner, and since all models are visualized in a single 3D scene, inter-model-connections are visible, too. I really like the idea and his presentation, but since I was his tutor, I'm not really an impartial observer ;-)
Fig 1: Kristian's 3D GMF Mapping Editor
Martin Esser then demonstrated a small editor he has created with GMF (unfortunately without Kristian's 3D editor ;-) ) for variability models (aka feature diagrams). The challenge is to implement these arcs connecting edges of optional or alternative features. I usually draw these diagrams using OmniGraffle, and although there is a stencil for these diagrams available, it is painful to adjust these arcs -- a GMF based editor would be a great help. Maybe Kang et al designed these diagrams in order to test the abilities of programmers of graphical editors... (BTW, I just stumbled over this Eclipse project proposal, they want to provide a graphical editor as well) It seems as if there is an Xtext presentation at every democamp ;-), and of course there was a talk about it in Berlin, too. Peter Friese gave a short overview of this really nice tool for creating editors (and more) for textual DSLs. I have attend a couple of Xtext presentations (e.g. at ESE 2008 and 2009), and although all these presentations used different slides and were presented by different persons, the slides are always magnificent (with great photos and nicely illustrated). Maybe they have an Xtext based tool for generating these slides... hmm... Ralph Müller mentioned something about Ytext in his keynote, maybe this is a new tool for generating presentations? After a short break (with Eclipse sponsored sandwiches and soft drinks ;-)), Marcus Engelhardt demonstrated a tool called Metrino for measuring models. Metrics can be defined using OCL for UML or Ecore based DSLs, and the tool creates nice reports and radar charts. IMHO model metrics are an important tool for modelers, and I always have the plan to create tool with GEF3D in order to display the metrics on top of the diagrams (using the city metaphor) as illustrated in Figure 2... Marcus, if you'd like to combine that with Metrino, I would be happy to assist you!
Fig. 2: Visualization of metrics with GEF3D
Joachim Hänsel and Jaroslav Svacina introduced their tool EvoTest for creating tests automatically with evolutionary techniques. I know that tests are very important, but (shame on me) I'm neither an expert for testing nor for evolutionary techniques. However, their tool looks very interesting, and I was impressed by the ability of that tool to generate white box tests considering path coverage. Next, Martin Köster talked (slides are available here) about how he created an Eclipse based IDE for the Clojure language, a Lisp (that language driving you crazy with brakets) dialect for the JVM. It was very interesting to learn how he used the DLTK in combination with his own ANTLR parser. There seem to be a lot of issues addressed by Xtext and DLTK. I wonder if it would make sense to integrate these two tools. Stephan Herrmann then presented a new episode of the famous Object Teams series. After a short "previously on OT" flash back (summarizing OT as a great tool for reusing existing components in a flexible (via AOP techniques) yet controlled (via modules and a role-team-concept) manner), the story continued with a special and important problem: persistence. He explained how it is possible to use OT in combination with O/R-mappers -- with minimal effort by using OT techniques. Actually, this topic was subject of a diploma thesis (in german language) by Olaf Otto. OT is a proposed Eclipse project, and I hope it will become a real Eclipse project, soon. Igor Novakovic demonstrated a powerful tool called SMILA. It is an Eclipse project for setting up tool chains for querying, processing and extracting data. In his demo, he showed how to combine some cool Fraunhofer tools for image recognition with a search engine (Lucene) (probably other tools were involved as well). The nice thing about SMILA is that in the end all these tools are combined in a way that some data can be queried in a very simple manner (e.g. HTML formular). In the demo, some chemical structures could be searched in a collection of research papers. Simply querying the text content wouldn't be that thrilling, but in that case, a tool analyzed the figures in the papers and presented the molecules using Jmol. Some years ago I integrated a search engine (Verity Information Server) with a CMS based on StoryServer, and a tool like SMILA would have been a great help (I assume that with SMILA I would have spent days instead of months to accomplish that task...). Finally, Enrico Schnepel presented a small framework called emf.observables for simplifying EMF databinding. His cool (Prezi based) slides are available, too. Actually it is a small yet helpful tool, generating some plugins next to your EMF model implementation with some wrapper classes hiding the complexity of EMFObservables/IObservable. It was a really interesting and inspiring evening: A big thank you to all the presenters. See you all again at the next Berlin Eclipse DemoCamp (or Day ;-)).

Monday, November 24, 2008

Eclipse Summit Europe 2008, Retrospect

This was my first Eclipse event so I was very excited. Unfortunately, my proposal about GEF3D was rejected. But anyway, it was a lot of fun! Here are some thoughts about some talks. Quentin Glineur (Obeo) talked about model-to-model transformations with ATL and QVT. Quentin listed several use cases for m2m-transformations: validation, refactoring, reexpression and consistency checking. He is currently implementing an engine for Declarative QVT based on the ATL virtual machine. Later I had a long discussion with him about model transformations. I'm really exited about his QVT implementation, which will be available at Eclipse.org. Jan Köhnlein, Peter Friese, and Sven Efftige (itemis) presented xText. Since I'm fighting (but don't get me wrong: xText is a super cool tool) with xText for a couple of weeks to get my own transformation language working, I was happy to hear thet the upcoming version of xText will solve a lot of problems, such as easier implementation of expressions or mixing of languages and grammars (I could use that for including OCL). I had lunch with Sven and it seems as if every little problem (or bug) will be solved in the next version of xText, it's a pitty that I have to finish my project before it will be released (feature freeze will be in March 2009, it will be part of Galileo). Since I have heard so much about Jazz I was especially interested in the talk by André Weinand (IBM). For me Jazz looks pretty much like a tool set for Scrum, but this could be a misinterpretation by a Scrum master (like the guy with the hammer for whom anything looks like a nail). I'm still a little bit confused whether Jazz is freely available or not. But it looks definitely impressing and I'd like to work with it. For example, it introduces certain "isolation levels", for example a (personal) repository workspace. That is each developer has his own individual repository for backup and so on, and there will be a separated project repository. Thinking about that I'm wondering how I will work with my code when it has been moved to the Eclipse repository... A while ago I posted a master thesis proposal about visualizing model differences using EMF Compare and GEF3D. Unfortunately, no student has shown interest in this thing. Nevertheless it was very interesting listening to Cédric Brun (Obeo) about EMF Compare. While seeing his slides I was thinking about new cool visualizations, for example an Apple Time Machine like view of different model versions in a repository. One of the highlights was Ed Merks' talk about "The Unbearable Stupidity of Modeling". While I agree with most of his statements, for example the exaggerated use of the prefix "meta", I noticed (again) this little difference between the american and european conception of "model" and related terms such as "abstraction". I have to work out that thing in my thesis ;-). Maybe it's because the great book on model theory by Herber Stachowiak was never translated into English... For people using the word "meta" too much, and thus maybe suffer under some kind of modeling fever (my adviser once diagnosed me with "Modellithis"), I can recommend the very funny article "UML Fever" by Alex E. Bell, or (in severe cases) you may consider joining the Meta-Modellers Anonymous. Or simply read Ed's blog regularly. Scott Lewis and Marcelo Mayworm's talk about ECF (the Eclipse Communication Framework) was quite nice, they showed a small video about two pair programmers editing a single piece of source code using ECF. It's very similar to my beloved editor SubEthaEdit. I'm wondering whether it would be possible to integrate ECD ad GEF3D somehow? David Sciamma (Anyware Technologies) presented the new and upcoming features of the Ecore tools, that is the Ecore diagram editor. Version 0.9 will be part of Galileo. Kristian Duske is currently working on a 3D port of these tools, so it would be nice to keep contact with the 2D developers. To be honest I didn't knew David is also developing these Ecore tools, I "only" knew him as the author of the TopCased UML editor. So it would be the second work of David which will be ported to 3D, since his TopCased UML editor is the one I'm using in all my transformation chain examples (thinking about that I'm reminded to update my 3D version to the latest TopCased UML 2 version, I hope that will be as quickly done as the previous version, that is within two days ;-) ). Having all that trouble with licenses and third party libraries in the context of the review of LWJGL and JOGL (and even my initial GEF3D contribution), I was very curious aboutJanet Campbell's talk about "IP for Eclipse Committers". IP is short for "intellectual property" and this talk was especially about the "due diligence". That is (for people not that familiar with the Eclipse Process) there are a bunch of rules to be followed before code can be committed to Eclipse (i.e. some Eclipse CVS or SVN). IMHO this is an outstanding quality of Eclipse to be very concerned about these issues, even if I have a lot of trouble because of that (LWJGL was not approved, and JOGL is only to be approved because SGI has changed its free license agreements two months ago). So, a big thank you to all that people doing these legal reviews and stuff! It was a really great experience to meet all that people I knew only from their email addresses, and especially people I was mailing with so much for the last couple of weeks because of GEF3D: Chris Aniszczyk, Ed Merks and Anne Jacko. It's a really great community, and I'm happy to be a part of it :-)

Wednesday, November 5, 2008

The Eclipse gef3d project is complete!

Three and a half months ago Chris Aniszczyk asked me if I would like to make GEF3D an Eclipse project. At that time I had released a first version of GEF3D under the EPL. I started GEF3D as part of my Ph.D. project (which still is an ongoing project), but I thought that others may be interested in that framework, so I released it. I didn't knew what I did when I agreed on moving GEF3D to Eclipse. Oh guys, this paperwork and this whole Eclipse process is overwhelming. Fortunately I have two great mentors, Chris Aniszczyk and Ed Merks, helping me through all this just like Virgil guided Dante through all the nine circles of Hell. Thank you, guys! After writing the proposal, creating the creation review slides, filling out several requests and other paperwork, I received a mail today with this subject: "The Eclipse gef3d project is complete!". That's cool! So, in the near future you will find GEF3D at http://www.eclipse.org/gef3d. It will need some time to find code there, because the code has to get clearance first. So long, you will find GEF3D at it's old location at http://gef3d.org. Stay tuned!
PS: I thought I've read about Dante in Ed's Blog some time ago, I vaguely remember a slide on which he compared meta models (and meta meta models) with the circles of Hell, but I cannot find it anymore (I also remember his advice not to go there...). This was when Dante's trip to Hell came back into my mind. Actually I've never read the Divine Comedy. My girl friend once wrote a paper about Botticelli's Dante illustrations, and she made me love this special "comic strip" from the 15th century (and she explained me how to "read" it). I even had the luck to see the original drawings in Berlin once).