Thursday, August 17, 2017

Two new Eclipse projects: Xpect and Xsemantics

Yesterday, two new Eclipse projects had been created: Both project are around for quite some time and they both are based on Eclipse Xtext, the famous framework to create editors with all state-of-the-art features (parser, linker, validators, content assist, etc.) for your own textual DSLs, simply based on a grammar.

Xpect is written by Moritz Eysholdt, who is also a committer of Xtext. It is a unit- and integration-testing framework to be used for Xtext-based languages. Instead of writing fiddly JUnit tests, you can simply write things like

// XPECT errors --> "cannot divide two strings"
"hello" / "world"
assuming your language supports division, strings, and types.

And types is the topic of Xsemantics, written by Lorenzo Bettini, author of the book "Implementing DSLs with Xtext and Xtend". It is a DSL (implemented in Xtext itself) for writing type systems, reduction rules, interpreters and general relation rules for languages implemented in Xtext. So you can write rules like that:

rule subtypeUnion_Left
 G|- UnionTypeExpression U <: TypeRef S
from {
 U.typeRefs.forall[T| G |- T <: S]
Well, you probably need to know type theory a little bit to see the beauty in that.

Both of these frameworks are heavily used by Eclipse N4JS: it contains more than 10.000 Xpect tests (e.g., all specification tests) and a Java 8 like type system defined in Xsemantics, and this is also why I act as kind of "assisting project lead" to help the original authors to bring these great tools to Eclipse. The authors and the N4JS team at enfore are now working on bringing the code to Eclipse as soon as possible, to simplify the usage of these frameworks for all Xtext users!

Keep on modelling!

I once said that modelling without graphical editors is like Tour de France without mountains. Well, I have to correct myself: a great test suite and a complicated type system are just as exciting ;-)

Thursday, November 3, 2016

BoF @ ECE2016: Eclipse and AsciiDoc

At NumberFour, we use AsciiDoc for our specifications and documentation. We are even working on producing AsciiDoc API documentation from JSDoc-like comments (similar to JavaDoc) for our JavaScript language N4JS. Since we ran into a couple of problems with AsciiDoctor, I thought that it might be interesting to learn what other people in the Eclipse Community think about AsciiDoc(tor) and if we might be able to "join forces" to overcome certain obstacles. So I organized a "Birds of Feather" about "Documentation with AsciiDoc and Eclipse" at EclipseCon 2016 Europe. We were about 10 participants and this blog post serves more or less as a kind of meeting minutes of this BoF combined with our findings at NumberFour.

AsciiDoc is a lightweight markup language, similar to Markdown. The original AsciiDoc was written in Python, but Asciidoctor, probably the most popular tool nowadays, is written in Ruby with a variant AsciidoctorJ which provides Java support (via JRuby) and Asciidoctor.js for JavaScript (via Opal). From an Asciidoc document (.adoc) you can either create HTML, PDF or even EPub. Unfortunately, the PDF support is lacking some major features (e.g., footnotes are not supported yet), but there is a common workaround to produce DocBook XML files from AsciiDoctor and then use FOP to eventually create a beautiful PDF. Lars Vogel wrote a nice tutorial about AsciiDoc and AsciiDoctor. One of the great things about AsciiDoc is that GitHub provides basic support for AsciiDoc as well!

Writing AsciiDoc

The first question is how to write AsciiDoc. One solution is to use stand-alone editors such as Sublime (there is an AsciiDoc plug-in for syntax highlighting and coce completion which, together with the preview plug-in, is a really great solution).

But usually, you do not want to switch editors when you are already using Eclipse. Fortunately, there's an existing AsciiDoc editor which is part of Mylyn Wiki Text. This editor has some syntax highlighting, an outline view and even a preview.

This editor seems to be a good starting point for further support of AsciiDoc on the Eclipse platform, although it might have some shortcomings with regard to features and design. Regardless, the BoF participants (including Torkild, a Mylyn Docs committer) decided to use the mylyn-docs developers mailing list for further communication about AsciiDoc and Eclipse.

Apparently, having a good editor, possibly even with WYSIWYG or at least WYSIWYM support is one of the most important things required.

Building AsciiDoc

There are several ways of building AsciiDoc. The most simple one is about using the AsciiDoctor tools directly from the command line. There is also a Maven plugin provided by the AsciiDoctor project.

We haven't discussed that topic any further though. It seems as if the existing tools are sufficient even though there are some shortcomings when using custom macros as described below. You may have a look at the N4JS documentation pom.xml to get an idea of how this could look like.

Converting to AsciiDoc

During the BoF, the question came up how to convert existing documentation written in other formats to Asciidoc. One solution for that is Pandoc, which we (at NumberFour) are using to convert our LaTeX specification to Asciidoc. Jeremie Bresson has already blogged about the BoF and even provided a solution for using Mylyn Wikitext to convert Eclipse MediaWiki to Asciidoc. Thank you very much, Jeremie!

Using AsciiDoc as a Single Source

I already mentioned above that you can generate HTML and PDF from AsciiDoc. However, HTML is not HTML. We use AsciiDoc to generate

  • the public web site (i.e., gh-pages)
  • the Eclipse help
  • the PDF specification

from a single asciidoc file. To give you an idea, have a look at an adoc source file and the generated web page. If you download the N4JS IDE, you can also have a look at the generated Eclipse help containing the same content. The PDF is not publicly available yet, we are currently working on migrating our language specification from LaTeX to AsciiDoc.

In order to generate Eclipse help, a table of contents file is required. At the moment, we generate them via the geneclipsetoc-maven-plugin Maven plugin, see our pom.xml for our configuration.

You may also use the Mylyn Docs to generate code, which is described in the Mylyn Docs Eclipse help. This also provides support for generating EPub, the first steps towards this support has already been described 2011 in Torkild's blog! It’s now a stable part of Mylyn Docs and has been for a few years. Currently it is in maintenance mode as there are few bugs and there is not much to add – apart for EPUB 3.0 support. You can find an EPUB examples (and presentations about Mylyn Docs in general) on Torkild's GitHub page.

Customizing AsciiDoc

Although AsciiDoc provides a lot of features ("macros" in Asciidoc terminology) for technical documentation, such as source blocks, warning blocks etc., we (at NumberFour) missed some functionality. We required additional support for

  1. definitions -- a typical feature required in specifications
  2. BibTex cites and bibliographies -- we already have a large BibTex database which we want to reuse
  3. todos -- well, also a typical feature required in specifications and documentations
  4. links to source code -- e.g., for adding links to either GitHub, or, in case of Eclipse help, to the files in the workspace
  5. larger documents, e.g., includes similar to LaTeX's chapter folders

One problem we found is that Ruby-based custom macros calling native code cannot be used with AscidoctorJ. E.g., there already exists a custom Ruby macro which provides support for BibTex, but this does not work with AscidoctorJ. We want to write the macros in Java, since we plan to provide some editor support in Eclipse as well -- and then Java would be the better solution. Thus we already wrote our own Java-based, BibTex macro (fixing some smaller AsciidoctorJ problems on the way) and we are also working on solutions for the other topics. If you are interested in that, or if you have written Java-based macros for AsciiDoc as well -- let me know! We haven't open sourced our custom macros yet since they are still under (heavy) development. But we will probably do that once they are in a state to make them public.

Edit: Updated description of EPUB support -- thank you Torkild for the information!

Thursday, July 21, 2016

From Xcore/ecore to OmniGraffle

Some years ago I wrote a small tool for creating OmniGraffle UML diagrams directly from Java source code. Visualizing Java is nice, but since I'm often use ecore/Xcore to define my models, I wanted a tool to also nicely visualize EMF based models.

I have now extended my tool, j2og, to also create UML class diagrams from ecore or Xcore models. Below you see a (manually layouted) version of an automatically generated diagram of the ecore library example.

j2og does not layout the diagram, since OmniGraffle provides some nice layout algorithms anyway. When creating the diagram, you can tweak the output with several settings. For example

  1. show or hide attribute and operation compartments
  2. show context, optionally grayed out -- the context are classifiers defined in an external package
  3. show package names, omit common package prefixes etc.
  4. and more

Note that besides OmniGraffle, you can open the diagrams with other tools (Diagrammix, Lucidchart) as well. See the j2og github page for details. You can install the tool via update site or Eclipse marketplace link.

The following image (click to enlarge) is the result of exporting a large Xcore model defining the AST of N4JS, a statically typed version of JavaScript. I have exported it and applied the hierarchy layout algorithm -- no other manual tweaks were applied. Of course, this diagram is probably too large to be really useable, but it is a great start to document (parts) of the model. Well, in case of an AST you probably prefer using an EBNF grammar ;-)

PS: Of course you could use ecoretools to create an UML diagram. I usually need the diagrams for documentation purposes. In that case, OmniGraffle simply is so much better since it is easier to use and the diagrams look so much nicer, (sorry, ecoretools).

Monday, March 14, 2016

N4JS: Emphasising the Java in JavaScript

For the last three years, I have worked at NumberFour leading a team which creates a type-safe extension for ECMAScript (aka JavaScript) called N4JS. I'm happy to tell you that yesterday, NumberFour announced N4JS to go open source (press release, PDF). You can find the project home page at:

N4JS bridges the strengths of JavaScript and Java; the result is a typed JavaScript superset that is dynamic, flexible and type-safe. This first release mainly targets Node.js developers, enabling them to create large and maintainable projects.

The idea behind N4JS is to bring Java's type system to JavaScript and to make JavaScript really as type-safe as Java. This is probably also the main difference to TypeScript, which focuses on ease of transition from untyped to typed JavaScript. I don't want to go into the details here, we already created a longer description of the differences to TypeScript

As a result, N4JS brings many features known from Java to ECMAScript. It is based on ECMAScript 2015 (almost all new features are supported, missing things will be added soon). It therefore supports classes as in ECMAScript 2015 (or Java). It also introduces interfaces with default methods similar to Java 8. Also the generics are quite similar to Java, including support for generic types, generic methods, and wildcards (i.e. use-site variance). We have actually ported the type inference algorithm for type variables introduced with Java 8 to N4JS. Of course we had to adjust the type system to match the semantics of ECMAScript. N4JS therefore also supports union and intersection types or "this"-type (great for builder pattern).

The most notable thing about N4JS is that it not only supports Java's nominal typing but also TypeScript's structural typing. It even supports different variants of structural typing, e.g., only taking fields into account. We also have created a short introduction to this very special feature of N4JS.

There are many other bells and whistles in N4JS such as dependency injection (as in JSR330/Google Guice) or testing with annotations (as in JUnit). And this is only the beginning. With the foundation finished, we are looking forward to adding more sophisticated features such as more program analysis.

Although we call the current release a "public alpha", N4JS is pretty stable already. We are using N4JS internally for over a year now and we have a rather large code base (and more than 70.000 tests!). Of course, there are a lot of known issues (we will migrate them to GitHub issues soon)...

N4JS comes with an Eclipse-based IDE. Our goal is to provide the same IDE support for N4JS as JDT for Java. You can either download the N4JS IDE as a standalone product or use the update site to add N4JS to your Eclipse Mars installation. We also provide an Oomph script to set-up an Eclipse IDE ready for developing N4JS itself. See the GitHub pages for source code and details.

Well, this has become a rather long posting already. I will post more about N4JS, its features and new developments in the future either here or on the N4JS developer blog. Last but not least, I want to thank the people behind N4JS. In particular I want to thank the N4JS team at NumberFour -- you did a great job and I'm looking forward to working on N4JS and other things with you! In place of the many helpers I want to thank Sebastian Zarnekow -- you did an amazing job (readers may want to look at the grammar file to get an idea of what kind of magic he had to add to Xtext to enable that)!

Update (14.3.2016, 19:00): The N4JS Eclipse project proposal is now public!

Sunday, October 18, 2015

j2og -- Java To OmniGraffle, Updated

Three years ago, I wrote a small plugin called "j2og" (for Java To OmniGraffle) which can create OmniGraffle drawings from your existing Java files. At that time, I used AppleScript to create the drawing. Unfortunately this mechanism was a little bit fragile, and somehow it didn't work with new versions of OmniGraffle / Mac OS X. Instead of fixing the AppleScript problem, I rewrote the export.

The new version now directly exports to OmniGraffle files. That means OmniGraffle is no longer needed to export your files to an OmniGraffle drawing. Actually, there are several tools which can open the exported drawings now:
  • OmniGraffle, of course; only available for Mac OS X
  • Diagrammix, via "Import"; only available for Mac OS X
  • Lucidchar, not tested yet; available for Mac OS X and Windows
The overall functionality was not changed. E.g., you can export classes and interfaces with or without attributes or operations, attributes can be transformed to associations, context of classes can be exported as well. The screenshot shows a sample drawing exported from the j2og sources. Since j2og does not layout the drawing (I was too lazy to write an layout algorithm), you have to manually layout the diagram. E.g., use the auto-layout of OmniGraffle as I did for the sample screenshot.

You can either install j2og from the Eclipse marketplace or the update site (see j2og homepage at github for details).

Thursday, March 19, 2015

CfP: Workshop on Methodical Development of Modeling Tools (ModTools15)

Same procedure as last year? Same procedure as every year... but this time it's Australia. This is where EDOC 2015 is located, and this is where the following workshop is located this year:

3rd International 
Workshop on Methodical Development of Modeling Tools (ModTools15)
on the 19th IEEE International Enterprise Computing Conference EDOC 2015

This year, EDOC takes place in Adelaide, Australia. You will find the call for paper and other information at the workshop's hompage: Submission deadline is April 15th 2015.
The workshop Methodical Development of Modeling Tools focuses on procedures and architectural principles related to the creation of software for presenting, editing, or analyzing models.
Software tools for modeling are required for scientific and practical applications of modeling methods and modeling languages in enterprise computing. To test and exemplify new modeling approaches, research prototypes of model editors are required, as well as tools for presenting and analyzing models. To be able to efficiently develop such modeling tools, it is desirable to methodically guide the development of modeling tools, and elaborate procedures to align their design with the conceptualization of new modeling languages and modeling methods.

Monday, February 17, 2014

CfP: Workshop on Methodical Development of Modeling Tools

How time flies... only recently I posted about a workshop (held at EDOC 2013), and today I can announce the 2014 version, held at EDOC 2014. It's the

2nd International 
Workshop on Methodical Development of Modeling Tools (ModTools14)
on the 17th IEEE International Enterprise Computing Conference EDOC 2014

This year, EDOC takes place in Ulm, Germany. You will find the call for paper and other information at the workshop's hompage: Submission deadline is April 1st 2014 (really, no kidding).

Update (10/4/2014): Submission Deadline extended: 2014-04-28 (final extension by the main conference)

Although I'm not working at the university anymore, I still think that a workshop like this is quite important because it tries to bridge the gap between pure scientific research and real world requirements. If you look at scientific conferences, many researchers present tools in order to evaulate there approach. From my own experience I know that often you will find dragons when you try to actually implement these tools. These dragons, once disturbed, may even threaten the whole theoretically nice approach. The workshop tries to give the brave knights---and since you are reading an Eclipse related blog, that's probably you!---fighting these dragons a place to exchange thoughts, methods, and ideas. And, last but not least, it gives you an opportunity to publish about that kind of work (the workshop proceedings are published together with the conference proceedings at IEEE).