Language integration

Question

I may be the minority here, but it seems through my entire academic/professional career I've been taught varying languages. During this time, syntax and programming paradigms were the focus, but at no point were we taught about integrating systems written using varying languages and the proper way to make this decision.

Now for the record, I'm not talking about the canonical web stack or the newer, sexier, JVM-friendly languages. What I'm wondering is if there are "known resources" where I could learn about the decision making processes behind binding languages like, Java and C++, for example.

Granted, tools like XML, JSON and XMPP come to mind. Then again, I've seen systems binding Java and C++ using serialization. I'm not looking for a one-fix-for-all type of solution. I'm more interested in learning about the varying solutions and how I should be going about making such decisions.

My apologies if this is far too broad for this forum, but at least I'm not asking folks to correct or rewrite my botched code ;)

Answer 1

There are four different models that I can think of:

• Embed a dynamic language inside an app that is primarily written in a more "systems" language, like Lua, Python or Javascript embedded in a Java, C++ or C# app. This is used primarily for a scripting / customization component to the app. This will be accomplished by exposing some of the host applications data types in a format that the dynamic language can make use of.

• Write native (or C# or Java) extensions for a language with performance problems. Python and Ruby have lots of these extensions. This differs from the above in that the dynamic language is the framework. This is accomplished by writing the native libraries (or a wrapper around other native libraries) to conform to the calling conventions of the client language. This is also the same general structure when intermixing assembler with C in systems code.

• Run the applications in different address spaces and communicate over sockets or pipes. In this case, it's just a coincidence that the applications are running on the same machine at all -- they could just as well be talking over the network.

• Develop the app using multiple languages that share the same platform and calling conventions. For example, Java and Scala can be intermixed freely.

Answer 2

Lua is a scripting language that integrates very nicely with C within the same process. It uses a stack model to bridge the impedance mismatch between the two languages / run-times. The Lua implementation is open-source and relatively compact; so it may be good candidate for studying.

D-Bus is an example of using IPC. It can be used to integrate separate processes implemented in different languages. It supports asynchronous messaging as well as synchronous method invocation. There are D-Bus implementations for Gtk, Qt, python etc...

The Parrot Virtual Machine is a VM that is aiming for language inter-operation.

Yet another approach is the one taken by GLib (used by Gtk). From the GObject manual:

The solution used by GLib is to use the GType library which holds at runtime a description of all the objects manipulated by the programmer. This so-called dynamic type 1 library is then used by special generic glue code to automatically convert function parameters and function calling conventions between different runtime domains.

Answer 3

The safe solution is a multiprocess solution where each language runs in its own address space, and they communicate over sockets or some other multiprocess abstraction (e.g., Linda tuples).

If two languages run in the same address space, either they have to be designed together on top of a common runtime (as Digital did beautifully with its VAX VMS family of languages, and as Microsoft has tried hard to do with its Common Language Runtime for .NET) or one has to be "in charge". There are many languages (OCaml, Standard ML, Haskell, Lua, and probably also Perl, Ruby) which interoperate nicely with C provided they are in charge. They don't interoperate with each other.

Usual the most fundamental difficulty is automatic memory management (the "managed heap"). Some one entity has to be in charge of deciding when it is safe to reclaim dead objects, and if two different languages both decide they are the ones who should answer that question, separate processes loom.

Answer 4

I think Swig is what you need. Also see this page, where you can find various tools to integrate Python with a host of other languages.

Answer 5

Diverging a bit on the decision making process...

When building a system, you split it up in functionally independent subsystems. For each of the subsystems, in theory, you should find out on what framework you will build it. Aspects that matter: * what framework matches the subsystem's functionality best * what frameworks are your developers comfortable with * future support: what if the framework vendor stops supporting it (as MS almost did with unmanaged C++) *

When the system becomes heterogenous, it's time to think about the intra-subsystem glue. This, too, can be regarded as a subsystem on it's own, and can be built on several inter-process communication protocols like MPI, pipes, ... or something home brewn.