Why am I doing this sort of thing?

Michael Foord (aka Fuzzyman) has wondered, while linking me, about why one would want to talk to Scala from IronPython.

Partly, it is the intellectual challenge of that particular sub-problem. But the broader goal is to see how far it is possible, in this polyglot age, to build a language stack -- along the lines the Ola Bini mused about at the start of the year -- that one could drop onto the JVM or .Net using essentially the same code base, give or take a thin C# layer to provide compatible APIs in .Net to those needed in Java but not present in J#.

While the Fan language is developed to be exactly so code-compatible across the JVM and CLR, it appears to be resolutely monoglot, with the platform hidden behind an abstraction layer.

The ulterior motive is to take a codebase that I already have in Java/J#, the first stages of a reimplementation in a VM targeted language, of my old CTClib project -- and be able to complete the reimplementation/upgrade to RFC2440 spec, in a platform neutral way, and without having to write any more Java, even if the IronPython/J#+Swing interface will be rather less Pythonic than I could manage if I were just doing it in Jython.

And why Scala as well? For any new static layer code, especially where being explicit about the types involved, such as around the crypto, will make things clearer.

One correction, though, to the remarks : ATM, I'm using the scala-net compiler, which is part of the Scala bazaar, and seems (from its failure modes when the J# libraries are in the loop) to be a different generator to that in IKVM. Not being IKVM means I'm also spared having to drag in the GNU Classpath library, where the server stuff has priority, and desktop UI support is still primitive -- its AWT is less functional, in many important respects, than Java 1.0 was.

Scaling the tower of Babel II

Finally, a little digital extraction. Scala from IronPython, works as expected. In file operator.scala:

package com_ravnaandtines.operator

trait Process {
 def op(x:Int, y:Int) : Int
}

class Multiply extends Process {
  def op(x:Int, y:Int) : Int =
   x * y
}

class Perform69 {
 def sixByNine(x: Process) =
  x.op(6,9)
}

Set up as in the previous post, and build using %SCALA_HOME%\bin\scalac-net.bat operator.scala to get operator.msil -- no assembly is generated at this point. Edit the .msil file as required to point at the appropriate versions of the runtime (see previous post). Then ilasm .\Operator.msil /dll /output=operator.dll. Now fire up IronPython:

>>> import clr
>>> clr.AddReference("operator.dll")
>>> from com_ravnaandtines.operator import *
>>> dir()
['Multiply', 'Perform69', 'Process', '__builtins__', '__doc__', '__name__', 'clr']
>>> m = Multiply()
>>> p69 = Perform69()
>>> p69.sixByNine(m)
54
>>> class Adder(Process):
...   def op(self,a,b):
...     return a+b
...
>>> add = Adder()
>>> p69.sixByNine(add)
15
>>>

So long as we get our interfaces/traits from Scala, all is fine.

In order to do the same in the JVM world, I need to have done set CLASSPATH=%CLASSPATH%;C:\scala-2.7.1.final\lib\scala-library.jar; and then I can do exactly as above, but without the first two lines with clr. Those I can put in a try/except, for common coding.

Note -- it is neither necessary nor sufficient to put the Scala library jar in java.class.path or python.path properties.

Scaling the tower of Babel

A few months ago, I was playing around with seeing how I could adapt an existing Java code base so that I could finish doing the work in better languages -- and have the code work on both the JVM and the CLR.

It seems that frustration will inevitably follow any attempt to incorporate any J# into the .Net versions of the stack (both IKVM and the Scala-net compilers bork when they see another Java in the stack) -- even though you can call Jython into Scala into Java on the JVM, this Tower of Babel cannot stand on the CLR with the J# runtime in the picture -- Java e.g. a Swing-based GUI using the J# extensions, has to stand off to one side.

That said, the Scala on .Net "hello world" with Scala 2.7.1 is comparatively painless.

1. Install the scala-msil package (sbaz install scala-msil)
2. Create file Hello.scala as
   

   import System.Console
   
   object test extends Application {
     Console.WriteLine("Hello world!")
   }

3. In a Visual Studio command prompt, issue %SCALA_HOME%\bin\scalac-net.bat hello.scala
4. Copy %SCALA_HOME%\lib\*.dll to the current directory
5. Run the resulting test.exe

This uses the 1.0 version of mscorlib.dll; as a post-processing step you may wish to re-base against the .Net 2.0 version -- for this, you need to rebuild predef.dll and scalaruntime.dll using the same technique as needed to strong-name them after the fact; only this time you want to reset the mscorlib reference from

.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 1:0:5000:0
}

to

.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 2:0:0:0
}

as well as doing any strong-naming you may desire.

To set the assembly name, add -Xassem [assembly name] to the compile command, to generate hello.exe.

To control the type of the generated assembly (.dll rather than .exe), I've not found a way other than taking the generated [assembly name].msil and ilasming it manually. In particular trying to skip generating the unwanted .exe will mean figuring why using scalac.bat -target:msil always gives

error: fatal error: class scala.runtime.BoxedBooleanArray not found.

whatever I do with -Xassem-path or tweaking environment variables.

Integrating .Net with Erlang, part 8

Gratuitous update on 08/08/08 with a small increment of unit testing -- now using Rhino Mocks 3.4 as well as home-grown ones, and NUnit 2.5 (alpha 3) for the neat

Assert.That( [delegate expression], Throws.Exception<[type]>())

syntax.

Drop here, as usual.

Erlang and Python

Following the suggestion that I try TwOtp, I've tried the "kicking the tyres" with that library.

Adapting the sample code to work against my own local node seemed easy enough:

from twisted.internet import reactor
from erlang import OneShotPortMapperFactory, buildNodeName

def gotConnection(inst):
    return inst.factory.callRemote(inst, "file", "get_cwd"
        ).addCallback(gotResult)

def gotResult(resp):
    print "Got response", resp
    reactor.stop()

def eb(error):
    print "Got error", error
    reactor.stop()

epmd = OneShotPortMapperFactory('client@chloe.ravnaandtines.com', "cookie")
epmd.connectToNode("servernode@chloe.ravnaandtines.com").addCallback(gotConnection).addErrback(eb)
reactor.run()

which involves rather more baggage than using IronPython against my tidying of the C# port of jinterface, for just a "hello world!", given that the jinterface-derived "hello world!" actually does more work (passing arguments, testing results):

import clr
clr.AddReference("OtpErlang.dll")
from System import Array

from Tinesware.Otp import *
import Tinesware.Otp.Erlang as Erlang

cNode = Self("clientnode@chloe.ravnaandtines.com", "cookie")
sNode = Peer("servernode@chloe.ravnaandtines.com")
connection = cNode.Connect(sNode)

args = Array[Erlang.IObject]([Erlang.Integer(1), Erlang.Integer(2)])

connection.SendRpc("mathserver", "add", args)
sum = connection.ReceiveRpc()

if  sum.Value.ToInt32() != 3:
  print "Assertion failed, returned = " + sum.Value
else:
  print "OK!"

And, like py_interface, I find that this doesn't "just work" either:

C:\Documents and Settings\Steve\My Documents\code\babel\erlang>python twotphw.py

Got error [Failure instance: Traceback (failure with no frames): <class 'erlang.epmd.NodeNotFound'>: 1]

which will require a similar amount of rolling sleeves up and getting hands dirty to resolve.

Erlang with Python and Ruby

Having played a bit with Erlang for real, in the sense of something non-trivial, even if not a complete product, I find I'm in the mood for more. But while it comes with some UI widgets, it's something more for headless use, and some client(s) in other languages doing the display.

To date, I can easily do Swing or WinForms as the UI toolkit -- but what about wxPython or FXRuby?

There are a number of bridges that I've found, but the closest to jinterface seem to be:

• py_interface for Python (about 2 years old)
• Un-named code drop on Google for Ruby (about 1 year old)

Neither of them have "just worked" for me in the way that the Java and C# versions have done against the simple mathserver example. Some of it might be down to my not figuring how the differences in the APIs are meant to work.

For py_interface, I had to comment out the line

[__import__(item) for item in __all__]

from init.py to get imports to work at all; and even then, the simple

from otp import *

def Handler(Result):
    print Result

cNode = erl_node.ErlNode("clientnode@chloe.ravnaandtines.com", erl_opts.ErlNodeOpts(cookie="cookie"))
print cNode
print cNode.Publish()
mBox = cNode.CreateMBox()
print mBox
mBox.SendRPC("servernode@chloe.ravnaandtines.com", 
    "mathserver", "add", [1, 2], Handler)
raw_input("wait")

just sits there; while the Ruby code

require 'erlang_node'
 
s_node = Erlang::Node.new("servernode@chloe.ravnaandtines.com")
c_node = Erlang::LocalNode.new("clientnode@chloe.ravnaandtines.com", "cookie")
connection = Erlang::Connection.new(s_node, c_node)
connection.sendRPC("mathserver", "add", [1,2])
sum = connection.receiveRPC()
puts "Fail" unless sum == 3

throws a different exception every time I run it, from some place or other inside the connection initializer.

So they both need deeper inspection than this simple kicking of the tyres, maybe modifying for R12 nodes, maybe just reverse engineering the kinks out of the driver programs.

Integrating .Net with Erlang, part 7

Another update, after work during the holiday weekend.

By using a lot of spare cycles, I found a test case for the division code not previously exercised:

9659964175213286916418497243082984930780677040655778513759952405592597918851617939992802011783376153721017703858175 / 3248063261698017931917474008545947342556579729240422363527 = 2974068974926080862795323621370295193198999568425344221053

Also made the astoundingly obvious discovery that I needed the real Erlang port mapper dæmon running, so put a unit test in which spawns an Erlang process to contact. It works fine in NUnit-gui, but fails every time in the build time coverage plus test cycle, so is labelled as [Explicit]. I need to to write at least one test that handles the call in the other direction, but this is getting close to as far as can easily be done in unit test form.

And it's getting close to my usual summer break from coding in the evenings, so this is probably close to being "it" for a while.