J# and Ruby.Net — first steps

This seems to work

//C# assembly referencing the Ruby.NET.Runtime assembly
namespace Inspect
{
    public class Access
    {
        public static Ruby.Class getRubyPerson()
        {
            ClassLibrary2.nested.Person p = new ClassLibrary2.nested.Person("", 0);
            return Ruby.Class.CLASS_OF(p);
        }
    }
}

## Ruby 
x = Inspect::Access.getRubyPerson()
fred = x.new(name, age)
fred.print

It's a pity that I need an object and cannot just send a System.Type into Interop.CLRClass.Load(type, null, false);, which this ends up doing, but that method is internal to the Runtime DLL.

It should be possible, though, to have a require_jsharp('assembly.dll') that does the tedious running around back and forth through the Ruby/C# boundary in an anutmated fashion, and creates appropriate proxies for lowercase package-names in a tree-like object.

J# and Ruby.Net — not all is rosy in the garden

Ruby expects modules to be constants (capitalised). Just like .Net namespace naming conventions dictate.

Java, OTOH, uses lower-case package names which map directly to .Net namespaces, which cause Ruby.Net to bork when trying to instantiate classes. In fact, this is a general issue — it affects any .Net module names which are not capitalised, like this C# code

namespace ClassLibrary2.nested
{
    public class Person
    {
        private String name;
        private int age;

        public Person(String name, int age)
        {
            this.name = name;
            this.age = age;
        }

        public void print()
        {
            Console.WriteLine("Hello " + name + " " + age);
        }
    }
}

which won't work when invoked by

require 'ClassLibrary2.dll'

name = 'Fred'
age = 42
fred = ClassLibrary2::nested::Person.new(name, age)
fred.print

But if you change nested to Nested, all is well.

*le sigh* — that means an adapter layer between Java-style names and Ruby/.Net ones. At least that should be doable in Java/J#, even if it makes things more painful than I had hoped.

What's wrong with this code?

Spent today getting about 2/3 coverage on the Bignums class, including rewriting the modularInverse function to complete in a finite time, based on the recursive method given in Wikipedia -- I can always redo it as iterative if there are problems, but for now I have achieved RSA.

So everything was running fine in the JVM; go to .Net, and … infinite loop trying to divide 40! by 20! (in a bit taken from the Ruby bignum unit tests. Eventually, I tracked it down to this:

    private static void s_m_mult_sub(
        int[] result, int ro,
        int[] multi, int mo,
        int scalar, int length)
    {
        long borrow = 0;
        int index = 0;

        /* Is basically practical => proceed */
        while(index < length)
        {
            long product = multUnitxUnit(scalar, multi[mo+index]) + borrow;
            borrow = product >>> 32;
            product &= MASK;
            if(product > (result[ro+index]&MASK))
                ++borrow;
            result[ro+index++] -= (int)product;
        }
        result[ro+index] -= (int)(borrow&MASK);
    }

which Java -- and the original 'C' from which this was taken -- runs "as you'd expect" but J# handles as if it were

result[ro+index++] = result[ro+index++] - (int)product;

and increments index twice, as well as splattering other mayhem around the structure.

Next up, getting JZlib tidied and turning its tests into JUnit form, and adding the PGP needed twiddles; then a first release to replace my old crypt.zip.

The original 'C' code:

static void s_m_mult_sub(unit result[], unit multi[], unit scalar, unit length)
{
    register unit high, low1, low2;
    unit borrow = 0;
    unit index = 0;

    /* Is basically practical => proceed */
    while(index < length)
    {
        multUnitxUnit(high, low1, scalar, multi[(size_t)index]);
        low2 = low1 + borrow;
        if(low2 < low1) high++;
        if(low2 > result[(size_t)index])
            borrow = high + 1;
        else
            borrow = high;
        result[(size_t)(index++)] -= low2;
    }
    assert(result[(size_t)index] >= borrow);
    result[(size_t)index] -= borrow;
}

Another step

Package	com.ravnaandtines.crypt.mda	com.ravnaandtines.crypt.cea
# Classes	8	31
Line Coverage	95% 1051/1104	99% 1641/1659
Branch Coverage	81% 170/210	95% 399/422
Complexity	2.369	2.146

That includes real test vectors for TEA (and fixing the implementation at last, so it's incompatible with what I used to have for Java, but is real for the first time -- problems with operator precedence and signed vs unsigned shift), as well as things like Safer not implementing the common interface and a few problems with non-zero array offsets and with triple encryption.

Just need to get bignums and ZLib sorted out with tests, and it'll be time for a proper drop.

Moore at work

The CAST5 algorithm defines a maintenance test:

B.2. Full Maintenance Test

    A maintenance test for CAST-128 has been defined to verify the
    correctness of implementations.  It is defined in pseudo-code as
    follows, where a and b are 128-bit vectors, aL and aR are the
    leftmost and rightmost halves of a, bL and bR are the leftmost and
    rightmost halves of b, and encrypt(d,k) is the encryption in ECB mode
    of block d under key k.

    Initial a = 01 23 45 67 12 34 56 78 23 45 67 89 34 56 78 9A (hex)
    Initial b = 01 23 45 67 12 34 56 78 23 45 67 89 34 56 78 9A (hex)
 
    do 1,000,000 times
    {
        aL = encrypt(aL,b)
        aR = encrypt(aR,b)
        bL = encrypt(bL,a)
        bR = encrypt(bR,a)
    }

 Verify a == EE A9 D0 A2 49 FD 3B A6 B3 43 6F B8 9D 6D CA 92 (hex)
 Verify b == B2 C9 5E B0 0C 31 AD 71 80 AC 05 B8 E8 3D 69 6E (hex)

In my code I had a comment "takes about an hour to run", dating from about the end of 1998. Trying today, I got

• In shiny new NetBeans 6.0, run from a JUnit 3.8.2 unit test with cobertura coverage instrumentation: 885s
• With instrumentation disabled: 6s
• In VS 2005 as a post-build run of NCover 1.5.8 running the ported test as a debug build: 34s
• In VS 2005 as a post-build run of NCover 1.5.8 running the ported test as a release build: 17s
• In VS 2005 as a command-line run of the test as a release build (100 lines of output piped to a file, rather than the console): 6s

So, you pay through the nose (~150 times slower) for your branch coverage instrumentation, as opposed to line coverage (~6 times). And both JVM and CLR are seriously faster, even allowing for the maybe 20x faster processor, than what I was running a decade ago.

Oh, yeah, today's unit test…

It passes.

Making a hash of it

Wrote one (meta-)test today. Find every class in com.ravnaandtines.crypt.mda, instantiate through the default constructor, and check that putting in a short sequence of bytes one at a time, as an array or as an array-slice all gave the same results.

On the up-side, this ups coverage:

Package	com.ravnaandtines.crypt.mda
# Classes	8
Line Coverage	95% 1051/1104
Branch Coverage	81% 170/210
Complexity	2.369

and found one bug, in BlockCypherHash, where partial blocks were being lost (I wasn't updating the count of pending bytes when I should).

On the down side, while the test driver was easy to write for J#

MD5 prototype = new MD5(); // an arbitrary class in the right assembly
System.Reflection.Assembly algorithms = prototype.GetType().get_Assembly();
System.Type[] types = algorithms.GetTypes();
System.Type mda = algorithms.GetType("com.ravnaandtines.crypt.mda.MDA");

for (int k = 0; k < types.length; ++k )
{
 final System.Type underTest = types[k];
 if (!mda.IsAssignableFrom(underTest))
 {
  continue;
 }
…

but for Java, it requires some magic to do similar blind enumeration of classes. The following fudge (based off this one, with the fix noted below) gets the all loaded classes in a named package (stuck with Java 1.1 for convenience when cross compiling)

    public static java.util.Vector find(final String pckgname) {
        // Translate the package name into an absolute path
        String name = pckgname;
        if (!name.startsWith("/")) {
            name = "/" + name;
        }
        name = name.replace('.', '/');
        // Get a File object for the package
        final java.net.URL url = // get an instance of the current class to hook from
                new CryptTestUtils().getClass().getResource(name);

        // NOTE -- this url will put %20 for spaces in your path 
        // It needs to be cleared up before creating a File object
        final String dirpath = java.net.URLDecoder.decode(// 1.2 class easily stubbed for J#
                url.getFile()); 

        final java.io.File directory = new java.io.File(dirpath);
        final java.util.Vector contents = new java.util.Vector();
        if (directory.exists()) {
            // Get the list of the files contained in the package
            final String[] files = directory.list();
            for (int i = 0; i < files.length; i++) {

                // we are only interested in .class files
                if (files[i].endsWith(".class")) {
                    // removes the .class extension
                    final String classname = 
                            files[i].substring(0, files[i].length() - 6);
                    try {
                        // Try to create an instance of the object
                        final Object obj = 
                                Class.forName(pckgname + "." + classname).newInstance();
                        contents.add(obj.getClass()); // save off the class
                    } catch (ClassNotFoundException cnfex) { 
                    } catch (InstantiationException iex) { 
                    // We try to instantiate an interface
                    // or an object that does not have a 
                    // default constructor
                    } catch (IllegalAccessException iaex) { 
                    // The class is not public
                    }
                }
            }
        }
            return contents;
    }

and then we do

final java.util.Vector classes = 
                com.ravnaandtines.crypt.CryptTestUtils.find(
                "com.ravnaandtines.crypt.mda"
                );       

for (final java.util.Enumeration iter = classes.elements(); iter.hasMoreElements();)
{
      final Class underTest = (Class) iter.nextElement();

      final Object test = underTest.newInstance();
      if(!(test instanceof com.ravnaandtines.crypt.mda.MDA ))
      {
          continue;
      }
…

So, that's all the hash algorithms done. Next is the similarly mindless conversion of the crypto algorithm test vectors (and a similar meta-test). Then turning the JZlib system tests into a set of unit tests, and porting something like the Ruby Bignum unit tests. Maybe by the turn of the year, I shall actually have a platform based on existing work that can be taken forward.

Tinkering away

The standard test vectors for the various algorithms do a good job of shaking down the code for an implementation.

Package	com.ravnaandtines.crypt.mda
# Classes	8
Line Coverage	93% 1024/1103
Branch Coverage	76% 159/210
Complexity	2.369

Coverage would be higher if there weren't a still few CTC-related entry-points, or code-paths like compensating for a broken Haval implementation in a PGP extension (EPB)…