My hobbyist coding updates and releases as the mysterious "Mr. Tines"

Friday 28 December 2007

Phase 2 — “The best code is code someone else has already written and debugged.”

So goes the old saying.

And looking at the almost decade old Angerona project code, I see there's a lot of code that is written, was lightly debugged, probably works, and just needs refactoring. After all, it was the lack of a ZLib that stopped that effort dead, not anything with the existing code.

So, the next phase will be to put together an infrastructure library (jar/dll) from the useful bits, which should be enough to get to the point of being able to parse a key-ring or similar binary file to OpenPGP format.

At that point, some weeks hence, it will make sense to put a first bit of GUI on top; and start filling in the gaps in a less clunky language than Java where possible.

Tuesday 25 December 2007

angerona.algorithms 1.0.2915.29952

Here. and a back-up here if Demon's DNS flakes out.

A complete replacement for the old crypt.zip, with source and pre built .jar and .dll, unit tests, and a .Net build of junit 3.8.2 -- includes:

  • JZlib (inflate/deflate)
  • Blowfish, CAST5, DES (including s3' and key-dependent variants), IDEA, Safer, Square, TEA, ThreeWay, TripleDES (encryption)
  • BlockCypherHash, Haval, MD5, RIPEM160, SHA0, SHA1 (message digests); and
  • Bignum (mutable multiple precision integers).

Needs the J# redistributables for .Net use,

Happy holidays! and 100th post!

Thursday 20 December 2007

J# and Ruby.Net — string and baling wire

A proof of concept

//Factory.java
public class Factory
{
    private Factory(){}

    public static Object Build(String classname) 
        throws InstantiationException, IllegalAccessException, 
        ClassNotFoundException
    {
        Class target = Class.forName(classname);
        return target.newInstance();
    }
}
//Utility.java
package Tinesware.Adapter;

public class Utility
{
    private Utility() { }

    public static void exit(int status)
    {
        java.lang.System.exit(status);
    }
}
//WindowAdapter.java
package Tinesware.Adapter;

public class WindowAdapter implements java11.awt.event.WindowListener  
{
    private final Ruby.Object payload;

    public WindowAdapter(Ruby.Object arg)
    {
        payload = arg;
    }

    public void windowActivated(java11.awt.event.WindowEvent e)
    {
    }

    public void windowClosed(java11.awt.event.WindowEvent e)
    {
        System.out.println("window Closed");
        Object[] args = {e};
        Ruby.Runtime.Eval.Call(payload, "windowClosed", args);
    }

    public void windowClosing(java11.awt.event.WindowEvent e)
    {
        System.out.println("window Closing");
        Object[] args = { e };
        Ruby.Runtime.Eval.Call(payload, "windowClosing", args);
    }

    public void windowDeactivated(java11.awt.event.WindowEvent e)
    {
    }

    public void windowDeiconified(java11.awt.event.WindowEvent e)
    {
    }

    public void windowIconified(java11.awt.event.WindowEvent e)
    {
    }

    public void windowOpened(java11.awt.event.WindowEvent e)
    {
    }
}
##Program.rb
require 'Tinesware.Adapter.dll'
require 'gwt221.dll'

window = Tinesware::Adapter::Factory.Build("dtai.gwt.GadgetFrame");
window.setSize (800, 600)
window.setTitle 'Proof of Concept'
window.show

class WindowHandler
  def initialize(w)
    @window = w
   end

  def windowClosing(e)
    @window.hide
  end
  def windowClosed(e)
    Tinesware::Adapter::Utility.exit 0
  end
end

adapter = Tinesware::Adapter::WindowAdapter.new(WindowHandler.new(window))
window.addWindowListener adapter

It only works grace of Ruby.Runtime.Eval.Call() being public, but that's all I need; that the event argument gets through is a bonus.

It is a mild hack, it does involve repetitive boilerplate coding, but you only have to do it for a few interface classes. And it will need some smoothing over on the Ruby side to make it look less awkward with code that's supposed to interoperate with Java as JRuby.

Extra bonus -- if the Call()'d method doesn't exist, it does nothing silently, so you only have to implement the parts of an interface you actually want.

Monday 17 December 2007

J# and Ruby.Net — foiled again

Carrying on from last post…

// C#
        public static void typeTest(object arg)
        {
            System.Type wtf = arg.GetType();
            Console.WriteLine(wtf.ToString());
            Console.WriteLine("Base = "+wtf.BaseType.ToString());

            System.Type[] iface = wtf.GetInterfaces();
            Console.WriteLine("# interfaces = " + iface.Length);
            foreach (System.Type t in iface)
                Console.WriteLine(":: "+t.ToString())
        }
##Ruby caller
q = Observer.new
JsharpToRuby::ClassDictionary.typeTest q

yields disappointing results

Observer
Base = Ruby.Object
# interfaces = 0

Oh dear! it doesn't seem to work the naïve way for proper interfaces

// C#
namespace JsharpToRuby
{
    public interface ITest
    {
        Boolean isPresent();
    }
##Ruby 
class EyeTest 
  include JsharpToRuby::ITest
  def isPresent
    1
  end
end
q = EyeTest.new
puts q.class
q.class.ancestors.each { |x| puts x }
JsharpToRuby::ClassDictionary.typeTest q

yields

EyeTest
EyeTest
ITest
Object
Kernel
EyeTest
Base = Ruby.Object
# interfaces = 0

This is of course the bit missed out in the interop tutorial, alas; I think for the good and simple reason that it is not yet implemented (from a quick inspection of the code).

*le sigh*

Maybe not all is lost. It would mean more hand-rolled adapter classes to wrap the objects and delegate by reflection.

But later, definitely, later.

Sunday 16 December 2007

J# and Ruby.Net — further steps

So what do we need Java classes for in a JRuby/Ruby.Net environment?

Things that we have to new because that defines system services in an unambiguous fashion — I'm thinking encodings for I/O and GUI components here, as well as libraries already written that we don't want to port just yet (my algorithms library); and interfaces/abstract classes we want to make concrete (needing to be an ActionListener or similar).

Concrete things are easy; abstractions are trickier, but not impossible.

//C# code to generate a Ruby class type
        public static Ruby.Class getThing()
        {
            object o = new java.awt.Panel();
            return Ruby.Class.CLASS_OF(o);
        }
##Ruby code to use it
thing = JsharpToRuby::ClassDictionary.getThing

puts thing.to_s

puts thing.ancestors

panel = thing.new
puts panel.to_s

which generates

Panel
Container
Component
Serializable
MenuContainer
ImageObserver
Object
Object
Kernel
java.awt.Panel[panel1,0,0,0x0,invalid,layout=java.awt.FlowLayout]

which goes … java.lang.Object, Ruby Object, …

So now try

io = nil
thing.ancestors.each { |x| io = x if x.to_s.eql? "ImageObserver" }
MyImageObserver = io

class Observer 
  include MyImageObserver
  def imageUpdate(img, infoflags, x, y, width, height) 
    nil
   end
end

q = Observer.new
puts q.class.ancestors

to get

Observer
ImageObserver
Object
Kernel

which means the task is not immediately insuperable; but may need some hand-rolling of concrete classes for some of the interesting interfaces, rather than being something that can be totally read-only…

J# and Ruby.Net — first steps

This seems to work

//C# assembly referencing the Ruby.NET.Runtime assembly
namespace Inspect
{
    public class Access
    {
        public static Ruby.Class getRubyPerson()
        {
            ClassLibrary2.nested.Person p = new ClassLibrary2.nested.Person("", 0);
            return Ruby.Class.CLASS_OF(p);
        }
    }
}
## Ruby 
x = Inspect::Access.getRubyPerson()
fred = x.new(name, age)
fred.print

It's a pity that I need an object and cannot just send a System.Type into Interop.CLRClass.Load(type, null, false);, which this ends up doing, but that method is internal to the Runtime DLL.

It should be possible, though, to have a require_jsharp('assembly.dll') that does the tedious running around back and forth through the Ruby/C# boundary in an anutmated fashion, and creates appropriate proxies for lowercase package-names in a tree-like object.

Thursday 13 December 2007

J# and Ruby.Net — not all is rosy in the garden

Ruby expects modules to be constants (capitalised). Just like .Net namespace naming conventions dictate.

Java, OTOH, uses lower-case package names which map directly to .Net namespaces, which cause Ruby.Net to bork when trying to instantiate classes. In fact, this is a general issue — it affects any .Net module names which are not capitalised, like this C# code

namespace ClassLibrary2.nested
{
    public class Person
    {
        private String name;
        private int age;

        public Person(String name, int age)
        {
            this.name = name;
            this.age = age;
        }

        public void print()
        {
            Console.WriteLine("Hello " + name + " " + age);
        }
    }
}

which won't work when invoked by

require 'ClassLibrary2.dll'

name = 'Fred'
age = 42
fred = ClassLibrary2::nested::Person.new(name, age)
fred.print

But if you change nested to Nested, all is well.

*le sigh* — that means an adapter layer between Java-style names and Ruby/.Net ones. At least that should be doable in Java/J#, even if it makes things more painful than I had hoped.

Sunday 9 December 2007

What's wrong with this code?

Spent today getting about 2/3 coverage on the Bignums class, including rewriting the modularInverse function to complete in a finite time, based on the recursive method given in Wikipedia -- I can always redo it as iterative if there are problems, but for now I have achieved RSA.

So everything was running fine in the JVM; go to .Net, and … infinite loop trying to divide 40! by 20! (in a bit taken from the Ruby bignum unit tests. Eventually, I tracked it down to this:

    private static void s_m_mult_sub(
        int[] result, int ro,
        int[] multi, int mo,
        int scalar, int length)
    {
        long borrow = 0;
        int index = 0;

        /* Is basically practical => proceed */
        while(index < length)
        {
            long product = multUnitxUnit(scalar, multi[mo+index]) + borrow;
            borrow = product >>> 32;
            product &= MASK;
            if(product > (result[ro+index]&MASK))
                ++borrow;
            result[ro+index++] -= (int)product;
        }
        result[ro+index] -= (int)(borrow&MASK);
    }

which Java -- and the original 'C' from which this was taken -- runs "as you'd expect" but J# handles as if it were

result[ro+index++] = result[ro+index++] - (int)product;
and increments index twice, as well as splattering other mayhem around the structure.

Next up, getting JZlib tidied and turning its tests into JUnit form, and adding the PGP needed twiddles; then a first release to replace my old crypt.zip.

The original 'C' code:

static void s_m_mult_sub(unit result[], unit multi[], unit scalar, unit length)
{
    register unit high, low1, low2;
    unit borrow = 0;
    unit index = 0;

    /* Is basically practical => proceed */
    while(index < length)
    {
        multUnitxUnit(high, low1, scalar, multi[(size_t)index]);
        low2 = low1 + borrow;
        if(low2 < low1) high++;
        if(low2 > result[(size_t)index])
            borrow = high + 1;
        else
            borrow = high;
        result[(size_t)(index++)] -= low2;
    }
    assert(result[(size_t)index] >= borrow);
    result[(size_t)index] -= borrow;
}

Another step

Packagecom.ravnaandtines.crypt.mdacom.ravnaandtines.crypt.cea
# Classes831
Line Coverage95% 1051/110499% 1641/1659
Branch Coverage81% 170/21095% 399/422
Complexity2.3692.146

That includes real test vectors for TEA (and fixing the implementation at last, so it's incompatible with what I used to have for Java, but is real for the first time -- problems with operator precedence and signed vs unsigned shift), as well as things like Safer not implementing the common interface and a few problems with non-zero array offsets and with triple encryption.

Just need to get bignums and ZLib sorted out with tests, and it'll be time for a proper drop.

Tuesday 4 December 2007

Moore at work

The CAST5 algorithm defines a maintenance test:

B.2. Full Maintenance Test

    A maintenance test for CAST-128 has been defined to verify the
    correctness of implementations.  It is defined in pseudo-code as
    follows, where a and b are 128-bit vectors, aL and aR are the
    leftmost and rightmost halves of a, bL and bR are the leftmost and
    rightmost halves of b, and encrypt(d,k) is the encryption in ECB mode
    of block d under key k.

    Initial a = 01 23 45 67 12 34 56 78 23 45 67 89 34 56 78 9A (hex)
    Initial b = 01 23 45 67 12 34 56 78 23 45 67 89 34 56 78 9A (hex)
 
    do 1,000,000 times
    {
        aL = encrypt(aL,b)
        aR = encrypt(aR,b)
        bL = encrypt(bL,a)
        bR = encrypt(bR,a)
    }

 Verify a == EE A9 D0 A2 49 FD 3B A6 B3 43 6F B8 9D 6D CA 92 (hex)
 Verify b == B2 C9 5E B0 0C 31 AD 71 80 AC 05 B8 E8 3D 69 6E (hex)

In my code I had a comment "takes about an hour to run", dating from about the end of 1998. Trying today, I got

  • In shiny new NetBeans 6.0, run from a JUnit 3.8.2 unit test with cobertura coverage instrumentation: 885s
  • With instrumentation disabled: 6s
  • In VS 2005 as a post-build run of NCover 1.5.8 running the ported test as a debug build: 34s
  • In VS 2005 as a post-build run of NCover 1.5.8 running the ported test as a release build: 17s
  • In VS 2005 as a command-line run of the test as a release build (100 lines of output piped to a file, rather than the console): 6s

So, you pay through the nose (~150 times slower) for your branch coverage instrumentation, as opposed to line coverage (~6 times). And both JVM and CLR are seriously faster, even allowing for the maybe 20x faster processor, than what I was running a decade ago.

Oh, yeah, today's unit test…

It passes.

Sunday 2 December 2007

Making a hash of it

Wrote one (meta-)test today. Find every class in com.ravnaandtines.crypt.mda, instantiate through the default constructor, and check that putting in a short sequence of bytes one at a time, as an array or as an array-slice all gave the same results.

On the up-side, this ups coverage:

Packagecom.ravnaandtines.crypt.mda
# Classes8
Line Coverage95% 1051/1104
Branch Coverage81% 170/210
Complexity2.369

and found one bug, in BlockCypherHash, where partial blocks were being lost (I wasn't updating the count of pending bytes when I should).

On the down side, while the test driver was easy to write for J#

MD5 prototype = new MD5(); // an arbitrary class in the right assembly
System.Reflection.Assembly algorithms = prototype.GetType().get_Assembly();
System.Type[] types = algorithms.GetTypes();
System.Type mda = algorithms.GetType("com.ravnaandtines.crypt.mda.MDA");

for (int k = 0; k < types.length; ++k )
{
 final System.Type underTest = types[k];
 if (!mda.IsAssignableFrom(underTest))
 {
  continue;
 }
…

but for Java, it requires some magic to do similar blind enumeration of classes. The following fudge (based off this one, with the fix noted below) gets the all loaded classes in a named package (stuck with Java 1.1 for convenience when cross compiling)

    public static java.util.Vector find(final String pckgname) {
        // Translate the package name into an absolute path
        String name = pckgname;
        if (!name.startsWith("/")) {
            name = "/" + name;
        }
        name = name.replace('.', '/');
        // Get a File object for the package
        final java.net.URL url = // get an instance of the current class to hook from
                new CryptTestUtils().getClass().getResource(name);

        // NOTE -- this url will put %20 for spaces in your path 
        // It needs to be cleared up before creating a File object
        final String dirpath = java.net.URLDecoder.decode(// 1.2 class easily stubbed for J#
                url.getFile()); 

        final java.io.File directory = new java.io.File(dirpath);
        final java.util.Vector contents = new java.util.Vector();
        if (directory.exists()) {
            // Get the list of the files contained in the package
            final String[] files = directory.list();
            for (int i = 0; i < files.length; i++) {

                // we are only interested in .class files
                if (files[i].endsWith(".class")) {
                    // removes the .class extension
                    final String classname = 
                            files[i].substring(0, files[i].length() - 6);
                    try {
                        // Try to create an instance of the object
                        final Object obj = 
                                Class.forName(pckgname + "." + classname).newInstance();
                        contents.add(obj.getClass()); // save off the class
                    } catch (ClassNotFoundException cnfex) { 
                    } catch (InstantiationException iex) { 
                    // We try to instantiate an interface
                    // or an object that does not have a 
                    // default constructor
                    } catch (IllegalAccessException iaex) { 
                    // The class is not public
                    }
                }
            }
        }
            return contents;
    }

and then we do

final java.util.Vector classes = 
                com.ravnaandtines.crypt.CryptTestUtils.find(
                "com.ravnaandtines.crypt.mda"
                );       

for (final java.util.Enumeration iter = classes.elements(); iter.hasMoreElements();)
{
      final Class underTest = (Class) iter.nextElement();

      final Object test = underTest.newInstance();
      if(!(test instanceof com.ravnaandtines.crypt.mda.MDA ))
      {
          continue;
      }
…

So, that's all the hash algorithms done. Next is the similarly mindless conversion of the crypto algorithm test vectors (and a similar meta-test). Then turning the JZlib system tests into a set of unit tests, and porting something like the Ruby Bignum unit tests. Maybe by the turn of the year, I shall actually have a platform based on existing work that can be taken forward.

Saturday 1 December 2007

Tinkering away

The standard test vectors for the various algorithms do a good job of shaking down the code for an implementation.

Packagecom.ravnaandtines.crypt.mda
# Classes8
Line Coverage93% 1024/1103
Branch Coverage76% 159/210
Complexity2.369

Coverage would be higher if there weren't a still few CTC-related entry-points, or code-paths like compensating for a broken Haval implementation in a PGP extension (EPB)…