10 Minute refactor

After a long weekend in the garden...

module Com_ravnaandtines
  module Zlib
    class Adler32
      # largest prime smaller than 65536
      @@ADLER_BASE = 65521
    
      def initialize
        @value = 1
      end
      
      def update(buffer)
        if not buffer
          return @value
        end
        ## build up the checksum
        low = @value & 0xffff
        high = (@value >> 16) & 0xffff
        buffer.each do |x| 
          low += (x.to_i & 0xff)
          high += low
        end

        ## collapse into modular parts
        low %= @@ADLER_BASE
        high %= @@ADLER_BASE
        @value = (high << 16) | low      
      end
      
      def reset
        @value = 1
      end
      
    end
  end
end

and tweak the tests thus:

  def basic_test_engine(seq, expected)
    ## string to array
    ## otherwise expect an each method to yield integers
    if seq.respond_to? :unpack
      seq = seq.unpack("C*")
    end
    adler = Com_ravnaandtines::Zlib::Adler32.new()
    a = adler.update(seq)
    assert_equal(expected, a)
  end

  def test_boundary
    basic_test_engine(nil, 1)  
  end

That feels better. No special cases, no leakage of state. Looking at JZlib, most of the work will be in doing the encapsulation properly.

Next up, though, when time and energy combine, more on the IronPython FTP client.

Selection Factors

I had started porting CTClib to managed C++ to do an Iron<something> piecemeal conversion. But at the moment IronPython doesn't have a good deployment story, IronRuby isn't all there; and I still much prefer Java's UI model to WinForms or WPF… So what about JRuby -- familiar UI style and run from jar -- then?

The stumbling block as always is PhilZ's choice of deflate with a 2^13 bit window (as opposed to Zlib's fixed 2^15 bit window size for Ruby or java.util.zip) for compression. This was where I paused my first Java port -- JZlib which could do the job has appeared since I last looked, around the turn of the century. Even so, for sake of portability I've decided to bite the bullet, and do a minimal zlib/deflate implementation in Ruby for the purpose, to do something meaningful with the language, using JZlib as a guide. Inflate can, of course, have the larger window (as in current CTClib-C builds), and use the built-in version, be it the C version from native Ruby or JRuby's java.util.zip wrapper.

So, the easy bit first -- Adler 32 checksum…

module Com_ravnaandtines
  module Zlib
    # largest prime smaller than 65536
    ADLER_BASE = 65521
    # Adler32 checksum : takes a seed (usually 1), and a byte sequence, 
    # returns 32-bit integer
    def adler32(adler, buffer)
      if not buffer
        return 1
      end
      
      ## string to array
      ## otherwise expect an each method to yield integers
      if buffer.respond_to? :unpack
        buffer = buffer.unpack("C*")
      end
      
      ## build up the checksum
      low = adler & 0xffff
      high = (adler >> 16) & 0xffff
      buffer.each do |x| 
        low += (x.to_i & 0xff)
        high += low
      end

      ## collapse into modular parts
      low %= ADLER_BASE
      high %= ADLER_BASE
      (high << 16) | low
    end
  end
end

Test vectors for the unit tests had to be scavenged from the internet:

require 'test/unit'
require 'tinesware_zlib'
include Com_ravnaandtines::Zlib

class AdlerTest < Test::Unit::TestCase

  def basic_test_engine(seq, expected)
    a = Com_ravnaandtines::Zlib.adler32(1, seq)
    assert_equal(expected, a)
  end

  def test_boundary
    a = Com_ravnaandtines::Zlib.adler32(0, nil)
    assert_equal(1, a)    
  end

  def test_simple_0
    basic_test_engine("Mark Adler", 0x13070394)
  end
  
  def test_simple_1
    basic_test_engine("\x00\x01\x02\x03", 0x000e0007)
  end
 
  def test_simple_2
    basic_test_engine("\x00\x01\x02\x03\x04\x05\x06\x07",  0x005c001d)
  end

  def test_simple_3
    basic_test_engine("\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f", 0x02b80079)
  end

  def test_simple_4
    basic_test_engine("\x41\x41\x41\x41", 0x028e0105)
  end

  def test_simple_5
    basic_test_engine("\x42\x42\x42\x42\x42\x42\x42\x42", 0x09500211)
  end

  def test_simple_6
    basic_test_engine("\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43", 0x23a80431)
  end

  class Vector #arrays filled with value = (byte) index
    def initialize(size)
      @size = size
    end
    def each
      index = 0
      while index < @size
        yield index & 0xff
        index += 1
      end
    end
  end

  def test_total
    index = 0
    results = [  486795068,
                1525910894,
                3543032800,
                2483946130,
                4150712693,
                3878123687,
                3650897945,
                1682829244,
                1842395054,
                 460416992,
                3287492690,
                 479453429,
                3960773095,
                2008242969,
                4130540683,
                1021367854,
                4065361952,
                2081116754,
                4033606837,
                1162071911 ]

    
    while index < 20
      size = 5*index + 1
      xx = Vector.new(1000*size)
      basic_test_engine(xx, results[index])
      index += 1
    end
  end

end

That was one evening. Now, how long will the rest of it take?

A wet weekend

The FTP client code is pretty much there, though I still need to write code to drain headers for the HTTP proxy case. And then the rest is pretty much just plumbing it all together -- of which there will be a fair volume.

Today's screen capture is how the UI will look during a transfer -- a float-up panel with a live throbber in front of the disabled directory views.

After a break

Last week I was at Recombination so nothing done then.

Now I've started to wire up the logon dialog. This included a bit of shuffling of the main UI, and preparing to split the DirectoryPanel into UI (repeated) and Directory Model classes (local and FTP); and starting the process of writing an FTP client against the .Net socket APIs.

Writing the logon dialog exposed one Mono quirk. The UseSystemPasswordChar property on the text-box doesn't seem to do anything. I resorted to self.pwdTB.PasswordChar = u'\u2022' as a portable solution.

Also, you can use the Data Protection API for more safely storing passwords:

import clr
clr.AddReference("System.Security")
from System.Security.Cryptography import *
import System.Text
import System

…

  def getSecureValue(self, section, key, default):
    raw = self.getValue(section, key, None)
    if raw== None:
      return default
    try:
      array = System.Convert.FromBase64String(raw)
      chars = ProtectedData.Unprotect(array,
        None, DataProtectionScope.CurrentUser)
      result = System.Text.Encoding.UTF8.GetString(chars)
      for i in range(chars.Length):
        chars[i] = 0
      return result
    except System.Exception, ex:
      print ex.ToString()
      return default

  def setSecureValue(self, section, key, value):
    bytes = System.Text.Encoding.UTF8.GetBytes(value)
    try:
      safed = ProtectedData.Protect(bytes,
        None, DataProtectionScope.CurrentUser)
      for i in range(bytes.Length):
        bytes[i] = 0
      string = System.Convert.ToBase64String(safed)
      for i in range(safed.Length):
        safed[i] = 0
      self.setValue(section, key, string)
    except System.Exception, ex:
      print ex.ToString()

which stores the password as an encrypted Base64 blob -- and works with Mono on Win32 as well. The weak point is the password kept in memory as a string (immutable) -- if you can use it as char array, you can wipe that when done.

Meets minimum

After a bit more fiddling with P/Invoke code, I have hooked into the Win32 Recycle or Delete APIs; though at the moment multiple file delete is done one file at a time. But the directory pane is roughly meets-minimum standard.

Next I can get to the object of the exercise, making an FTP connection and getting the remote directory information back to its own version of DirectoryPane. This will be an interesting combination of BackgroundWorker and asynchronous I/O, with pauses built in so as to throttle the uplink to what my router is prepared to handle.

And then I get to tweak it -- like switching to WPF if available -- and port it -- Mono on Debian working around the P/Invoke stuff.

Directory pane almost there

Some of the problems I was cursing Mono for turn out to be IronPython 1.x-isms -- mainly the insisting on having Images rather than Icons. Mono has other peculiarities, like in how it actually lays out widgets (not that .Net is much better) at runtime, so for the moment I'm forcing the issue by trapping any size change to the directory pane so that it fits inside the viewport.

Apart from wiring up the delete context menu, the directory pane is done.

More Mono

A lot of work done for the benefit of Mono, redoing how I wrap files to use os.path rather than .Net directly; and rewriting the P/Invoke layer from scratch (and from pinvoke.net) --including finding that Bitmap.FromHicon() is the simplest way of getting the transparent pixels roughly honoured as well as feeding that system what it wants.

I said that the DirectoryPane would be the worst of it, and it's doing its best to prove me right.