Tinesware

Infrastructure Helper 1.3

2010-11-19T20:31:00.003Z

Mainly a bug-fix release. The big bug being that the release distro contained the release build of Tinesware.Recorder.dll which had in-lined the use of coverage file location property so that the overwrite of the value was having no effect.

Also fixed the usage output, and the case of a read-only strong-naming key, and.

The new release also includes some more interoperation with PartCover (using resource /trunk/Infrastructure/Scripts/Partcover.xsl to perform the conversion to NCover style output), and an FxCop rule to enforce proper re-throwing of exceptions (so as not to lose stack information); and adds exemption reasons to the XSLT generated reports.

PartCover work is still on-going. It is working albeit with some inefficiencies with determining which overloaded method coverage refers to when overloads are encountered, but the big bug was the reason for a new release.

Infrastructure Helper 1.2

2010-06-06T11:23:00.004+01:00

The new release includes

A coverage tool that works by rewriting IL (so should be compatible with both the 2.0 and 4.0 .net runtimes) based on a slightly less feature-rich C# tool; this is not nCover, though it generates compatible output, and so I've not attempted to make it look like nCover at the command line.
A class-level coverage exemption attribute to allow exemptions to be applied to functions that cannot just be attributed as they stand -- a typical case would be a function that is defined as the first or last expression inside an F# function, and any funs it contains; and also preventing the coverage reporting from throwing when encountering these functions.
At the source code level, now uses F# 2.0.0.0 for .net 3.5; and allows tool locations to be specified via environment variables.
Includes Mono.Cecil 0.9.2, Mono.Options, and some other Mono tools for use in the coverage instrumentation.

The new coverage tool excludes all compiler-generated lines, so may report slightly different results to the old nCover 1.5.8; however the differences will be minor where they exist.

Looking ahead, I see the following likely features for the post 1.2 development:

extended XSLT to collate reports of the reasons given for code being excluded
an nCoverExplorer like tool that distinguishes exemptions from coverage
More code comments -- maybe even proper help files
And of course bug fixing and refactoring as needed.

PartCover/ nCover 1.5.8 support is dead; but if I ever find a FOSS .net 4 coverage tool that uses the profiling APIs I would consider using that.

Later (13-Jul-10) : the current updated version of PartCover does meet this criterion, so I shall be working to use its output format as well.

EasyWPF 0.3.93

2010-03-28T23:19:00.001+01:00

Having been made aware of the Python EasyGui project while reviewing "Hello World", I thought that maybe there should be an equivalent version for IronPython. So here is one (with just a minimal amount of fudging in places):

"""
@version: 0.3.93(2010-Mar-28)
@note:
EasyGui provides an easy-to-use interface for simple GUI interaction
with a user.  It does not require the programmer to know anything about
tkinter, frames, widgets, callbacks or lambda.  All GUI interactions are
invoked by simple function calls that return results.

EasyWPF is an API-compatible implementation for IronPython written against 
the Windows Presentation Foundation

Note that EasyWPF requires .net release 3.0 or greater.

EasyGui (http://easygui.sourceforge.net)
is licensed under the Creative Commons Attribution 2.0 License
http://creativecommons.org/licenses/by/2.0/

You are free to copy, distribute, and display the work, and to make derivative 
works (including translations). If you do, you must give the original author 
(http://easygui.sourceforge.net) credit. The author specifically permits (and 
encourages) teachers to post, reproduce, and distribute some or all of this 
material for use in their classes or by their students.

EasyWPF is licensed under the same terms.
"""
egversion = __doc__.split()[1]

__all__ = ['ynbox'
    , 'ccbox'
    , 'boolbox'
    , 'indexbox'
    , 'msgbox'
    , 'buttonbox'
    , 'integerbox'
    , 'multenterbox'
    , 'enterbox'
    , 'exceptionbox'
    , 'choicebox'
    , 'codebox'
    , 'textbox'
    , 'diropenbox'
    , 'fileopenbox'
    , 'filesavebox'
    , 'passwordbox'
    , 'multpasswordbox'
    , 'multchoicebox'
    , 'abouteasywpf'
    , 'egversion'
    , 'egdemo'
    , 'EgStore'
    ]

import sys
#-------------------------------------------------------------------
# Console I/O helpers
#-------------------------------------------------------------------

def write(*args):
    args = [str(arg) for arg in args]
    args = " ".join(args)
    sys.stdout.write(args)
    
def writeln(*args):
    write(*args)
    sys.stdout.write("\n")
    
def dq(s):
    return '"%s"' % s

#--------------------------------------------------
# check python version and take appropriate action
#--------------------------------------------------

if not ('subversion' in dir(sys)) or not (sys.subversion[0] == 'IronPython'):
    stars = "*"*75  
    writeln("""\n\n\n""" + stars + """
You are running Python version: 
\t\t"""+sys.version+"""
You must be using IronPython 2.6 or later to use EasyWPF.
Terminating.
""" + stars + """\n\n\n""")
    sys.exit(0)  

import os, pickle,traceback

import  clr
clr.AddReference('PresentationFramework')
clr.AddReference('PresentationCore')
clr.AddReference('System.Xml')
clr.AddReference('System.Windows.Forms')
from System import String, StringComparison, Console, Environment, Uri, Char
from System.Xml import XmlTextReader
from System.IO import StringReader, StringWriter, FileInfo, Path, BinaryReader, File
import System.Windows
from System.Windows import Thickness
from System.Windows.Markup import XamlReader
import System.Windows.Forms
import System.Windows.Controls
from Microsoft.Win32 import OpenFileDialog, SaveFileDialog
from System.Windows.Media import FontFamily

Vista = Environment.OSVersion.Version.Major >= 6

#-------------------------------------------------------------------
# Vista glass support
#-------------------------------------------------------------------

if Vista:
  from ctypes import *
  from System.Windows.Interop import WindowInteropHelper, HwndSource
  from System import IntPtr
  from System.Windows.Media import Brushes, Colors, Brush
  dwmapi = windll.dwmapi
  class __MARGINS(Structure):
    _fields_ = [("Left", c_int), ("Right", c_int), ("Top", c_int), ("Bottom", c_int)]
    
  def ExtendGlassFrame(window, margin):
    val = c_int()
    result = dwmapi.DwmIsCompositionEnabled(byref(val))
    success = result == 0 and not (val == 0)
    if not success:
      return False
      
    hwnd = WindowInteropHelper(window).Handle
    if (hwnd == IntPtr.Zero):
      return False
        
    window.Background = Brushes.Transparent
    HwndSource.FromHwnd(hwnd).CompositionTarget.BackgroundColor = Colors.Transparent
    margins = __MARGINS(int(margin.Left), 
          int(margin.Right), 
          int(margin.Top), 
          int(margin.Bottom))

    result = dwmapi.DwmExtendFrameIntoClientArea(hwnd, byref(margins))
    return result == 0
    
  class __GlassHelper:
    def __init__(self, w, t):
      self.window = w
      self.thickness = t
    
    def ApplyGlass(self):
      val = ExtendGlassFrame(self.window, self.thickness)
      hwnd = WindowInteropHelper(self.window).Handle
      HwndSource.FromHwnd(hwnd).AddHook(self.__wndproc)
    
    def __wndproc(self, hwnd, msg, wparam, lparam, handled):
      if msg == 0x031E: ## composition changed
        handled.Value = True ## set ref argument
        ExtendGlassFrame(self.window, self.thickness)
      return IntPtr.Zero
else:
  from System.Windows import SystemColors
    
#-------------------------------------------------------------------
# Xaml support
#-------------------------------------------------------------------

def LoadXaml(xaml):
    sr = StringReader(xaml)
    xr = XmlTextReader(sr)
    return XamlReader.Load(xr)
    
from System.IO.Compression import GZipStream, CompressionMode
from System.IO import MemoryStream
from System.Windows.Media.Imaging import BitmapImage, BitmapFrame

#-------------------------------------------------------------------
# unpacking my icon from the string
#-------------------------------------------------------------------

__icon64 = ('H4sIAAAAAAAEAO29B2AcSZYlJi9tynt/SvVK1+B0oQiAYBMk2JBAEOzBiM3mkuwdaUcjKasqgcplVmV'+
'dZhZAzO2dvPfee++999577733ujudTif33/8/XGZkAWz2zkrayZ4hgKrIHz9+fB8/In6NX+PX/DV+rV/'+
'jN/vNfrNf49f4NX6dX2Pr1/w1fo1fSL/xn7/Gb/drzH/tX+PXeEGfbeEz+n/6a6D9r4Mvf40/6Ndwz2+'+
'm/5fnD5Iv/6A/SH7gP3xE//xD/9A/RJ/83/y/X+P//r/lB/7DR/i78wi831R+oR//E/9NP39T+YB//k+'+
'/xm/6P/2m/xMA0b//EwPiFvLz/8Z7/ze+05//94bn1/jAZ+vX8On028mHv/ZA4/d+/q//6//6P//P//P'+
'GZvT8l//lf/k3/81/89/wN/wNf9af9Wf9jX/j3/gX/oV/If0in/wD/8A/8C/8C//CP/lP/pP/+D/+j//'+
'L//K/bF/57/67/+7P+/P+vD/uj/vj/ol/4p/4a//av/YP/8P/8D/jz/gz/qQ/6U/6Y/6YP4Y+/Fv+lr+'+
'Feve7+J/+p//pT/1T/9Q/5A/5Q/6UP+VP+cv/8r/8L/vL/rL/+D/+j/+Zf+afoV/+yr/yr6RX/uf/+X/'+
'22//z//w/T/3+w//wP/wP/oP/4L/2r/1rf/Ff/BdT+z/9T//T/9a/9W/9q/6qv+rv+Xv+nn/v3/v3/rf'+
'/7X+z7QnUv/qv/qv0yy/7Zb+MkL++vv7P//P//Jf8kl/yl/6lf+l//V//1//Vf/Vf/Uv/0r/0b//b/7Z'+
'gRf/+U//UP/Xv//v/vrz7B//BfzDBpzaE/F/xV/wV8uEv/+W//G/6m/4mIgK9Tu2JJoS59EgjpSHQh3/'+
'H3/F3/NP/9D9tcaCWf9ff9XcRYvT7v/vv/rvU46/8lb9ScCPq0S/0yb/z7/w7tv3QQyT9V/6Vf4V++Tf'+
'+jX+D8Pwf/of/YXN7mhciEf3yH/6H/+Ef+Uf+kX/1X/1Xb2hMiP11f91fRzS0n/yv/+v/Otz81/hf/pf'+
'/hUbUmZ0f8gNJ+xchbSQbf9Cvr/+n3/8k0jW/3c6v8Wv8Bge/xq/xR/8ev8av8at+F/n///wf0v//b3k'+
'PUvr/APisn8C2BAAA')
from System import Convert

__icoFile = Path.ChangeExtension(Path.GetTempFileName(), '.ico')
__iconzip = Convert.FromBase64String(__icon64)

def __GetIcon():
  mem = MemoryStream(__iconzip)
  unzip = GZipStream(mem, CompressionMode.Decompress, False)
  image = BitmapImage()
  image.BeginInit()
  image.StreamSource = unzip
  image.EndInit()
  return image

def __WriteIcon():
  mem = MemoryStream(__iconzip)
  unzip = GZipStream(mem, CompressionMode.Decompress, False)
  reader = BinaryReader(unzip)
  data = reader.ReadBytes(1206)
  reader.Close()
  unzip.Close()
  File.WriteAllBytes(__icoFile, data)

__WriteIcon()

#-------------------------------------------------------------------
# Making minimal hand-drawn title bar
#-------------------------------------------------------------------

def __FurnishWindow(title):
    boxRoot.Title = title
    boxRoot.FindName('title').Content = title
    boxRoot.FindName('icon').Source = __GetIcon()
    
    uri = Uri(__icoFile)
    boxRoot.Icon = BitmapFrame.Create(uri)
    
    boxRoot.MouseLeftButtonDown  += lambda s,e : boxRoot.DragMove()
      
    if Vista:
      boxRoot.AllowsTransparency = False
      boxRoot.Background = Brushes.Transparent
      boxRoot.FindName('title').Background = Brushes.Transparent
      thickness = Thickness(0.0, float(boxRoot.FindName('title').Height), 0.0, 0.0)
      helper = __GlassHelper(boxRoot, thickness)
      boxRoot.SourceInitialized += lambda s,e : helper.ApplyGlass()
    else:
      def activeColor():
        boxRoot.FindName('title').Background = SystemColors.ActiveBorderBrush  
        boxRoot.FindName('banner').Background = SystemColors.ActiveBorderBrush  
        boxRoot.FindName('title').Foreground = SystemColors.ActiveCaptionTextBrush
      def inactiveColor():
        boxRoot.FindName('title').Background = SystemColors.InactiveBorderBrush  
        boxRoot.FindName('banner').Background = SystemColors.InactiveBorderBrush  
        boxRoot.FindName('title').Foreground = SystemColors.InactiveCaptionTextBrush
      
      boxRoot.Activated += lambda s,e: activeColor()
      boxRoot.Deactivated += lambda s,e: inactiveColor()
      activeColor()    

#-------------------------------------------------------------------
# repeated code in EasyGUI -- for compatibility, still limit to GIF support
#-------------------------------------------------------------------

def __ValidateImage(image, msg):
    if image:
        image = os.path.normpath(image)
        junk,ext = os.path.splitext(image)
        if ext.lower() == ".gif":
            if os.path.exists(image):
                pass
            else:
                msg += ImageErrorMsg % (image, "Image file not found.")
                image = None
        else:
            msg += ImageErrorMsg % (image, "Image file is not a .gif file.")
            image = None
    return (image, msg)


# Initialize some global variables that will be reset later
__choiceboxMultipleSelect = None
__widgetTexts = None
__replyButtonText = None
__choiceboxResults = None
__firstWidget = None
__enterboxText = None
__enterboxDefaultText=""
__multenterboxText = ""
choiceboxChoices = None
choiceboxWidget = None
entryWidget = None
boxRoot = None
ImageErrorMsg = (
    "\n\n---------------------------------------------\n"
    "Error: %s\n%s")
root = None    
#-------------------------------------------------------------------
# various boxes built on top of the basic buttonbox
#-----------------------------------------------------------------------

#-----------------------------------------------------------------------
# ynbox
#-----------------------------------------------------------------------
def ynbox(msg="Shall I continue?"
    , title=" "
    , choices=("Yes", "No")
    , image=None
    ):
    """
    Display a msgbox with choices of Yes and No.

    The default is "Yes".

    The returned value is calculated this way::
        if the first choice ("Yes") is chosen, or if the dialog is cancelled:
            return 1
        else:
            return 0

    If invoked without a msg argument, displays a generic request for a confirmation
    that the user wishes to continue.  So it can be used this way::
        if ynbox(): pass # continue
        else: sys.exit(0)  # exit the program

    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg choices: a list or tuple of the choices to be displayed
    """
    return boolbox(msg, title, choices, image=image)


#-----------------------------------------------------------------------
# ccbox
#-----------------------------------------------------------------------
def ccbox(msg="Shall I continue?"
    , title=" "
    , choices=("Continue", "Cancel")
    , image=None
    ):
    """
    Display a msgbox with choices of Continue and Cancel.

    The default is "Continue".

    The returned value is calculated this way::
        if the first choice ("Continue") is chosen, or if the dialog is cancelled:
            return 1
        else:
            return 0

    If invoked without a msg argument, displays a generic request for a confirmation
    that the user wishes to continue.  So it can be used this way::

        if ccbox():
            pass # continue
        else:
            sys.exit(0)  # exit the program

    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg choices: a list or tuple of the choices to be displayed
    """
    return boolbox(msg, title, choices, image=image)


#-----------------------------------------------------------------------
# boolbox
#-----------------------------------------------------------------------
def boolbox(msg="Shall I continue?"
    , title=" "
    , choices=("Yes","No")
    , image=None
    ):
    """
    Display a boolean msgbox.

    The default is the first choice.

    The returned value is calculated this way::
        if the first choice is chosen, or if the dialog is cancelled:
            returns 1
        else:
            returns 0
    """
    reply = buttonbox(msg=msg, choices=choices, title=title, image=image)
    if reply == choices[0]: return 1
    else: return 0


#-----------------------------------------------------------------------
# indexbox
#-----------------------------------------------------------------------
def indexbox(msg="Shall I continue?"
    , title=" "
    , choices=("Yes","No")
    , image=None
    ):
    """
    Display a buttonbox with the specified choices.
    Return the index of the choice selected.
    """
    reply = buttonbox(msg=msg, choices=choices, title=title, image=image)
    index = -1
    for choice in choices:
        index = index + 1
        if reply == choice: return index
    raise AssertionError(
        "There is a program logic error in the EasyGui code for indexbox.")


#-----------------------------------------------------------------------
# msgbox
#-----------------------------------------------------------------------
def msgbox(msg="(Your message goes here)", title=" ", ok_button="OK",image=None,root=None):
    """
    Display a messagebox
    """
    if type(ok_button) != type("OK"):
        raise AssertionError("The 'ok_button' argument to msgbox must be a string.")

    return buttonbox(msg=msg, title=title, choices=[ok_button], image=image,root=root)

#-------------------------------------------------------------------
# buttonbox
#-------------------------------------------------------------------
def buttonbox(msg="",title=" "
    ,choices=("Button1", "Button2", "Button3")
    , image=None
    , root=None
    ):
    """
    Display a msg, a title, and a set of buttons.
    The buttons are defined by the members of the choices list.
    Return the text of the button that the user selected.

    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg choices: a list or tuple of the choices to be displayed
    """
    global boxRoot, __replyButtonText, __widgetTexts, buttonsFrame
    
    # Initialize __replyButtonText to the first choice.
    # This is what will be used if the window is closed by the close button.
    __replyButtonText = choices[0]
    
    xaml = '''<Window
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window2" SizeToContent="WidthAndHeight" MinWidth="400" MinHeight="150"  WindowStyle="None" AllowsTransparency="True">
    <Grid>
        <Grid Name="banner">
            <Image Name="icon" Stretch="Fill" HorizontalAlignment="Left" VerticalAlignment="Top" Width="20" Height="20" Margin="4,4,0,0" />
            <Label Height="28" Name="title" VerticalAlignment="Top" Background="WhiteSmoke" Margin="28,0,0,0">Label</Label>
        </Grid>
        <DockPanel Margin="0,28,0,0" Name="backdrop">
          <StackPanel Name="buttons" DockPanel.Dock="Bottom" HorizontalAlignment="Center" Orientation="Horizontal">
            <!-- Button Height="23" Name="button1" Width="75" Margin="2">Button</Button -->
          </StackPanel>

          <StackPanel  Name="body" VerticalAlignment="Top">
            <TextBlock Name="message" VerticalAlignment="Top" Margin="5" />
            <Image Name="image" Stretch="None" />
          </StackPanel>
        </DockPanel>
    </Grid>
</Window>'''
    boxRoot = LoadXaml(xaml)
    if root:
      boxRoot.Owner = root
      boxRoot.Top = root.Top
      boxRoot.Left = root.Left
      root.Visibility = System.Windows.Visibility.Hidden
    
    __FurnishWindow(title)
    
    if Vista:
      brush = Brushes.White
    else:
      brush = SystemColors.WindowBrush
    
    for name in ('body', 'buttons', 'backdrop'):
       boxRoot.FindName(name).Background = brush

    (image, msg) = __ValidateImage(image, msg)
    boxRoot.FindName('message').Text = msg

    if image:
      bi = BitmapImage()
      bi.BeginInit()
      image = FileInfo(image).FullName
      bi.UriSource = Uri(image)
      bi.EndInit()
      
      pane = boxRoot.FindName('image')
      pane.BeginInit()
      pane.Source = bi
      pane.EndInit()
    else:
      boxRoot.FindName('image').Visibility = System.Windows.Visibility.Collapsed
      
    if len(choices) < 1:
      choices = ('Ok')
      
    first = None
    for choice in choices:
      x = System.Windows.Controls.Button()
      x.Content = choice
      x.Click += __buttonEvent
      x.MinHeight=23 
      x.MinWidth=75 
      x.Margin= Thickness(2)
      x.HorizontalAlignment = System.Windows.HorizontalAlignment.Center
      boxRoot.FindName('buttons').Children.Add(x)
      if not first:
        first = x
        
    first.Focus()
    boxRoot.ShowDialog()
    
    if root:
      root.Visibility = System.Windows.Visibility.Visible
    return __replyButtonText

#-------------------------------------------------------------------
# integerbox
#-------------------------------------------------------------------
def integerbox(msg=""
    , title=" "
    , default=""
    , argLowerBound=0
    , argUpperBound=99
    , image = None
    , root  = None
    ):
    """
    Show a box in which a user can enter an integer.

    In addition to arguments for msg and title, this function accepts
    integer arguments for default_value, lowerbound, and upperbound.

    The default_value argument may be None.

    When the user enters some text, the text is checked to verify
    that it can be converted to an integer between the lowerbound and upperbound.

    If it can be, the integer (not the text) is returned.

    If it cannot, then an error msg is displayed, and the integerbox is
    redisplayed.

    If the user cancels the operation, None is returned.
    """
    if default != "":
        if type(default) != type(1):
            raise AssertionError(
                "integerbox received a non-integer value for "
                + "default of " + dq(str(default)) , "Error")

    if type(argLowerBound) != type(1):
        raise AssertionError(
            "integerbox received a non-integer value for "
            + "argLowerBound of " + dq(str(argLowerBound)) , "Error")

    if type(argUpperBound) != type(1):
        raise AssertionError(
            "integerbox received a non-integer value for "
            + "argUpperBound of " + dq(str(argUpperBound)) , "Error")

    if msg == "":
        msg = ("Enter an integer between " + str(argLowerBound)
            + " and "
            + str(argUpperBound)
            )

    while 1:
        reply = enterbox(msg, title, str(default), image=image, root=root)
        if reply == None: return None

        try:
            reply = int(reply)
        except:
            msgbox ("The value that you entered:\n\t%s\nis not an integer." % dq(str(reply))
                    , "Error")
            continue

        if reply < argLowerBound:
            msgbox ("The value that you entered is less than the lower bound of "
                + str(argLowerBound) + ".", "Error")
            continue

        if reply > argUpperBound:
            msgbox ("The value that you entered is greater than the upper bound of "
                + str(argUpperBound) + ".", "Error")
            continue

        # reply has passed all validation checks.
        # It is an integer between the specified bounds.
        return reply

#-------------------------------------------------------------------
# multenterbox
#-------------------------------------------------------------------
def multenterbox(msg="Fill in values for the fields."
    , title=" "
    , fields=()
    , values=()
    ):
    r"""
    Show screen with multiple data entry fields.
    
    If there are fewer values than names, the list of values is padded with
    empty strings until the number of values is the same as the number of names.
    
    If there are more values than names, the list of values
    is truncated so that there are as many values as names.
    
    Returns a list of the values of the fields,
    or None if the user cancels the operation.
    
    Here is some example code, that shows how values returned from
    multenterbox can be checked for validity before they are accepted::
        ----------------------------------------------------------------------
        msg = "Enter your personal information"
        title = "Credit Card Application"
        fieldNames = ["Name","Street Address","City","State","ZipCode"]
        fieldValues = []  # we start with blanks for the values
        fieldValues = multenterbox(msg,title, fieldNames)
    
        # make sure that none of the fields was left blank
        while 1:
            if fieldValues == None: break
            errmsg = ""
            for i in range(len(fieldNames)):
                if fieldValues[i].strip() == "":
                    errmsg += ('"%s" is a required field.\n\n' % fieldNames[i])
            if errmsg == "":
                break # no problems found
            fieldValues = multenterbox(errmsg, title, fieldNames, fieldValues)
        
        writeln("Reply was: %s" % str(fieldValues))
        ----------------------------------------------------------------------
    
    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg fields: a list of fieldnames.
    @arg values:  a list of field values
    """
    return __multfillablebox(msg,title,fields,values,None)


#-----------------------------------------------------------------------
# multpasswordbox
#-----------------------------------------------------------------------
def multpasswordbox(msg="Fill in values for the fields."
    , title=" "
    , fields=tuple()
    ,values=tuple()
    ):
    r"""
    Same interface as multenterbox.  But in multpassword box,
    the last of the fields is assumed to be a password, and
    is masked with asterisks.

    Example
    =======

    Here is some example code, that shows how values returned from
    multpasswordbox can be checked for validity before they are accepted::
        msg = "Enter logon information"
        title = "Demo of multpasswordbox"
        fieldNames = ["Server ID", "User ID", "Password"]
        fieldValues = []  # we start with blanks for the values
        fieldValues = multpasswordbox(msg,title, fieldNames)

        # make sure that none of the fields was left blank
        while 1:
            if fieldValues == None: break
            errmsg = ""
            for i in range(len(fieldNames)):
                if fieldValues[i].strip() == "":
                    errmsg = errmsg + ('"%s" is a required field.\n\n' % fieldNames[i])
                if errmsg == "": break # no problems found
            fieldValues = multpasswordbox(errmsg, title, fieldNames, fieldValues)
        
        writeln("Reply was: %s" % str(fieldValues))
    """
    return __multfillablebox(msg,title,fields,values,"*")

#-----------------------------------------------------------------------
# __multfillablebox
#-----------------------------------------------------------------------
def __multfillablebox(msg="Fill in values for the fields."
    , title=" "
    , fields=()
    , values=()
    , mask = None
    ):
    global boxRoot, __multenterboxText, __multenterboxDefaultText, cancelButton, entryWidget, okButton

    choices = ["OK", "Cancel"]
    if len(fields) == 0: return None

    fields = list(fields[:])  # convert possible tuples to a list
    values = list(values[:])  # convert possible tuples to a list

    if   len(values) == len(fields): pass
    elif len(values) >  len(fields):
        fields = fields[0:len(values)]
    else:
        while len(values) < len(fields):
            values.append("")
            
    xaml = '''<Window
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window2" SizeToContent="WidthAndHeight" MinWidth="400" MinHeight="150"  WindowStyle="None" AllowsTransparency="True">
    <Grid>
        <Grid Name="banner">
            <Image Name="icon" Stretch="Fill" HorizontalAlignment="Left" VerticalAlignment="Top" Width="20" Height="20" Margin="4,4,0,0" />
            <Label Height="28" Name="title" VerticalAlignment="Top" Background="WhiteSmoke" Margin="28,0,0,0">Label</Label>
        </Grid>
        <DockPanel Margin="0,28,0,0" Name="backdrop">
          <StackPanel Name="buttons" DockPanel.Dock="Bottom" HorizontalAlignment="Center" Orientation="Horizontal">
          </StackPanel>
          <TextBlock Name="message" DockPanel.Dock="Top" VerticalAlignment="Top" Margin="5" />
          <DockPanel Name="entries">
            <StackPanel Name="fields" DockPanel.Dock="Left" >
            </StackPanel>
            <StackPanel Name="values" >
            </StackPanel>            
          </DockPanel>
        </DockPanel>
    </Grid>
</Window>'''
    boxRoot = LoadXaml(xaml)
    if root:
      boxRoot.Owner = root
      boxRoot.Top = root.Top
      boxRoot.Left = root.Left
      root.Visibility = System.Windows.Visibility.Hidden
    
    __FurnishWindow(title)
    
    if Vista:
      brush = Brushes.White
    else:
      brush = SystemColors.WindowBrush
    
    #for name in ('backdrop',):
    #   boxRoot.FindName(name).Background = brush
    boxRoot.FindName('backdrop').Background = brush
    boxRoot.FindName('message').Text = msg
      
    buttons = []
    for choice in ("Ok", "Cancel"):
      x = System.Windows.Controls.Button()
      x.Content = choice
      x.Height=23 
      x.Width=75 
      x.Margin= Thickness(2)
      boxRoot.FindName('buttons').Children.Add(x)
      buttons.append(x)

    buttons[0].Click += lambda s, e: __multenterboxGetText(mask)
    buttons[0].IsDefault = True
    buttons[1].Click += lambda s, e: __multenterboxCancel()
    buttons[1].IsCancel = True

    global entryWidgets
    entryWidgets = []
    labelWidgets = []
    first = None
    
    lastWidgetIndex = len(fields) - 1
    left = boxRoot.FindName('fields')
    right = boxRoot.FindName('values')

    for widgetIndex in range(len(fields)):
        argFieldName  = fields[widgetIndex]
        argFieldValue = values[widgetIndex]
        
        label = System.Windows.Controls.Label()
        label.Content = argFieldName
        left.Children.Add(label)
        labelWidgets.append(label)
        label.Margin = Thickness(2)
        
        if mask and (widgetIndex == lastWidgetIndex):
            entry = System.Windows.Controls.PasswordBox()
            entry.PasswordChar = mask
            entry.Password = argFieldValue
        else:
            entry = System.Windows.Controls.TextBox()
            entry.Text = argFieldValue
            
        if not first:
            first = entry
            
        entry.Margin = Thickness(2)
        right.Children.Add(entry)
        entryWidgets.append(entry)
        

    # ------------------- time for action! -----------------
    first.Focus()    # put the focus on the entryWidget
    
    boxRoot.SizeChanged += lambda s,e : __multienteradjust(entryWidgets, labelWidgets)

    boxRoot.ShowDialog()
    if root:
      root.Visibility = System.Windows.Visibility.Visible
    return __multenterboxText
    
#-----------------------------------------------------------------------
# __multienteradjust -- fiddle to get the heights to match up
#-----------------------------------------------------------------------
def __multienteradjust(entryWidgets, labelWidgets):
  h = max (entryWidgets[0].ActualHeight, labelWidgets[0].ActualHeight)
  for e in entryWidgets:
    e.Height = h
    e.MinHeight = h
  for l in labelWidgets:
    l.Height = h
    l.MinHeight = h 
 
#-----------------------------------------------------------------------
# __multenterboxGetText
#-----------------------------------------------------------------------
def __multenterboxGetText(mask):
    global __multenterboxText

    __multenterboxText = []
    for entryWidget in entryWidgets[:-1]:
        __multenterboxText.append(entryWidget.Text)
        
    entry = entryWidgets[-1]
    __multenterboxText.append(entry.Password if mask else entry.Text)       
    __finish()


def __multenterboxCancel():
    global __multenterboxText
    __multenterboxText = None
    __finish()


#-------------------------------------------------------------------
# enterbox
#-------------------------------------------------------------------
def enterbox(msg="Enter something."
    , title=" "
    , default=""
    , strip=True
    , image=None
    , root=None
    ):
    """
    Show a box in which a user can enter some text.

    You may optionally specify some default text, which will appear in the
    enterbox when it is displayed.

    Returns the text that the user entered, or None if he cancels the operation.

    By default, enterbox strips its result (i.e. removes leading and trailing
    whitespace).  (If you want it not to strip, use keyword argument: strip=False.)
    This makes it easier to test the results of the call::

        reply = enterbox(....)
        if reply:
            ...
        else:
            ...
    """
    result = __fillablebox(msg, title, default=default, mask=None,image=image,root=root)
    if result and strip:
        result = result.strip()
    return result


def passwordbox(msg="Enter your password."
    , title=" "
    , default=""
    , image=None
    , root=None
    ):
    """
    Show a box in which a user can enter a password.
    The text is masked with asterisks, so the password is not displayed.
    Returns the text that the user entered, or None if he cancels the operation.
    """
    return __fillablebox(msg, title, default, mask="*",image=image,root=root)


def __fillablebox(msg
    , title=""
    , default=""
    , mask=None
    , image=None
    , root=None
    ):
    """
    Show a box in which a user can enter some text.
    You may optionally specify some default text, which will appear in the
    enterbox when it is displayed.
    Returns the text that the user entered, or None if he cancels the operation.
    """

    global boxRoot, __enterboxText, __enterboxDefaultText
    global cancelButton, entryWidget, okButton

    if title == None: title == ""
    if default == None: default = ""
    __enterboxDefaultText = default
    __enterboxText        = __enterboxDefaultText

    xaml = '''<Window
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window2" SizeToContent="WidthAndHeight" MinWidth="400" MinHeight="150"  WindowStyle="None" AllowsTransparency="True">
    <Grid>
        <Grid Name="banner">
            <Image Name="icon" Stretch="Fill" HorizontalAlignment="Left" VerticalAlignment="Top" Width="20" Height="20" Margin="4,4,0,0" />
            <Label Height="28" Name="title" VerticalAlignment="Top" Background="WhiteSmoke" Margin="28,0,0,0">Label</Label>
        </Grid>
        <DockPanel Margin="0,28,0,0" Name="backdrop">
          <StackPanel Name="buttons" DockPanel.Dock="Bottom" HorizontalAlignment="Center" Orientation="Horizontal">
            <!-- Button Height="23" Name="button1" Width="75" Margin="2">Button</Button -->
          </StackPanel>

          <StackPanel  Name="body" VerticalAlignment="Top">
            <TextBlock Name="message" VerticalAlignment="Top" Margin="5" />
            <Image Name="image" Stretch="None" />
          </StackPanel>
        </DockPanel>
    </Grid>
</Window>'''
    boxRoot = LoadXaml(xaml)
    if root:
      boxRoot.Owner = root
      boxRoot.Top = root.Top
      boxRoot.Left = root.Left
      root.Visibility = System.Windows.Visibility.Hidden
      
    __FurnishWindow(title)
    
    if Vista:
      brush = Brushes.White
    else:
      brush = SystemColors.WindowBrush
    
    for name in ('body', 'buttons', 'backdrop'):
       boxRoot.FindName(name).Background = brush

    (image, msg) = __ValidateImage(image, msg)
    boxRoot.FindName('message').Text = msg
    
    if image:
      bi = BitmapImage()
      bi.BeginInit()
      image = FileInfo(image).FullName
      bi.UriSource = Uri(image)
      bi.EndInit()
      
      pane = boxRoot.FindName('image')
      pane.BeginInit()
      pane.Source = bi
      pane.EndInit()
    else:
      boxRoot.FindName('image').Visibility = System.Windows.Visibility.Collapsed

    if mask:
      entry = System.Windows.Controls.PasswordBox()
      entry.PasswordChar = mask
      entry.Password = __enterboxDefaultText
    else:
      entry = System.Windows.Controls.TextBox()
      entry.Text = __enterboxDefaultText
      
    entry.Height = 22
    entry.Margin = Thickness(5)
    boxRoot.FindName('body').Children.Add(entry)
    entry.Focus()
    
    buttons = []
    for choice in ("Ok", "Cancel"):
      x = System.Windows.Controls.Button()
      x.Content = choice
      x.Height=23 
      x.Width=75 
      x.Margin= Thickness(2)
      boxRoot.FindName('buttons').Children.Add(x)
      buttons.append(x)

    buttons[0].Click += lambda s, e: __enterboxGetText(entry, mask)
    buttons[0].IsDefault = True
    buttons[1].Click += lambda s, e: __enterboxCancel()
    buttons[1].IsCancel = True

    boxRoot.ShowDialog()
    if root:
      root.Visibility = System.Windows.Visibility.Visible
    return __enterboxText

def __finish():
    global boxRoot
    boxRoot.Hide()
    boxRoot.Close()
    boxRoot = None  

def __enterboxGetText(entry, mask):
    global __enterboxText
    __enterboxText = entry.Password if mask else entry.Text   
    __finish()


def __enterboxCancel():
    global __enterboxText
    __enterboxText = None
    __finish()

#-------------------------------------------------------------------
# multchoicebox
#-------------------------------------------------------------------
def multchoicebox(msg="Pick as many items as you like."
    , title=" "
    , choices=()
    , **kwargs
    ):
    """
    Present the user with a list of choices.
    allow him to select multiple items and return them in a list.
    if the user doesn't choose anything from the list, return the empty list.
    return None if he cancelled selection.

    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg choices: a list or tuple of the choices to be displayed
    """
    if len(choices) == 0: choices = ["Program logic error - no choices were specified."]

    global __choiceboxMultipleSelect
    __choiceboxMultipleSelect = 1
    return __choicebox(msg, title, choices)


#-----------------------------------------------------------------------
# choicebox
#-----------------------------------------------------------------------
def choicebox(msg="Pick something."
    , title=" "
    , choices=()
    , buttons=()
    ):
    """
    Present the user with a list of choices.
    return the choice that he selects.
    return None if he cancels the selection selection.

    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg choices: a list or tuple of the choices to be displayed
    """
    if len(choices) == 0: choices = ["Program logic error - no choices were specified."]

    global __choiceboxMultipleSelect
    __choiceboxMultipleSelect = 0
    return __choicebox(msg,title,choices,buttons)


#-----------------------------------------------------------------------
# __choicebox
#-----------------------------------------------------------------------
def __choicebox(msg
    , title
    , choices
    ,  buttons=()
    ):
    """
    internal routine to support choicebox() and multchoicebox()
    """
    global boxRoot, __choiceboxResults, choiceboxWidget, defaultText
    global choiceboxChoices
    #-------------------------------------------------------------------
    # If choices is a tuple, we make it a list so we can sort it.
    # If choices is already a list, we make a new list, so that when
    # we sort the choices, we don't affect the list object that we
    # were given.
    #-------------------------------------------------------------------
    choices = list(choices[:])
    if len(choices) == 0:
        choices = ["Program logic error - no choices were specified."]
    defaultButtons = ["OK", "Cancel"]

    # make sure all choices are strings
    choices = [x.ToString() for x in choices]

    if buttons:
        if type(buttons) == type("abc"): # user sent a string
            choiceboxButtons = [buttons]
        else: # we assume user sent in a list or tuple of strings
            choiceboxButtons = [x.ToString() for x in buttons]
    else:
        choiceboxButtons = defaultButtons

    if title == None: title = ""

    # Initialize __choiceboxResults
    # This is the value that will be returned if the user clicks the close icon
    __choiceboxResults = None

    xaml = '''<Window
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    Title="Window1" SizeToContent="WidthAndHeight" MinWidth="400" MinHeight="150" WindowStyle="None" AllowsTransparency="True">
    <Grid>
        <Grid Name="banner">
            <Image Name="icon" Stretch="Fill" HorizontalAlignment="Left" VerticalAlignment="Top" Width="20" Height="20" Margin="4,4,0,0" />
            <Label Height="28" Name="title" VerticalAlignment="Top" Background="WhiteSmoke" Margin="28,0,0,0">Label</Label>
        </Grid>
        <DockPanel Margin="0,28,0,0" Name='backdrop'>
          <ListBox Name="listBox" DockPanel.Dock="Bottom" MaxHeight="300" />
        
          <StackPanel Name="stackPanel" VerticalAlignment="Top" HorizontalAlignment="Right" DockPanel.Dock="Right"  MinHeight="100" >
            <Button MinHeight="23" Name="ok" MinWidth="73" Margin="1">OK</Button>
            <Button MinHeight="23" Name="cancel" MinWidth="73" Margin="1">Cancel</Button>
            <Button MinHeight="23" Name="select" MinWidth="73" Margin="1">Select All</Button>
            <Button Height="23" Name="clear" MinWidth="73" Margin="1">Clear All</Button>
          </StackPanel>
          <TextBlock Name="message" VerticalAlignment="Top" Text="message"  Padding="5" TextWrapping="Wrap" MinHeight="100" />
        </DockPanel>
    </Grid>
</Window>'''
    boxRoot = LoadXaml(xaml)

    boxRoot.FindName('message').Text = msg

    if Vista:
      brush = Brushes.White
    else:
      brush = SystemColors.WindowBrush
      
    for control in ('message', 'stackPanel', 'backdrop'):
      boxRoot.FindName(control).Background = brush

    __FurnishWindow(title)
      
    #---------------------------------------------------
    # sort the choices
    # eliminate duplicates
    # put the choices into the choiceboxWidget
    #---------------------------------------------------
    choices = set(choices)
    choices = [x for x in choices]
    choices.sort(cmp = lambda s1, s2 : String.Compare(s1, s2, StringComparison.CurrentCultureIgnoreCase))
    
    listBox = boxRoot.FindName('listBox')
    listBox.ItemsSource = choices
    choiceboxWidget = listBox
    choiceboxChoices = choices

    if __choiceboxMultipleSelect:
        listBox.SelectionMode = System.Windows.Controls.SelectionMode.Extended
    
    if len(choices) == 0:
      boxRoot.FindName('ok').Visibility = System.Windows.Visibility.Hidden
      listBox.MouseDoubleClick += __choiceboxCancel
    else:
      listBox.MouseDoubleClick += __choiceboxGetChoice
      listBox.SelectedItem = choices[0]
      listBox.Focus()
      
    if len(choices) == 0 or not __choiceboxMultipleSelect:
      boxRoot.FindName('select').Visibility = System.Windows.Visibility.Hidden
      boxRoot.FindName('clear').Visibility = System.Windows.Visibility.Hidden
    else:
      boxRoot.FindName('select').Click += __choiceboxSelectAll
      boxRoot.FindName('clear').Click += __choiceboxClearAll
      
    boxRoot.FindName('cancel').Click += __choiceboxCancel
    boxRoot.FindName('cancel').IsCancel = True
    boxRoot.FindName('ok').Click += __choiceboxGetChoice
    boxRoot.FindName('ok').IsDefault = True
    
    boxRoot.KeyUp += KeyboardListener
    
    # --- run it! -----
    boxRoot.ShowDialog()
    return __choiceboxResults

def __choiceboxGetChoice(source, event):
    global boxRoot, __choiceboxResults, choiceboxWidget
    if __choiceboxMultipleSelect:
        __choiceboxResults = [x for x in choiceboxWidget.SelectedItems]
    else:
        __choiceboxResults = choiceboxWidget.SelectedItem
    __finish()

def __choiceboxSelectAll(source, event):
    global choiceboxWidget, choiceboxChoices
    choiceboxWidget.SelectAll()

def __choiceboxClearAll(source, event):
    global choiceboxWidget, choiceboxChoices
    choiceboxWidget.SelectedIndex = -1

def __choiceboxCancel(source, event):
    global boxRoot, __choiceboxResults
    __choiceboxResults = None
    __finish()


def KeyboardListener(source, event):
    global choiceboxChoices, choiceboxWidget
    key = event.Key.ToString()
    if len(key) <= 1:
        if not Char.IsControl(key):
            # Find the key in the list.
            # before we clear the list, remember the selected member
            try:
                start_n = choiceboxWidget.SelectedIndex
            except IndexError:
                start_n = -1

            ## clear the selection.
            choiceboxWidget.SelectedIndex = -1

            ## start from previous selection +1
            for n in range(start_n+1, len(choiceboxChoices)):
                item = choiceboxChoices[n]
                if item[0].lower() == key.lower():
                    choiceboxWidget.SelectedIndex = n
                    choiceboxWidget.ScrollIntoView(item)
                    return
            else:
                # has not found it so loop from top
                for n in range(len(choiceboxChoices)):
                    item = choiceboxChoices[n]
                    if item[0].lower() == key.lower():
                        choiceboxWidget.SelectedIndex = n
                        choiceboxWidget.ScrollIntoView(item)
                        return

                # nothing matched -- we'll look for the next logical choice
                for n in range(len(choiceboxChoices)):
                    item = choiceboxChoices[n]
                    if item[0].lower() > key.lower():
                        if n > 0:
                            choiceboxWidget.SelectedIndex = (n-1)
                        else:
                            choiceboxWidget.SelectedIndex = 0
                        choiceboxWidget.ScrollIntoView(item)
                        return

                # still no match (nothing was greater than the key)
                # we set the selection to the first item in the list
                lastIndex = len(choiceboxChoices)-1
                choiceboxWidget.SelectedIndex = lastIndex
                item = choiceboxChoices[lastIndex]
                choiceboxWidget.ScrollIntoView(item)
                return

#-----------------------------------------------------------------------
# exception_format
#-----------------------------------------------------------------------
def exception_format():
    """
    Convert exception info into a string suitable for display.
    """
    return "".join(traceback.format_exception(
           sys.exc_info()[0]
        ,  sys.exc_info()[1]
        ,  sys.exc_info()[2]
        ))

#-----------------------------------------------------------------------
# exceptionbox
#-----------------------------------------------------------------------
def exceptionbox(msg=None, title=None):
    """
    Display a box that gives information about
    an exception that has just been raised.

    The caller may optionally pass in a title for the window, or a
    msg to accompany the error information.

    Note that you do not need to (and cannot) pass an exception object
    as an argument.  The latest exception will automatically be used.
    """
    if title == None: title = "Error Report"
    if msg == None:
        msg = "An error (exception) has occurred in the program."

    codebox(msg, title, exception_format())

#-------------------------------------------------------------------
# codebox
#-------------------------------------------------------------------

def codebox(msg=""
    , title=" "
    , text=""
    ):
    """
    Display some text in a monospaced font, with no line wrapping.
    This function is suitable for displaying code and text that is
    formatted using spaces.

    The text parameter should be a string, or a list or tuple of lines to be
    displayed in the textbox.
    """
    textbox(msg, title, text, codebox=1 )

#-------------------------------------------------------------------
# textbox
#-------------------------------------------------------------------
def textbox(msg=""
    , title=" "
    , text=""
    , codebox=0
    ):
    """
    Display some text in a proportional font with line wrapping at word breaks.
    This function is suitable for displaying general written text.

    The text parameter should be a string, or a list or tuple of lines to be
    displayed in the textbox.
    """

    if msg == None: msg = ""
    if title == None: title = ""

    global boxRoot, __replyButtonText, __widgetTexts, buttonsFrame
    global rootWindowPosition

    xaml = '''<Window
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    Title="Window1" SizeToContent="WidthAndHeight" MinWidth="400" MinHeight="150" WindowStyle="None" AllowsTransparency="True">
    <Grid>
        <Grid Name="banner">
            <Image Name="icon" Stretch="Fill" HorizontalAlignment="Left" VerticalAlignment="Top" Width="20" Height="20" Margin="4,4,0,0" />
            <Label Height="28" Name="title" VerticalAlignment="Top" Background="WhiteSmoke" Margin="28,0,0,0">Label</Label>
        </Grid>
        <DockPanel Margin="0,28,0,0" Name='backdrop'>
          <ScrollViewer DockPanel.Dock="Bottom" MaxHeight="300" >
            <TextBlock Name="content" />
          </ScrollViewer>        
          <StackPanel Name="stackPanel" VerticalAlignment="Top" HorizontalAlignment="Right" DockPanel.Dock="Right"  MinHeight="100" >
            <Button MinHeight="23" Name="ok" MinWidth="73" Margin="1">OK</Button>
          </StackPanel>
          <TextBlock Name="message" VerticalAlignment="Top" Text="message"  Padding="5" TextWrapping="Wrap" MinHeight="100" />
        </DockPanel>
    </Grid>
</Window>'''
    boxRoot = LoadXaml(xaml)

    boxRoot.FindName('message').Text = msg

    __FurnishWindow(title)
    content = boxRoot.FindName('content')
    
    if Vista:
      boxRoot.FindName('message').Background = Brushes.White
      boxRoot.FindName('stackPanel').Background = Brushes.White
      boxRoot.FindName('backdrop').Background = Brushes.White
      content.Background = Brushes.White
      content.Foreground = Brushes.Black
    else:
      boxRoot.FindName('message').Background = SystemColors.WindowBrush
      boxRoot.FindName('stackPanel').Background = SystemColors.WindowBrush
      boxRoot.FindName('backdrop').Background = SystemColors.WindowBrush
      content.Background = SystemColors.WindowBrush
      content.Background = SystemColors.WindowTextBrush
      
    # put a textbox in the top frame

    if codebox:
      content.TextWrapping = System.Windows.TextWrapping.NoWrap
      content.FontFamily = FontFamily('GlobalMonospace.CompositeFont')
    else:
      content.TextWrapping = System.Windows.TextWrapping.WrapWithOverflow
      content.FontFamily = FontFamily('GlobalSanSerif.CompositeFont')

    okButton = boxRoot.FindName('ok')
    okButton.Click += __textboxOK
    okButton.IsDefault = True
    okButton.IsCancel = True

    # ----------------- the action begins ----------------------------------------
    try:
        # load the text into the textbox
        if type(text) == type("abc"): pass
        else:
            try:
                text = "".join(text)  # convert a list or a tuple to a string
            except:
                msgbox("Exception when trying to convert "+ str(type(text)) + " to text in textbox")
                sys.exit(16)

        ##print "setting text"
        ##print text
        content.Text = text
    except:
        msgbox("Exception when trying to load the textbox.")
        sys.exit(16)

    try:
        okButton.Focus()
    except:
        msgbox("Exception when trying to put focus on okButton.")
        sys.exit(16)

    boxRoot.ShowDialog()
    __finish()
    return __replyButtonText

#-------------------------------------------------------------------
# __textboxOK
#-------------------------------------------------------------------
def __textboxOK(source, event):
    global boxRoot
    boxRoot.Hide()



#-------------------------------------------------------------------
# diropenbox
#-------------------------------------------------------------------
def diropenbox(msg=None
    , title=None
    , default=None
    ):
    """
    A dialog to get a directory name.
    Note that the msg argument, if specified, is ignored.

    Returns the name of a directory, or None if user chose to cancel.

    If the "default" argument specifies a directory name, and that
    directory exists, then the dialog box will start with that directory.
    """
    
    dialog = System.Windows.Forms.FolderBrowserDialog()
    
    dialog.Description=getFileDialogTitle(msg,title) 
    if default:
        dialog.SelectedPath = FileInfo(default).FullName
    
    state = dialog.ShowDialog()
    f = dialog.SelectedPath if state == System.Windows.Forms.DialogResult.OK else None
      
    if not f: return None
    return os.path.normpath(f)



#-------------------------------------------------------------------
# getFileDialogTitle
#-------------------------------------------------------------------
def getFileDialogTitle(msg
    , title
    ):
    if msg and title: return "%s - %s" % (title,msg)
    if msg and not title: return str(msg)
    if title and not msg: return str(title)
    return None # no message and no title

#-------------------------------------------------------------------
# class FileTypeObject for use with fileopenbox
#-------------------------------------------------------------------
class FileTypeObject:
    def __init__(self,filemask):
        if len(filemask) == 0:
            raise AssertionError('Filetype argument is empty.')
            
        self.masks = []
        
        if type(filemask) == type("abc"):  # a string
            self.initializeFromString(filemask)
            
        elif type(filemask) == type([]): # a list
            if len(filemask) < 2:
                raise AssertionError('Invalid filemask.\n'
                +'List contains less than 2 members: "%s"' % filemask)
            else:
                self.name  = filemask[-1]
                self.masks = list(filemask[:-1] )
        else:
            raise AssertionError('Invalid filemask: "%s"' % filemask)

    def __eq__(self,other):
        if self.name == other.name: return True
        return False
    
    def add(self,other):
        for mask in other.masks:
            if mask in self.masks: pass
            else: self.masks.append(mask)

    def toTuple(self):
        return (self.name,tuple(self.masks))
        
    def isAll(self):
        if self.name == "All files": return True
        return False
        
    def initializeFromString(self, filemask):
        # remove everything except the extension from the filemask
        self.ext = os.path.splitext(filemask)[1]
        if self.ext == "" : self.ext = ".*"
        if self.ext == ".": self.ext = ".*"
        self.name = self.getName()
        self.masks = ["*" + self.ext]
        
    def getName(self):
        e = self.ext
        if e == ".*"  : return "All files"
        if e == ".txt": return "Text files"
        if e == ".py" : return "Python files"
        if e == ".pyc" : return "Python files"
        if e == ".xls": return "Excel files"
        if e.startswith("."): 
            return e[1:].upper() + " files"
        return e.upper() + " files"


#-------------------------------------------------------------------
# fileopenbox
#-------------------------------------------------------------------
def fileopenbox(msg=None
    , title=None
    , default="*"
    , filetypes=None
    ):
    """
    A dialog to get a file name.
    
    About the "default" argument
    ============================
        The "default" argument specifies a filepath that (normally)
        contains one or more wildcards.
        fileopenbox will display only files that match the default filepath.
        If omitted, defaults to "*" (all files in the current directory).
    
        WINDOWS EXAMPLE::
            ...default="c:/myjunk/*.py"  
        will open in directory c:\myjunk\ and show all Python files.

        WINDOWS EXAMPLE::
            ...default="c:/myjunk/test*.py"  
        will open in directory c:\myjunk\ and show all Python files
        whose names begin with "test".
        
        
        Note that on Windows, fileopenbox automatically changes the path
        separator to the Windows path separator (backslash).

    About the "filetypes" argument
    ==============================
        If specified, it should contain a list of items,
        where each item is either::
            - a string containing a filemask          # e.g. "*.txt"        
            - a list of strings, where all of the strings except the last one
                are filemasks (each beginning with "*.",
                such as "*.txt" for text files, "*.py" for Python files, etc.).
                and the last string contains a filetype description
            
        EXAMPLE::
            filetypes = ["*.css", ["*.htm", "*.html", "HTML files"]  ]
        
    NOTE THAT
    =========

        If the filetypes list does not contain ("All files","*"),
        it will be added.

        If the filetypes list does not contain a filemask that includes
        the extension of the "default" argument, it will be added.
        For example, if     default="*abc.py"
        and no filetypes argument was specified, then
        "*.py" will automatically be added to the filetypes argument.

    @rtype: string or None
    @return: the name of a file, or None if user chose to cancel

    @arg msg: the msg to be displayed.
    @arg title: the window title
    @arg default: filepath with wildcards
    @arg filetypes: filemasks that a user can choose, e.g. "*.txt"
    """
    dialog = OpenFileDialog()
    ## filter = Image Files(*.BMP;*.JPG;*.GIF)|*.BMP;*.JPG;*.GIF|All files (*.*)|*.*

    initialbase, initialfile, initialdir, filetypes = fileboxSetup(default,filetypes)
    dialog.InitialDirectory = initialdir
    
    filtering = [ '%s (%s) | %s' % (x[0], ';'.join(x[1]), ';'.join(x[1])) for x in filetypes]
    dialog.Filter = '|'.join(filtering)

    #------------------------------------------------------------
    # if initialfile contains no wildcards; we don't want an
    # initial file. It won't be used anyway.
    # Also: if initialbase is simply "*", we don't want an
    # initialfile; it is not doing any useful work.
    #------------------------------------------------------------
    if (initialfile.find("*") < 0) and (initialfile.find("?") < 0):
        initialfile = None
    elif initialbase == "*":
        initialfile = None
    else:
        initialfile = FileInfo(Path.Combine(initialdir, initialfile)).FullName
    dialog.FileName = initialfile
    dialog.Title = getFileDialogTitle(msg,title)

    f = None
    if dialog.ShowDialog():
      f = dialog.FileName

    if not f: return None
    return os.path.normpath(f)

#-------------------------------------------------------------------
# filesavebox
#-------------------------------------------------------------------
def filesavebox(msg=None
    , title=None
    , default=""
    , filetypes=None
    ):
    """
    A file to get the name of a file to save.
    Returns the name of a file, or None if user chose to cancel.

    The "default" argument should contain a filename (i.e. the
    current name of the file to be saved).  It may also be empty,
    or contain a filemask that includes wildcards.
    
    The "filetypes" argument works like the "filetypes" argument to
    fileopenbox.
    """
    dialog = SaveFileDialog()


    initialbase, initialfile, initialdir, filetypes = fileboxSetup(default,filetypes)
    dialog.InitialDirectory = initialdir
    
    filtering = [ '%s (%s) | %s' % (x[0], ';'.join(x[1]), ';'.join(x[1])) for x in filetypes]
    dialog.Filter = '|'.join(filtering)

    initialfile = FileInfo(Path.Combine(initialdir, initialfile)).FullName
    dialog.FileName = initialfile
    dialog.Title = getFileDialogTitle(msg,title)
    
    f = None
    if dialog.ShowDialog():
      f = dialog.FileName
    if not f: return None
    return os.path.normpath(f)


#-------------------------------------------------------------------
#
# fileboxSetup
#
#-------------------------------------------------------------------
def fileboxSetup(default,filetypes):    
    if not default: default = os.path.join(".","*")
    initialdir, initialfile = os.path.split(default)
    if not initialdir : initialdir  = "."
    if not initialfile: initialfile = "*"
    initialbase, initialext = os.path.splitext(initialfile)
    initialFileTypeObject = FileTypeObject(initialfile)
    
    allFileTypeObject = FileTypeObject("*")
    ALL_filetypes_was_specified = False  

    if not filetypes: filetypes= []
    filetypeObjects = []

    for filemask in filetypes:
        fto = FileTypeObject(filemask)
        
        if fto.isAll(): 
            ALL_filetypes_was_specified = True # remember this
            
        if fto == initialFileTypeObject:
            initialFileTypeObject.add(fto) # add fto to initialFileTypeObject
        else:
            filetypeObjects.append(fto)

    #------------------------------------------------------------------
    # make sure that the list of filetypes includes the ALL FILES type.
    #------------------------------------------------------------------
    if ALL_filetypes_was_specified: 
        pass
    elif allFileTypeObject == initialFileTypeObject:
        pass
    else:
        filetypeObjects.insert(0,allFileTypeObject)
    #------------------------------------------------------------------
    # Make sure that the list includes the initialFileTypeObject
    # in the position in the list that will make it the default.
    # This changed between Python version 2.5 and 2.6
    #------------------------------------------------------------------ 
    if len(filetypeObjects) == 0:
        filetypeObjects.append(initialFileTypeObject)

    if initialFileTypeObject in (filetypeObjects[0], filetypeObjects[-1]):
        pass
    else:
      filetypeObjects.append(initialFileTypeObject)   
        
    filetypes = [fto.toTuple() for fto in filetypeObjects]

    return initialbase, initialfile, initialdir, filetypes

#-------------------------------------------------------------------
# utility routines
#-------------------------------------------------------------------
# These routines are used by several other functions in the EasyGui module.

def __buttonEvent(s, e):
    """
    Handle an event that is generated by a person clicking a button.
    """
    global  boxRoot, __widgetTexts, __replyButtonText
    __replyButtonText = s.Content
    __finish()

#-----------------------------------------------------------------------
#
#     class EgStore
#
#-----------------------------------------------------------------------
class EgStore:
    r"""
A class to support persistent storage.

You can use EgStore to support the storage and retrieval
of user settings for an EasyGui application.


# Example A
#-----------------------------------------------------------------------
# define a class named Settings as a subclass of EgStore
#-----------------------------------------------------------------------
class Settings(EgStore):

    def __init__(self, filename):  # filename is required
        #-------------------------------------------------
        # Specify default/initial values for variables that
        # this particular application wants to remember.
        #-------------------------------------------------
        self.userId = ""
        self.targetServer = ""

        #-------------------------------------------------
        # For subclasses of EgStore, these must be
        # the last two statements in  __init__
        #-------------------------------------------------
        self.filename = filename  # this is required
        self.restore()            # restore values from the storage file if possible



# Example B
#-----------------------------------------------------------------------
# create settings, a persistent Settings object
#-----------------------------------------------------------------------
settingsFile = "myApp_settings.txt"
settings = Settings(settingsFile)

user    = "obama_barak"
server  = "whitehouse1"
settings.userId = user
settings.targetServer = server
settings.store()    # persist the settings

# run code that gets a new value for userId, and persist the settings
user    = "biden_joe"
settings.userId = user
settings.store()


# Example C
#-----------------------------------------------------------------------
# recover the Settings instance, change an attribute, and store it again.
#-----------------------------------------------------------------------
settings = Settings(settingsFile)
settings.userId = "vanrossum_g"
settings.store()

"""
    def __init__(self, filename):  # obtaining filename is required
        raise NotImplementedError()

    def restore(self):
        """
        Set the values of whatever attributes are recoverable
        from the pickle file.

        Populate the attributes (the __dict__) of the EgStore object
        from     the attributes (the __dict__) of the pickled object.

        If the pickled object has attributes that have been initialized
        in the EgStore object, then those attributes of the EgStore object
        will be replaced by the values of the corresponding attributes
        in the pickled object.

        If the pickled object is missing some attributes that have
        been initialized in the EgStore object, then those attributes
        of the EgStore object will retain the values that they were
        initialized with.

        If the pickled object has some attributes that were not
        initialized in the EgStore object, then those attributes
        will be ignored.

        IN SUMMARY:

        After the recover() operation, the EgStore object will have all,
        and only, the attributes that it had when it was initialized.

        Where possible, those attributes will have values recovered
        from the pickled object.
        """
        if not os.path.exists(self.filename): return self
        if not os.path.isfile(self.filename): return self

        try:
            f = open(self.filename)
            unpickledObject = pickle.load(f)
            f.close()

            for key in list(self.__dict__.keys()):
                default = self.__dict__[key]
                self.__dict__[key] = unpickledObject.__dict__.get(key,default)
        except:
            pass

        return self

    def store(self):
        """
        Save the attributes of the EgStore object to a pickle file.
        Note that if the directory for the pickle file does not already exist,
        the store operation will fail.
        """
        f = open(self.filename, "w")
        pickle.dump(self, f)
        f.close()


    def kill(self):
        """
        Delete my persistent file (i.e. pickle file), if it exists.
        """
        if os.path.isfile(self.filename):
            os.remove(self.filename)
        return

    def __str__(self):
        """
        return my contents as a string in an easy-to-read format.
        """
        # find the length of the longest attribute name
        longest_key_length = 0
        keys = []
        for key in self.__dict__.keys():
            keys.append(key)
            longest_key_length = max(longest_key_length, len(key))

        keys.sort()  # sort the attribute names
        lines = []
        for key in keys:
            value = self.__dict__[key]
            key = key.ljust(longest_key_length)
            lines.append("%s : %s\n" % (key,repr(value))  )
        return "".join(lines)  # return a string showing the attributes




#-----------------------------------------------------------------------
#
# test/demo easygui
#
#-----------------------------------------------------------------------
def egdemo():
    """
    Run the EasyGui demo.
    """
    # clear the console
    writeln("\n" * 100)
    
    # ============================= define a code snippet =========================
    code_snippet = ("dafsdfa dasflkj pp[oadsij asdfp;ij asdfpjkop asdfpok asdfpok asdfpok"*3) +"\n"+\
"""# here is some dummy Python code
for someItem in myListOfStuff:
    do something(someItem)
    do something()
    do something()
    if somethingElse(someItem):
        doSomethingEvenMoreInteresting()

"""*16
    #======================== end of code snippet ==============================

    #================================= some text ===========================
    text_snippet = ((\
"""It was the best of times, and it was the worst of times.  The rich ate cake, and the poor had cake recommended to them, but wished only for enough cash to buy bread.  The time was ripe for revolution! """ \
*5)+"\n\n")*10

    #===========================end of text ================================

    intro_message = '''
Pick the kind of box that you wish to demo.
 * IronPython version %s
 * EasyWPF version %s''' % (sys.version, egversion)
 
    #========================================== END DEMONSTRATION DATA


    while 1: # do forever
        choices = [
            "msgbox",
            "buttonbox",
            "buttonbox(image) -- a buttonbox that displays an image",
            "choicebox",
            "multchoicebox",
            "textbox",
            "ynbox",
            "ccbox",
            "enterbox",
            "enterbox(image) -- an enterbox that displays an image",
            "exceptionbox",
            "codebox",
            "integerbox",
            "boolbox",
            "indexbox",
            "filesavebox",
            "fileopenbox",
            "passwordbox",
            "multenterbox",
            "multpasswordbox",
            "diropenbox",
            "About EasyGui",
            " Help"
            ]
        choice = choicebox(msg=intro_message
            , title="EasyWPF " + egversion
            , choices=choices)

        if not choice: return

        reply = choice.split()

        if   reply[0] == "msgbox":
            reply = msgbox("short msg", "This is a long title")
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "About":
            reply = abouteasywpf()

        elif reply[0] == "Help":
            _demo_help()

        elif reply[0] == "buttonbox":
            reply = buttonbox()
            writeln("Reply was: %s" % repr(reply))

            title = "Demo of Buttonbox with many, many buttons!"
            msg = "This buttonbox shows what happens when you specify too many buttons."
            reply = buttonbox(msg=msg, title=title, choices=choices)
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "buttonbox(image)":
            _demo_buttonbox_with_image()

        elif reply[0] == "boolbox":
            reply = boolbox()
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "enterbox":
            image = "python_and_check_logo.gif"
            message = "Enter the name of your best friend."\
                      "\n(Result will be stripped.)"
            reply = enterbox(message, "Love!", "     Suzy Smith     ")
            writeln("Reply was: %s" % repr(reply))

            message = "Enter the name of your best friend."\
                      "\n(Result will NOT be stripped.)"
            reply = enterbox(message, "Love!", "     Suzy Smith     ",strip=False)
            writeln("Reply was: %s" % repr(reply))

            reply = enterbox("Enter the name of your worst enemy:", "Hate!")
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "enterbox(image)":
            image = "python_and_check_logo.gif"
            message = "What kind of snake is this?"
            reply = enterbox(message, "Quiz",image=image)
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "exceptionbox":
            try:
                thisWillCauseADivideByZeroException = 1/0
            except:
                exceptionbox()
            
        elif reply[0] == "integerbox":
            reply = integerbox(
                "Enter a number between 3 and 333",
                "Demo: integerbox WITH a default value",
                222, 3, 333)
            writeln("Reply was: %s" % repr(reply))

            reply = integerbox(
                "Enter a number between 0 and 99",
                "Demo: integerbox WITHOUT a default value"
                )
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "diropenbox" : _demo_diropenbox()
        elif reply[0] == "fileopenbox": _demo_fileopenbox()
        elif reply[0] == "filesavebox": _demo_filesavebox()

        elif reply[0] == "indexbox":
            title = reply[0]
            msg   =  "Demo of " + reply[0]
            choices = ["Choice1", "Choice2", "Choice3", "Choice4"]
            reply = indexbox(msg, title, choices)
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "passwordbox":
            reply = passwordbox("Demo of password box WITHOUT default"
                + "\n\nEnter your secret password", "Member Logon")
            writeln("Reply was: %s" % str(reply))

            reply = passwordbox("Demo of password box WITH default"
                + "\n\nEnter your secret password", "Member Logon", "alfie")
            writeln("Reply was: %s" % str(reply))

        elif reply[0] == "multenterbox":
            msg = "Enter your personal information"
            title = "Credit Card Application"
            fieldNames = ["Name","Street Address","City","State","ZipCode"]
            fieldValues = []  # we start with blanks for the values
            fieldValues = multenterbox(msg,title, fieldNames)

            # make sure that none of the fields was left blank
            while 1:
                if fieldValues == None: break
                errmsg = ""
                for i in range(len(fieldNames)):
                    if fieldValues[i].strip() == "":
                        errmsg = errmsg + ('"%s" is a required field.\n\n' % fieldNames[i])
                if errmsg == "": break # no problems found
                fieldValues = multenterbox(errmsg, title, fieldNames, fieldValues)

            writeln("Reply was: %s" % str(fieldValues))

        elif reply[0] == "multpasswordbox":
            msg = "Enter logon information"
            title = "Demo of multpasswordbox"
            fieldNames = ["Server ID", "User ID", "Password"]
            fieldValues = []  # we start with blanks for the values
            fieldValues = multpasswordbox(msg,title, fieldNames)

            # make sure that none of the fields was left blank
            while 1:
                if fieldValues == None: break
                errmsg = ""
                for i in range(len(fieldNames)):
                    if fieldValues[i].strip() == "":
                        errmsg = errmsg + ('"%s" is a required field.\n\n' % fieldNames[i])
                if errmsg == "": break # no problems found
                fieldValues = multpasswordbox(errmsg, title, fieldNames, fieldValues)

            writeln("Reply was: %s" % str(fieldValues))

        elif reply[0] == "ynbox":
            title = "Demo of ynbox"
            msg = "Were you expecting the Spanish Inquisition?"
            reply = ynbox(msg, title)
            writeln("Reply was: %s" % repr(reply))
            if reply:
                msgbox("NOBODY expects the Spanish Inquisition!", "Wrong!")

        elif reply[0] == "ccbox":
            title = "Demo of ccbox"
            msg = "Fix EasyGUI bug by initializing mag"
            reply = ccbox(msg,title)
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "choicebox":
            title = "Demo of choicebox"
            longchoice = "This is an example of a very long option which you may or may not wish to choose."*2
            listChoices = ["nnn", "ddd", "eee", "fff", "aaa", longchoice
                    , "aaa", "bbb", "ccc", "ggg", "hhh", "iii", "jjj", "kkk", "LLL", "mmm" , "nnn", "ooo", "ppp", "qqq", "rrr", "sss", "ttt", "uuu", "vvv"]

            msg = "Pick something. " + ("A wrapable sentence of text ?! "*30) + "\nA separate line of text."*6
            reply = choicebox(msg=msg, choices=listChoices)
            writeln("Reply was: %s" % repr(reply))

            msg = "Pick something. "
            reply = choicebox(msg=msg, title=title, choices=listChoices)
            writeln("Reply was: %s" % repr(reply))

            msg = "Pick something. "
            reply = choicebox(msg="The list of choices is empty!", choices=[])
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "multchoicebox":
            listChoices = ["aaa", "bbb", "ccc", "ggg", "hhh", "iii", "jjj", "kkk"
                , "LLL", "mmm" , "nnn", "ooo", "ppp", "qqq"
                , "rrr", "sss", "ttt", "uuu", "vvv"]

            msg = "Pick as many choices as you wish."
            reply = multchoicebox(msg,"Demo of multchoicebox", listChoices)
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "textbox":
            title = "Demo of textbox"
            msg = "Here is some sample text. " * 16
            reply = textbox(msg, "Text Sample", text_snippet)
            writeln("Reply was: %s" % repr(reply))

        elif reply[0] == "codebox":
            msg = "Here is some sample code. " * 16
            reply = codebox(msg, "Code Sample", code_snippet)
            writeln("Reply was: %s" % repr(reply))

        else:
            msgbox("Choice\n\n" + choice + "\n\nis not recognized", "Program Logic Error")
            return



def _demo_buttonbox_with_image():
    image = "python_and_check_logo.gif"

    msg   = "Pretty nice, huh!"
    reply=msgbox(msg,image=image, ok_button="Wow!")
    writeln("Reply was: %s" % repr(reply))

    msg   = "Do you like this picture?"
    choices = ["Yes","No","No opinion"]

    reply=buttonbox(msg,image=image,choices=choices)
    writeln("Reply was: %s" % repr(reply))

    image = os.path.normpath("python_and_check_logo.png")
    reply=buttonbox(msg,image=image, choices=choices)
    writeln("Reply was: %s" % repr(reply))

    image = os.path.normpath("zzzzz.gif")
    reply=buttonbox(msg,image=image, choices=choices)
    writeln("Reply was: %s" % repr(reply))


def _demo_help():
    from StringIO import StringIO
    savedStdout = sys.stdout    # save the sys.stdout file object
    sys.stdout = capturedOutput = StringIO()
    help("EasyWPF")
    sys.stdout = savedStdout   # restore the sys.stdout file object
    codebox("EasyWPF Help",text=capturedOutput.getvalue())

def _demo_filesavebox():    
    filename = "myNewFile.txt"
    title = "File SaveAs"
    msg ="Save file as:"

    f = filesavebox(msg,title,default=filename)
    writeln("You chose to save file: %s" % f)

def _demo_diropenbox(): 
    title = "Demo of diropenbox"
    msg = "Pick the directory that you wish to open."
    d = diropenbox(msg, title)
    writeln("You chose directory...: %s" % d)

    d = diropenbox(msg, title,default="./")
    writeln("You chose directory...: %s" % d)
    
    d = diropenbox(msg, title,default="c:/")
    writeln("You chose directory...: %s" % d)
    
    
def _demo_fileopenbox():        
    msg  = "Python files"
    title = "Open files"
    default="*.py"
    f = fileopenbox(msg,title,default=default)
    writeln("You chose to open file: %s" % f)

    default="./*.gif"
    filetypes = ["*.jpg",["*.zip","*.tgs","*.gz", "Archive files"],["*.htm", "*.html","HTML files"]]
    f = fileopenbox(msg,title,default=default,filetypes=filetypes)
    writeln("You chose to open file: %s" % f)
    
    """#deadcode -- testing ----------------------------------------
    f = fileopenbox(None,None,default=default)
    writeln("You chose to open file: %s" % f)

    f = fileopenbox(None,title,default=default)
    writeln("You chose to open file: %s" % f)

    f = fileopenbox(msg,None,default=default)
    writeln("You chose to open file: %s" % f)

    f = fileopenbox(default=default)
    writeln("You chose to open file: %s" % f)

    f = fileopenbox(default=None)
    writeln("You chose to open file: %s" % f)
    #----------------------------------------------------deadcode """


def _dummy():
    pass

EASYWPF_ABOUT_INFORMATION = '''
========================================================================
0.3.93(2010-Mar-28)
========================================================================

ENHANCEMENTS
------------------------------------------------------

 * Make the WNDPROC callback for handling the Composition Changed message
   actually set the "handled" parameter

========================================================================
0.2.93(2010-Mar-08)
========================================================================

ENHANCEMENTS
------------------------------------------------------

 * Complete port to apparent parity with EasyGui 0.93

========================================================================
0.1.93(2010-Mar-07)
========================================================================

ENHANCEMENTS
------------------------------------------------------

 * First partial implementation against WPF of EasyGui 0.93
'''

def abouteasywpf():
    """
    shows the easywpf revision history
    """
    codebox("About EasyWPF\n"+egversion,"EasyWPF",EASYWPF_ABOUT_INFORMATION)
    return None



if __name__ == '__main__':
    if len(sys.argv) == 1:
        egdemo()
    else:
        # test the new root feature 
        xaml = '''<Window
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window3" Height="300" Width="300">
    <Grid>
        <TextBox Name="textBox1" />
    </Grid>
</Window>'''        
        
        root = LoadXaml(xaml)
        root.FindName('textBox1').Text = """This is a test of a main WPF window in which we will place an easygui msgbox.
                It will be an interesting experiment.\n\n"""
        root.Show()
        msgbox("this is a test of passing in boxRoot", root=root)
        msgbox("this is a second test of passing in boxRoot", root=root)
        
        reply = enterbox("Enter something", root=root)
        writeln("You wrote:", reply)

        reply = enterbox("Enter something else", root=root)
        writeln("You wrote:", reply)
        root.Hide()
        root.Close()
        root = None

Plain unformatted code for easy cutting and pasting.

Save this as EasyWPF.py, and use

from EasyWPF import *

in place of

from easygui import *

EasyWPF 0.2.93

2010-03-08T20:59:00.005Z

See next post.

Infrastructure Helper 1.1

2010-02-14T19:45:00.002Z

Mainly a build process update to v1.0, but with minor tweaks for the February 2010 CTP of F# (version 1.9.9.9).

The Infrastructure project should now build inside VS 2008 shell with the F# extension -- all the C# compilation needed to test that language's lambdas, and to show that even with the coverage annotation, setting the release build as default can still let a C# assembly be deployed without dragging F# along -- are done as BeforeBuild targets invoking the CSC task.

The manual steps of checking FxCop output are similarly automated as AfterBuild targets, though none of these are yet abstracted into a shared .targets file.

Source is now hosted at Google Code., with a first release available -- 7-zip archive containing PGP signature and nested archive.

Adopting PartCover

2010-01-26T20:52:00.004Z

NOTE: the current version of PartCover (for .net 4) as of 13-Jul-10 appears to resolve this issue.

Starting to think about Infrastructure Helper next release, I've started by running up PartCover to try that out; and noting the necessary transformations, like for a post-build step execution

%NCOVER% %NUNIT% $(TargetFileName) /noshadow //reg //a "Tinesware.Infrastructure;Tinesware.InfrastructureTests;Tinesware.Rules;Tinesware.TestData;CSharpTests"

becoming

%PCOVER% --target "%NUNIT%"  --target-work-dir $(TargetDir) --target-args "$(TargetFileName) /noshadow" --include [Tinesware.*]* --include [CSharpTests]* --output $(TargetDir)PCoverage.xml

where the %THING% symbols point to the relevant executables.

Next step is to unpick and start consuming the PartCover xml; and also to see how flaky it actually is when running over F# code. I'm not immediately optimistic, as when running coverage over the operational tests, I get

[01591] [05764] GetClassIDInfo failed for .NewRight
[01591] [05764] GetClassIDInfo failed for ..ctor
[01591] [05764] GetClassIDInfo failed for ..ctor
[01591] [05764] GetClassIDInfo failed for .get_Item
[01607] [05764] GetClassIDInfo failed for ..ctor
[01607] [05764] GetClassIDInfo failed for .Invoke
[01622] [05764] GetClassIDInfo failed for ..ctor
[01622] [05764] GetClassIDInfo failed for .Invoke
System.InvalidProgramException: Common Language Runtime detected an invalid program.
   at Tinesware.Rules.Assist.Local.LineRange(XElement node)
   at Tinesware.Rules.Assist.Local.MatchNode @ 234.Invoke(a candidate) in Infrastructure\TineswareRules\Assist.fs:line 235
...

which is some Partcover noise from instrumenting the F# code; and then a barf in code that unpicks NCover format data (and which worked just fine under NCover earlier in the same post-build operation).

Infrastructure Helper 1.0

2009-11-22T20:48:00.005Z

After a lot of refactoring, some bug-fixing, and some more catching up with F#, the project has got to the point where I feel it deserves a 1.0 label.

Code and built binaries as usual from the Mediafire drop.

Legend

Uncovered code
User exempted
Generated code
Statically analyzed
Covered code

Modules summary

CSharpTests

100%

Tinesware.Infrastructure

10.53%

78.95%

Tinesware.InfrastructureTests

3.57%

7.14%

25%

64.29%

Tinesware.Rules

9.31%

2.09%

0.48%

88.12%

Tinesware.TestData

3.23%

24.19%

12.9%

62.9%

3.23%

No new features since 0.9, but fixes for analysing F# Sum and Record types, code marked with the [GeneratedCode] attribute, and the changed naming convention for the generated classes that represent F# funs in the October CTP.

Looking ahead, I see the following likely features for the post 1.0 development:

Support for PartCover report files
Some integration with NDepend (a suppress-message style attribute and modified CQL rules)
extended XSLT to collate reports of the reasons given for code being excluded
Support for more F# language features as necessary (I haven't tried computation expressions at all yet, and expect there might be some more wrinkles under the covers there),
And of course bug fixing and refactoring as needed.

If anyone else out there does decide to use this, I don't promise prompt turn around with bug reports, but I do welcome them.

Infrastructure Helper v0.9

2009-11-03T21:29:00.003Z

Around life being rather hectic here, some progress on fun things -- Version 0.9 of the FxCop coverage analyser now uploaded to Mediafire.

This adopts the October CTP, and makes a number of changes in the static coverage analysis as a result of the changes in code generation. It also much improves the estimation of which range of lines an FxCop Method covers, and adds a couple of simple programming-by-contract methods for C# coding.

A little more help

2009-10-02T16:58:00.002+01:00

Version 0.8 of the FxCop coverage analyser now uploaded to Mediafire

Changes since 0.7 include

Code in C# lambdas is added to the coverage point count of the enclosing method
more trivial methods are statically analysed above and beyond simple field-backed properties. These include
- Empty methods
- methods that return a constant or a field
- methods that contain just the one statement to
  - failwith
  - raise
  - or simply throw
  an exeception

As before I assume that such analyses are run against debug builds, and consequently cater for the oddities that sometimes arise in that build target type.

A little help for .net (emphasis F#) development.

2009-09-16T07:35:00.005+01:00

One of the things that is always an annoyance when trying to get good unit test coverage of your code -- especially in F# -- is the amount of red that can come from compiler generated code, as well as things like trivial properties that just expose a field. Do you really want to write great swathes of unit tests like

[Test]
public void TrivialPointsPropertyIsTrival()
{
  var subject = new MyObj();
  int nonDefaultValue = 23;
  subject.Points = nonDefaultValue;
  Assert.That(subject.Points, Is.EqualTo(nonDefaultValue), "What goes in should come out");
}

rather than ones that exercise real logic?

Of course not. So I've been putting together a little project in F#, mainly for use with F# (thought there are some nods to C# code), to take raw coverage data in NCover format and separate sheep from goats.

It works as an FxCop rule, analysing a coverage.xml file in conjunction with the code, and producing both warnings about code that is not covered, and a revised coverage file that categorises uncovered code. As an adjunct, the bundle includes a method-level attribute CoverageExemption, to allow the developer to declare some part of a method as uncovered and why, like:

  [< Test >]
  [< SuppressMessage("Microsoft.Performance", "CA1822:MarkMembersAsStatic", Justification = "Unit tests cannot be static") >]
  [< CoverageExemption( Points = 1, Justification = "None branch required for warning-free compilation") >] 
  member self.ExpandFileWithSensibleWorks() =
    let result = CoverageExemptionTest.MakeOption (ExpandFile CoverageExemptionTest.Resolver "dummy.txt")
    match result with
    | Some x -> Assert.That( x.EndsWith("\\dummy.txt", StringComparison.OrdinalIgnoreCase), Is.True, "should be a valid file name")
    | None -> Assert.Fail()

where we declare -- for code review purposes -- why we cannot get coverage of that final pattern when building with the expected /W4 /WX restrictions.

The sort of report we get from this (after applying a modified form of the XSLT file that processes it for the browser) looks like:

Legend

Uncovered code
User exempted
Generated code
Statically analyzed
Covered code

Modules summary

Tinesware.Infrastructure

25%

75%

Tinesware.InfrastructureTests

4.17%

8.33%

79.17%

Tinesware.Rules

14.66%

2.82%

82.52%

Tinesware.TestData

26.67%

20%

6.67%

40%

6.67%

etc., where code is split up into

Orange -- code marked by the user with CoverageExemption as uncoverable, i.e. the bits where review should concentrate
Yellow -- compiler generated code; this is currently either types beginning with "<", marked with the CompilerGenerated or GeneratedCode attributes, automatic code from algebraic types, which have CompilationMapping() with either the SourceConstructFlags.NonpublicRepresentation or SourceConstructFlags.UnionCase flags, funs, and C# lambdas (code-points for funs are coalesced with the containing function, so are not totally exempted at this point; C# lambdas needs some work -- the process is heuristic based on observed generated type names)
Silver -- simple code (such as properties that map onto a field, or trivial constructors, typically generated implicitly, that simply invoke another constructor) whose stereotypical structure can be matched against known patterns within FxCop.

The CoverageExemption attribute in the Tinesware.Infrastructure assembly is itself marked with [< System.Diagnostics.Conditional("CODE_ANALYSIS") >]; this assumes that you build debug builds for analysis with that value set -- omitting it in the release build to avoid adding this declarative clutter to what you finally publish. This also means that you can use it in C# code without having to also require the F# libraries to be present to use the application.

Source and binaries for what I've labelled the 0.7 version is on Mediafire in the WorksInProgress folder; the source uses, but does not include FxCop 1.36, NCover 1.5.8 and NUnit 2.5.2.9222 as support. In the binaries drop, CoverageExemption attribute is in Tinesware.Infrastructure.dll, the FxCop rule (along with the initial five-finger exercise of porting the much travelled "FxCop suppressions must have a justification" rule) in Tinesware.Rules.dll, and the modified XSLT file is included too.

The code could still be polished a lot further, but it's now at least at a useful minimum of functionality.

The lock-free queue revisited

2009-07-24T20:43:00.005+01:00

The previous entry gave the queue itself, and didn't worry about locks that might or might not be taken out by the ancillary code. To avoid allocations and deallocations of queue nodes in the steady state, get rid of the STL list and keep them on a stack --

/*
            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
                    Version 2, December 2004

 Copyright (C) 2004 Sam Hocevar
  14 rue de Plaisance, 75014 Paris, France
 Everyone is permitted to copy and distribute verbatim or modified
 copies of this license document, and changing it is allowed as long
 as the name is changed.

            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. You just DO WHAT THE FUCK YOU WANT TO. */

/* This program is free software. It comes without any warranty, to
 * the extent permitted by applicable law. You can redistribute it
 * and/or modify it under the terms of the Do What The Fuck You Want
 * To Public License, Version 2, as published by Sam Hocevar. See
 * http://sam.zoy.org/wtfpl/COPYING for more details. */ 

#pragma once

// suppress 64-bit ready warning about casting on the win32 platform (only)
#pragma warning(disable:4311)
#pragma warning(disable:4312)

// After Sutter in Dr. Dobbs -- http://ddj.com/cpp/210604448?pgno=2
#include <list>
#include "Windows.h"

template <typename T>
class SimpleQueue {
public:
    typedef T* Tptr;                                // should really be a smart_pointer<T>
    typedef const T * Tcptr;

private:
    SimpleQueue(const SimpleQueue &);               // Not copyable
     SimpleQueue & operator= (const SimpleQueue &); // Not assignable

  struct Node {
     Node( Tptr val ) : value(val), next(NULL) { }
    Node() : next(NULL) { value = NULL; }
    Tptr value;
    Node* next;
  };

  Node * freeStack;             // for producer only
  Node* first;                  // for producer only
  Node *divider, *last;         // shared -- Use explicit atomic compares only

  // Allocator/Deallocator for nodes -- 
  // only used in the producer thread
  // OR in the destructor.
  Node * Get(Tptr val)
  {
     if(freeStack)
     {
        // Clean because of Release
        Node * next = freeStack;
        freeStack = next->next;
        next->value = val;
        return next;
     }

     // clean by construction
     return new Node(val);     
  }

  // Avoids costly free() while running
  void Release(Node * node)
  {
        // reset the node to clean before shelving it
        node->value = NULL;
        node->next = freeStack;
        freeStack = node;
  }


public:
    SimpleQueue() : freeStack(NULL) {
    first = divider = last = Get(NULL);                         // add dummy separator
  }

  ~SimpleQueue() {
    while( first != NULL ) {               // release the list
      Node* tmp = first;
      first = tmp->next;
      delete tmp;
    }

     // Require -- Producer thread calls this or is dead
     while(freeStack)
     {
          delete Get(NULL);
     }
  }

  // Produce is called on the producer thread only:
  void Produce( Tptr t ) {
    last->next = Get(t);                            // add the new item
    InterlockedExchangePointer(&last, last->next);  // publish it : atomic last = last->next;


    // Burn the consumed part of the queue
    for( PVOID looper = first;                     // non-null; pointer read is atomic; atomic while( first != divider )
           InterlockedCompareExchangePointer(&looper, NULL, divider), looper;
           looper = first)
     {
      Node* tmp = first;
      first = first->next;

      Release(tmp);
    }
  }

  // Consume is called on the consumer thread only:
  bool Consume( Tptr & result ) {

     PVOID choice = divider;                                  // non-null; pointer read is atomic
     InterlockedCompareExchangePointer(&choice, NULL, last);

     if(choice)
     {
        if(!divider || ! divider->next || ! divider->next->value) 
            printf("help!");

        result = divider->next->value;                        // C: copy it back
        choice = divider;
        InterlockedExchangePointer(&divider, divider->next);  // D: publish that we took it : atomic divider = divider->next;
        reinterpret_cast<Node*>(choice)->next = NULL;
        return true;                                          // and report success
    }

    return false;                                             // else report empty
  }
};

#pragma warning(default:4312)
#pragma warning(default:4311)

Next add a recycling class to avoid having to keep allocating and reallocating payload objects. C++ sucks as a functional language (without all the heavyweight Boost machinery), so import the functions we need via static polymorphic constraints on templated types.

/*
            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
                    Version 2, December 2004

 Copyright (C) 2004 Sam Hocevar
  14 rue de Plaisance, 75014 Paris, France
 Everyone is permitted to copy and distribute verbatim or modified
 copies of this license document, and changing it is allowed as long
 as the name is changed.

            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. You just DO WHAT THE FUCK YOU WANT TO. */

/* This program is free software. It comes without any warranty, to
 * the extent permitted by applicable law. You can redistribute it
 * and/or modify it under the terms of the Do What The Fuck You Want
 * To Public License, Version 2, as published by Sam Hocevar. See
 * http://sam.zoy.org/wtfpl/COPYING for more details. */ 
#pragma once

#include "SimpleQueue.h"

template <typename T, typename TSource, typename TReceiver>
class ConveyorBelt {
    SimpleQueue<T> forward;
    SimpleQueue<T> reverse;

public:
    

    ConveyorBelt() {}
    ~ConveyorBelt() 
    {
        SimpleQueue<T>::Tptr t;
        while(forward.Consume(t))
        {
            delete t;
        }
        while(reverse.Consume(t))
        {
            delete t;
        }
    }

    void Accept(TSource & source)
    {
        SimpleQueue<T>::Tptr t;
        if(!reverse.Consume(t))
        {
            t = new T();
        }
        source.Set(t);
        forward.Produce(t);
    }

    bool Display(TReceiver & sink)
    {
        SimpleQueue<T>::Tptr t;
        if(!forward.Consume(t))
            return false;

        SimpleQueue<T>::Tcptr tconst = t;
        sink.Use(tconst);
        t->Reset();


        reverse.Produce(t);
        return true;
    }
};

All T records are owned by the ConveyorBelt, and we just loan them out to the TSource and TReceiver objects to write into or read from. Once read, they are recycled back to the producing thread.

A test harness looks like this --

/*
            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
                    Version 2, December 2004

 Copyright (C) 2004 Sam Hocevar
  14 rue de Plaisance, 75014 Paris, France
 Everyone is permitted to copy and distribute verbatim or modified
 copies of this license document, and changing it is allowed as long
 as the name is changed.

            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. You just DO WHAT THE FUCK YOU WANT TO. */

/* This program is free software. It comes without any warranty, to
 * the extent permitted by applicable law. You can redistribute it
 * and/or modify it under the terms of the Do What The Fuck You Want
 * To Public License, Version 2, as published by Sam Hocevar. See
 * http://sam.zoy.org/wtfpl/COPYING for more details. */ 

#include "stdafx.h"
#include "SimpleQueue.h"
#include "ConveyorBelt.h"

using namespace System;
using namespace System::Threading;


class MarkedBuffer {
public:
    size_t start;
    size_t end;
    char data[16384];

    MarkedBuffer() : start(0), end(0) 
    { Reset(); }

    void Reset()
    { memset(data, 0, sizeof(data)); }
};

class Endpoints {
public:
    size_t source;
    size_t sink;

    Endpoints() : source(0), sink(0) {}
    void Set(MarkedBuffer * t)
    {
        t->start = source;
    }
    void Use(MarkedBuffer const * t)
    {
        sink = t->start;
    }
};


ref class Work
{
public:

   static ConveyorBelt<MarkedBuffer, Endpoints, Endpoints> * q2;
   static Endpoints * e;

   static void Produce2()
   {
      Console::WriteLine( "Producer thread procedure." );
       for(int i=0; i<32768; ++i)
       {
            e->source = i;
            q2->Accept(*e);
            if(0 == (i%100)) Thread::Sleep(200);
            else Thread::Sleep(0);
       }
      Console::WriteLine( "Producer thread finishhd." );
   }

   static void Consume2()
   {
      Console::WriteLine( "Consumer thread procedure." );
       int latest = -1;
       while(latest < 32767)
       {
           if(q2->Display(*e))
            {
                 if(e->sink != size_t(latest+1))
                 {
      Console::WriteLine( "Consumer thread procedure -- failure." );
                 }
                 latest = int(e->sink);
            }
       }
      Console::WriteLine( "Consumer thread finished." );
   }
};


int main(array<System::String ^> ^)
{

     {
          ConveyorBelt<MarkedBuffer, Endpoints, Endpoints> queue;
          Endpoints e;

          Work::q2 = &queue;
          Work::e = &e;

          ThreadStart ^ producerDelegate = gcnew ThreadStart( &Work::Produce2 );
          Thread ^ producerThread = gcnew Thread( producerDelegate );
          ThreadStart ^ consumerDelegate = gcnew ThreadStart( &Work::Consume2 );
          Thread ^ consumerThread = gcnew Thread( consumerDelegate );


          producerThread->Start();
          consumerThread->Start();

          producerThread->Join();
          consumerThread->Join();
     }

      Console::WriteLine( "Completed." );

    return 0;
}

Where the MarkedBuffer type is representative of the uses I have made of such a queue inside a proxying application -- a packet read from the network into the data array or composed for writing to the network, with start and end offsets as marked; the data being conveyed between threads service client facing and server facing sockets.

Using the ConveyorBelt to carry data, allocations will take place until a steady state is reached, and may infrequently happen as spikes load the queue; all deallocations are deferred to the destruction of the system.

A simple LockFree Queue (C++, Windows)

2009-07-22T21:20:00.007+01:00

Prompted by some discussion on StackOverflow today...

This is the simple single-producer, single-consumer version of the Lock-Free Queue as published by Herb Sutter in DDJ, but adapted to replace the C++0x style atomic<Node*> with Win32 API calls to InterlockedExchangePointer and InterlockedCompareExchangePointer. Converting the multiple producers/consumers case is left as a trivial exercise for the reader.

So is doing the free-list manually, as a stack of Nodes rather than bringing in a gratuitous std::list.

/*
            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
                    Version 2, December 2004

 Copyright (C) 2004 Sam Hocevar
  14 rue de Plaisance, 75014 Paris, France
 Everyone is permitted to copy and distribute verbatim or modified
 copies of this license document, and changing it is allowed as long
 as the name is changed.

            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. You just DO WHAT THE FUCK YOU WANT TO. */

/* This program is free software. It comes without any warranty, to
 * the extent permitted by applicable law. You can redistribute it
 * and/or modify it under the terms of the Do What The Fuck You Want
 * To Public License, Version 2, as published by Sam Hocevar. See
 * http://sam.zoy.org/wtfpl/COPYING for more details. */ 

#pragma once

// suppress 64-bit ready warning about casting on the win32 platform (only)
#pragma warning(disable:4311)
#pragma warning(disable:4312)

// After Sutter in Dr. Dobbs -- http://ddj.com/cpp/210604448?pgno=2
#include <list>
#include "Windows.h"

template <typename T>
class SimpleQueue {

    typedef T* Tptr;                                // should really be a smart_pointer<T>

private:
    SimpleQueue(const SimpleQueue &);               // Not copyable
     SimpleQueue & operator= (const SimpleQueue &); // Not assignable

  struct Node {
     Node( Tptr val ) : value(val), next(NULL) { }
    Node() : next(NULL) { value = NULL; }
    Tptr value;
    Node* next;
  };

  std::list<Node *> freeList;   // for producer only
  Node* first;                  // for producer only
  Node *divider, *last;         // shared -- Use explicit atomic compares only

  // Allocator/Deallocator for nodes -- 
  // only used in the producer thread
  // OR in the destructor.
  Node * Get(Tptr val)
  {
     if(!freeList.empty())
     {
        // Clean because of Release
        Node * next = freeList.front();
        freeList.pop_front();
        next->value = val;
        return next;
     }

     // clean by construction
     return new Node(val);     
  }

  // Avoids costly free() while running
  void Release(Node * node)
  {
        // reset the node to clean before shelving it
        node->value = NULL;
        node->next = NULL;
        freeList.push_back(node);
  }


public:
  SimpleQueue() {
    first = divider = last = Get(NULL);                         // add dummy separator
  }

  ~SimpleQueue() {
    while( first != NULL ) {               // release the list
      Node* tmp = first;
      first = tmp->next;
      delete tmp;
    }

     // Require -- Producer thread calls this or is dead
     while(!freeList.empty())
     {
          delete Get(NULL);
     }
  }

  // Produce is called on the producer thread only:
  void Produce( Tptr t ) {
    last->next = Get(t);                            // add the new item
    InterlockedExchangePointer(&last, last->next);  // publish it

    // Burn the consumed part of the queue
    for( PVOID looper = first;                     // non-null; pointer read is atomic
           InterlockedCompareExchangePointer(&looper, NULL, divider), looper;
           looper = first)
     {
      Node* tmp = first;
      first = first->next;
      Release(tmp);
    }
  }

  // Consume is called on the consumer thread only:
  bool Consume( Tptr & result ) {

     PVOID choice = divider;                                  // non-null; pointer read is atomic
     InterlockedCompareExchangePointer(&choice, NULL, last);

     if(choice)
     {
        result = divider->next->value;                        // C: copy it back
        choice = divider;

        InterlockedExchangePointer(&divider, divider->next);  // D: publish that we took it
        reinterpret_cast<Node*>(choice)->next = NULL;
        return true;                                          // and report success
    }

    return false;                                             // else report empty
  }
};

#pragma warning(default:4312)
#pragma warning(default:4311)

And a simple test harness using C++/CLI to make the threading simpler to write

/*
            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
                    Version 2, December 2004

 Copyright (C) 2004 Sam Hocevar
  14 rue de Plaisance, 75014 Paris, France
 Everyone is permitted to copy and distribute verbatim or modified
 copies of this license document, and changing it is allowed as long
 as the name is changed.

            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. You just DO WHAT THE FUCK YOU WANT TO. */

/* This program is free software. It comes without any warranty, to
 * the extent permitted by applicable law. You can redistribute it
 * and/or modify it under the terms of the Do What The Fuck You Want
 * To Public License, Version 2, as published by Sam Hocevar. See
 * http://sam.zoy.org/wtfpl/COPYING for more details. */ 

#include "stdafx.h"
#include "SimpleQueue.h"

using namespace System;
using namespace System::Threading;
ref class Work
{
public:
   static void Produce()
   {
      Console::WriteLine( "Producer thread procedure." );
       for(int i=0; i<32768; ++i)
       {
            int * data = reinterpret_cast<int*>(malloc(sizeof(int)));
            *data = i;
            q->Produce(data);
            if(0 == (i%1000)) Thread::Sleep(20);
       }
      Console::WriteLine( "Producer thread finished." );
   }

   static void Consume()
   {
      Console::WriteLine( "Consumer thread procedure." );
       int latest = -1;
       while(latest < 32767)
       {
            int * data;
            if(q->Consume(data))
            {
                 if(data[0] != latest+1)
                 {
      Console::WriteLine( "Consumer thread procedure -- failure." );
                 }
                 latest = data[0];
                 free(data);
            }
       }
      Console::WriteLine( "Consumer thread finished." );
   }

   static SimpleQueue<int> * q;

};


int main(array<System::String ^> ^)
{
     {
          SimpleQueue<int> queue;
          Work::q = &queue;

          ThreadStart ^ producerDelegate = gcnew ThreadStart( &Work::Produce );
          Thread ^ producerThread = gcnew Thread( producerDelegate );
          ThreadStart ^ consumerDelegate = gcnew ThreadStart( &Work::Consume );
          Thread ^ consumerThread = gcnew Thread( consumerDelegate );


          producerThread->Start();
          consumerThread->Start();
          producerThread->Join();
          consumerThread->Join();
     }

      Console::WriteLine( "Completed." );

    return 0;
}

The Jython/Scala stack revisited

2009-05-26T18:46:00.004+01:00

After last year's experiments, another refinement in the process of getting the two languages to talk, this time from inside NetBeans.

Last year, doing everything from the command line, I got across the language bridge by adding the scala jar to the classpath. Inside NetBeans, you get a little way along the road by adding the scala-library jar and the jars from the scala layer of the stack to the project Python Path. This will let you import types, and see that traits have the expected members. Classes, however, appear as None valued.

For development purposes, you can, instead of adding the jars to the classpath, add them to sys.path instead, like:

sys.path.append(r'C:\Users\Tines\.netbeans\6.5\scala\scala-2.7.3.final\lib\scala-library.jar')
sys.path.append(r'C:\Users\Tines\Documents\FTPStack\FTPlib\dist\FTPLib.jar')

For a real app, rather than hard-coding, these file paths could be supplied as arguments.

Scala companion objects, with names ending in $ don't show up at all in the namespaces when imported into Jython, so anything static-like will have to be accessed through a throwaway class instance.

Scala is not C#

2009-05-11T19:51:00.004+01:00

Took a while to debug what was going wrong here

        val tmp = toUnsigned(leftSide.digits(i)) * toUnsigned(rightSide.digits(j)))
           +  toUnsigned(da(k)) + carry

because it compiles, silently discarding the fact that this is two statements, the second with no immediately apparent side-effects.

It should be more like

        val tmp = (
          (toUnsigned(leftSide.digits(i)) * toUnsigned(rightSide.digits(j))) +
          toUnsigned(da(k)) + carry)

where the parentheses and the trailing operators give a belt-and-braces approach to delineating the statement boundaries.

Integrating Scala with Erlang -- Initial steps

2009-05-09T22:27:00.008+01:00

So, I've started a parallel implementation of jinterface 1.5 in Scala, the idea being

To teach myself Scala by doing
To experiment with the actor-based concurrency as a counterparty to the Erlang model
To experiment with Scala as a dual VM technology

So far, just the term-representation classes, and the start of the Input/Decode and Output/Encode part; and some unit tests, only for Atom so far.

I am impressed at how much I've been able to do without bringing in platform types that I will need to abstract out. What was memory-based I/O in Java or C# becomes Seq[Byte] or Queue[Byte] as appropriate and basing all exceptions off the shared name Exception root class gets rid of any dependence on concrete subclasses. It's only BigInt that is a known problem so far, where I'll need to factor out a platform abstraction later. Of course abstracting ZLib support is a task lurking in the near future, as well.

The first noticeable difference from trying to tidy the C#2 port of an older Java version is that the functional style and rich library support make a lot of previously long-winded support methods significantly more expressive. Contrast the Erlang list (of integer values) to String

public System.String StringValue
        {
            get
            {
                char[] body = new char[Arity];
                ReadOnlyCollection<IObject> e = Elements;
                for (int i = 0; i < Arity; ++i)
                {
                    Integer v = e[i] as Integer;
                    if (null == v) body[i] = '?';
                    else
                    {
                        System.Diagnostics.Debug.Assert(v != null && v.Value != null);
                        char? c = v.Value.ToChar();
                        if (null == c) c = '?';
                        body[i] = (char)c;
                    }
                }
                return new string(body);
            }
        }
// and elsewhere
        public char? ToChar()
        {
            try
            {
                return (char)System.UInt16.Parse(this.ToString(), System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture);
            }
            catch (OverflowException)
            {
                return null;
            }
        }

with

def AsString =
  {
    (Elements map List.AsChar).mkString
  }
// and elsewhere
  def AsChar(obj : IObject) =
    obj match
    {
      case c : Integer if c.Value < (Integer.Zero + Char.MinValue.toInt) => '?'
      case c : Integer if c.Value >= (Integer.Zero + Char.MaxValue.toInt) => '?'
      case c : Integer => c.Value.intValue.toChar
      case _ => '?'
    }

Yes, using C#3's functional extensions, that too could be simplified; this is really the Java equivalent of doing that.

Code drop on the MediaFire page.

ZLIB and .Net

2009-04-27T21:43:00.008+01:00

A recurring problem in several of the projects I have in the pipeline is the matter of handling ZLIB. Java, through java.util.zip offers ZLIB compression with a 32k byte window (but no means of tuning the window) with the DeflaterOutputStream. The .Net framework doesn't offer direct ZLIB at all, but provides naked Deflate via System.IO.Compression.DeflateStream.

That gives us enough to be able to reflate the output of a ZLIB deflation, since a ZLIB is a 2 byte header, a deflate section and finally a 4 byte checksum:

import clr    
clr.AddReference( 'vjslib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' )
from java.io import *
from java.util.zip import *
from System.IO import *
from System.IO.Compression import *
from System import Array, Byte, SByte, Int64
from System.Text import Encoding

def readFileToSByteArray(name):
  # open a file via Java I/O
  raw = RandomAccessFile(name, 'r')
  # print "File size in bytes = %d "  % (raw.length())

  # read into signed byte array
  dim = Array[Int64]([raw.length()])
  helper = Array[SByte]([0])
  buffer = Array.CreateInstance(helper[0].GetType(), dim)
  raw.readFully(buffer)
  raw.close()
  return buffer
  
def sbyteArrayToUbyteArray(buffer, offset=0, length=-1):
  if length < 0:
    length = buffer.Length
 
  dim = Array[Int64]([length])
  helper = Array[Byte]([0])
  ubuffer = Array.CreateInstance(helper[0].GetType(), dim)
  
  # copy into unsigned byte array
  for i in range(0,length):
    ubuffer[i] = buffer[i+offset] & 0xff
  return ubuffer
  
def jdeflate(buffer):
  sink0 = ByteArrayOutputStream()
  sink = DeflaterOutputStream(sink0)
  sink.write(buffer, 0, buffer.Length)
  sink.close()
  sink0.close()
  return sink0.toByteArray()
  
def update_adler(adler, buffer):
  s1 = adler & 0xffff
  s2 = (adler >> 16) & 0xffff
  BASE = 65521
  for n in range(0,buffer.Length):
          s1 = (s1 + buffer[n]) % BASE
          s2 = (s2 + s1)     % BASE
  return (s2 << 16) + s1
  
def ninflate(buffer):
  mem = MemoryStream(buffer)
  inflate = DeflateStream(mem, CompressionMode.Decompress)

  mem = MemoryStream()
  while True:
    x = inflate.ReadByte()
    if x < 0:
      break
    mem.WriteByte(x)
 
  inflate.Close()
  mem.Close()
  return mem.ToArray()
  
 
##==========================================

# select a file
name = r'..\..\sidereal.py'

# make signed, unsigend buffers and string
buffer = readFileToSByteArray(name)
print "Constructed buffer size %d" % (buffer.Length)
ubuffer = sbyteArrayToUbyteArray(buffer)
instring = Encoding.Default.GetString(ubuffer)

# compute the Adler32 checksum
adler = update_adler(1, ubuffer)
print "Adler32 = %d" % (adler)
 
# deflate in Java
deflated = jdeflate(buffer)

# check the ZLIB header -- expect 120, 156 actually (120,-100)
first = (deflated[0] & 0xff)
second = (deflated[1] & 0xff)
print "header value EXPECTED 120:156 ACTUAL %d:%d" % (first, second)

i = deflated.Length-4
x = ((deflated[i]&0xff) << 24) | ((deflated[i+1]&0xff) << 16) | ((deflated[i+2]&0xff) << 8) | (deflated[i+3]&0xff)
print "Java Adler32 = %d" % (x)

# discard header and Adler32 tail
newbuffer = sbyteArrayToUbyteArray(deflated, 2, deflated.Length-6)
print "ZLIB length %d" % (deflated.Length)
print "deflate section length %d" % (newbuffer.Length)

# reflate
out = ninflate(newbuffer)

# compare with input
print "reflated buffer length : %d" % (out.Length)
outstring = Encoding.Default.GetString(out)
same = outstring.Equals(instring)
print "Output == Input ?? %s" %(str(same))

That works fine; but the converse, taking a .Net deflate and adding the appropriate top and tail took a bit of getting to work.

First pitfall -- the InflaterInputStream has to be read in chunks as large as feasible, rather than byte at a time, so as not to throw a premature EOFException. That overcome, I got a result of the right length, but differing in the final few characters for the file under test, which I resolved by doing a belt-and-braces closing of the deflate operation. The un-refactored code I currently have continues from the above as:

# now the other way around
sink0 = MemoryStream()
sink = DeflateStream(sink0, CompressionMode.Compress)
sink.Write(ubuffer, 0, ubuffer.Length)
## overkill the flush and close here
sink.Flush()
sink.Close()
sink0.Flush()
sink0.Close()

deflated = sink0.ToArray()
print "deflate section length %d" % (deflated.Length)

#now inflate
dim = Array[Int64]([deflated.Length+6])
jbuffer = Array.CreateInstance(buffer[0].GetType(), dim)

jbuffer[0] = 120 # 0111 0100 = 120 : 32 kbit window, Deflate
jbuffer[1] = -100 # 10 0 ????? = 128 + x : default compress, no dict, checksum

def sbyte(x):
  y = x & 0xff
  if y > 127:
    return y - 256
  return y
 
for i  in range(0,deflated.Length):
    jbuffer[i+2] = sbyte(deflated[i])
 
i = deflated.Length+2
jbuffer[i] = sbyte(adler>>24) 
jbuffer[i+1] = sbyte(adler>>16) 
jbuffer[i+2] = sbyte(adler>>8)
jbuffer[i+3] = sbyte( adler )
 
 
source0 = ByteArrayInputStream(jbuffer)
source = InflaterInputStream(source0)

dim = Array[Int64]([ubuffer.Length])
helper = Array[SByte]([0])
xbuffer = Array.CreateInstance(helper[0].GetType(), dim)

## take big bites
offset = 0
try:
 while True:
  x = source.read(xbuffer, offset, xbuffer.Length-offset)
  if x < 0:
    break
  offset += x
except EOFException:
  pass

reflated = sbyteArrayToUbyteArray(xbuffer)

print "reflated length is %d" % (reflated.Length)
outstring = Encoding.Default.GetString(reflated)

same = outstring.Equals(instring)
print same

This should allow me to simplify the baggage accumulated for the C#/Erlang bridge, which currently uses a second-generation port of the original 'C' ZLIB for this sort of interoperation.

Later — I have updated the C#/Erlang bridge in the light of this.

End of an era

2009-04-26T10:05:00.005+01:00

After the Yahoo! announcement, time to find another home for files I've stashed there, and I've gone for MediaFire -- link at the right, except for those on my Demon site

pyFtp.zip : IronPython FTP client rework — far from complete, just a repository
CTCJava2beta2.7z — 7zip archive
ctcjlib.2.3.2181.36359.zip — last native DLL build
ctclib_231.zip — last source code drop
crypt.zip — A complete replacement for the old crypt.zip, with source and pre built .jar and .dll (angerona.algorithms version 1.0.2915.29952), unit tests, and a .Net build of junit 3.8.2 -- includes:
- JZlib (inflate/deflate)
- Blowfish, CAST5, DES (including s3' and key-dependent variants), IDEA, Safer, Square, TEA, ThreeWay, TripleDES (encryption)
- BlockCypherHash, Haval, MD5, RIPEM160, SHA0, SHA1 (message digests); and
- Bignum (mutable multiple precision integers).
Requires the next item if using the .Net version:
vjredist2nd.zip — J# redistributables 2nd edition for .Net frameworks 2.0 and 3.0 (for use with Java/Ruby JVM/CLR hybrid code, digital extraction permitting.)
util.zip — source for ancient Java utilities (1.0/1.1 timescale)
util_jar.zip —JAR file of the above in a .Zip
utildocs.zip — old-school Javadocs
angerona.redcoffee.zip -- Current drop of JVM/CLR hybrid library. Archive is 546,797 bytes bytes; MD5 e235de70 113f0147 b1809ec0 df7823bc & SHA-1 4d421c00 d4b3b45d be0244e3 649b19bd 04923a94.
GWT 2.2 - 1 -- A build of the old, old, gwt2_2.zip Gadget Windowing Toolkit (an LGPL'd Swing for Java 1.0 and 1.1, javadocs and such in the latter archive) for Java 5 and J# 2.0 (though it needs an auxiliary C#-based library). Archive is 257,055 bytes; MD5 7335e89a 171560f3 003f225e d97a3b3c & SHA-1 3069e971 e75dfbf9 44324637 bf0d730b 97d6f2c9.
jinterface.zip — (MD5 : edf5f0e2 61c4282e 1a937d6d a34e5c79) Project files for building the Erlang jinterface 1.4 code under J#; plus the built .dll (MD5 : edf5f0e2 61c4282e 1a937d6d a34e5c79) version 1.4.3030.33797. Needs the VJ# 2.0 distributables
erlang.zip — (MD5 : d5baedd9 90157eb9 a217e51a bb5d2f12) -- My fork of the Jungerl version of jinterface for C#, plus the jinterface 1.4 code and the above extensions for J# (code only), plus some initial unit tests for the Otp.Net fork. The tests only so far deal with some of the utilities and the wire types used for decoding data encoded e.g. by jinterface, and converting back again, but are essentially sufficient for those. This now includes compression as well, with both the Erlang code and the Zlib port needed for compression having been scrubbed with FxCop. Needs the VJ# 2.0 distributables to run the tests, as well as NUnit 2.5 and Rhino Mock 3.4 as of 08-08-08.

Belatedly, more IronPython + Silverlight

2009-03-22T20:14:00.003Z

Domestic happenings and the demands of the day-job have kept me away from the codeface for playtime for a while; but at last, following up from last year, a SilverLight2/IronPython2 orrery-clock. Back-ported from F#, I confess, hence the possibly less than Pythonic idioms at times.

Silverlight example.

Python swinging both ways

2009-01-11T20:48:00.008Z

Following up to the last post I made in my quest for a cross-VM (JVM and CLR), single code-base, polyglot language stack, which doesn't involve any more Java than can be helped. Especially when there is Scala to play with, a language which a reading of the stairway book shows is very nifty indeed.

To summarise where we are, and what we have discovered to date --

Python can call Scala and Java happily
Java calling into Python is funky and platform dependent; doubly so for Scala
Scala cannot both call into java.* classes and compile on .net
Low-level Scala pre-defs differ in the Byte type (signed in Java/J#/Scala-JVM, unsigned in Scala-msil)
Ruby's predilection for Pascal-case namespaces (fudged in JRuby to match Java's lower-case convention) would require a lot of plumbing around in C# to mate IronRuby to J# or Scala-msil (as the rest of .net uses Pascal-casing).

And then there is this weekend's discovery.

As J# is a barely JDK1.2 implementation at best, I can't use the latest Java splash-screen feature, and have to fall back on older implementations. I have one to hand, back from c. JDK 1.4 days. It uses synchronized(), Object.wait() and Object.notifyAll().

The first is easy enough to deal with when wanting to put as much as possible of this GUI-driving code into Python -- define an interface to hold a code block

public interface ISynchronizedAction
{
  void performAction();
}

and then have a class which knows about the object to synchronize on, and can execute the block in a synchronized scope

public class Synchronized {
  private Object lock;

  public Synchronized(Object lock) {this.lock = lock;}

  public void Lock(ISynchronizedAction action)
  {
    synchronized(lock) {action.performAction();}
  }
}

Except in IronPython, I also discover by dumping the dir() of an object sub-classing java.awt.Window -- inheritance being needed for the splash-screen window class rather than composition, in order to hook the update/repaint APIs -- that neither Object.wait() nor Object.notifyAll() are present to be called. Fortunately the object I want these methods on is the same one I want to synchronize on, the splash-screen window itself, so I was able to just extend Synchronized with

  public void delegateWait() throws InterruptedException
  {
    lock.wait();
  }

  public void delegateNotifyAll()
  {
    lock.notifyAll();
  }

and make the there-and-back-again calls as required.

Then I just had to work around the older dialect's lack of support for .png ~~(by using a .gif)~~ by using the handy Java-1.1 compatible javapng-1.3.0 library to actually paint the image into the splash screen.

As usual, Jython needs the helper .jar file in the CLASSPATH, while IronPython can reference the equivalent .dll from code by assembly file name.

Towards Scala/.net 2.0

2009-01-10T19:37:00.005Z

A propos of my last bit of banging head against a wall... Oh, and these snippets are definitely offered under the WTFPL.

The main things that the scala-msil compilation balks at when looking at .Net assemblies post framework 1.x are generics; and things marked as var. Looking at the source in http://lampsvn.epfl.ch/svn-repos/scala/msil/trunk/src/, the first changes to make are, first in msil/util/Signature.java, add the extra byte code values

    public static final int ELEMENT_TYPE_VAR = 0x13;     // a class type variable VAR
    /***/
    public static final int  ELEMENT_TYPE_GENERICINST    = 0x15;     // GENERICINST    ...

and then start to do something about parsing them in msil/PEFile.java, in method PEFile.Sig.decodeType0()

        case ELEMENT_TYPE_GENERICINST:
        // Followed by <type> token
        //ELEMENT_TYPE_GENERICINST <an mdTypeDef metadata token> <argument Count> <arg1> ... <argN>
        {
            // base type
            int tmp = readByte();
            if((tmp != ELEMENT_TYPE_VALUETYPE) && (tmp != ELEMENT_TYPE_CLASS))
                throw new RuntimeException("*1*>> "+ byte2hex(tmp) +
        "@" + pos() + " in " + this);   
            int id = decodeInt();
            type = pemodule.getTypeDefOrRef(id);
            if (type == null)   throw new RuntimeException("**>> "+ byte2hex(desc) +
        "@" + pos() + " in " + this);
            // number of type arguments
            int num = decodeInt();
            System.out.println("num = "+num);
            for(int i=0;i<num;++i)
            {
                Type t = decodeType0();
                //TODO: use this information
            }
        }
        break;

        case ELEMENT_TYPE_VAR:
        {
            int index = decodeInt();
            System.out.println("var of type "+index);
            // TODO: actually look up what the index means 
            return Type.GetType("System.Object");
        }

This is necessary but not sufficient -- this still leaves problems with types involving templated subtypes or interfaces, for which the first TODO is probably meaningful. As a consequence we're still a long way from a "hello .net 2.0 world!" example yet; let alone a rebuilt scala-compiler.jar, alas.

More Silverlight Clock

2008-12-16T20:35:00.002Z

I could, I suppose, have implemented this project in JavaScript -- that language has all the tools to get the current URL query string and set a permalink to a custom URL, one that wouldn't need clicking, but could be right-clicked; and save the self-inflicted DLR download hit.

If I wanted to, I could put the permalink URL out to an element in the DOM with Silverlight, too, which negates one perceived advantage. And using Silverlight gives me something extra -- multithreading.

Yes, there are JavaScript clocks, like this annoying mouse-following one, but they have to fake such things because they are running full-time in the UI thread. By using a BackgroundWorker to prompt a redraw at intervals (and, later, perform updates on the positions of celestial bodies), with Silverlight, I can offload everything but the painting from the UI thread.

Persisting user settings in Silverlight

2008-12-14T17:30:00.007Z

Many years ago, the first GUI application I wrote (in X with Motif) was a little orrery-cum-clock thing, with the clock face doubling as a view of the sky, showing the classical planets. Some time in the intermediate past, I thought about doing this as a Java applet, but it never got anywhere, because out on the web, it would have to be settable to the user's geographic location, and I never figured a good way to persist that data between page visits.

Having done the basic tinkering with IronPython and Silverlight to get the two working together, I thought about resurrecting the project. And the first thing to solve is the settings persistence.

But now, I have an easy API to get at the current URL and such via System.Windows.Browser.HtmlPage.Document -- so I can read the values from its query string; and set the values in a hyperlink for bookmarking purposes, thus (app.py):

from System.Windows import Application
from System.Windows.Controls import Canvas
import System
import System.Windows.Browser

xaml = Application.Current.LoadRootVisual(Canvas(), "astroclock.xaml")
xaml.hyperlink.NavigateUri = System.Windows.Browser.HtmlPage.Document.DocumentUri
thisPage = System.Windows.Browser.HtmlPage.Document.DocumentUri.GetComponents(
  System.UriComponents.SchemeAndServer | System.UriComponents.Path,
  System.UriFormat.SafeUnescaped)
query = System.Windows.Browser.HtmlPage.Document.QueryString
  
try:
  xaml.slider1.Value = float(query['lat'])
except:
  pass
try:
  xaml.slider2.Value = float(query['long'])
except:
  pass
 
def latValueChanged(s, e):
  v = xaml.slider1.Value
  if v > 0:
 xaml.label1.Text = "Latitude %.2fN" % (v)
  elif v < 0:
 xaml.label1.Text = "Latitude %.2fS" % (-v)
  else: 
 xaml.label1.Text = "Latitude 0"
 xaml.hyperlink.NavigateUri = System.Uri("%s?lat=%f&long=%f" % (thisPage, v, xaml.slider2.Value))

xaml.slider1.ValueChanged += latValueChanged
latValueChanged(None, None)

def longValueChanged(s, e):
  v = xaml.slider2.Value
  if v > 0:
 xaml.label2.Text = "Longitude %.2fE" % (v)
  elif v < 0:
 xaml.label2.Text = "Longitude %.2fW" % (-v)
  else:
 xaml.label2.Text = "Longitude 0"
 xaml.hyperlink.NavigateUri = System.Uri("%s?lat=%f&long=%f" % (thisPage, xaml.slider1.Value, v))
 
xaml.slider2.ValueChanged += longValueChanged
longValueChanged(None, None)

driving some UI described by (astroclock.xaml)

<Canvas 
  x:Class="System.Windows.Controls.Canvas"
  xmlns="http://schemas.microsoft.com/client/2007"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  x:Name="parentCanvas">
    <Grid>
        <Canvas Name="canvas1" Margin="0,0,0,129" Background="Black" Width="480" Height="240"/>
        <Slider Height="22" Margin="128,0,0,93" Name="slider1" VerticalAlignment="Bottom" Maximum="90" Minimum="-90" Value="52" LargeChange="10" SmallChange="1" Width="360"/>
        <TextBlock Height="28" Margin="0,0,0,87" Name="label1" VerticalAlignment="Bottom" HorizontalAlignment="Left" Width="120">Latitude</TextBlock>
        <TextBlock Height="28" HorizontalAlignment="Left" Margin="0,0,0,62" Name="label2" VerticalAlignment="Bottom" Width="120">Longitude</TextBlock>
        <Slider Height="22" Margin="128,0,0,65" Name="slider2" VerticalAlignment="Bottom" Maximum="180" Minimum="-180" LargeChange="10" SmallChange="1" Width="360"/>
        <HyperlinkButton Height="28" HorizontalAlignment="Left" Margin="0,0,0,17" Name="hyperlink" VerticalAlignment="Bottom" Width="87" NavigateUri="astroclock.html" Content="Permalink" />
    </Grid>
</Canvas>

Evolving implementation here.

Syntax highlighting

2008-12-02T21:54:00.008Z

Following on from Scott Hanselman's post about the SyntaxHighlighter script, here's the secret sauce I needed to get it to work.

The separate hosting is the easy bit -- the trick was getting the code I needed to run the repainting into a window.onload, and to keep Blogger from mutilating the code by use of a CDATA section.

Et voilà!

<link href='.../SyntaxHighlighter.css' rel='stylesheet' type='text/css'/>
<script src='.../shCore.js' type='text/javascript'/>
<script src='.../shBrushPython.js' type='text/javascript'/>
<script src='.../shBrushRuby.js' type='text/javascript'/>
...
<script type='text/javascript'>//<![CDATA[
window.onload = function () {
    dp.SyntaxHighlighter.ClipboardSwf = ".../clipboard.swf";
    dp.SyntaxHighlighter.BloggerMode();
    dp.SyntaxHighlighter.HighlightAll("code",true,false,false,1,false);
}
//]]></script>

There is also a Scala brush for this -- but F# or Erlang will need to be hand-cranked first.

Later: hand cranking done & linked.

Python Swings; Ruby still doesn't

2008-11-29T19:23:00.014Z

Here's a proof-of-concept of running a Swing UI from Python on the CLR and the JVM, following up on earlier experiments (which I had forgotten about).

try:  
  # IronPython must load the J# redistributable libraries  
  # Using the FullName of the assemblies is probably overkill  
  import clr  
  clr.AddReference( 'vjslib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' )  
  clr.AddReference( 'vjssupuilib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' )  
except ImportError:  
  pass  
     
import javax.swing  
import java.awt.event  
import java.lang  
  
class Exit(java.awt.event.WindowAdapter):  
  def windowClosing(self, evt):  
    evt.getWindow().setVisible(False)
    evt.getWindow().dispose()
  def windowClosed(self, evt):  
    java.lang.System.exit(0) 
    
# With the 'import's above, we can now refer to things that are part of the  
# standard Java platform via their full paths.  
frame = javax.swing.JFrame("Window") # Creating a Java JFrame.  
label = javax.swing.JLabel("Hello")  
     
# We can transparently call Java methods on Java objects, just as if they were defined in Python.  
frame.getContentPane().add(label)  # Invoking the Java method 'getContentPane'.  
frame.addWindowListener(Exit()) # Need to do this for IronPython/J# which is barely JDK 1.2 level  
frame.pack()  
frame.setVisible(True)

which works happily with java -jar jython-complete.jar HelloSwing.py using Jython 2.5β0 and with ipy.exe HelloSwing.py for IronPython2.0RC2 as well.

I was briefly conflicted earlier this week when I saw that IronRuby had reached α2, and wondered about a JRuby/IronRuby UI, but the nigh-equivalent

begin  
  require 'vjslib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'  
  require 'vjssupuilib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'  
rescue LoadError  
  # This is the 'magical Java require line'.  
  require 'java'  
end    
        
class Exit < java.awt.event.WindowAdapter
  def windowClosing evt
    evt.getWindow.setVisible(false)
    evt.getWindow.dispose # make Closed
  end
  def windowClosed evt
    java.lang.System.exit 0 
  end
end

# With the 'require' above, we can now refer to things that are part of the  
# standard Java platform via their full paths.  
frame = javax.swing.JFrame.new("Window") # Creating a Java JFrame.  
label = javax.swing.JLabel.new("Hello")  
     
# We can transparently call Java methods on Java objects, just as if they were defined in Ruby.  
frame.getContentPane.add(label)  # Invoking the Java method 'getContentPane'.  
frame.addWindowListener(Exit.new()) # Need to do this for IronRuby/J# which is barely JDK 1.2 level  
frame.pack  
frame.setVisible(true)

which works on JRuby 1.1.5, fails in IronRuby α2 when trying to load Swing classes, and I've not found a dodge that will get me at even class metadata for the Swing files. Ruby's casing rules may make it impossible for IronRuby to recognise the lower-case Java namespaces (I had fun with that earlier with Ruby.Net).

LATER: I could do something VM dependent that takes the class name and argument list for construction in {Iron|J}Ruby. What is more tricky is inheritance like

class Exit < java.awt.event.WindowAdapter

which is a major use-case. I wonder if there might be a metaprogramming technique I can use for this...

LATER YET: The problem is that there is no exposed mechanism to wrap a System.Type as a Ruby Class; I could manually subclass everything that is not final that I want in a C# assembly with a My.JavaNamespace and emulate the way that JRuby maps namespace Java::JavaAwtEvent to a value named java.awt.event, but the nearest to automation I can see is to Reflection.Emit a whole ton of stuff.

So, for the moment, consider it a dead end.

Sign of the times -- Impedence mismatch of Scala and J#

2008-10-19T22:52:00.003+01:00

Following up on the Scala/.Net experiments, I took the interfaces for hashes and cyphers from my Java port; and replaced them in the Java world with a set of Scala traits, like:

trait MDA
{
    def update(data:Array[byte], offset:int, length:int): unit

    def update(data:byte) : unit

    def digest() : Array[byte] 
}

and it builds happily (though NetBeans 6.5 nightly complains it can't find the interface definition from the library in the code view).

Over to Visual Studio, with a suitable script to build the equivalent .dll -- and the implementing classes in J# fail to build because the Array[byte] maps to ubyte[] while a J# byte[] is an sbyte[], and so the J# compilation fails to match the signature of the Scala.Net trait -- and there's no Java ubyte type I can force everything to. And as byte arrays -- or at least bytes -- are the fundamental building block for streams, it looks like all the Java code has to get the elbow.

Why am I doing this sort of thing?

2008-09-25T18:57:00.004+01:00

Michael Foord (aka Fuzzyman) has wondered, while linking me, about why one would want to talk to Scala from IronPython.

Partly, it is the intellectual challenge of that particular sub-problem. But the broader goal is to see how far it is possible, in this polyglot age, to build a language stack -- along the lines the Ola Bini mused about at the start of the year -- that one could drop onto the JVM or .Net using essentially the same code base, give or take a thin C# layer to provide compatible APIs in .Net to those needed in Java but not present in J#.

While the Fan language is developed to be exactly so code-compatible across the JVM and CLR, it appears to be resolutely monoglot, with the platform hidden behind an abstraction layer.

The ulterior motive is to take a codebase that I already have in Java/J#, the first stages of a reimplementation in a VM targeted language, of my old CTClib project -- and be able to complete the reimplementation/upgrade to RFC2440 spec, in a platform neutral way, and without having to write any more Java, even if the IronPython/J#+Swing interface will be rather less Pythonic than I could manage if I were just doing it in Jython.

And why Scala as well? For any new static layer code, especially where being explicit about the types involved, such as around the crypto, will make things clearer.

One correction, though, to the remarks : ATM, I'm using the scala-net compiler, which is part of the Scala bazaar, and seems (from its failure modes when the J# libraries are in the loop) to be a different generator to that in IKVM. Not being IKVM means I'm also spared having to drag in the GNU Classpath library, where the server stuff has priority, and desktop UI support is still primitive -- its AWT is less functional, in many important respects, than Java 1.0 was.

Scaling the tower of Babel II

2008-09-24T15:38:00.009+01:00

Finally, a little digital extraction. Scala from IronPython, works as expected. In file operator.scala:

package com_ravnaandtines.operator

trait Process {
 def op(x:Int, y:Int) : Int
}

class Multiply extends Process {
  def op(x:Int, y:Int) : Int =
   x * y
}

class Perform69 {
 def sixByNine(x: Process) =
  x.op(6,9)
}

Set up as in the previous post, and build using %SCALA_HOME%\bin\scalac-net.bat operator.scala to get operator.msil -- no assembly is generated at this point. Edit the .msil file as required to point at the appropriate versions of the runtime (see previous post). Then ilasm .\Operator.msil /dll /output=operator.dll. Now fire up IronPython:

>>> import clr
>>> clr.AddReference("operator.dll")
>>> from com_ravnaandtines.operator import *
>>> dir()
['Multiply', 'Perform69', 'Process', '__builtins__', '__doc__', '__name__', 'clr']
>>> m = Multiply()
>>> p69 = Perform69()
>>> p69.sixByNine(m)
54
>>> class Adder(Process):
...   def op(self,a,b):
...     return a+b
...
>>> add = Adder()
>>> p69.sixByNine(add)
15
>>>

So long as we get our interfaces/traits from Scala, all is fine.

In order to do the same in the JVM world, I need to have done set CLASSPATH=%CLASSPATH%;C:\scala-2.7.1.final\lib\scala-library.jar; and then I can do exactly as above, but without the first two lines with clr. Those I can put in a try/except, for common coding.

Note -- it is neither necessary nor sufficient to put the Scala library jar in java.class.path or python.path properties.

Scaling the tower of Babel

2008-08-23T20:04:00.011+01:00

A few months ago, I was playing around with seeing how I could adapt an existing Java code base so that I could finish doing the work in better languages -- and have the code work on both the JVM and the CLR.

It seems that frustration will inevitably follow any attempt to incorporate any J# into the .Net versions of the stack (both IKVM and the Scala-net compilers bork when they see another Java in the stack) -- even though you can call Jython into Scala into Java on the JVM, this Tower of Babel cannot stand on the CLR with the J# runtime in the picture -- Java e.g. a Swing-based GUI using the J# extensions, has to stand off to one side.

That said, the Scala on .Net "hello world" with Scala 2.7.1 is comparatively painless.

Install the scala-msil package (sbaz install scala-msil)

Create file Hello.scala as

import System.Console

object test extends Application {
  Console.WriteLine("Hello world!")
}

In a Visual Studio command prompt, issue %SCALA_HOME%\bin\scalac-net.bat hello.scala
Copy %SCALA_HOME%\lib\*.dll to the current directory
Run the resulting test.exe

This uses the 1.0 version of mscorlib.dll; as a post-processing step you may wish to re-base against the .Net 2.0 version -- for this, you need to rebuild predef.dll and scalaruntime.dll using the same technique as needed to strong-name them after the fact; only this time you want to reset the mscorlib reference from

.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 1:0:5000:0
}

.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 2:0:0:0
}

as well as doing any strong-naming you may desire.

To set the assembly name, add -Xassem [assembly name] to the compile command, to generate hello.exe.

To control the type of the generated assembly (.dll rather than .exe), I've not found a way other than taking the generated [assembly name].msil and ilasming it manually. In particular trying to skip generating the unwanted .exe will mean figuring why using scalac.bat -target:msil always gives

error: fatal error: class scala.runtime.BoxedBooleanArray not found.

whatever I do with -Xassem-path or tweaking environment variables.

Integrating .Net with Erlang, part 8

2008-08-08T20:54:00.003+01:00

Gratuitous update on 08/08/08 with a small increment of unit testing -- now using Rhino Mocks 3.4 as well as home-grown ones, and NUnit 2.5 (alpha 3) for the neat

Assert.That( [delegate expression], Throws.Exception<[type]>())

syntax.

Drop here, as usual.

Erlang and Python

2008-06-07T11:03:00.007+01:00

Following the suggestion that I try TwOtp, I've tried the "kicking the tyres" with that library.

Adapting the sample code to work against my own local node seemed easy enough:

from twisted.internet import reactor
from erlang import OneShotPortMapperFactory, buildNodeName

def gotConnection(inst):
    return inst.factory.callRemote(inst, "file", "get_cwd"
        ).addCallback(gotResult)

def gotResult(resp):
    print "Got response", resp
    reactor.stop()

def eb(error):
    print "Got error", error
    reactor.stop()

epmd = OneShotPortMapperFactory('client@chloe.ravnaandtines.com', "cookie")
epmd.connectToNode("servernode@chloe.ravnaandtines.com").addCallback(gotConnection).addErrback(eb)
reactor.run()

which involves rather more baggage than using IronPython against my tidying of the C# port of jinterface, for just a "hello world!", given that the jinterface-derived "hello world!" actually does more work (passing arguments, testing results):

import clr
clr.AddReference("OtpErlang.dll")
from System import Array

from Tinesware.Otp import *
import Tinesware.Otp.Erlang as Erlang

cNode = Self("clientnode@chloe.ravnaandtines.com", "cookie")
sNode = Peer("servernode@chloe.ravnaandtines.com")
connection = cNode.Connect(sNode)

args = Array[Erlang.IObject]([Erlang.Integer(1), Erlang.Integer(2)])

connection.SendRpc("mathserver", "add", args)
sum = connection.ReceiveRpc()

if  sum.Value.ToInt32() != 3:
  print "Assertion failed, returned = " + sum.Value
else:
  print "OK!"

And, like py_interface, I find that this doesn't "just work" either:

C:\Documents and Settings\Steve\My Documents\code\babel\erlang>python twotphw.py

Got error [Failure instance: Traceback (failure with no frames): <class 'erlang.epmd.NodeNotFound'>: 1]

which will require a similar amount of rolling sleeves up and getting hands dirty to resolve.

Erlang with Python and Ruby

2008-06-06T21:19:00.006+01:00

Having played a bit with Erlang for real, in the sense of something non-trivial, even if not a complete product, I find I'm in the mood for more. But while it comes with some UI widgets, it's something more for headless use, and some client(s) in other languages doing the display.

To date, I can easily do Swing or WinForms as the UI toolkit -- but what about wxPython or FXRuby?

There are a number of bridges that I've found, but the closest to jinterface seem to be:

py_interface for Python (about 2 years old)
Un-named code drop on Google for Ruby (about 1 year old)

Neither of them have "just worked" for me in the way that the Java and C# versions have done against the simple mathserver example. Some of it might be down to my not figuring how the differences in the APIs are meant to work.

For py_interface, I had to comment out the line

[__import__(item) for item in __all__]

from init.py to get imports to work at all; and even then, the simple

from otp import *

def Handler(Result):
    print Result

cNode = erl_node.ErlNode("clientnode@chloe.ravnaandtines.com", erl_opts.ErlNodeOpts(cookie="cookie"))
print cNode
print cNode.Publish()
mBox = cNode.CreateMBox()
print mBox
mBox.SendRPC("servernode@chloe.ravnaandtines.com", 
    "mathserver", "add", [1, 2], Handler)
raw_input("wait")

just sits there; while the Ruby code

require 'erlang_node'
 
s_node = Erlang::Node.new("servernode@chloe.ravnaandtines.com")
c_node = Erlang::LocalNode.new("clientnode@chloe.ravnaandtines.com", "cookie")
connection = Erlang::Connection.new(s_node, c_node)
connection.sendRPC("mathserver", "add", [1,2])
sum = connection.receiveRPC()
puts "Fail" unless sum == 3

throws a different exception every time I run it, from some place or other inside the connection initializer.

So they both need deeper inspection than this simple kicking of the tyres, maybe modifying for R12 nodes, maybe just reverse engineering the kinks out of the driver programs.

Integrating .Net with Erlang, part 7

2008-05-27T18:24:00.005+01:00

Another update, after work during the holiday weekend.

By using a lot of spare cycles, I found a test case for the division code not previously exercised:

9659964175213286916418497243082984930780677040655778513759952405592597918851617939992802011783376153721017703858175 / 3248063261698017931917474008545947342556579729240422363527 = 2974068974926080862795323621370295193198999568425344221053

Also made the astoundingly obvious discovery that I needed the real Erlang port mapper dæmon running, so put a unit test in which spawns an Erlang process to contact. It works fine in NUnit-gui, but fails every time in the build time coverage plus test cycle, so is labelled as [Explicit]. I need to to write at least one test that handles the call in the other direction, but this is getting close to as far as can easily be done in unit test form.

And it's getting close to my usual summer break from coding in the evenings, so this is probably close to being "it" for a while.

Integrating .Net with Erlang, part 6

2008-05-20T19:46:00.005+01:00

Time for a major drop of the code.

It's been tidied up to remove most of the deprecation and to start unifying the comments as .Net XML. The BigInteger class has had a number of bugs found and fixed by unit testing (initialization, big left shifts, hex-string), and been made into a struct to reflect its value nature.

Pretty much all (say, 80%) of the stuff that can be tested statically in isolation has been, and the code has been revised with dependency injection and interfaces to do that (including a bug-fixed, back ported (to C# 2.0) version of NullObject.For<T>). Further testing will require some more extensive test set-up, such as spinning up a Java peer in a separate thread and talking TCP with it.

Code coverage is currently just over 60% for the Otp library, just over 70% for the ZLib code, for the static testing; beyond upgrading the wire types and the constants, the upgrade to 1.4 spec from 1.2 is not really begun.

Integrating .Net with Erlang, part 5

2008-04-23T20:30:00.011+01:00

I have a first drop that has all the C# input code written, and a start of some unit tests for interoperability with the J# build of jinterface 1.4. Nowhere near enough tests are written yet, and a couple of those that have been (encoding a 64-bit integer in J# and unpacking in C#) don't yet work. But at least it's a start.

Still to do are the full gamut of integer types, and the complex ones -- lists, tuples, ports and such, then replicate for writing by C# and reading in J#.

One fix to the tests was for my code -- I needed to take the surprising little-endian wire format for an extended integer, and reverse it to fit into the BigInteger class I got from Codeplex; the second was a fix to that class:

381c381,384
<    
---
> 
>             // BUGFIX -- use an accumulator rather than |=ing into
>             // m_digits[m_digits.DataUsed], which will be zero
>             DType accumulator = 0;
384c387,389
<     m_digits[m_digits.DataUsed] |= (DType)(array[offset + leftOver - i] << ((i - 1) * 8));
---
>                 DType digit = array[offset + leftOver - i];
>                 digit = (digit << ((i - 1) * 8));
>                 accumulator |= digit;
385a391
>             m_digits[m_digits.DataUsed] = accumulator;

Code drop for the curious at the usual site. The project assumes you have NCover and NUnit around your system to run the tests and take coverage stats.

Update: More tests, and a few more fixes, refactorings and extensions added 24-Apr. Doubles now get encoded in the new style and integers that fall into the Int64 range are handled so the jinterface code can handle them.

Update: 26-Apr -- Added an export to little-endian byte array for BigInteger and added unit tests for short and long BigInteger representations.

Update: 27-Apr -- A first pass of Java type to C# and back for all the Java wire types, and a start of interface breaking changes to shut FxCop up a bit. Still needs a lot of like-for-like based unit tests, and detailed code cleaning.

Update: 28-Apr -- Unit tests about complete for all the wire types. Now the fun really begins.

Update: 4-May -- Unit tests extended to the input and output classes, including adding compression via a modified version of the ComponentAce port JZlib. All code now clean after FxCop scrubbing. Still needs all the node related classes being compared with the jinterface 1.4 versions and tested.

Integrating .Net with Erlang, part 4

2008-04-22T23:01:00.004+01:00

So where are we going from here with the Otp.Net code? First off, if we bring the codebase into .Net 2.0 from its '04-'06 vintage, and make all the data classes in the Otp.Erlang namespace immutable -- returning ReadOnlyConnection(of T)rather than arrays as needed -- we can get rid of the ICloneable support. That lets us make Otp.Erlang.Object into an interface containing just the encode method(dropping the decode method, or making it an extension method as per C# 3.0).

Then we can use some subset of the fields used in equality tests to do GetHashCode properly, and replace arrays with growable collections.

Next, fix the range check on Double's floatValue method, and make it return a nullable value:

    public float? floatValue()
    {
        if (d > Single.MaxValue || d < Single.MinValue)
            return null;

        return (float) d;
    }

and rename Long as Integer, kill both their derived classes and make Integer carry a BigInteger to which we can delegate conversions returning nullables.

That will mean some fixing up of input and output, but we need that anyway to handle new-style float values encoded as 8-byte big-endian blobs (and the change of tense means that this is where we move past what I've done so far).

To come after that will be making GenericQueue delegate to Queue(of IObject), and then a general scrub done in comparison with jinterface 1.4, before it will be sorta-releasable.

At the moment it builds, but will not work, so no interim archive is yet publicly available.

Integrating .Net with Erlang, part 3

2008-04-19T23:36:00.008+01:00

A kind anonymous has pointed out that someone has already done the job of porting an earlier version (1.2.1.1) of the jinterface code, even though it has not made its way into the standard distro. Via this blog post, one eventually gets to the sourceforge repository details for it.

This actually does work with IronPython (after the trivial edits to load the appropriate assembly and the slightly different class names) and without the J# runtime being involved.

I spotted the comment after spending a little while porting the 1.4 code. Actually the underlying Java code is quite horrid -- to start with there are lots of methods with the same name as fields; and a lot of identically named fields shadowing ones elsewhere in the inheritance hierarchy; and methods and fields are used interchangeably. Then there are blocks of cut-and-paste and tweak code; plenty of swallowing of exceptions and blurred responsibilities (where one object performs a series of manipulations on a member of another object). And there there is this (where d is a double):

  public float floatValue()
    throws OtpErlangRangeException {
    float f = (float)d;

    if (f != d) {
      throw new OtpErlangRangeException("Value too large for float: " + d);
    }

    return f;
  }

which doesn't look too tested to me -- what's this (f != d), in case you couldn't spot it. This is carried over into otp.net, and, I think, shows how little floating point is used.

I fixed up the name clashes and some of the blurred responsibility in the Java, then ran the conversion over; and by taking out the BigInteger stuff (not there in otp.net's Long class -- haven't checked the rest of these), compression, most of the floating point, got what was left to build, though it currently fails in the challenge/response handshake (where otp.net has replaced the roll-your-own MD5 with calls into System.Cryptography).

Where could we go from here?

Two main things need doing. The otp.net code needs to be brought up to the 1.4 standard and make better use of .Net 2.0 e.g. for tracing; and it needs to be refactored to clean up inherited code smells.

Ideally, some of the changes should be back-ported into jinterface, too.

To date, this includes fallback to Java's file based location for cookie and a replacement for some of the non-ported tracing code (logging the remote endpoint).

Integrating .Net with Erlang, part 2

2008-04-18T20:39:00.007+01:00

So, we have pure J#. Where do we go from here?

C#

The body of the driver class carries over unchanged but for the Exception type. The driver project has to itself reference vjslib, as well as the jinterface assembly but is otherwise a standard C# class-with-main, and "just works".

using System;
using com.ericsson.otp.erlang;

public class ClientNode
{
 
 [STAThread]
 public static void  Main(System.String[] _args)
 {
  
  OtpSelf cNode = new OtpSelf("clientnode", "cookie");
  OtpPeer sNode = new OtpPeer("servernode@chloe.ravnaandtines.com");
  OtpConnection connection = cNode.connect(sNode);
  
  OtpErlangObject[] args = new OtpErlangObject[]{new OtpErlangLong(1), new OtpErlangLong(2)};
  connection.sendRPC("mathserver", "add", args);
  OtpErlangLong sum = (OtpErlangLong) connection.receiveRPC();
  if (sum.intValue() != 3)
  {
   throw new System.SystemException("Assertion failed, returned = " + sum.intValue());
  }
  System.Console.Out.WriteLine("OK!");
 }
}

How do we divest ourselves of the dependency on the J# runtime?

The Java Language Conversion Assistant generates C# code that doesn't compile, and would require a moderate amount of patching up for differences between superficially similar API classes. It is by no means a one evening exercise.

The OtpErlang.jar similarly fails to be converted by the jbImp tool, but IKVM converts it nicely; so at the penalty of having 27Mb worth of IKVM.OpenJDK.ClassLibrary.dll and IKVM.Runtime.dll instead of 3.7Mb of J#, driving the ~100kb dll, you can build the C# program and it passes this simple proof of concept.

IronPython and the DLR

Porting the code is easy enough

import clr
clr.AddReference("jinterface.dll")
from System import Array

from com.ericsson.otp.erlang import *

cNode = OtpSelf("clientnode", "cookie")
sNode = OtpPeer("servernode@chloe.ravnaandtines.com")
connection = cNode.connect(sNode);

args = Array[OtpErlangObject]([OtpErlangLong(1), OtpErlangLong(2)])
connection.sendRPC("mathserver", "add", args)
sum = connection.receiveRPC()
if (sum.intValue() != 3):
  print "Assertion failed, returned = " + sum.intValue()
else:
  print "OK!"

A direct conversion, the only slightly tricky bit being coercing the list to an Array.

This works fine when calling into the J# built library, as expected. IKVM needs the latest version -- before 0.36.0.11 (see comments), though, is a different story.

>\IronPython-2.0B1\ipy.exe Program.py
Traceback (most recent call last):
  File Snippets, line unknown, in Initialize
  File OtpErlang, line unknown, in connect
  File OtpErlang, line unknown, in .ctor
  File OtpErlang, line unknown, in .ctor
  File IKVM.OpenJDK.ClassLibrary, line unknown, in .ctor
  File IKVM.OpenJDK.ClassLibrary, line unknown, in init
  File IKVM.OpenJDK.ClassLibrary, line unknown, in getContext
  File IKVM.OpenJDK.ClassLibrary, line unknown, in getStackAccessControlContext
  File IKVM.Runtime, line unknown, in getStackAccessControlContext
  File IKVM.Runtime, line unknown, in GetProtectionDomainFromType
  File IKVM.Runtime, line unknown, in getProtectionDomain0
  File IKVM.Runtime, line unknown, in GetProtectionDomain
SystemError: The invoked member is not supported in a dynamic assembly.

Trace taken with 0.36.0.5, from last December. Fortunately, that wasn't some fundamental architectural thing being run into.

Integrating .Net with Erlang, part 1

2008-04-18T18:46:00.010+01:00

So, you can integrate Java with Erlang, using the jinterface code supplied as part of the standard Erlang distribution. It's Java, so can we do this with .Net?

Let's start with J#. We set up the erlang source

-module(mathserver).
-export([start/0, add/2]).

start() ->
   Pid = spawn(fun() -> loop() end),
   register(mathserver, Pid).

loop() ->
   receive
      {From, {add, First, Second}} ->
        From ! {mathserver, First + Second},
        loop()
   end.

add(First, Second) ->
   mathserver ! {self(), {add, First, Second}},
   receive
      {mathserver, Reply} -> Reply
   end.

and test it with

>erl -name servernode@chloe.ravnaandtines.com -setcookie cookie
Eshell V5.6  (abort with ^G)
(servernode@chloe.ravnaandtines.com)1> node().
'servernode@chloe.ravnaandtines.com'
(servernode@chloe.ravnaandtines.com)2> pwd().
jinterface          mathserver.erl
ok
(servernode@chloe.ravnaandtines.com)3> c(mathserver).
l{ok,mathserver}
(servernode@chloe.ravnaandtines.com)4> s().
jinterface          mathserver.beam     mathserver.erl
ok
(servernode@chloe.ravnaandtines.com)5> mathserver:start().
true
(servernode@chloe.ravnaandtines.com)6> mathserver:add(1,2).
3
(servernode@chloe.ravnaandtines.com)7>

Now fire up Visual Studio, and create a J# class library project. Add existing items to copy the jinterface code into the project, and build it.

Cannot find type 'java.lang.ref.WeakReference'

Well, that's not too bad. Give it a stub, and build again:

package java.lang.ref;
import System.*;

public class WeakReference extends System.WeakReference
{
    public WeakReference()
    {
    }
}

gives us

error VJS1227: Cannot find constructor 'java.lang.ref.WeakReference(com.ericsson.otp.erlang.OtpMbox)'
error VJS1223: Cannot find method 'get()' in 'java.lang.ref.WeakReference'

So give it what it needs:

package java.lang.ref;
import System.*;

public class WeakReference extends System.WeakReference
{
    public WeakReference(Object o)
    {
        super(o);
    }
    public Object get()
    {
        try
        {
            return this.get_Target();
        }
        catch (InvalidOperationException ioe)
        {
            return null;
        }
    }
}

And it builds. Now put in the driver program:

import com.ericsson.otp.erlang.*;

public class ClientNode
{

    public static void main(String[] _args) throws Exception
    {

        OtpSelf cNode = new OtpSelf("clientnode", "cookie");
        OtpPeer sNode = new OtpPeer("servernode"+"@"+"chloe.ravnaandtines.com");
        OtpConnection connection = cNode.connect(sNode);

        OtpErlangObject[] args = new OtpErlangObject[]{
            new OtpErlangLong(1), new OtpErlangLong(2)};
        connection.sendRPC("mathserver", "add", args);
        OtpErlangLong sum = (OtpErlangLong)connection.receiveRPC();
        if (sum.intValue() != 3)
        {
                throw new IllegalStateException("Assertion failed, returned = " + sum.intValue());
        }
        System.out.println("OK!");

    }

}

Now, run that and get:

OK!

So, we have first light!

Get the goodies from here.

Blast from the past

2008-04-08T20:48:00.003+01:00

I had need to play with .ico files for the first time in ages the other day, and pulled out my old Java IcoImage utility to start with -- only to realise that it didn't handle modern 32-bit depth icons.

So I fixed it.

Latest versions here.

PassiveFTP 1.11

2008-02-19T21:24:00.003Z

Here or here.

PassiveFTP 1.11 — build 1.11.2971.37821 — mitigates the upload rate problem I've been having, by doing some unexpectedly effective tweaks to socket parameters.

This came after having do do some throughput optimizations on XP at work for code that worked just dandy on Windows server OSs from NT4 up. There it was just a case of upping the send buffer size. Here, that was ineffective, but I did get a big boost from by sending only small packets and disabling Nagle coalescence.

The new default upload rate is 128kb/s (1ms sleep between successful sends of 128-byte packets), which gave the best rate for me, up from an old and unreliable 2kb/s.

This will of course feed back into the FePy version whenever I get the enthusiasm for doing all the plumbing.

If only...

2008-02-04T19:37:00.000Z

Ultimately, the big splat from yesterday tracks down to calling the stub of


JbImp error: Couldn't find class 'java.lang.InheritableThreadLocal'

from the class constructor of Console$. Now it would not be beyond the wit of man to emulate that new class in a library shim and leave everything else as I found it, but... More annoying are the extensions to fundamental classes dating back to JDK 1.0 days, especially final ones like String


JbImp error: Unresolved Method 'public static java.lang.String java.lang.Character.toString(char)'
JbImp error: Unresolved Method 'public static java.lang.String[] java.lang.String.split(java.lang.String)'

where (absent some incantation to splice J#'s AWT into the namespace of an OpenJDK dll or similar), tackling it means modifying the Scala library code to retrofit. Oh, yes, and even then there are some places where the Scala bytecode manages to throw jbimp for a wobbly.

But if I keep to a fairly restricted subset (and for the moment I'd be interested in using it when writing a parser of OpenPGP), I might get away with a very small port of the runtime and library.

Oh, well, like I said, hands on.

Under my nose all the time

2008-02-03T23:28:00.001Z

Another of those hidden things about J# -- the JbImp tool, which takes .class files and spits out J# assemblies. Just what I wanted, but was looking in the general freeware community for.

Now the full scala-library.jar has a lot of post-J# Java API calls, but by cutting out a lot of the code I can get

>"\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\Jbimp.exe" /t:library /usestubrefs /out:scala-lib.dll /recurse *.class

to work on the remainder of it, and then

>"\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\Jbimp.exe" /t:library /usestubrefs /r:..\JavaLayer.dll ..\scala-lib.dll /out:SClass.dll /recurse *.class

And then build an IronPython project against it and get…

>PythonLayer.exe
In class com_ravnaandtines.jlayer.JClass1.doSomething(), printing from Jython
11
In class com_ravnaandtines.jlayer.JClass1.doSomething(), printing Hello Babel!
Traceback (most recent call last):
  File scala-lib, line unknown, in .cctor
  File vjslib, line unknown, in checkAndThrowException
  File src\Program.py, line 24, in Initialize
  File , line 0, in NonDefaultNew##27
  File , line 0, in .ctor##29
  File SClass, line unknown, in .ctor
SystemError: The type initializer for 'scala.Console$' threw an exception.

which is progress, and most of what the JVM version does; but as this

    private int len;
    private JClass1 in = new JClass1();

    public SClass(String token)
    {
        len = in().doSomething(token);
        Console$.MODULE$.println(BoxesRunTime.boxToInteger(len()));
    }

is the relevant part of the SClass code decompiled, the failure means that although it has executed some Scala derived code, it has fallen over at the first sniff of runtime.

I hope that a more measured winnowing of the runtime will fix this. But not tonight.

No love for client-side Java

2008-02-03T17:56:00.000Z

It seemed a nice idea, having a deeper stack than just Python on Java that could build cross-VM apps from one code-base. But alas…

The only Java-on-.Net runtime I've found that has even half-way decent support for client-side apps (enough of AWT to support the GWT, or any flavour of Swing) is MSFT's J#, especially at 2.0
The obvious cross-compilers (scala-net, if I could get it to work; IKVM) want to use anything else they can except the J# runtime, and don't offer the facility to choose the one you want
Meanwhile scala's bytecode is just plain beyond decompilation to Java by obvious decompilers

So, I'd have to write the tools myself if I want to do this. It wouldn't even be a simple fork of IKVM to re-point references, and there are also its own special attributes decorating the classes. It might be simpler to tweak the scala compiler, since the .Net-bit islooked smaller in that (and it would improve my hands-on with the language).

Later:

Building a replacement scalaruntime.dll based on the .Java files from the scala-library source jar (one tweak for the older Java dialect with doesn't have the JDK 1.4 Character.toString()), plus a J# version of scala.runtime.SymtabAttribute based on the C# one in SVN, easy. Finding that the scala compiler source jar omits all the MSIL compiler files (from https://lampsvn.epfl.ch/trac/scala/browser/msil/trunk), more of a pain.

With a mscorlib 2.0 and my new runtime, and the predef.dll moved out of the way, I have to figure out why System.String is causing a wobbly:


java.lang.RuntimeException: PEModule.getTypeDefOrRef(): TypeSpec
        at ch.epfl.lamp.compiler.msil.PEModule.getTypeDefOrRef(PEModule.java:206)
        at ch.epfl.lamp.compiler.msil.PEType.loadInterfaces(PEType.java:309)
        at ch.epfl.lamp.compiler.msil.Type.initInterfaces(Type.java:944)
        at ch.epfl.lamp.compiler.msil.Type.getInterfaces(Type.java:907)
        at scala.tools.nsc.symtab.clr.TypeParser.parseClass(TypeParser.scala:88)
        at scala.tools.nsc.symtab.clr.TypeParser.parse(TypeParser.scala:58)
...
java.io.IOException: type 'System.String' is broken

I'd probably have to rework a whole lot of stuff that shims a virtual Java library layer. Which would have to start with identifying and encapsulating the shimming.

One step forwards...

2008-02-01T18:24:00.000Z

Based on the helpful comment on my last post, I can run IKVM against 2.0 assemblies, but hit a different obstacle -- a critical failure linking against the J# runtime, leading to a bug report.

It seems to run OK auto-binding to the OpenJDK, but then I need to frob the reference somehow. Perhaps it is back to the scala-net class loader issue...

Le sigh

Later...

I wondered. We won't have much of Swing, but do we have enough of AWT to run the GWT? So I converted TestGadget.jar and gwt221.jar and put all the images and .dlls into the same folder and ran it and...

>TestGadget.exe
Exception during event dispatch:
cli.System.NotImplementedException: The method or operation is not implemented.
        at cli.ikvm.awt.NetComponentPeer.createImage(Unknown Source)
        at java.awt.Component.createImage(Component.java:2522)
        at dtai.gwt.Gadget.createImage(Gadget.java:2045)
        at dtai.gwt.ImageGadget.validateImage(ImageGadget.java:255)
        at dtai.gwt.ImageGadget.paintForeground(ImageGadget.java:283)
...

and while much of the applet showed, the (re-)paint behaviour was terrible, and the applet could easily get into a state where it was spinning at almost 100% CPU.

Le sigh²

Confusion of Babel

2008-01-31T22:07:00.001Z

I really would like to put a functional layer in the cross-VM stack, but...

Ocamljava and F# might be code compatible, but the former has a cumbersome build and interop process (and doesn't seem to advertise call-from-Java); while Bigloo seems entirely silent on interop.

Cross-compiling Scala class bytecodes to CLR doesn't seem to be a goer -- IKVM balks at referencing CLR 2.0 assemblies. I might be able decompile the classes and rebuild them via J# -- the hiccup there is that I would have to translate the runtime .jar as well -- my simple "Hello World!" level class decompiles to invoke scala.runtime.BoxesRunTime, but that's not in any of the scalac-net assemblies. On the up-side that would remove any nagging issues about CLR 2.0 compatibility, though; on the down-side, it would mean that I need a decompiler able to handle the generated code... Jode -- which I happened to have to hand -- balked at half the classes in the runtime; Jad seemed to have dealt with them all, though with a lot of goto statements replacing things like jumps out of loops -- but leaves a lot of JVM INSTR monitorenter;...JVM INSTR monitorexit; where a synchronized{} block should be, and possibly other horrors.

Meanwhile I took another prod at scalac-net, using the magic -Ydebug flag:

>\scala-2.6.1-final\bin\scalac-net -d classes -Xassem-path JavaLayer.dll SClass.
scala  -Ydebug
[running phase parser on SClass.scala]
[running phase namer on SClass.scala]
java.lang.ClassCastException: scala.tools.nsc.symtab.Symbols$ClassSymbol cannot
be cast to scala.runtime.Nothing$
        at scala.tools.nsc.symtab.clr.TypeParser.parseClass(TypeParser.scala:77)

        at scala.tools.nsc.symtab.clr.TypeParser.parse(TypeParser.scala:58)
        at scala.tools.nsc.symtab.SymbolLoaders$MSILTypeLoader.doComplete(Symbol
Loaders.scala:263)
        at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoad
ers.scala:54)
...
java.io.IOException: type 'scala.Predef' is broken
(scala.tools.nsc.symtab.Symbols$ClassSymbol cannot be cast to scala.runtime.Nothing$)
...

Kudos to the scala implementors -- the line numbers match up down the stack, and show the culprit to be

      unpickler.unpickle(symtab, 0, clazz.asInstanceOf, staticModule.asInstanceOf, typ.FullName);

which looks like it's doing the right sort of thing; and there's nothing particularly odd about the scala.Predef class in the assembly; and I get the same error trying to compile some of the examples in the distro...

Next diversion, rebuilding scalac with more instrumentation, piling the tower higher.

A tower of Babel

2008-01-30T22:09:00.001Z

// JClass1.java
package com_ravnaandtines.jlayer;
public class JClass1
{
    public JClass1()
    {
    }
    public int doSomething(String arg)
    {
        System.out.println("In "+this.getClass().toString()+".doSomething(), printing "+arg);
        return arg.length();
    }
}

//SClass.scala
package com_ravnaandtines.slayer

import com_ravnaandtines.jlayer

class SClass(token: String) {
 private val in = new com_ravnaandtines.jlayer.JClass1()
 private val len = in.doSomething(token)
 Console.println(len)
}

//Python.py
import com_ravnaandtines.jlayer.JClass1

direct = com_ravnaandtines.jlayer.JClass1()
val = direct.doSomething("from Jython")
print val

import com_ravnaandtines.slayer.SClass
x = com_ravnaandtines.slayer.SClass("Hello Babel!")

Build the first into JavaLayer.jar and copy it alongside the second

\scala-2.6.1-final\bin\scalac -classpath .JavaLayer.jar -d classes SClass.scala
cd classes
"\Program Files\Java\jdk1.6.0\bin\jar.exe" -cf ..\ScalaLayer.jar .

Then run the last with the two jars and the scala runtime jar in your path

In class com_ravnaandtines.jlayer.JClass1.doSomething(), printing from Jython
11
In class com_ravnaandtines.jlayer.JClass1.doSomething(), printing Hello Babel!
12

I had hoped that I could do the same in .Net, but having built the J# JavaLayer.dll, I got

>\scala-2.6.1-final\bin\scalac-net -d classes -Xassem-path JavaLayer.dll SClass.
scala
error: error while loading Predef, type 'scala.Predef' is broken
(scala.tools.nsc.symtab.Symbols$ClassSymbol cannot be cast to scala.runtime.Noth
ing$)
one error found

so haven't yet managed to make a Scala-derived DLL for IronPython to consume. And that doesn't look like something I might obviously have done wrong.

Steps forwards, steps back.

2008-01-23T22:49:00.000Z

It would be nice to have the getX()/setX() map to a property, syntactically, in general use Jython as well, along the lines of the previous essay. Alas, that simple "X indirects to getX()" method doesn't seem to work as is. I wonder if I'm relying on things that are a more modern version than Jython yet supports. I need to investigate more.

Meanwhile, on another front, even using the invaluable vjsresgen tool, I have to have platform specific interludes to handle resources -- especially to convert from .png format to something that the back-version Java support understands.

Colour me slowpoke ... Swing on J#

2008-01-23T15:49:00.000Z

I'd not caught this before:

The Microsoft Supplemental UI Library provides much of the functionality described in the Java 2 JFC Swing specification. It also adds support for much of the functionality found in the JDK 1.2 java.util package.

…

Version 2.0 of the Supplemental UI Library is included as part of the Visual J# Version 2.0 Redistributable Package. No additional download is required.

…

To access the functionality in the Supplemental UI Library, simply add a reference to VJSSupUILib.dll in your project.

Betterer and betterer…

Of course this throws a little bit of a spanner in the works of the Jythonizer effort -- I might have subclassed javax.swing.JFrame, but not the component that I get from calling its shiny new contentPane attribute delegating to getContentPane(). Oh, well. At least I might be able to get some cute tricks along the lines of some of the ideas in Swing Hacks instead.

Making IronPython and J# more Jythonic -- Part 1

2008-01-22T23:08:00.001Z

For the Nine Unknown Men who might also be attempting this sort of thing.

Persistance pays…

import dtai.gwt.GadgetFrame

def my__getattr__(self, name):
  if name.startswith("__"):
    raise AttributeError(name)
  getter = "get"+name.capitalize()
  if getter in dir(self):
    tmp = getattr(self, getter)
    return tmp()
  iser = "is"+name.capitalize()
  if iser in dir(self):
    tmp = getattr(self, iser)
    return tmp()
  raise AttributeError(name)
        
class Local:    
  def getFred(self):
    return "Fred!"
  
Local.__getattr__ = my__getattr__
   
test = Local()   
print test.fred

class WrappedGadgetFrame(dtai.gwt.GadgetFrame):
  pass

WrappedGadgetFrame.__getattr__ = my__getattr__
dtai.gwt.GadgetFrame = WrappedGadgetFrame

window =  dtai.gwt.GadgetFrame()
print window.visible

when executed, yields

C:...>PyInterop.exe
Fred!
False

which is the get() side of attributes done at the cost of a pass of subclassing. The equivalent set() shouldn't be too difficult, but automating the subclassing will be the next challenge.

Then adding the listener-based attributes; and finally, hiding it from Jython…

How Jythonic can it get?

2008-01-21T21:09:00.000Z

Following up from my previous post, I've been thinking that it would be nice if I could make the code there a bit more Jythonic -- wrapping Java-style bean properties and event handlers transparently in the IronPython/J# world, where the inherent properties will be the Capitalized .Net ones, and we only have the .Net delegate-style handlers.

I will need to experiment to see exactly how open the IronPython proxy classes are to extension, first.

So many projects, so little free time. And I really need to do more with F# and PowerShell, not to mention wanting to try writing a generic protocol-terminating proxy framework in Erlang, probably with SOCKS5 as an exemplar protocol.

LATER

As I suspected, setting Local.__getattr__ = my__getattr__ for a function of suitable signature works where Local is a pure IronPython class; but for an imported .Net class I get


TypeError: can't set '__getattr__' in dictproxy

even with the indirect approach of something like dtai.gwt.GadgetFrame.__dict__["__getattr__"] = my__getattr__.

So Java/J# idioms at the interface it may have to be.

Pushing at an open door…

2008-01-16T22:35:00.003Z

Compared with the travails required to shim Ruby.Net and JRuby onto a shared Java/J# code platform, doing the same for Jython (in NetBeans 5.0)/IronPython (using the DevStudio extension build for IronPython 1.1) was trivial. In fact most of the work was re-gearing my mind to Python after thinking in Ruby for some weeks.

Apart from the fact that that the IronPython build ignores any directory structure and smooshes all classes into the one assembly and considers that the package, which will impact on the broader code shape for any project, there is no shimming. Just build your assembly with references to the appropriate external files and you don't even need all the clr.AddReference to do it by hand

Doing the same as the J# and Ruby.Net — string and baling wire example, in a project that references the appropriate assemblies just requires the same Python code as a Jython build with the equivalent .jar files on the classpath

## pyinterop.py
import dtai.gwt.GadgetFrame
import java11.awt.event.WindowListener
import java.lang.System

window = dtai.gwt.GadgetFrame()
window.setSize (800, 600)
window.setTitle("Proof of Concept")

class Handler(java11.awt.event.WindowListener):
  def __init__(self, window):
    self.window = window

  def windowClosing(self, e):
    self.window.hide()

  def windowClosed(self, e):
    java.lang.System.exit(0)

  def windowActivated(self, e):
    pass

  def windowDeactivated(self, e):
    pass

  def windowDeiconified(self, e):
    pass

  def windowIconified(self, e):
    pass

  def windowOpened(self, e):
    pass

adapter = Handler(window)
window.addWindowListener(adapter)
window.show()

It still works in either case, if I simply use the WindowAdapter class and just override the methods I need. W00t! Indeed, so far it almost seem too easy!

Now I just have to keep working around the weird problems I ran into last time around this loop. Keeping Java beneath everything this time around should help.

Ruby.Net and Ruby.Net — foiled again

2008-01-15T20:25:00.000Z

A reboot (of the machine, and myself) later, I realised that the obscure "Wrong argument type String (Class expected)" was the result of a sleep-addled 'C'-drenched brain putting include 'Wizard' where require 'Wizard' was expected.

Not so good with the memory issue. After a couple of hours debugging and finally getting the code to interoperate, DevStudio was peaking at over 370Mb of memory, and still thrashing (on a 512Mb machine), while building a project with < 1000 lines of Ruby. NetBeans, by comparison has about 150Mb (if I have a lot of files open)in the IDE and and under 40Mb in the spawned application.

I've put the Interop example code -- showing how to call from shared code JRuby/Ruby.Net to shared-code Java/J# libraries. main.rb calls Wizard.rb which calls through the interop layer defined by .jsl files for Ruby.Net->J#, and Interfaces.rb plus the .java files for JRuby->Java -- on one of my sites, for those who are interested in taking this further.

Until I upgrade my hardware to something that permits more than a toy Ruby.Net layer, though, I don't think I'll be taking the ideal of a shared code/multi-VM project further with Ruby. Which sucks, because that has the best IDE support (I don't even know if the Jython add-in for NetBeans still works at 6.0).

Still a little experimentation is needed with that, and with Scala, to get to an equivalent proof of concept stage.

Ruby.Net and Ruby.Net — first steps

2008-01-14T22:14:00.001Z

Not a typo, alas.

I was testing calling from a Ruby.Net .exe file into a Ruby.Net .dll, to see if partitioning like that would work; though not in such a simple setup as that : I was trying to test a Ruby implementation of a wizard dialog using the GWT... So I create a Testbed.Net.exe that refers to gwt_ruby.dll (which is built from a single source file Wizard.rb).

That fails with a Key Not Found Exception, and looking at the Ruby.Net code, and ildasm-ing the assembly, I discern that it was looking for a gwt_ruby.rb file having been built into the project. So I give it one to satisfy it

require 'Wizard'
class Gwt_Ruby
end

Later...

The Ruby.Net build is being a voracious memory hog on my 3 year old machine with only 512Mb RAM -- even for only a few hundred lines of Ruby code each build and test is taking more than half an hour of thrashing madly at almost 0% CPU. And I thought that NetBeans was memory hungry -- that I can work on in realtime, with the same code in JRuby. I'm not sure that testing this is going to be practical, even without having to work around packaging quirks.

It's almost enough to drive me to Scala!

Phase 2 — “The best code is code someone else has already written and debugged.”

2007-12-28T21:30:00.000Z

So goes the old saying.

And looking at the almost decade old Angerona project code, I see there's a lot of code that is written, was lightly debugged, probably works, and just needs refactoring. After all, it was the lack of a ZLib that stopped that effort dead, not anything with the existing code.

So, the next phase will be to put together an infrastructure library (jar/dll) from the useful bits, which should be enough to get to the point of being able to parse a key-ring or similar binary file to OpenPGP format.

At that point, some weeks hence, it will make sense to put a first bit of GUI on top; and start filling in the gaps in a less clunky language than Java where possible.

angerona.algorithms 1.0.2915.29952

2007-12-25T17:16:00.000Z

Here. and a back-up here if Demon's DNS flakes out.

A complete replacement for the old crypt.zip, with source and pre built .jar and .dll, unit tests, and a .Net build of junit 3.8.2 -- includes:

JZlib (inflate/deflate)
Blowfish, CAST5, DES (including s3' and key-dependent variants), IDEA, Safer, Square, TEA, ThreeWay, TripleDES (encryption)
BlockCypherHash, Haval, MD5, RIPEM160, SHA0, SHA1 (message digests); and
Bignum (mutable multiple precision integers).

Needs the J# redistributables for .Net use,

Happy holidays! and 100th post!

J# and Ruby.Net — string and baling wire

2007-12-20T20:01:00.001Z

A proof of concept

//Factory.java
public class Factory
{
    private Factory(){}

    public static Object Build(String classname) 
        throws InstantiationException, IllegalAccessException, 
        ClassNotFoundException
    {
        Class target = Class.forName(classname);
        return target.newInstance();
    }
}

//Utility.java
package Tinesware.Adapter;

public class Utility
{
    private Utility() { }

    public static void exit(int status)
    {
        java.lang.System.exit(status);
    }
}

//WindowAdapter.java
package Tinesware.Adapter;

public class WindowAdapter implements java11.awt.event.WindowListener  
{
    private final Ruby.Object payload;

    public WindowAdapter(Ruby.Object arg)
    {
        payload = arg;
    }

    public void windowActivated(java11.awt.event.WindowEvent e)
    {
    }

    public void windowClosed(java11.awt.event.WindowEvent e)
    {
        System.out.println("window Closed");
        Object[] args = {e};
        Ruby.Runtime.Eval.Call(payload, "windowClosed", args);
    }

    public void windowClosing(java11.awt.event.WindowEvent e)
    {
        System.out.println("window Closing");
        Object[] args = { e };
        Ruby.Runtime.Eval.Call(payload, "windowClosing", args);
    }

    public void windowDeactivated(java11.awt.event.WindowEvent e)
    {
    }

    public void windowDeiconified(java11.awt.event.WindowEvent e)
    {
    }

    public void windowIconified(java11.awt.event.WindowEvent e)
    {
    }

    public void windowOpened(java11.awt.event.WindowEvent e)
    {
    }
}

##Program.rb
require 'Tinesware.Adapter.dll'
require 'gwt221.dll'

window = Tinesware::Adapter::Factory.Build("dtai.gwt.GadgetFrame");
window.setSize (800, 600)
window.setTitle 'Proof of Concept'
window.show

class WindowHandler
  def initialize(w)
    @window = w
   end

  def windowClosing(e)
    @window.hide
  end
  def windowClosed(e)
    Tinesware::Adapter::Utility.exit 0
  end
end

adapter = Tinesware::Adapter::WindowAdapter.new(WindowHandler.new(window))
window.addWindowListener adapter

It only works grace of Ruby.Runtime.Eval.Call() being public, but that's all I need; that the event argument gets through is a bonus.

It is a mild hack, it does involve repetitive boilerplate coding, but you only have to do it for a few interface classes. And it will need some smoothing over on the Ruby side to make it look less awkward with code that's supposed to interoperate with Java as JRuby.

Extra bonus -- if the Call()'d method doesn't exist, it does nothing silently, so you only have to implement the parts of an interface you actually want.

J# and Ruby.Net — foiled again

2007-12-17T20:07:00.001Z

Carrying on from last post…

// C#
        public static void typeTest(object arg)
        {
            System.Type wtf = arg.GetType();
            Console.WriteLine(wtf.ToString());
            Console.WriteLine("Base = "+wtf.BaseType.ToString());

            System.Type[] iface = wtf.GetInterfaces();
            Console.WriteLine("# interfaces = " + iface.Length);
            foreach (System.Type t in iface)
                Console.WriteLine(":: "+t.ToString())
        }

##Ruby caller
q = Observer.new
JsharpToRuby::ClassDictionary.typeTest q

yields disappointing results

Observer
Base = Ruby.Object
# interfaces = 0

Oh dear! it doesn't seem to work the naïve way for proper interfaces

// C#
namespace JsharpToRuby
{
    public interface ITest
    {
        Boolean isPresent();
    }

##Ruby 
class EyeTest 
  include JsharpToRuby::ITest
  def isPresent
    1
  end
end
q = EyeTest.new
puts q.class
q.class.ancestors.each { |x| puts x }
JsharpToRuby::ClassDictionary.typeTest q

yields

EyeTest
EyeTest
ITest
Object
Kernel
EyeTest
Base = Ruby.Object
# interfaces = 0

This is of course the bit missed out in the interop tutorial, alas; I think for the good and simple reason that it is not yet implemented (from a quick inspection of the code).

*le sigh*

Maybe not all is lost. It would mean more hand-rolled adapter classes to wrap the objects and delegate by reflection.

But later, definitely, later.

J# and Ruby.Net — further steps

2007-12-16T20:53:00.001Z

So what do we need Java classes for in a JRuby/Ruby.Net environment?

Things that we have to new because that defines system services in an unambiguous fashion — I'm thinking encodings for I/O and GUI components here, as well as libraries already written that we don't want to port just yet (my algorithms library); and interfaces/abstract classes we want to make concrete (needing to be an ActionListener or similar).

Concrete things are easy; abstractions are trickier, but not impossible.

//C# code to generate a Ruby class type
        public static Ruby.Class getThing()
        {
            object o = new java.awt.Panel();
            return Ruby.Class.CLASS_OF(o);
        }

##Ruby code to use it
thing = JsharpToRuby::ClassDictionary.getThing

puts thing.to_s

puts thing.ancestors

panel = thing.new
puts panel.to_s

which generates

Panel
Container
Component
Serializable
MenuContainer
ImageObserver
Object
Object
Kernel
java.awt.Panel[panel1,0,0,0x0,invalid,layout=java.awt.FlowLayout]

which goes … java.lang.Object, Ruby Object, …

So now try

io = nil
thing.ancestors.each { |x| io = x if x.to_s.eql? "ImageObserver" }
MyImageObserver = io

class Observer 
  include MyImageObserver
  def imageUpdate(img, infoflags, x, y, width, height) 
    nil
   end
end

q = Observer.new
puts q.class.ancestors

to get

Observer
ImageObserver
Object
Kernel

which means the task is not immediately insuperable; but may need some hand-rolling of concrete classes for some of the interesting interfaces, rather than being something that can be totally read-only…

J# and Ruby.Net — first steps

2007-12-16T10:44:00.001Z

This seems to work

//C# assembly referencing the Ruby.NET.Runtime assembly
namespace Inspect
{
    public class Access
    {
        public static Ruby.Class getRubyPerson()
        {
            ClassLibrary2.nested.Person p = new ClassLibrary2.nested.Person("", 0);
            return Ruby.Class.CLASS_OF(p);
        }
    }
}

## Ruby 
x = Inspect::Access.getRubyPerson()
fred = x.new(name, age)
fred.print

It's a pity that I need an object and cannot just send a System.Type into Interop.CLRClass.Load(type, null, false);, which this ends up doing, but that method is internal to the Runtime DLL.

It should be possible, though, to have a require_jsharp('assembly.dll') that does the tedious running around back and forth through the Ruby/C# boundary in an anutmated fashion, and creates appropriate proxies for lowercase package-names in a tree-like object.

J# and Ruby.Net — not all is rosy in the garden

2007-12-13T00:34:00.001Z

Ruby expects modules to be constants (capitalised). Just like .Net namespace naming conventions dictate.

Java, OTOH, uses lower-case package names which map directly to .Net namespaces, which cause Ruby.Net to bork when trying to instantiate classes. In fact, this is a general issue — it affects any .Net module names which are not capitalised, like this C# code

namespace ClassLibrary2.nested
{
    public class Person
    {
        private String name;
        private int age;

        public Person(String name, int age)
        {
            this.name = name;
            this.age = age;
        }

        public void print()
        {
            Console.WriteLine("Hello " + name + " " + age);
        }
    }
}

which won't work when invoked by

require 'ClassLibrary2.dll'

name = 'Fred'
age = 42
fred = ClassLibrary2::nested::Person.new(name, age)
fred.print

But if you change nested to Nested, all is well.

*le sigh* — that means an adapter layer between Java-style names and Ruby/.Net ones. At least that should be doable in Java/J#, even if it makes things more painful than I had hoped.

What's wrong with this code?

2007-12-09T22:48:00.002Z

Spent today getting about 2/3 coverage on the Bignums class, including rewriting the modularInverse function to complete in a finite time, based on the recursive method given in Wikipedia -- I can always redo it as iterative if there are problems, but for now I have achieved RSA.

So everything was running fine in the JVM; go to .Net, and … infinite loop trying to divide 40! by 20! (in a bit taken from the Ruby bignum unit tests. Eventually, I tracked it down to this:

    private static void s_m_mult_sub(
        int[] result, int ro,
        int[] multi, int mo,
        int scalar, int length)
    {
        long borrow = 0;
        int index = 0;

        /* Is basically practical => proceed */
        while(index < length)
        {
            long product = multUnitxUnit(scalar, multi[mo+index]) + borrow;
            borrow = product >>> 32;
            product &= MASK;
            if(product > (result[ro+index]&MASK))
                ++borrow;
            result[ro+index++] -= (int)product;
        }
        result[ro+index] -= (int)(borrow&MASK);
    }

which Java -- and the original 'C' from which this was taken -- runs "as you'd expect" but J# handles as if it were

result[ro+index++] = result[ro+index++] - (int)product;

and increments index twice, as well as splattering other mayhem around the structure.

Next up, getting JZlib tidied and turning its tests into JUnit form, and adding the PGP needed twiddles; then a first release to replace my old crypt.zip.

The original 'C' code:

static void s_m_mult_sub(unit result[], unit multi[], unit scalar, unit length)
{
    register unit high, low1, low2;
    unit borrow = 0;
    unit index = 0;

    /* Is basically practical => proceed */
    while(index < length)
    {
        multUnitxUnit(high, low1, scalar, multi[(size_t)index]);
        low2 = low1 + borrow;
        if(low2 < low1) high++;
        if(low2 > result[(size_t)index])
            borrow = high + 1;
        else
            borrow = high;
        result[(size_t)(index++)] -= low2;
    }
    assert(result[(size_t)index] >= borrow);
    result[(size_t)index] -= borrow;
}

Another step

2007-12-09T00:06:00.000Z

Package	com.ravnaandtines.crypt.mda	com.ravnaandtines.crypt.cea
# Classes	8	31
Line Coverage	95% 1051/1104	99% 1641/1659
Branch Coverage	81% 170/210	95% 399/422
Complexity	2.369	2.146

That includes real test vectors for TEA (and fixing the implementation at last, so it's incompatible with what I used to have for Java, but is real for the first time -- problems with operator precedence and signed vs unsigned shift), as well as things like Safer not implementing the common interface and a few problems with non-zero array offsets and with triple encryption.

Just need to get bignums and ZLib sorted out with tests, and it'll be time for a proper drop.

Moore at work

2007-12-04T22:52:00.000Z

The CAST5 algorithm defines a maintenance test:

B.2. Full Maintenance Test

    A maintenance test for CAST-128 has been defined to verify the
    correctness of implementations.  It is defined in pseudo-code as
    follows, where a and b are 128-bit vectors, aL and aR are the
    leftmost and rightmost halves of a, bL and bR are the leftmost and
    rightmost halves of b, and encrypt(d,k) is the encryption in ECB mode
    of block d under key k.

    Initial a = 01 23 45 67 12 34 56 78 23 45 67 89 34 56 78 9A (hex)
    Initial b = 01 23 45 67 12 34 56 78 23 45 67 89 34 56 78 9A (hex)
 
    do 1,000,000 times
    {
        aL = encrypt(aL,b)
        aR = encrypt(aR,b)
        bL = encrypt(bL,a)
        bR = encrypt(bR,a)
    }

 Verify a == EE A9 D0 A2 49 FD 3B A6 B3 43 6F B8 9D 6D CA 92 (hex)
 Verify b == B2 C9 5E B0 0C 31 AD 71 80 AC 05 B8 E8 3D 69 6E (hex)

In my code I had a comment "takes about an hour to run", dating from about the end of 1998. Trying today, I got

In shiny new NetBeans 6.0, run from a JUnit 3.8.2 unit test with cobertura coverage instrumentation: 885s
With instrumentation disabled: 6s
In VS 2005 as a post-build run of NCover 1.5.8 running the ported test as a debug build: 34s
In VS 2005 as a post-build run of NCover 1.5.8 running the ported test as a release build: 17s
In VS 2005 as a command-line run of the test as a release build (100 lines of output piped to a file, rather than the console): 6s

So, you pay through the nose (~150 times slower) for your branch coverage instrumentation, as opposed to line coverage (~6 times). And both JVM and CLR are seriously faster, even allowing for the maybe 20x faster processor, than what I was running a decade ago.

Oh, yeah, today's unit test…

It passes.

Making a hash of it

2007-12-02T16:58:00.001Z

Wrote one (meta-)test today. Find every class in com.ravnaandtines.crypt.mda, instantiate through the default constructor, and check that putting in a short sequence of bytes one at a time, as an array or as an array-slice all gave the same results.

On the up-side, this ups coverage:

Package	com.ravnaandtines.crypt.mda
# Classes	8
Line Coverage	95% 1051/1104
Branch Coverage	81% 170/210
Complexity	2.369

and found one bug, in BlockCypherHash, where partial blocks were being lost (I wasn't updating the count of pending bytes when I should).

On the down side, while the test driver was easy to write for J#

MD5 prototype = new MD5(); // an arbitrary class in the right assembly
System.Reflection.Assembly algorithms = prototype.GetType().get_Assembly();
System.Type[] types = algorithms.GetTypes();
System.Type mda = algorithms.GetType("com.ravnaandtines.crypt.mda.MDA");

for (int k = 0; k < types.length; ++k )
{
 final System.Type underTest = types[k];
 if (!mda.IsAssignableFrom(underTest))
 {
  continue;
 }
…

but for Java, it requires some magic to do similar blind enumeration of classes. The following fudge (based off this one, with the fix noted below) gets the all loaded classes in a named package (stuck with Java 1.1 for convenience when cross compiling)

    public static java.util.Vector find(final String pckgname) {
        // Translate the package name into an absolute path
        String name = pckgname;
        if (!name.startsWith("/")) {
            name = "/" + name;
        }
        name = name.replace('.', '/');
        // Get a File object for the package
        final java.net.URL url = // get an instance of the current class to hook from
                new CryptTestUtils().getClass().getResource(name);

        // NOTE -- this url will put %20 for spaces in your path 
        // It needs to be cleared up before creating a File object
        final String dirpath = java.net.URLDecoder.decode(// 1.2 class easily stubbed for J#
                url.getFile()); 

        final java.io.File directory = new java.io.File(dirpath);
        final java.util.Vector contents = new java.util.Vector();
        if (directory.exists()) {
            // Get the list of the files contained in the package
            final String[] files = directory.list();
            for (int i = 0; i < files.length; i++) {

                // we are only interested in .class files
                if (files[i].endsWith(".class")) {
                    // removes the .class extension
                    final String classname = 
                            files[i].substring(0, files[i].length() - 6);
                    try {
                        // Try to create an instance of the object
                        final Object obj = 
                                Class.forName(pckgname + "." + classname).newInstance();
                        contents.add(obj.getClass()); // save off the class
                    } catch (ClassNotFoundException cnfex) { 
                    } catch (InstantiationException iex) { 
                    // We try to instantiate an interface
                    // or an object that does not have a 
                    // default constructor
                    } catch (IllegalAccessException iaex) { 
                    // The class is not public
                    }
                }
            }
        }
            return contents;
    }

and then we do

final java.util.Vector classes = 
                com.ravnaandtines.crypt.CryptTestUtils.find(
                "com.ravnaandtines.crypt.mda"
                );       

for (final java.util.Enumeration iter = classes.elements(); iter.hasMoreElements();)
{
      final Class underTest = (Class) iter.nextElement();

      final Object test = underTest.newInstance();
      if(!(test instanceof com.ravnaandtines.crypt.mda.MDA ))
      {
          continue;
      }
…

So, that's all the hash algorithms done. Next is the similarly mindless conversion of the crypto algorithm test vectors (and a similar meta-test). Then turning the JZlib system tests into a set of unit tests, and porting something like the Ruby Bignum unit tests. Maybe by the turn of the year, I shall actually have a platform based on existing work that can be taken forward.

Tinkering away

2007-12-01T20:57:00.000Z

The standard test vectors for the various algorithms do a good job of shaking down the code for an implementation.

Package	com.ravnaandtines.crypt.mda
# Classes	8
Line Coverage	93% 1024/1103
Branch Coverage	76% 159/210
Complexity	2.369

Coverage would be higher if there weren't a still few CTC-related entry-points, or code-paths like compensating for a broken Haval implementation in a PGP extension (EPB)…

Life in the old dog yet…

2007-11-30T20:13:00.000Z

Contradicting my earlier assertion that “it's only when you want GUI that you have to get off the fence.” — I've built the Java 1.0-based Gadget Windowing Toolkit against JVM for Java 5 and (with just a little added C#) CLR.

It might not be “teh shiny”, and won't give you mouse wheel events, but it's the fastest way to a common GUI toolkit for both VMs.

Weaning it off the Java 1.0 event model that it uses inside a 1.1 emulator, and using Container instead of its own Gadget class (written because Container and Component had protected constructors at Java 1.0) would make it a lot leaner, and more robust, but it's not a simple matter of mechanically replacing the classes. There is a lot of code dedicated to managing focus and graphics contexts — including managing the masks and offsets — that would have to be carefully moved across.

And of course, there's only the one huge demo/test, rather than unit tests.

Smelling the coffee…

2007-11-27T20:24:00.000Z

… before work goes crazy in the 4 weeks left to holidays — a first drop of angerona.redcoffee.zip; what may (ha! ha!) become a hybrid Java/Ruby CTClib equivalent for both JVM and CLR.

This contains code for various symmetric crypto algorithms, strong hashes, erasable big numbers and Zlib compression (based on JZlib 1.0.7); it contains some initial JUnit tests aimed at JUnit 3.8.2 in NetBeans 6.0 (currently using RC1). It also contains a project to build JUnit 3.8.2 (almost) as a J# project generating a command line executable (.exe also included). The "almost" is because the assertEquals() methods for Double and boolean have been taken from 3.8.1 to permit them to compile under J#.

The NetBeans project has been amended to use cobertura 1.9 to perform coverage analysis as the unit tests are run -- assumed to be in C:\cobertura-1.9, adjust build.xml to fit your location; or use it as a prototype for other projects. The VS 2005 solution builds the JUnit ~3.8.2 executable, the angerona.algorithms.dll library, and the unit tests as three projects; and as a post-build step for the last runs the tests under NCover (assumed to be installed to directory C:\Program Files\NCover) -- I'm using 1.5.4, having had some problems with 1.5.8 run over managed C++ code not contained in gc classes.

Also included is an FxCop project; I shall be using that run manually, and PMD as a live plug-in while the code is being groomed; they will get incorporated into the build process later when the noise level has been considerably reduced.

Note that the two build environments are set up to use the same source file structures, though J# has to be hand-held to point it at the files that NetBeans just picks up automagically.

The only files in any sort of stable state at the moment, the proofs of concept, are SHA0.java and SHA1.java, which have unit tests based on the FIPS PUB 180-1 test vectors. These tests all pass and give 100% coverage (including branch coverage) to the base SHA class and the two wrappers. The first phase will be getting the state of no PMD warnings, no FxCop warnings (except where naming conventions are involved, and there, Java conventions set out by PMD will win if there is a conflict), unit tests with ideally 100% coverage; this will overlap with doing some refactoring.

Then with a stable foundation, the plan will be to gradually migrate the rest of CTClib, primarily to Ruby, but probably with a few Java interface types to make the external interface more easily usable by other JVM or CLR projects.

Archive is 257,055 bytes; MD5 7335e89a 171560f3 003f225e d97a3b3c & SHA-1 4d421c00 d4b3b45d be0244e3 649b19bd 04923a94.

Shaking some dust off…

2007-11-26T10:06:00.000Z

After a long distraction by paying work, I'm starting to turn attention back to projects. The IronPython FTP client still needs wiring up (and bug fixing), but a few developments in recent months -- JRuby, and now Ruby.Net 0.9, have led me to turn my attention once again to taking my old PGP-emulator library, currently languishing as 'C' and a number of abortive ports, out of the 1970s.

I have several key bits -- crypto algorithm implementations I rolled myself, and, now the JZlib port of the key ZLib component (for reasons best known to PRZ, the version in PGP 2.x uses an 8k window, and no standard APIs give you control over that so as to be able to emit compatible compression) -- in Java. But now I could build the rest of the system in Ruby and build it to the CLR as well as the JVM.

Then it's only when you want GUI that you have to get off the fence.

Of course, uploading the library will mean I shall want to have completed my improved FTP client, so win-win,… or something

Meanwhile a long term-goal for self-development stuff at work will involve some F#…

One small step…

2007-09-28T21:05:00.000+01:00

A first part of the refactoring of the file system representation for the local files in the FTP client, removing all use of module os (to simply deployment by taking out all dependencies on FePy's extended library code), and encapsulating all references to the file data in a LocalFile class.

TODO list includes:

saving settings
hooking a remote file system class to the FTP client class
linking the FTP client and any low-level error feedback to the UI
Ensuring that file transfers but not directory listings get caught by the progress bar
Handling reconnects intelligently, including recording current directory state on the remote site
…

Drop here again.

Coding formula

2007-09-16T20:23:00.000+01:00

Normal service has resumed -- i.e. hobby coding is light again, as I'm deep into the dev cycle on the current project at work, the weather has improved since the height of the summer so I'm out at weekends, except I've been being distracted by my teeth, and have just been moping.

10 Minute refactor

2007-08-28T20:17:00.002+01:00

After a long weekend in the garden...

module Com_ravnaandtines
  module Zlib
    class Adler32
      # largest prime smaller than 65536
      @@ADLER_BASE = 65521
    
      def initialize
        @value = 1
      end
      
      def update(buffer)
        if not buffer
          return @value
        end
        ## build up the checksum
        low = @value & 0xffff
        high = (@value >> 16) & 0xffff
        buffer.each do |x| 
          low += (x.to_i & 0xff)
          high += low
        end

        ## collapse into modular parts
        low %= @@ADLER_BASE
        high %= @@ADLER_BASE
        @value = (high << 16) | low      
      end
      
      def reset
        @value = 1
      end
      
    end
  end
end

and tweak the tests thus:

  def basic_test_engine(seq, expected)
    ## string to array
    ## otherwise expect an each method to yield integers
    if seq.respond_to? :unpack
      seq = seq.unpack("C*")
    end
    adler = Com_ravnaandtines::Zlib::Adler32.new()
    a = adler.update(seq)
    assert_equal(expected, a)
  end

  def test_boundary
    basic_test_engine(nil, 1)  
  end

That feels better. No special cases, no leakage of state. Looking at JZlib, most of the work will be in doing the encapsulation properly.

Next up, though, when time and energy combine, more on the IronPython FTP client.

Selection Factors

2007-08-24T21:30:00.002+01:00

I had started porting CTClib to managed C++ to do an Iron<something> piecemeal conversion. But at the moment IronPython doesn't have a good deployment story, IronRuby isn't all there; and I still much prefer Java's UI model to WinForms or WPF… So what about JRuby -- familiar UI style and run from jar -- then?

The stumbling block as always is PhilZ's choice of deflate with a 2^13 bit window (as opposed to Zlib's fixed 2^15 bit window size for Ruby or java.util.zip) for compression. This was where I paused my first Java port -- JZlib which could do the job has appeared since I last looked, around the turn of the century. Even so, for sake of portability I've decided to bite the bullet, and do a minimal zlib/deflate implementation in Ruby for the purpose, to do something meaningful with the language, using JZlib as a guide. Inflate can, of course, have the larger window (as in current CTClib-C builds), and use the built-in version, be it the C version from native Ruby or JRuby's java.util.zip wrapper.

So, the easy bit first -- Adler 32 checksum…

module Com_ravnaandtines
  module Zlib
    # largest prime smaller than 65536
    ADLER_BASE = 65521
    # Adler32 checksum : takes a seed (usually 1), and a byte sequence, 
    # returns 32-bit integer
    def adler32(adler, buffer)
      if not buffer
        return 1
      end
      
      ## string to array
      ## otherwise expect an each method to yield integers
      if buffer.respond_to? :unpack
        buffer = buffer.unpack("C*")
      end
      
      ## build up the checksum
      low = adler & 0xffff
      high = (adler >> 16) & 0xffff
      buffer.each do |x| 
        low += (x.to_i & 0xff)
        high += low
      end

      ## collapse into modular parts
      low %= ADLER_BASE
      high %= ADLER_BASE
      (high << 16) | low
    end
  end
end

Test vectors for the unit tests had to be scavenged from the internet:

require 'test/unit'
require 'tinesware_zlib'
include Com_ravnaandtines::Zlib

class AdlerTest < Test::Unit::TestCase

  def basic_test_engine(seq, expected)
    a = Com_ravnaandtines::Zlib.adler32(1, seq)
    assert_equal(expected, a)
  end

  def test_boundary
    a = Com_ravnaandtines::Zlib.adler32(0, nil)
    assert_equal(1, a)    
  end

  def test_simple_0
    basic_test_engine("Mark Adler", 0x13070394)
  end
  
  def test_simple_1
    basic_test_engine("\x00\x01\x02\x03", 0x000e0007)
  end
 
  def test_simple_2
    basic_test_engine("\x00\x01\x02\x03\x04\x05\x06\x07",  0x005c001d)
  end

  def test_simple_3
    basic_test_engine("\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f", 0x02b80079)
  end

  def test_simple_4
    basic_test_engine("\x41\x41\x41\x41", 0x028e0105)
  end

  def test_simple_5
    basic_test_engine("\x42\x42\x42\x42\x42\x42\x42\x42", 0x09500211)
  end

  def test_simple_6
    basic_test_engine("\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43\x43", 0x23a80431)
  end

  class Vector #arrays filled with value = (byte) index
    def initialize(size)
      @size = size
    end
    def each
      index = 0
      while index < @size
        yield index & 0xff
        index += 1
      end
    end
  end

  def test_total
    index = 0
    results = [  486795068,
                1525910894,
                3543032800,
                2483946130,
                4150712693,
                3878123687,
                3650897945,
                1682829244,
                1842395054,
                 460416992,
                3287492690,
                 479453429,
                3960773095,
                2008242969,
                4130540683,
                1021367854,
                4065361952,
                2081116754,
                4033606837,
                1162071911 ]

    
    while index < 20
      size = 5*index + 1
      xx = Vector.new(1000*size)
      basic_test_engine(xx, results[index])
      index += 1
    end
  end

end

That was one evening. Now, how long will the rest of it take?

A wet weekend

2007-08-19T20:09:00.001+01:00

The FTP client code is pretty much there, though I still need to write code to drain headers for the HTTP proxy case. And then the rest is pretty much just plumbing it all together -- of which there will be a fair volume.

Today's screen capture is how the UI will look during a transfer -- a float-up panel with a live throbber in front of the disabled directory views.

After a break

2007-08-18T18:11:00.001+01:00

Last week I was at Recombination so nothing done then.

Now I've started to wire up the logon dialog. This included a bit of shuffling of the main UI, and preparing to split the DirectoryPanel into UI (repeated) and Directory Model classes (local and FTP); and starting the process of writing an FTP client against the .Net socket APIs.

Writing the logon dialog exposed one Mono quirk. The UseSystemPasswordChar property on the text-box doesn't seem to do anything. I resorted to self.pwdTB.PasswordChar = u'\u2022' as a portable solution.

Also, you can use the Data Protection API for more safely storing passwords:

import clr
clr.AddReference("System.Security")
from System.Security.Cryptography import *
import System.Text
import System

…

  def getSecureValue(self, section, key, default):
    raw = self.getValue(section, key, None)
    if raw== None:
      return default
    try:
      array = System.Convert.FromBase64String(raw)
      chars = ProtectedData.Unprotect(array,
        None, DataProtectionScope.CurrentUser)
      result = System.Text.Encoding.UTF8.GetString(chars)
      for i in range(chars.Length):
        chars[i] = 0
      return result
    except System.Exception, ex:
      print ex.ToString()
      return default

  def setSecureValue(self, section, key, value):
    bytes = System.Text.Encoding.UTF8.GetBytes(value)
    try:
      safed = ProtectedData.Protect(bytes,
        None, DataProtectionScope.CurrentUser)
      for i in range(bytes.Length):
        bytes[i] = 0
      string = System.Convert.ToBase64String(safed)
      for i in range(safed.Length):
        safed[i] = 0
      self.setValue(section, key, string)
    except System.Exception, ex:
      print ex.ToString()

which stores the password as an encrypted Base64 blob -- and works with Mono on Win32 as well. The weak point is the password kept in memory as a string (immutable) -- if you can use it as char array, you can wipe that when done.

Meets minimum

2007-08-07T19:41:00.000+01:00

After a bit more fiddling with P/Invoke code, I have hooked into the Win32 Recycle or Delete APIs; though at the moment multiple file delete is done one file at a time. But the directory pane is roughly meets-minimum standard.

Next I can get to the object of the exercise, making an FTP connection and getting the remote directory information back to its own version of DirectoryPane. This will be an interesting combination of BackgroundWorker and asynchronous I/O, with pauses built in so as to throttle the uplink to what my router is prepared to handle.

And then I get to tweak it -- like switching to WPF if available -- and port it -- Mono on Debian working around the P/Invoke stuff.

Directory pane almost there

2007-08-05T16:38:00.001+01:00

Some of the problems I was cursing Mono for turn out to be IronPython 1.x-isms -- mainly the insisting on having Images rather than Icons. Mono has other peculiarities, like in how it actually lays out widgets (not that .Net is much better) at runtime, so for the moment I'm forcing the issue by trapping any size change to the directory pane so that it fits inside the viewport.

Apart from wiring up the delete context menu, the directory pane is done.

More Mono

2007-08-04T21:17:00.000+01:00

A lot of work done for the benefit of Mono, redoing how I wrap files to use os.path rather than .Net directly; and rewriting the P/Invoke layer from scratch (and from pinvoke.net) --including finding that Bitmap.FromHicon() is the simplest way of getting the transparent pixels roughly honoured as well as feeding that system what it wants.

I said that the DirectoryPane would be the worst of it, and it's doing its best to prove me right.

Not quite so pretty

2007-07-30T21:19:00.000+01:00

After remarkably little work, it runs under Mono 1.2.4 and IPCE-r6. The main changes are that as Mono doesn't support System.Management (and the extra work wasn't useful for the icon related info), I fell back to Directory.GetLogicalDrives(), and had to do a bit of type tweaking (explicitly converting to Bitmap to draw the icons).

Everything else is non-visible : the lower left sector has been factored out into a separate class and modules have been rationalised, ahead of making a next step in functionality.

First drop for the curious and for back-up distribution. Now updated.

You move 16 tons...

2007-07-29T17:17:00.000+01:00

The gaps in the Win32 API support in .Net are random and frustrating. You can get all the drive letters (and the "DriveType" or "MediaType" properties which together give the offset into Shell32.dll for the drive icon), but short of unpicking the PE format by hand, Google gives no way to get the icons without using P/Invoke.

So with a combination of existing recipes for wrapping SHGetFileInfo in C#, and feeding it through Fuzzyman's dynamic compiler utility, I have some approximately script-like behaviour for icon extraction -- as can be seen from the left-hand dropdown in the screenshot.

I still need to add some extra hooks to use SHGetFileInfo to check if a file is executable or not, so I can get custom icons, and then refactor out a proper utility. And there's still more work on the generic directory pane widget before it'll be something stand-alone (sorted file list, hooking up events); though I hope that should not require any more exploting of exotic APIs to achieve.

Sneak Peek

2007-07-28T19:18:00.001+01:00

I've starting recoding my FTP client in IronPython, mainly as a self-teaching exercise (but hoping it might work in Mono/IPCE too). So far, this is the shell of the main UI, with a ribbon-style bar at the top.

The biggest piece of work is likely to be the Directory Pane component that Fox just gave to me on a plate.

PassiveFTP 1.10

2007-06-29T20:35:00.001+01:00

Now available on the home server too.

PassiveFTP 1.10

2007-06-27T22:46:00.000+01:00

As promised…

PassiveFTP 1.10 — build 1.10.2734.38706 — squeezes in an up-one-level button into the remote site list, for FTP sites that manage to conceal their ".." directories. Or for people who'd prefer it anyway.

From the GeoCities Windows page at the moment. I need to do a physical rather than a remote logon to my home server before I can upload to that, so in a day or two.

PassiveFTP -- new revision

2007-06-26T21:38:00.001+01:00

As the revised Demon homepages site doesn't provide ".." as a directory in the default ls response, I shall have to add an explicit "go up one level" button to the server side.

Expect version 1.10 soonish.

Current directions

2007-06-24T16:55:00.000+01:00

CTClib/Angerona is taking a strange turn.

I want to use IronPython for .Net and Mono UI; but for the core library keep it as pure Python as feasible. But to get there, I'm porting via C++/CLI.

I have most of it building (I have not ported I/O, or deflate as yet), and I'm separating out the test code into a more formal set of unit tests.

The plan goes get it into "managed 'C'" with test coverage (side release 1); refactor to provide CLR interfaces at suitable places (side release 2); then pull code through the language barrier, one module at a time, to achieve a pyCTC library (library release 1), followed by a WinForms GUI on top. Or maybe Jython for the GUI.

No promises, as usual.

Drophash in Python dialects

2007-04-27T16:50:00.001+01:00

Iron Python 1.1

Now for IronPython 1.1 — annoyingly the WebBrowser control doesn't accept normal DnD, nor does it let you paint it to a bitmap to blit on another control, so this one uses a ListView.

import clr
clr.AddReferenceByPartialName("System.Windows.Forms")
clr.AddReferenceByPartialName("System.Drawing")

from System.Windows.Forms import *
from System.Drawing import *

from md5 import *
from sha import *

pane = None
hashes = [md5, sha]
nbsp = list(" ")


def getHash(file, hashOp, delimiter) :

    try :
        reader = open(file, 'rb')
    except :
        return ""
   
    try :
        digest = hashOp()
        while True :
            chunk = reader.read(1024)
            if not chunk :
                break
            digest.update(chunk)
    finally:
        reader.close()

    raw = digest.hexdigest()
    work = []
    i = 0
    while i < len(raw) :
        work.append(raw[i])
        i += 1
        if 0 == (i%8) :
            work = work + delimiter

    return ''.join(work)

def dragOver(sender, evt):
  evt.Effect = DragDropEffects.Link
 
def dragDrop(sender, evt):
  global pane
  data = evt.Data
  data = data.GetData(DataFormats.FileDrop)
  num = pane.Items.Count
 
  for file in data:
    pane.Items.Add(file)
    for hashOp in hashes :
      anHash = getHash(file, hashOp, nbsp)
      pane.Items[num].SubItems.Add(anHash)
    num += 1
      
    


def decorate(frame):
    global pane
    frame.Text = "IronPython - Drophash"
    ## optional icon
    ## ico = Image.FromFile("drophash.ico")
    ## frame.Icon = Icon.FromHandle(ico.GetHicon());
    pane = ListView()
    pane.Dock = DockStyle.Fill;
    frame.Controls.Add(pane)
    pane.AllowDrop = True
    pane.DragOver += dragOver
    pane.DragEnter += dragOver
    pane.DragDrop += dragDrop
    pane.View = View.Details
    pane.Columns.Add("File name", 100, HorizontalAlignment.Left)
    pane.Columns.Add("MD5", 200, HorizontalAlignment.Left)
    pane.Columns.Add("SHA-1", 200, HorizontalAlignment.Left)


if __name__ == "__main__":
    frame = Form()
    decorate(frame)
    Application.Run(frame)

Jython 2.2

Jython next; but rather than coding the messy .ico to image transformation again, use a PNG icon (save as drophash.png) instead:—

import sys
import os.path
sys.packageManager.makeJavaPackage("javax.swing", "JWindow", None)
sys.packageManager.makeJavaPackage("java.awt", "Window", None)
sys.packageManager.makeJavaPackage("java.awt.dnd", "DropTargetListener", None)
sys.packageManager.makeJavaPackage("java.awt.datatransfer", "DataFlavor", None)
import javax.swing
import java.lang
import java.awt.dnd
import java.awt.datatransfer

from md5 import *
from sha import *

pane = None
hashes = [md5, sha]
nbsp = list(" ")
target = None

def getHash(file, hashOp, delimiter) :

  try :
    reader = open(file, 'rb')
  except :
    return ""
  
  try :
    digest = hashOp()

  while True :
    chunk = reader.read(1024)
    if not chunk :
      break

      digest.update(chunk)
  finally:
    reader.close()
        
  raw = digest.hexdigest()
  work = []
  i = 0
  while i < len(raw) :
    work.append(raw[i])
    i += 1
    if 0 == (i%8) :
      work = work + delimiter

  return ''.join(work)
  
def OnDropFiles(filenames) :
  global pane, nbsp, hashes
  pane.text += "<table border=1><tr><th>File</th><th>MD5</th><th>SHA-1</th></tr>"
  for file in filenames :
    pane.text += "<tr><td>%s</td>" % file

    for hashOp in hashes :
      anHash = getHash(str(file), hashOp, nbsp)
      pane.text += "<td><pre>%s</pre></td>" % anHash

  pane.text +="</tr>"
  pane.text +="</table>"

  

class dropHandler(java.awt.dnd.DropTargetAdapter):
  def __init__(self):
    java.awt.dnd.DropTargetAdapter.__init__(self)

  def drop(self, evt):
    if not evt.transferable.isDataFlavorSupported(java.awt.datatransfer.DataFlavor.javaFileListFlavor):
      return
    evt.acceptDrop(-1)
    names = evt.transferable.getTransferData(java.awt.datatransfer.DataFlavor.javaFileListFlavor)
    OnDropFiles(names)

def decorate(frame):
  global pane
  frame.setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE)
  frame.title = "Jython - Drophash"
  frame.layout = java.awt.BorderLayout()

  pane = javax.swing.JLabel()
  pane.border = javax.swing.BorderFactory.createEmptyBorder(5,5,5,5)
  frame.contentPane.add(javax.swing.JScrollPane(pane), java.awt.BorderLayout.CENTER)
  pane.text = "<html>"
  icon = java.awt.Toolkit.getDefaultToolkit().createImage("drophash.png")
  frame.setIconImage(icon)
  target = java.awt.dnd.DropTarget(pane, dropHandler())
  
  
  width = 800
  height = 600
  frame.setSize(width, height);
  screenDim = java.awt.Toolkit.getDefaultToolkit().getScreenSize();
  frame.setLocation(
        (screenDim.width - width) / 2,
        (screenDim.height - height) / 2
        )
  



if __name__ == "__main__":
  frame = javax.swing.JFrame()
  decorate(frame)
  frame.visible = True

Made with Cascading Style Sheets

Drophash in Python

2007-03-03T16:40:00.001Z

First little Python project, a little utility to take the MD5 and SHA hashes of a bunch of files dragged and dropped onto it. Same function as but better presentation that a Win32 version (on the Windows page) I wrote back in late '00 (and which I have still been making regular use of for checking MD5s of file downloads).

WxPython UI

This one is neater than the Win32 version since I can use the nifty HTML window widget and place the results for each new batch of files in a table, with the files and hashes cross-indexed. Simply cut, paste and save the text of the section below as drophash.py.

#!/usr/bin/env python
# -*- coding: ISO-8859-1 -*-

#MD5 test suite:
#MD5 ("") = d41d8cd98f00b204e9800998ecf8427e
#MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661
#MD5 ("abc") = 900150983cd24fb0d6963f7d28e17f72
#MD5 ("message digest") = f96b697d7cb7938d525a2f31aaf161d0
#MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b
#MD5 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = d174ab98d277d9f5a5611c2c9f419d9f
#MD5 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 57edf4a22be3c955ac49da2e2107b67a

#SHA test suite:
#SHA ("abc") = A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D
#SHA ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") = 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1
#SHA (A million repetitions of "a") = 34AA973C D4C4DAA4 F61EEB2B DBAD2731 6534016F


import wx
import wx.html as html
from md5 import *
from sha import *

def getHash(file, hashOp, delimiter) :

    try :
        reader = open(file, 'rb')
    except :
        return ""
    
    try :
        digest = hashOp()
        while True :
            chunk = reader.read(1024)
            if not chunk :
                break
            digest.update(chunk)
    finally:
        reader.close()
        
    raw = digest.hexdigest()
    work = []
    i = 0
    while i < len(raw) :
        work.append(raw[i])
        i += 1
        if 0 == (i%8) :
            work = work + delimiter

    return ''.join(work)

class FileDropTarget(wx.FileDropTarget):
    def __init__(self, window):
        wx.FileDropTarget.__init__(self)
        self.window = window

    hashes = (md5, sha)
    nbsp = list(" ")

    def OnDropFiles(self, x, y, filenames) :
        self.window.AppendToPage("<table border=1><tr><th>File</th><th>MD5</th><th>SHA-1</th></tr>")
        for file in filenames :
            self.window.AppendToPage("<tr><td>%s</td>" % file)

            for hashOp in self.hashes :
                anHash = getHash(file, hashOp, self.nbsp)
                self.window.AppendToPage("<td><pre>%s</pre></td>" % anHash)

            self.window.AppendToPage("</tr>")
        self.window.AppendToPage("</table>")

def decorate(frame):
    frame.SetTitle("wxPython - Drophash")
    ## optionally, an icon
    ## _icon = wx.EmptyIcon()
    ## _icon.CopyFromBitmap(wx.Bitmap("drophash.ico", wx.BITMAP_TYPE_ANY))
    ## frame.SetIcon(_icon)
    frame.style = wx.DEFAULT_FRAME_STYLE
    pane = html.HtmlWindow(frame, -1)
    dt = FileDropTarget(pane)
    pane.SetDropTarget(dt)


if __name__ == "__main__":
    theApp = wx.App(False)
    frame  = wx.Frame(None, -1)
    decorate(frame)
    theApp.SetTopWindow(frame)
    frame.Show()
    theApp.MainLoop()

Python + Tk UI

As an alternative, a pure Tk version. Not so fancy; you have to select the files you want (or put them on the command line). Just extract this section as tkhash.py and double click…

#!/usr/bin/env python
# -*- coding: ISO-8859-1 -*-

#MD5 test suite:
#MD5 ("") = d41d8cd98f00b204e9800998ecf8427e
#MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661
#MD5 ("abc") = 900150983cd24fb0d6963f7d28e17f72
#MD5 ("message digest") = f96b697d7cb7938d525a2f31aaf161d0
#MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b
#MD5 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = d174ab98d277d9f5a5611c2c9f419d9f
#MD5 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 57edf4a22be3c955ac49da2e2107b67a

#SHA test suite:
#SHA ("abc") = A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D
#SHA ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") = 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1
#SHA (A million repetitions of "a") = 34AA973C D4C4DAA4 F61EEB2B DBAD2731 6534016F

import Tkinter
import ScrolledText
import tkFileDialog
import sys
import tkFont

from md5 import *
from sha import *

hashes = {"MD5" : md5, "SHA" : sha}
nbsp = list(" ")

def getHash(file, hashOp, delimiter) :

    try :
        reader = open(file, 'rb')
    except :
        return ""
    
    try :
        digest = hashOp()
        while True :
            chunk = reader.read(1024)
            if not chunk :
                break
            digest.update(chunk)
    finally:
        reader.close()
        
    raw = digest.hexdigest()
    work = []
    i = 0
    while i < len(raw) :
        work.append(raw[i])
        i += 1
        if 0 == (i%8) :
            work = work + delimiter

    return ''.join(work)


def doGetHashes() :
  filenames = tkFileDialog.askopenfilenames()
  updateCanvas(filenames)
  
def updateCanvas(filenames):
  textarea.config(state=Tkinter.NORMAL)
  for file in filenames :
    textarea.insert(Tkinter.END, ("%s\n" % file))

    for hashOp in hashes :
      anHash = getHash(file, hashes[hashOp], nbsp)
      textarea.insert(Tkinter.END, ("%s " % hashOp) )

      textarea.insert(Tkinter.END, ("%s\n" % anHash), 'mono' )

    textarea.insert(Tkinter.END, "-------------------------\n")
  textarea.tag_config('mono', font='courier')
  textarea.config(state=Tkinter.DISABLED)

if __name__ == "__main__":
  root = Tkinter.Tk()
  root.title("tkFileHasher")

  textarea = ScrolledText.ScrolledText()
  textarea.pack()
  textarea.config(state=Tkinter.DISABLED)
  Tkinter.Button(text="Select files to hash...", command=doGetHashes).pack()
  updateCanvas(sys.argv[1:])
  Tkinter.mainloop()

Fun with JavaScript

2007-02-14T16:38:00.000Z

“E”-ddress encoder script 1.0 — a utility based on the the old Hiveware “Enkoder”, only this time done entirely in JavaScript. Because you can.

Also the same thing for Acrete script 1.0

This sort of thing doesn't need heavyweight stuff like a Java applet; though Acrete currently uses the <canvas> element which isn't supported in IE (though I'm investigating alternatives for that). And, besides, I might as well do something useful, rather than just playing, with dynamic languages like JavaScript, Ruby and Python. Java projects (at least, those not for work) have probably reached the end of the road as a result.

Acrete update

2006-10-30T16:37:00.000Z

A small tweak to the previous entry; code included in the Jar file. Inline demo and code download page. To run

java -jar acrete.jar

Blast from the past

2006-10-25T16:36:00.000+01:00

A Java port of the re-implementation I did of Dole's Acrete program for growing solar systems that first appeared in White Dwarf 57. The port is not elegant, and used a lot of “here's one I prepared earlier” for the animation rather than using Swing.

CTCJava 2.0 Beta state

2006-06-18T16:35:00.001+01:00

Meanwhile CTCJava 2.0 is (slowly) undergoing a massive scrubbing with the help of PMD, Cobertura, JUnit and MiniCppUnit, so declaring beta may have been a tad premature.

PassiveFTP 1.9

2006-06-18T16:35:00.000+01:00

PassiveFTP 1.9 — build 1.9.2360.19246 — is a minor usability tweak, moving status indications from dialogs to the main screen - progress at the lower-left, errors in the scrolling text at top right.

CTCJava 2.0 Beta 2 - update

2006-01-22T16:34:00.000Z

Testing in advance of Beta 3 has shown that there are some problems with managing keyrings in key generation. This includes some problems with the handle to the JVM environment through multiple levels of calls Java->native->Java->native...; and failing to pick up the keyring files once they have been updated. This will take more than simple refactoring or easy Swing UI tweaks.

If the bug turns out to be made more difficult to resolve because of the back and forth calling, I may end up pulling more of the OpenPGP functionality into the Java layer. I will certainly be making it more C++ with a C interface where that simplifies matters for me.

CTCJava 2.0 Beta 2

2006-01-14T16:33:00.000Z

Beta 2 of CTCJava 2.0 — a simpler archive structure (archive = archive2 + signature; archive2 = jar+java source zip+dll 2.3.2205.36688+ctcjlib JNI layer source). The main change is that documents are shown in the main window, by selecting the current document from the tree control, though there are other minor UI changes and behind the scenes updates.

There will be at least one more interation with UI tweaks before I declare RC.

CTCJava 2.0 Beta 1

2006-01-01T16:32:00.000Z

This release is a 7-zip format archive, and uses Java 5 (JRE 1.5) or later, rather than the Java 1.1 of the previous models. The main feature of this release is a replacement of the UI widgets with Java Swing components; and the start of a rework of the way in which multiple open files are managed.

This release archive contains a 7-zip archive with detached signature; within that 4 other such archives, each containing a file and its detached signature. The files in the innermost archives, the ones with accompanying signature, are

CTCJava.jar — the compiled file including icons (Both the “Dean's icons” set used before, and the preferred Tango Project set.
java.zip — the refactored source code
ctcjlib.dll — version 2.3.2192.30638; modified from the previous version to take into account changes in the names of classes and methods in the Java code
ctcjlib-src.zip — the modified source used to build the above.

I shan't make promises about the likely schedule of further updates.

CTCjlib -- IMPORTANT UPDATE!

2005-12-21T16:29:00.000Z

The previous CTCjlib update (2.3.2178.34472) was built off an experimental (and broken) code-base, the result of my having set the project aside some years (and computers) ago, and taking from the wrong repository.

I have now reverted the code-base to the last released archive from this site, and built CTCJlib.dll v 2.3.2181.36359 from that. As noted before, this incorporates the fix noted in the 26-Jun-05 entry below, plus the altered source files, and with detached signatures in the archive too.

Please accept my apologies, and — Enjoy!

Enkoder applet

2005-12-10T16:27:00.000Z

“E”-ddress encoder 1.0 — a utility based on the the old Hiveware “Enkoder”, only this one is a Java app/applet. Offered into the public domain.

PassiveFTP 1.8

2005-10-29T16:26:00.000+01:00

PassiveFTP 1.8 — build 1.8.2127.30224 — minor bug-fix release — only treats client-side executables as possibly having unique icons, so does not get icon creation failures when scanning client-side folders with vary many files. Also set default upload to dial-up compatible 2kb/s for best observed stability.

From the windows page

CTClib inflate issue

2005-06-26T16:04:00.000+01:00

The CTClib deflate implementation follows PGP 2.x in limiting the compression window to 8k; it also limits the decompression window similarly. Other implementations e.g. Bouncy Castle, do not heed this limitation. Andrew Paterson writes:

Since my last e-mail, I did a quick experiment and I have established that, as you suggested, the 8K window size was the culprit. As it was easier to modify CTC, I changed the definition of WSIZE back to 32K (0x8000) in gzip.h then defined WSIZE as 8K (0x2000) in deflate.c before gzip.h was included. That did the trick! Encryption/signing (deflating) uses an 8K window size whilst decryption (inflating) uses a 32K window size. Problem solved and, as a bonus, the compatibility of CTC is actually increased.

Unfortunately, it does mean that I will have to rebuild/redeploy all of my software that uses CTC but I think I prefer that to using an uneasy hybrid of Sun JDK, hacked Bouncy Castle JCE and hacked GNU Classpath zip classes. There are also licence implications in that the Bouncy Castle crypto libraries are not under the GPL whereas the GNU Classpath classes obviously are.

Look for an update build sometime.

PassiveFTP 1.7

2005-05-09T16:02:00.000+01:00

PassiveFTP version 1.7.1955.38236 — allows the upload rate to be limited for fussy servers.

From the Windows page, as usual.

PassiveFTP problem

2005-05-08T16:01:00.000+01:00

I have observed a flaw in the implementation of PassiveFTP. I intended to make the data channel socket non-blocking, but instead re-asserted that for the command channel. Also the host I usually FTP to has started to choke on uploads done at full ADSL rate, so I need to get the non-blocking behaviour fixed, so I can throttle the upload rate appropriately.

Look for an update in the next week or so.