Thursday, July 9, 2009

C++/Java Invocation API: Java App Meets Vista Logo Certification?

Attachmate wanted to prepare an update to their Windows-native product, Reflection X, in Java so it could be run on several previously unsupported operating systems. At the same time, another significant goal was supplied from Attachmate's Marketing department, which specified that Reflection X must pass Microsoft's Vista Logo Certification.

Vista Logo requires a candidate application to meet more than 30 Microsoft-specified requirements that are to be tested by a third party validator, in this case Veri-Test. Two items from the list of requirements are 1) the application must provide a fully capable MSI installer; and 2) the application must conform to the Windows Error Reporting (WER) initiative. In a nutshell, the WER requirement will be met when 1) serious runtime exceptions are recognized by the application; 2) the user is warned via a common Windows dialog that asks if the problem should be reported to Microsoft; 3) if the user selects OK, a minidump is transmitted to the Microsoft WER site where it can be bucketed for statistical purposes and from where the user can be redirected to a website that provides further information on a workaround or product update; & 4) an application restart action can be started.

I prepared a WIX MSI installer that installed the Reflection X jar files, documents, graphic images, language resources, and JVM, and then installed and started the Reflection X Server, the underlying Windows service. Difficulty with the WER requirement came from the fact that the Java JVM is a virtual machine that contains its internal threads within structured exception handlers (SEH). That means that any serious or fatal Java exception that originates in Reflection X -- the kind that should be reported via the WER -- would be caught by the JVM, never making its way to a WER handler in RX's Java launcher application.

The interesting solution to this problem was found in the precedence given to different exception handler types. As mentioned above, a SEH is defined to surround each Java thread and therefore catches all exceptions that originate within the thread. If an exception is not handled in an application's SEH then the JVM will eventually catch it with one of its own handlers, ensuring that none escape to an interested WER manager. Another exception type, a Vectored Exception Handler (VEH), is given the first look at every exception thrown within the application by the operating sytem before it makes it to any SEH. I added a VEH to Reflection X's Java launcher application that evaluates every exception Reflection X generates and reports serious or fatal failures to the WER before passing them on to the SEH that should eventually handle it.

As a result of this design, and I think much to Microsoft's chagrin, Reflection X 2008 is the only Java application to earn Vista Logo Certification.

Tuesday, January 27, 2009

C: Rocket System Reliability

I started working at Olin Rocket Research as an applications programmer soon after I graduated from the University of Washington. I was hired to maintain existing engineering applications that were written in Basic, Fortran, or Lotus macros for the Chemical and Reliability Engineering departments.
Among the projects I was assigned was the update of a program that calculated the reliability of a rocket system.  The program had been originally written by a mechanical engineer more than ten years earlier and then modified several times each year as needed for each new system variation.
The calculation for reliability of a hardware system is similar to that for resistance of an electrical system in that serial and parallel component sequences must be identified and counted differently. The calculated reliability of a series of hardware components is simply the product of the set of serial components' tested reliability ratings.  The calculated reliability of parallel components is a little trickier -- too simply stated as 1 - [(1-pc1)(1-pc2)] for a simple single redundancy system with parallel component reliability values of pc1 and pc2.
The problem as I saw it was twofold. First, since the components of each different system were different in number and type -- and each different component commonly possessed a different reliability value -- the original Fortran program had to be recompiled with a new data set each time a new system was to be evaluated. The other part of the problem was that each system that had a different number of components and a different redundancy arrangement had to be described in a different way in the code.  That is, each time the hardware system changed the calculation sequence had to be rewritten to reflect the new serial and parallel layout and then recompiled. That easily could be seen to have wasted months of effort over the program's lifetime.
I saw that the program needed to be changed so that it would no longer be necessary to recompile every time a new system was to be evaluated. I chose to drive the program with a table described in a data file that provided system shape and component reliability values. I also saw that a switch to the C programming language was necessary because Fortran did not offer dynamic memory allocation needed for variably-sized systems nor the recursion that would be required for handling all possible shapes of parallel circuits.
The next day I replaced the 2000 lines of Fortran sourcecode in a couple of hours with a 50 line, table-driven recursive C-language version.  From that time on, all a mechanical engineer had to do to calculate a system's reliability was feed it's shape and components' reliability values into a data file and open and run it with the program.  No code changes or compiler were needed from then on.

C: Cubic Splines Interpolation

I started working at Olin Rocket Research as an applications programmer soon after I graduated from the University of Washington.  I was hired to maintain existing engineering applications that were written in Basic, Fortran, or Lotus macros for the Chemical and Reliability Engineering departments.
Among the projects I was assigned was to perform data reduction on about seventeen years of test laboratory data that tracked, over time and temperature change, the reaction with hydrazine rocket fuel of different metals that might be used in rocket fuel tanks, plumbing, or nozzles.  The product of the work was to be two-dimensional line charts that showed the measure of the reaction recorded -- the amount of gas generated.
The problem with the tool available at the time, Lotus, was it rendered line charts in jagged, sawtooth patterns. The extremes of those charts reflect measured, accurate values, but nearly all of the points in between were not accurate. The slope of the curve approaching and leaving a naturally occurring measured data point is almost never sharply steep, flat, and then sharply steep in the other direction in the span of three infinitesimally small data points. Instead, the slope approaches and leaves zero comparably slowly at each maximum or minimum point, and curves gradually as it connects them.
To improve the quality of the visual rendering I wrote a new program that generated a large number of additional data points using cubic splines interpolation on the lab-measured data points. That involved predictive calculation of a more natural first and second derivative on both sides of each inflection point based on the slope of the surrounding lines' midpoints.  With the new data points the chart would be redrawn using many more data points than those actually measured, which would effectively correct the sawtooth pattern with curves, much like the effect of using a french curve drafting tool.  Each line reflected what near instantaneous measurement would've shown.

Sunday, January 25, 2009

C++/MFC: Extra! Personal Client Printer Emulator

The Printer Emulator feature of Attachmate's Extra! Personal Client product accepts an SNA datastream from an IBM mainframe computer printer and converts it for use by a Windows or HP EPL5 driver so the mainframe image or text can be printed on a PC printer.

Attachmate had recently purchased its largest rival, DCA, so I was lucky enough to find myself selected as a feature owner in the situation where the entire feature set of both vendors' products were to be rewritten for the benefit of both customer bases. That provided me with two significant design opportunities in the printer emulator project's UI. First was to combine the appearance of the two vendors' products into one that would be better than either, and perceived as familiar by both sets of users. That required deciding on which of the two companies' terminology was best for each title or label, as well as breaking one feature into two or more when one product offered greater configuration precision.

The printer UI is a combination of a text-based connection status window and the configuration property sheet with its seven property pages.

The configuration pages contain a wide range of options for which users' opinions will vary over which selection is best for their circumstances.

Saturday, January 24, 2009

C++/MFC: Integration of IIS Access Control with Apache Server

Attachmate's Reflection for the Web (RWeb) product is a Java-based application more often than not run on customers' Windows systems. It depends on an Apache server to serve servlets and validate user credentials. Customer administrators had asked for years that RWeb accept the validation already in effect on their Windows networks rather than require that they configure an additional Tomcat validation layer for RWeb alone. The technical need was an ISAPI redirector with a configurator fit to Tomcat.
When Windows Vista was near release several other changes to RWeb's installation and configuration became necessary and I found myself with an opportunity to clear the table of all enhancement requests. I wrote a pair of Windows native utilities, the IIS Utilities, to answer the requests for IIS-based user credential validation. With that, the RWeb Java product, its InstallAnywhere installer, and and the IIS redirection were simplified, easily tested, and focused on what they do best.
It has recently happened that, while they did not realize in 2008 that their customers running RWeb on a Vista or Windows 7 platform could not use IIS redirection without the IIS Utilities, RWeb Product Management released them, as is, as standalone products on June 10, 2011.
IIS6Utility.exe was written in C++ and uses the ABO API to talk with IIS 5 & 6. It installs an ISAPI filter definition to the IIS metabase for all metabase-oriented operating systems except Windows 2003 Server, where it instead installs an ISAPI extension definition. In addition it modifies two Tomcat configuration text files and four others for RWeb.

IIS7Utility.exe was written in C# and talks to IIS 7 using the .Net Web Administration API. It also installs an ISAPI filter definition to an IIS 7 application pool and modifies the same six files for Tomcat and RWeb configuration. I set it up to be deployed to users via Click-Once for the Attachmate test team's purposes, but when shipped to customers the intent was for it to be installed standalone as an option via RWeb's InstallAnywhere installer.
Both applications were thoroughly unit tested and localized for English, French, German, and Japanese. They both report interesting activity to the Windows event viewer.

Java/UI: External UI Handler for Multiplatform Installers

The multiplatform installer project was prepared as a prototype external UI handler for Attachmate's installers. Its goal was to demonstrate that a standardized, parameter driven generic user interface could be used for installation of all Attachmate products, regardless of target platform. It would miminize the effort necessary to maintain a standard look and feel as any change to an Attachmate company name, font, logo, icons, or graphic image would be reflected in all products' next version.

The prototype is a wizard written in Java, built with embedded JUnit tests, localized with ResourceBundles, driven with a pair of localized XML configuration files -- one for the installer wizard and one for the to-be-installed product, provides complete source code Javadocs, and logs runtime events. It is written to collect the user's preferences and then will invoke one of several type of install packages including a Windows MSI database, an InstallShield Multiplatform project, an HPUX package, or a UNIX install script.

In the case of an MSI installation, this external UI handler is registered with the installer service so it is available in Add\Remove Programs.

The Welcome screen contains an example of a standard graphic, parameterized localized text, and the first place in a standardized installation sequence.

The software license agreement contains parameterized company-standard legalese. It, too, maintains a place in the standard installation sequence, but may be preceded by an optional prerequisite panel.

The contents of the feature selection tree are one set of data that originates in the custom schema XML file prepared for the product. As it is constructed it reflects previously installed features (Yes, the string resource should say "press the Continue button". This was a prototype.)

C++/Many other languages: API Generator

The APIGen code generator prepares well written, ready to compile or interpret source code for many combinations of Attachmate API's, programming languages, and host terminal types. It is used internally at Attachmate by employees in Sales, Support, Quality Engineering, and Development. It has also been passed on to a few customers.
I conceived and designed it, and then implemented approximately 25 of the 30 generator classes now available.
The base classes of the large inheritance tree seen below, were designed to serve in a translation object server so other contributors (or students) could develop the generator for the remaining (over 200) combinations of API, host, and language. That is, it provides a means for other contributors to snap-in translation modules that rely on the base classes for the API's and languages that concern their field of expertise.  I feel funny bragging here about such a complex inheritance tree, especially in light of the software engineering best practice advice, "favor aggregation over inheritance", but the MS Visual Studio I used at the time provided a Visio modeling link that made code architecture and generation very easy and I wanted to exercise that approach.
Having finished the first 25 generators I have since endeavored to romance the coding of the remaining 200 with members of Attachmate's Developer Support and Sales Engineering teams.

Initial screen where user specifies choice of API and programming language.

Optional screen where controls allow user to specify function parameters.

Output screen shows generated source.

The UML class diagram below, recorded early in the project and not showing all generator classes, shows inheritance relationships in APIGen for the twenty-two Attachmate API's.

Each leaf level class symbol represents one combination of API, output language, and terminal type.

C++/SQL: Speedbar State Machine

Speedbar Case Study

Speedbar State Machine Case Study
Object Oriented Analysis - Winter 2005
Don Cannon
March 14, 2005

Supervisor/Agent Recorder/Player dialog with Speedbar
Problem Statement
A local software development company had a product on the market whose development featured a rate, process maturity, and team culture that produced a product that was capable of working only on its "happy path" of execution. Any venture off the well-marked trail would commonly be met with a Windows blue screen. When the director of Development for this company -- my development manager at my previous job -- first demonstrated the product to me, it blue screened four times during the demo -- unintentionally. Each time the PC had to be shut off and restarted, with the loss of many of the application's volatile configuration settings. I had been hired to stabilize the product overall, with elimination of these blue screens the highest priority.
Product Overview
The UI view for the application that possessed that behavior is a custom toolbar called "the Speedbar". The Speedbar controls recording, playing, and editing a combination of PC screen video and telephone audio that occurs between a call center agent and a customer. It presents a variety of function buttons needed in creating an audio/video recording such as video recording, audio recording, video and audio recording, playback, pause, fast forward, fast rewind, book mark, annotate, and append. Each of those buttons was to be presented in one of four states: enabled, disabled, enabled and flashing, and disabled and flashing. The recorded files were stored on a network server within a relational database so access to them could be controlled by the product administrator.

The Speedbar object's implementation accessed the memory addresses of various kinds of objects such as functions, the database connection, or recording files. Those objects are grouped and stored in different network locations and so sometimes it happens that some of them are not available during various activities the Recorder performs or due to network fluctuations. It is critically important that these memory pointer variables are guarded so that their properties or methods cannot be invoked when the actual object is not available, or after it has been destroyed. The primary failure of the Speedbar object was its propensity to access null objects in the software service's process space.
The enabled/disabled/flash state of each button on the Speedbar was established by approximately 30 variables -- the majority of which were declared to be static. An example is a Boolean value named bPlaying which stored whether or not a recording was currently being played back. A preliminary search of the files found that the variables were "assigned" values in approximately 2000 locations - across the product in several different class objects and three concurrently running modules. When the code logic was inspected which pulls together all of the variables to determine the current enabled or disabled state for each Speedbar button, it was immediately clear that the volatile nature of the various parameters provided a level of complexity that was way beyond what could be understood. The code was so complex that no person could be capable of concluding that it was ready for production. A preliminary McCabe's complexity evaluation of the mechanism that assigned the enabled/disabled decisions had a value of 57, which characterizes it as un-maintainable.

With the complexity and its nature realized it became clear that the key to the solution was to devise a way to reduce the multi-dimensional problem down to a level that could be more easily understood. I chose to reduce this multi-dimensional problem to two dimensions by reducing it to a two dimensional finite state machine. In so doing, each feasible and unique combination of variables was seen to be a theoretical node on a connected, directed graph, and the edges between the nodes represented the declared path to the next node resulting from invoking a specific action. It was common for a node to have multiple edges leaving it that corresponded to the number of button or runtime actions currently enabled. It was also common for a node to have multiple edges pointing to it that showed that several states were followed by that state if the right action was invoked. To implement the plan the following was done:
    1. Each unique and feasible combination of variable values was provided a state name.

    2. Every action capable of bumping one state to the next was provided an action or event name. Most of these were just pressing one of the buttons, and others were a preset recording length elapsed, a recording played to its end, or the connection to the recording database was lost.

    3. The names of all states discovered during analysis were listed across the X-axis of an Excel spreadsheet. The names of all actions realized in the analysis were listed on the Y-axis of the same spreadsheet. Into the row/column intersection of each state and action was written the state that was to result when the action was fired while in that state. This matrix was fairly sparse in that a large fraction of the intersection cells represented infeasible states.

    4. The matrix was sent to QA for review and then for use as the basis for their Speedbar test suite.

    5. The completed state/action matrix was copied into a header file for a new state machine class. As the state machine class was compiled the data was read into a matrix that was used to implement the logic specified in the spreadsheet.

    6. All code that involved an assignment of one of the independent variables was replaced with an action fired to the state machine.

    7. Every time an action was fired to the state machine, if the state changed, a procedure named uniquely for the resultant state would be called that would explicitly set the enabled/disabled state for every button. At project completion there were seventy-two unique states and twenty-four actions or events that would trigger the transition from one state to another.

Upon completion and insertion of the state machine the blue screens happened no more and the module's McCabe's value was 10. The context of a given enable/disable decision throughout the code was (almost) immediately obvious to an engineer, and so thereafter relatively little thought was required to properly modify the code.

C++: Cool Tool Automated Testing

The Cool Tool is an internally used test tool that concerns testing of host automation API's for Attachmate's flagship product. The UI presents a means to specify a sequence of API methods with their arguments and then execute them. It can record those steps, play them back later, and link them to the bug database -- which is how the API Technical Support team reported bugs. It also could translate the playback scripts to runnable C++ or Visual Basic source code.
An especially valuable feature was where it could convert a trace file generated by the application itself into a Cool Tool script that could be used as described above. That allowed us to translate customers' trace files to source code that could be replayed in Technical Support or Development so issues could be reproduced using the same steps the user ran.
Click the following link to see screen shots and runtime options in the user guide that was presented when the user selected the Cool Tool's help menu.
Click here to see the Cool Tool help file.

C++: MSI GUID Generator

NewGuids is a utility prepared for internal use at Attachmate that provides for quick and reliable upgrades to WIX MSI installer XML source files. It is useful where the set of component GUIDs must be different from one version of the product to the others. It has been most valuable in the upgrade of successive versions of installer modules for third party products such as the Sun Java JRE.

In the case of upgrading a typical Java JRE from one update to another this reduces the time previously required for manual replacement of approximately 4000 GUIDs from several hours to a matter of only seconds.

HTML/Javascript: Javascript SDK Guide for Java EAO

The Javascript SDK Guide for Enterprise Access Objects is the API manual for the Attachmate Enterprise Access Objects product. EAO is a set of ActiveX and Java classes that provide interfaces a client app programmer can use to control one or many mainframe sessions. The product originally released with a Javadoc generated from the code comments and was scoped to Java programmers only. For the third product release I hand-edited a new version so details and code samples relevant to Javascript would be provided. Click here to see the guide.