Tuesday, October 15, 2013

Commercial API's and the Open and Closed Principle (hypotheticals galore)

Suppose you work for a company that markets a desktop software product that is worth 10,000 times its price in operational savings to your customers. Because you must contend with some worthy competitors, your product earns revenue from only 50% of your potential market (and it doesn't sell at a price that is anywhere near its value).
Now, further suppose that you, yourself, wrote a program that can control your company's product by making use of its exposed programming interface, and automates its key features such that it can perform millions of operations per day and run your company's product 24/7 with almost no human interaction. You soon realize that your product's customers would like to develop and run a similar program that can provide speed and cost savings in their own companies. Because your application is specific to your company's business and won't work for everyone you prepare a library that can be loaded and used by applications that your customers develop for themselves. With your library, their applications possess your application's access to your company's product, but tailored to their needs.

Here are a few function prototypes that your library might offer:

List<string> getImportantData(String whichStuff)

bool storeImportantData(String storeStuff, String where)

String getCalculatedResults(String workStuff)

That three method library is an Applications Programming Interface -- an API. Now, your customers not only purchase your company's product, but most of them will probably want your API library, too.

Modification 1:
So, for the first couple of years your company's sales numbers increase. Now, including your library, you're selling two products. Changes in technology occur, and you might soon be losing market share to one of your competitors because they are nearing the release of a product that offers the same features as yours, including those provided in your API library, except it can be run with an application that features the use of a different programming technology. Instead of driving a monolithic desktop program -- that offers way more features than any one customer needs -- it can load and exercise only the monolith's component subset that it actually uses. That can translate to higher processing speed, less use of diskspace, and lower product cost because it's unnecessary to purchase the unused features.

You certainly do not want to lose market share, so your company sets out to match the competition's new offering by developing new software that offers the same features included in your original desktop product, but now those features can be purchased and loaded on an as-needed basis. This will provide the same technology and benefits as your competitor's product.

You are quick to remind everyone that a significant share of your product's sales have been due to the value provided in your API library, and there is every reason to expect that that will be the case when customers use the new product architecture, too. So, you now want to provide an API library to the new product that provides the same benefits as the original. The new product offers a different technology, but the actions that need to be programmable consist of the same basis: Customers will still want to retrieve and store data, and will still need the calculations obtained from your original product. There are a couple of new features made available by the change to the new architecture, and we will be serving our customers best if you make them available through our API.  Some customers may desire to automate the object-based nature of the new product using their own client applications designed using the same object-based technology.  Because the objects they use in your new product are not anchored to the monolith as before, it will be necessary to provide a little more information for your library that is loaded in their client applications to obtain a connection with the main product.  So, we'll introduce a slightly different API set to handle this new option:

List<string> getImportantStuff(String whichStuff, long serverLocation)

bool storeImportantStuff(String storeStuff, String where, long serverLocation)

String getProcessedResults(String workStuff, long serverLocation)

Excellent!  You're now providing the same capabilities to your customers that they found so compelling in your original monolithic desktop product. You are saving them disk space and product expense as well as increasing their processing speed because their client applications are making use of refactored, faster calls.


When Modification 1 was in design it would've been a good idea to provide what would be seen as the same library to customers that you offered in your original product. It would be loaded by customer client applications the same as with your original product, and it would offer the same set of methods.  On its backside however, it would interface with your new object-based product instead of your monolithic desktop application. In the case of this Modification 1, it might become necessary to bridge the gap between product generations by presetting new configuration options on the system that allow earlier generation API calls to function in the new environment. With that, your customers could continue to make use of the client applications they had written over time.  They would not find it necessary to rewrite the client applications they'd perfected over the years just so they could get the same benefit from your new product they've already had with one they're already using.
From my time working in a developer support role, I am aware of several scenarios where customers cannot update their client applications.  Sometimes the subject-expert application author has left the company or the application was written by a short-term contractor. Sometimes companies have lost the source code for applications they run regularly as people left, hardware was updated, or roles changes. Sometimes a change in development technology such as the programming language or development environment makes editing and recompiling a challenge: "There isn't anyone here who does that anymore!"
When the new technology provides your customers with a business case for investment in writing new client applications, their development team could take up that task -- one which is much simpler and less expensive than duplicating what was already working in the old technology.

This is where a software engineering best practice, the Open and Closed Principle is suggested. It can be stated as, "an interface may be open for extension, but closed to change".  Once published, a functional interface must remain exactly the same -- no changes allowed. Development operations expend a lot of time and money writing applications to an interface, and if the interface were to be changed some proportion of that time and expense would be spent again to react to the changes. When the interface developer wants to add new capabilities that might require a change to the interface they instead add new member functions as extensions. With that, any legacy applications will continue to work as before with every new release of the interface library. The new interface capabilities are available for in the same library when they provide a compelling reason, and not only because the rug was pulled out from under the earlier interface.
So, applying this wisdom to Modification 1, the new product could provide what, by nearly all indications is the original library, but one that also includes the new product's extension methods -- the 3 original methods together with the three new ones. For customers who intend to use only the new API methods and the new object based programming technology, the objects to which our API library talks on its back end will be directly available to their new object-based client applications.
With that, your customers are not forced into a situation where they must think about how much time will be required to rewrite their client automation applications. They can continue to enjoy the value of what they did in the past to run with your product, and not be forced into a situation where they wold be wise to look into whether or not your competitor might offer a less expensive or better way to get the job done. The cost arithmetic won't justify their moving to a competitor's product and changing all of their API client applications when they are currently running a suite of client applications that continue to work with every version of your product.

Modification 2:
After a company meeting you learn that your company has merged with another company that is your most successful direct competitor. After the merger your one larger company will have two large customer bases that are standardized on two similar products, but which of course have different API definitions. How should you handle this in order to retain the product loyalty of both customer sets? It seems inevitable that, rather than continue to develop two products that offer an almost exact feature match, one-half of your engineering resources will be retasked. Only one of the two previously separate vendors' products or a new hybrid product will be sold starting in the near future . As similarly asked during consideration of Modification 1, how do you retain the loyalty of customers who may not have used the surviving product or technology before the merger?
As mentioned above, in the future one of two approaches toward balancing the API problem are feasible. One, the company decides that one of the products will survive and the other will be phased out over the next couple of releases in order to provide that side of the customer base with migration time.  Or, you can help approximately one-half of the customers by either replacing their soon-to-be deprecated API client applications -- such as preparing a code compiler/translator -- or you can provide a new API library that provides the same interface as the sunset application's API but drives your surviving product.


If the surviving application was developed by your new step-company, then you will want to develop a new version of your standard API library that loads as before, and exposes the same methods as before, but which serves as an adapter to the new API.  With that your customers' client applications will continue to function as before, but with the new product. Again, no changes should be necessary on your customers' end other than installation and configuration of the new product.

If the management decision is instead that a new product is to be released that combines and replaces the offerings of each partner in the merger then, as you might guess, a new API library that exposes both your product's and your step-sibling's API's -- that will load and be used with no new customer effort. You can always extend the new library to offer features available in the new application or technology, but, for the reasons cited above in Modification 1, you must always preserve the customers' ability to continue use of the API client applications prepared and used in the past by offering an unchanged basis API.

Modification 3:
 Your development team learns that your company is preparing to develop and release a new type of system that is both similar and different to what you've been supporting up until now. The new system type is similar in that customers still desire the ability to automate sending, receiving, and receiving the results of calculations from the application -- a welcome acknowledgment to you and your development team that your API has been offering a solid basis for automating your technology.
When you think about preparing an API interface for this new product, consider whether or not your customers who have written client applications that automate your other product might be able to use those same applications with your new product variation. If you stick to the basis used so far, offering the same set of methods, would they work as written with no more than some configuration changes?

 Modification 4:
Customers and sales representatives continually see significant value in, or are coerced into, adjusting their operations so they can take advantage of newer technologies. For example, a product developed to run on a web browser can be quickly and inexpensively installed, distributed, and updated. Access security for such a system can be administered in a very general way. So, of course customers and sales reps are interested in what we can do for them in a mobile or cloud-based solution. As those questions are asked we are led to consider how we might benefit from knowledge of the open-closed rule when the product we will soon offer serves browsers from the web.


In the context of this discussion it is important to state that no matter the new technology obtained, customers will almost certainly continue to run their operation using the same type of system: Windows, OS/X, Linux, ...  While the new technology offers new features, for instance a JavaScript API to which customers or customers' server applications can write automation in their HTML forms, your first task is to protect your legacy customers by maintaining open-closed rule support providing an API library that your customers' client applications can load and use.The important fact is that you continue to provide automation access to your basis API so customers are never required to change their legacy applications.  The API library you provide to maintain support for legacy customer client apps in this new product might need to do some tricky stuff on its backend to be able to communicate the calls made into the standard library to the web-based objects, but you must not change the link between previous products to which your customers have standardized and your new product technology. As mentioned above, both technologies are running on the same system, where it is both possible and will provide the benefits of their workhorse applications to your customers.

Modification 5: In Modification 3 we were concerned with a jump to a web browser thin client offering.  You can also jump to a server-based web technology, which would require that the standard API library variation that you've provided for every product so far must now support the customers' legacy client application, but be able to communicate with the web server where your new server-based product is installed.


Modification 6:  By now, the pattern should be clear: With any version of your company's product, provide an API library with the same API interface exposed so your customers are always able to use their legacy client applications with your new generation product.  That would include a mobile phone or tablet-based offering.  It might involve a watch or a car, or some technology of which we are not yet aware.  The pattern is, preserve the basic interface, extend it as you wish, and sometimes consider the extensions a new part of the basis moving forward.


Conclusion: If you prepare an automation library that exposes the same API basis for every generation and variation of your product you save your customers from rewriting developed, tested, and proven applications. They do not need to pay for new planning, design, development, and testing of a new generation of automation applications. They already possess applications that they know and trust will deliver. When you release a new generation of your product that features the use of a new technology, smart customers will see that as a time when they should evaluate your competitors' products, too. Because the suite of automation applications that they need is already on your side of the balance, that evaluation period will likely be very short, and to your advantage.

When a potential customer evaluates your product, even without a suite of automation applications on the shelf based on your API, your reputation will help them realize that all applications they write to automate your product will be usable in all technologies supported by your products in the future.

When a company's development manager considers the design and implementation of automation applications that they will produce to drive your products they will write the suite with long term efficiency and quality in mind. They can write more efficient applications, applications and application components because they know they can rely on your basis API as a given.