In the context of modernizing character-based screens (a.k.a. 5250 screens in the IBM i world), the concept of “virtual users” refers to the technique of “wrapping” an existing 5250 interactive application with code that intercepts and controls the 5250 data stream. The technique can be implemented in a “batch” program, without any required access to a physical 5250 terminal or emulator. The “batch” program becomes the “virtual user” – a robot that can then do the bidding of another application.
The “virtual users” technique is commonly applied in the context of application modernization, and can offer compelling benefits including:
- Fast and incremental delivery
- Low risk and low cost, relative to alternatives
- Target strategic elements of the application that deliver the maximum business benefit
- Reuse business knowledge and rules that are already encoded in the 5250 application
When considering application modernization strategies, it would be unwise to overlook the potential of “virtual users.”
But what about applying “virtual user” techniques in your Business Process Integration (BPI) solutions as well? Is it valid to do so? Is it wise? Why would you do so, and what are the risks and rewards? Let me propose some answers . . .
Why?
The three key parts that comprise a BPI solution are transport, transformation and orchestration. All three of these should be core competencies of your BPI toolset, but transformation is the part of most interest in this discussion.
Transformation involves transforming data between the various formats in which it is represented both within your organisation and with trading partners. For example, you should be able to map data between standard EDI formats and your application database or from a spreadsheet to XML.
Whatever other formats are involved, a common part of many transformation requirements is getting data into or out of your IBM i application’s database. But what if it is not feasible to directly address the application’s database tables in your transformations?
If you scoff at that thought, then you are fortunate indeed and you probably need read no further. But many organisations today still employ extensive manual rekeying of data in the course of what should be routine business processes. Sometimes the reason is simply an apparent case of Hobson’s choice. A variety of factors may rule out the option of directly addressing the application’s database tables, such as:
- No specifications or source code
- Lack of domain knowledge or skills
- No application program interfaces
One of the objectives of Business Process Integration is to eliminate the manual re-keying of data. If re-keying is your only viable option, then the only way to automate the process will be to employ “virtual users” to perform the re-keying.
Consider this
Imagine a simple process involving three main steps:
- Receive sales orders from a trading partner via FTP
- Transform the sales orders from their XML format and write them to the ERP database
- Call an ERP application program to initiate the order, generate picking lists, adjust stock levels and invoice the customer
Now what if, for any of the reasons discussed, we cannot perform the transformation directly to the ERP database? We might still be able to make use of a “virtual user” to automate this process. In such a case, the BPI solution might employ a “virtual user” to:
- Connect to a 5250 session and logon
- Start the order entry 5250 application
- Navigate to the order entry screen
- “Type” the order details from the XML file into the order entry screen
- Verify the results and logoff
Sometimes a “virtual user” may be employed to achieve more than simply transformation. For example, it can be good BPI practice to receive data from external parties into a temporary or “staging” database and then validate it before accepting it into your core ERP application. Now we really want step 2 of our scenario to look like this:
2a. Transform the sales orders from their XML format and write to the “staging” database
2b. Call an application program interface (API) that contains business logic and validation rules that will check the data before it is inserted into the system
But what if the API for step 2b simply does not exist? Well, again, we might make use of a “virtual user” to automate this process. In this instance we are leveraging the business logic and validation rules that are already encoded in the 5250 application.
Risks
As a general rule, implementing “virtual users” is the least preferred integration technique if any of the more standard techniques are feasible. This is because it does carry risks that are prone to under-estimation and/or discounting as not “today’s problem.” Primarily the risks fall into two categories:
- Changes to the 5250 application
If the 5250 application that the “virtual user” will operate is subject to on-going change, then it is very important to clearly understand that such changes may also affect the BPI solution – because it depends on the 5250 “screens” having a defined layout and sequence. If the application is developed or maintained in-house, then, of course, such changes and their impact are entirely within your control, but you need to be sure that all involved are aware of the potential impacts. If the application is provided by a third-party, then you must be aware of the potential impact of patches and upgrades to the application.
- Unanticipated behaviour of the application
I am not referring to application failures, but simply intended responses to certain inputs that you were not aware of when you designed the BPI solution. For example, imagine if entering the discount code “9Y2Z” on a sales order the week before Thanksgiving resulted in an extra pop-up window prompting you to enter an optional promotional order code. If you were not aware of this exceptional behaviour, then your solution will not be designed to handle it and will probably fail when that particular combination of circumstances arises.
These risks have the potential to affect the reliability of the BPI solution. If your organization has come to rely upon the BPI solution to receive sales orders, then a failure in the process may well be critical. The risks are by no means unmanageable but you need to understand and plan for them.
Rewards
Calculating the rewards of implementing Business Process Integration using “virtual users” is relatively straightforward.
In the simplest case, it may very simply hold forth the possibility of completing that last link in the data chain that makes the solution possible or not.
In a more nuanced case, it may be the savings in time and cost of the alternative process of discovery, specification, implementation and testing that is necessary to implement a mapping solution in the absence of existing domain knowledge.
Either way, the bottom-line rewards are the same as offered by any Business Process Integration solution
Conclusion
In general, preferred BPI techniques for integrating with an existing IBM i application include:
- Direct transformation mapping to or from the application’s database and/or
- Integration via application program interfaces provided by the application
Using such techniques will generally provide the most resilient and the best performing solution as well as being the easiest to maintain.
However, if these techniques are not open to you, then you may have only the one option remaining. Implementing “virtual users” judiciously in your BPI solutions may provide the “last mile” of integration that could not otherwise be feasibly attained.
There are risks and rewards. If we were talking here about the FTSE100, I might be able to present some formulae to evaluate the risk versus reward, or a graph that plots one against the other. Alas, there is no simple calculation here – all I can offer are some considerations necessary for you to make your own judgement according to your own circumstances.
There is an oft-used truism: If the only tool you have is a hammer, every problem looks like a nail. Well, in this case, we might turn that on its head . . . if your problem is a nail, then you had better use a hammer to drive it home.
Suggestions
If you’re considering employing “virtual users” in your BPI solutions, here are some suggestions that might help along the way:
- You’ll need to have access to someone who knows the 5250 application inside-out. If that’s not you, then it may well be worth consulting the business users who most frequently use the system.
- Make sure that you document what has been done and that those who need to know, do know – so that in the event of changes or upgrades to the 5250 application, the impacts can be planned and managed from the start.
- Keep it simple – minimize navigation and data entry. Consider non-programmatic ways to minimize the navigation and data entry necessary to achieve a given result. For example, it might be that by creating a specific user-profile for the purpose, you are able to assign it a different entry menu or initial program that eliminates 5250 navigation that is not essential to the purpose.
- If the orchestration of the 5250 operations is non-trivial, or if you anticipate reusing it in other solutions, consider wrapping it in a separate callable program or function and calling it from your BPI solution.
- Generally, the performance of an automated 5250 application can be very good for many purposes, but since the navigation and data entry tends to be a very granular activity, beware of vectoring these activities through too many layers of your BPI toolset – especially if the solution will be subject to high-volume, high-throughput patterns of usage. Again, consider wrapping your “virtual user” in a program or function.
- Make sure you are familiar with the capabilities of the tools you will be using. This includes both your BPI toolset and the tools you use to implement your “virtual user” robots.
