Wednesday, August 22, 2018

Friday, August 03, 2018

Creating scalable RPA processes with Queueing

Often when I am talking with customers and prospects they ask about the concept of queueing transactions and then level loading across multiple agents/servers/bots so that their workloads are instantly scalable up or down as needed by adding additional agents into the mix.

Here's an example where queueing might be useful: Let's say a company needs to interact with a vendor or customer web site, an SAP application or some other UI based ERP or business application that can only process one transaction at a time. Often a single bot won't cut it because it's not fast enough because of application or other response time issues, so you need something that is infinitely scalable. Also a server in many cases is limited to one transaction at a time because it's automating using a user interface. Background processes are another story because a single server/agent/bot can handle many background process threads simultaneously.

By intercepting all transactions as they come in and funneling them to a single queue or inbox, one or more (pick a number) robotic processes can pick up the next transaction as it finishes the previous one. Think of it as being similar to picking up the next phone call in a call center or being next in line at the grocery store. First-in, first out using as many lines available.

So how does this queueing thing work ? Well the idea is that as a transaction comes in and gets added to a database or some other data store, a queue entry is written immediately so the next agent/process/bot can pick up the transaction and get it done. As the next sequential entry gets picked up it gets removed from the queue or marked as in-process or complete.

You might consider using a message queue system such as Microsoft Message Queue (MSMQ), WebSphereMQ, RabbitMQ, ActiveMQ or even a platform such as Apache Kafka. There are probably others that I missed. These queueing systems can be implemented simply and used with any RPA tool set that can call an API or web service. However they do add another layer to the complexity and overhead of your RPA process by adding another moving part for your application or RPA process to monitor: The message queue service.

Another simpler option perhaps might be to write queue entries to a database so that they can be processed easily and also the queue can be visualized via an SQL query as needed for reporting purposes. If you have read articles on table queueing theory on the web, there are lots of things written that recommend not using databases as queue mechanisms. I have researched a lot and found that SQL Server (and probably other databases as well) can work nicely as a queueing system if you're careful. The trick is when reading an entry, how do you make sure you process a particular database queue entry only one time ?

While researching on SQL Server methods for locking I learned about the READPAST and UPDLOCK flags. I won't regurgitate other articles so you can simply google the highlighted keywords for yourself and draw your own conclusions. However the upshot is that when these two flags are used on a query combination that reads and updates/deletes a record all in a single transaction, you should be able insure that each queue table entry is handled correctly and only once. When writing a new entry to a queue table you simply perform a standard record insert, so feeding the queue a bunch of transaction data that needs to get processed is very simple.

Hopefully you'll find this technique to be helpful when you're scaling your RPA, Workflow other process automation workloads. This technique should work with any automation toolset or custom process you create where queueing is required.

Sunday, June 24, 2018

Check out Open Source WinAppDriver UI Recorder

A new open-sourced tool is now available for the Windows Application Driver (WinAppDriver) community: the WinAppDriver UI Recorder tool. This tool will enable users to easily create automated UI tests.
For those of you not familiar with WinAppDriver, it is a UI automation service for Windows 10 that users can use to test their applications. 

Wednesday, June 20, 2018

Setting Up SSH Daemon on IBM i to Use Port Other Than 22

Log on to a 5250 session on your IBM i system. 

Edit the SSH Daemon config file.

On V7R3 the edit command would look as follows:


edtf '/QOpenSys/QIBM/UserData/SC1/OpenSSH/etc/sshd_config'

Once the edit screen comes up it may look as follows: 

# $OpenBSD: sshd_config,v 1.75 2007/03/19 01:01:29 djm Exp $               
                                                                           
# This is the sshd server system-wide configuration file.  See             
# sshd_config(5) for more information.                                     
                                                                           
# This sshd was compiled with PATH=/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys
                                                                           
# The strategy used for options in the default sshd_config shipped with    
# OpenSSH is to specify options with their default value where             
# possible, but leave them commented.  Uncommented options change a        
# default value.                                                           
                                                                           
 Port 2022                                                                 
 #Port22

Add a new entry for the new port. Ex: Port 2022 and comment (#) the od entry if not already commented. 

After adding the port entry for port 2022, you can press F3 to save and exit the config file editor.

Stop and restart the SSH daemon with the following commands

ENDTCPSVR *SSHD
Wait 10 seconds
STRTCPSVR *SSHD

Run the following command to make sure port 2022 is active:

NETSTAT OPTION(*CNN) 


If you see local port 2022 is active then you should be able to log in to the SSH server on port 2022. 

     Remote           Remote     Local                        
Opt  Address          Port       Port       Idle Time  State  
     *                *          as-admi >  100:11:23  Listen 
     *                *          as-admi >  100:17:44  Listen 
     *                *          as-admi >  100:17:44  Listen 
     *                *          as-admi >  100:17:44  Listen 

     *                *          2022       000:06:12  Listen 



Wednesday, June 06, 2018

MobaXterm X Server and SSH Client

If you have a need to connect to remote Linux servers via SSH, VNC, Telnet, RDP and more, check out the MobaXTerm application. 


Saturday, May 12, 2018

Who needs a chatbot when you can have a mailbot ?

While chatbots are the latest new craze, the concept of interacting with a business system has been around for a long time. Back in 2002 I wrote an email monitoring system for a retail customer who was sending reps and managers out into the field with Blackberry devices. When the manager was about to visit a set of stores they would simply send an email request to a selected address such as: storeinfo@domain.com along with a subject of: “STINFO store#”.

Once the request was received a background process would be triggered to parse the subject line, build a sales report and then format the data as HTML and send it back as a response. Theoretically a PDF, Excel, CSV or other attachment could be sent back as well in response to a query.

Fast forward sixteen years and we apply the term “bot” to just about any automation process. Since the term “bot” is used so heavily today you might call this request/response scenario an email-bot or mail-bot. I used to call it poor-man’s client/server because a request could be created and processed without opening up a web site or business systems directly to the web or other client application. This also allowed for a nice layer of security. As long as responses are always being returned back to the same authorized requestor email address and that address has been validated, chances are data can’t be compromised. Also since the mail-bot can only respond to a limited set of scenarios, it’s threat vector for being hackable is reduced.

A mail-bot pattern essentially works like this:

An email request is received by a monitored address containing a from address and a request in the subject line or email body.

Once the mailbox monitor process picks up an inbound email message, the message is parsed, information is validated and then a response is compiled if the request is from a valid email address.

A formatted response email message or file attachments are then returned to the requestor.

There’s a little more to it than what I described above, but if you’re thinking about creating a chat-bot, you might want to first do a proof of concept project using the mail-bot pattern to see if your logic is sound and if your customers will get value from such an information request method.

No matter whether you’re using an RPA tool or coding a custom business process the mail-bot pattern can help your team be successful interacting with vendors, employees and customers.

Sunday, April 08, 2018

Free Windows .Net Snippet Keeper Application

Free Snippet Keeper Application. This is my attempt at creating a useful Snipper keeper application. I also used a couple of notable technologies in this app: MetroUIFramework, SQLite for the data store and FastColoredtextBox for the text editing.
Download SnippetKeeper

NancyFx - A Lightweight Web Framework for .Net

I'm always looking for alternate technologies to use for creating easy to install and use app servers, web servers, micro-services and more.

A couple years back I came across the NancyFx framework for this. It works well with .Net and .Net Core to create self hosted web applications and services.
http://nancyfx.org/

Open Source Spreadsheets for your .Net Applications

If you're building .Net applications chances are you've had the need to convert data into spreadsheet format. Check out the following links for a couple of open source options:
NPOI
SpreadsheetLight
These also work nicely in conjunction with SQL Server, IBM i and other databases to quickly create Excel output.

VietOCR Open Source Front End for Tesseract OCR

Check out this OCR application if you need a basic open source front end to the Tesseract engine. VietOCR is a Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese and other languages supported by Tesseract. http://vietocr.sourceforge.net/

Ublu Midrange and Mainframe Life Cycle Extension Language

I haven't checked out this project yet, but it looks interesting. From the Github description: Ublu is an interpretive language for remote systems programming of midrange or mainframe hosts from a Java platform such as Linux, Mac, OpenBSD or Windows. It also can run natively on IBM i ®, IBM z/OS USS ® or any other reasonable Java platform. https://github.com/jwoehr/ublu

Windows Software Installation Tools

I have used this product for the past 6 years for building Windows and Web based software installations. Much more affordable than Installshield and works very well.

Advanced Installer

Another open source option is Nullsoft

Nullsoft Scriptable Install System