Reliable messaging can be hard to achieve in difficult terrains. This was apparent as we met up recently with a lead developer of a major freight company that specializes in shipping to and from far flung facilities in Alaska.
Rob Terpilowski, lead Java developer, Lynden, Inc., explained how he and his team used Apache MQ messaging middleware and other open source messaging software to replace homegrown tracking software. He said the original tracking software was built to overcome immediate needs without consideration for later modernization or integration requirements. Terpilowski described the project in a session titled “Building a reliable messaging system for an unreliable (and diverse) world that took place as part of the CamelOne conference last month in Boston.
The challenge was to combine disparate systems including a decades-old pre-relational database platform, Solaris 10, Glassfish application servers, both graphical and green-screen user interfaces and barcode scanners that are embedded devices running on Windows CE.
One of the major reasons the old homegrown system needed to be replaced, according to Terpilowski, was that it couldn’t scale up to meet the shipping company’s growing needs. The polling system that was in place ran into issues with growing database load and with endpoints getting overwhelmed with irrelevant data. Terpilowski and his team were able to add messaging via Active MQ and an additional MQ table that the existing system would write to, keeping the inflow of data fairly consistent. But the flow of data back out to the endpoints went from a constant pull of large amounts of data to a much more targeted push of relevant data via Active MQ and location-based topics.
There were also challenges associated with intermittent Wi-Fi connections at storage and freight routing facilities where workers with barcode scanners would constantly move in and out of thick-walled storage containers, breaking the connection. Early efforts at overcoming the connectivity issues centered on a .NET failover system on the devices themselves, but that failover system was deemed non-optimal.
Eventually the team decided on rolling its own proxy server. The main purpose of the proxy server was to maintain connections on behalf of the endpoints as they appeared and disappeared, and then close out the connection properly when it was appropriate. With the messaging proxy in place to handle connectivity issues, Terpilowski’s new system was able to monitor proxy related statistics, summarize client information and reset a client’s connection to the system when necessary.
Stay tuned for more on embedded middleware issues. Let us know about your experience with open source messaging software.