Predictably, the first client roll-out of our new .Net / U2 (aka IBM Universe) application hit a snag and, predictably, it wasn’t one that I saw coming.
And, boy, what a snag. The first time we tried to login to our application on-site, U2’s UniOLEDB driver threw a “Native Error 930065″ Exception. The very first time, an error that I couldn’t remember ever seeing before. Talk about a worst case scenario!
Like many things in the U2 world this particular error code isn’t documented, nor Googleable. (Go on, try Googling “UniOleDB” and “930065″. Feel Lucky? I bet you end up back on this page). However, Google Desktop came through when my real memory failed by dredging up an old e-mail from IBM support stating (inaccurately) that a 930065 error that we hit when evaluating their U2 10.2 version meant that we were out of licenses.
This time, the “out of licenses” scenario seemed more plausible, since this client only had a few spare licenses. But the very first user? While logging in? Our application is a strictly two-tiered architecture, and given that our clients have to pay for database licenses, we are reluctant to use U2’s add-on server-based connection pooling feature. So our database connections are old school — the user desktop connects, does its stuff, and immediately disconnects to free up the license.
UniOLEDB actually co-operates fairly well with the .Net Data Provider for OLE DB. You create your OleDBConnection object, set the connection string, login to the database, then immediately close the connection. Thereafter, the connection is automatically reopened when needed, such as when you call a .Fill method to run a SELECT statement, and (so I thought) automatically closed afterwards . This arrangement looked great during our many months of internal testing. Connection management was never a concern — it just worked.
It turns out that the arrangement wasn’t working as well as I thought. We have plenty of database licenses on our test server and we never come close to reaching the limit. As a result, I never noticed a bug/feature concerning the way U2 handles the database connections. In the course of handling a particular action (e.g. logging in), we might read from a dozen tables, resulting in the database connection being opened and closed a dozen times. There is only one OleDBConnection object on the client end (a public static variable), which means there is only 1 database connection per client application. But closing that connection does not immediately cause the database license to be freed up. Depending upon the speed of the desktop PC and the load on the server, those dozen connections might result in 8-10 database licenses being briefly devoted to a single client.
Actually, much of this is conjecture on my part. Since I couldn’t find any confirmation of this behaviour in IBM’s “Using UniOLEDB” manual, I e-mailed IBM support asking “when I close a database connection is it possible that the Universe license is not freed up instantaneously?” I received a rather philosophical response that “In reality, there is no such thing as instantaneous. Everything takes a period of time, no matter how tiny a period.”
The IBM tech support person went on to say that he tried closing his test application, then entering the U2 license command into a Telnet session, and he was never able to catch the license still being used. He suggested that, when the code encounters a 930065 error, it should pause to let Universe “reclaim” the license, then try to connect again.
I wasn’t keen on the concept of adding speed bumps to the application, but his use of the word “reclaim” was enough of a hint for me that closing the connection doesn’t free up the license synchronously. (Like that word better, Oh Zen Master of IBM?) There was a period of time that would elapse before the license was freed up again, somewhere between 0 and the amount of time required for an IBM tech support guy to type “uvlictool report_lic” into a Telnet session.
While a dozen connections might chew up 12 licenses during that time, 1 connection couldn’t possibly chew up more than 1 license. So, rather than relying on the (admittedly undocumented) ability of UniOLEDB to automatically connect and disconnect as needed, the safest solution seemed to be to take control of the connections in our code. This meant a long evening of furiously adding calls to our connect and disconnect method at the start of and end of each user “action”, and a long day of testing that code, but it fixed the problem. No more 930065 errors!
What I am still seeing, though, is that the license occasionally isn’t freed up at all. The UniOLEDB object gets closed, disposed and set to null and the license is still being used until the next database connection is made from that client. This isn’t a garbage collection timing issue, but a failure to let go. With so many layers of code sitting between the user and the server, it’s hard to know which layer is failing to pass along the request to drop the license.
