10.1B Changes Integer Math Wrapping Behavior
An encryption component built into one of Solvepoint’s products came to our attention in a regression test on 10.1B. The root cause was a change in integer math wrapping behavior. Notice, I said “change”. I did not say “improvement”.
Some background, first. In most major computing languages integers wrap when an operation overflows the maximum allowed integer.
- In C# int i = int.MaxValue + 1; wraps.
- In ANSI C unsigned integers wrap.
- In Java integers wrap.
Integer wrapping behavior is an embedded, predictable and necessary part of many applications including a number of encryption algorithms, communications stacks, and data verification algorithms.
What about Progress applications? Well,…
All versions of Progress prior to 10.1B wrapped integers.
With the coming of 64-bit integers someone at Progress realized that it was possible to do 32-bit arithmetic in 64-bits and actually know whether the result went beyond the maximum (or minimum) integer. But just because something could be done doesn’t mean it should be done. So even though the Decimal data-type exists as an easy alternative to those wanting to avoid classic integer behavior, Progress changed integer math in 10.1B to throw an error instead of wrapping. If you’re incredulous, see Solution ID P119716.
In 10.1B 32-bit integer wraps throw an error “Value integer too large to fit in INTEGER datatype. (13682)“. Be prepared for some code written long ago to break.
If we multiply an integer less than the maximum 32-bit integer by a multiplicand that will result in a value greater than the maximum 32-bit integer, we can see the issue played out. In the following example we will use 1234567891 (a large easy to remember number less than max 32-bit int, +2,147,483,647) and multiply it by 10. In Pre 10.1B the result is -539222978, but as of 10.1B it returns an error.
Looking at the binary underneath this, see where 10.1B detects the overflow within 64-bits and throws the 32-bit exception #13682:
But can we now declare all Progress integer math safe? Not really. Since 64-bit integers in 10.1B are as blind to the >64 bits as 32-bit integers were to the >32 bits before 10.1B, 64-bit integers in 10.1B wrap around the way 32-bit integers did before 10.1B. (read that sentence several time if need be ) To help you understand why, Progress says “Checking for 64bit overflow would be too expensive. To do it would require putting everything in 128 bits to do calculations and then to copy it all back, since >64bits are required to do 64bit overflow checking. This would cause all arithmetic in the 4gl to grind to a halt. There are therefore no plans to do anything about this.”
So you’ve been warned. When this bites you’ll be able to say, “Darn!, and I read something about that somewhere!”
PS: And yes, for all the hard-core C jocks, signed integer wrapping is implementation dependent, not guaranteed to demonstrate modulus behavior, but usually wraps. Use of a signed integers rather than unsigned integers when classic integer wrapping is needed in C is a no-no.
Dataserver for Oracle 8i De-supported.
According to Progress the “Next Major Release after OpenEdge 10.1A” will not have support for Dataserver for Oracle 8i. Progress Software lists Dataserver for Oracle 10g as the Replacement Feature. Extended Support ends December 2007.
Progress defines “De-supported” usage as: “De-support is used where changes in technology or standards have made a feature obsolete and it is removed from the OpenEdge product. De-supported features have replacement equivalents and have zero impact on backwards compatibility.”
If you are stuck with an older embedded version of Oracle and need to move beyond OpenEdge 10.1A it may be time to explore available alternatives.
Most Dataserver users are not affected by the decision to De-Support Oracle 8i as they are able to upgrade Oracle.
OpenEdge ASSIGN Statement – More than just performance.
Back in the early ’90s when I broke the news of the ASSIGN statement in a Profiles in Progress article, I had no idea what silliness would follow. I also had no idea that twenty years later we would be seeing newly written code that still gets it wrong.
For the few who are as yet unaware, wrapping consecutive assignments with an ASSIGN has multiple benefits.
a = 4.
b = 3.
c = 7.
isn’t as good as
ASSIGN a = 4
b = 3
c = 7.
Shortly after the Profiles in Progress article was published there were signs that some in the Progress community were getting it wrong. Even though the article clearly graphed variations in execution time, some performance pundits started running around saying the ASSIGN statement was 2.7163639281 times faster than not using the ASSIGN. I’m exaggerating the precision of the number, of course, to make a point. None of those digits (including the initial “2″) are significant. None.
The ASSIGN statement varies for many reasons, not the least of which are whether what’s being assigned are components of an index in common, or part of a key that fully qualify a unique index. Let’s demonstrate one of these two factors using the example above.
Let’s suppose that a, b, and c are components of an index a-b-c. And let’s start, for simplicity’s sake, with a, b and c all having the value of “1″. In the ASSIGN-less code snippet, the index a-b-c will move through two transitional values, 4-1-1 and 4-3-1, before it gets to 4-3-7. Using ASSIGN, index a-b-c becomes 4-3-7 directly. Not using ASSIGN causes three-fold the activity:
Further, if index a-b-c is unique and the transitional values 4-1-1 or 4-3-1 already exist for another row, then the code without the ASSIGN statement will fail. Yes, fail.
Here’s a diagram that illustrates this index key collision for the above example:
The ASSIGN statement is not just an optional performance improvement as some believe. In some contexts, likely those least considered, lack of ASSIGN will affect reliability. Reliability is more important than Performance. Performance is also impacted as, in the example above, the application database must now perform triple the number of index lookups, inserts, and removes. Not good.
So everyone is using the ASSIGN statement, right? Sadly, no. One would think that we should find proper use of the ASSIGN statement it in all code written since the 80’s. Sadly, this isn’t the case –even in newly written 2007 code. We’ve seen it with our own eyes. As a community of Progress users, I know we can do better.
While this article isn’t an exhaustive presentation of the reasons why using ASSIGN is better, I’m hoping that the reliability and performance example above is compelling enough. It should be.
Hope this helps.
lkrela – What OpenEdge 10.1B users need to know and why it matters.
Progress users have come to expect extreme reliability from the Progress DB. This is no accident and has taken much consistent work over many years by the Progress Engine Crew. It is amazing how the Progress DB keeps ticking given what’s thrown at it in the field.
If you aren’t already aware, you should know that something bad has slipped through Progress’ regression tests.
If you are a 10.1B user, especially if you aren’t yet on Service Pack 3, your transactions may be exposed to data corruption. Tom Harris, OpenEdge RDBMS Development, calls it “a bad bug”. In an e-mail to the PEG DBA Forum, Tom has asked for users to “please” use the lkrela startup parameter “if you are running 10.1B prior to service pack 3″.
On OpenEdge versions 10.1B through Service Pack 2 inclusive there are critical bugs in locking that affect how locks are released, which in turn affect transaction consistency.
For your reference, the link to Solution ID is P126982.
While there is a fix in Service Pack 3, until Progress updates its regression tests for this and possibly related conditions, we are recommending to our clients that they keep using the lkrela startup parameter on 10.1B.
On Reliability, Functionality, Flexibility, and Performance
“Reliability is more important than Functionality. Functionality is more important than Flexibility. Flexibility is more important than Performance. Performance is viability.” -Mark Karaman
Welcome from the Editor in Chief
Welcome to progress.solvepoint.com.
In the decades that I have been working with Progress technologies there has never been a more exciting time than now. Features rich with business value like auditing and object orientation are beginning to flow. Through Progress Software’s acquisitions, new blood appears to be injecting vigor into the layered and related products groups. There are signs that some of the new folk are also growing in their knowledge and respect for the core database and language. Tool-sets are being re-tooled. Yes, indeed. These are exciting times!
The pace with which Progress users must discern and learn is at its highest in history. Opportunities and risks abound.
This blog, progress.solvepoint.com, joins PSDN and the PEG as a helpful place for users who need to sort out the good from the bad and make wiser technical decisions.
Welcome!
Sincerely,
Mark Karaman
progress.solvepoint.com Editor-in-Chief
Sponsored by Solvepoint
This blog is sponsored by Solvepoint, a leader in Progress performance tuning, database administration, software development, maintenance and integration. More Information