OpenEdge ABL Phantom Error
At Solvepoint we’ve coined a new term, the Phantom Error. What is a Phantom Error you ask? Well, it is an undesirable circumstance where the Progress VM decides to raise the error condition but leaves error-status:get-message() set to the empty string.
This , of course, can lead to a host of ugly sorts of bugs not the least of which is not having any idea where the error happened or why.
Here is a simple example that demonstrates a Phantom Error:
DEFINE NEW GLOBAL SHARED VARIABLE myHandle AS HANDLE NO-UNDO. main: DO ON ERROR UNDO main, RETRY main: IF RETRY THEN do: MESSAGE RETURN-VALUE error-status:get-message(1) VIEW-AS ALERT-BOX. LEAVE main. END. RUN someProc IN myHandle. END.
You’ll notice that both return-value and get-message(1) return blank.
If you happen to be in a terminal based procedure editor, however, you will receive the error message in the “message area” at the bottom. Unfortunately, this doesn’t do much good for server code.
As a consequence, server code will typically swallow these Phantom Errors at worst, or report the error out of context at best.
So, what can be done? Diligent error trapping is called for. Protect the code by always testing handles before using them.
Unfortunately, this isn’t the only type of code that will produce a Phantom Error. We will discuss other Phantom Errors under the Tag “Phantom Errors” in other posts.
Little-known OpenEdge Database _User Table Security Behavior
Since it is not well known, I thought that I’d bring the following information to light regarding the OpenEdge _User Table Security Behaviour.
- OpenEdge Database security is implemented at the table level not the connection level. This means that, even with “blank user disabled”, a client may still connect. Security is only checked when that user attempts to access a table. Take careful note of the confirmation message:
You are about to prevent the blank userid from
accessing your database. Users who are not
listed in the data security fields will not
be able to compile procedures with this database.Notice it says nothing about connecting. Which brings us to point 2.
- Table level access is compiled into a .r. Therefore an _user without any access to that table may still run the .r successfully (even the client connected without a user id).
- The _user row may be modified or completely removed without affecting the running sessions (even after a STOP is raised).
- The _user password can only be changed when connected via that _user.
- The _user row , however, may be deleted by any session that has _can-delete on the _user table.
- The same _user row may then be created again with a different password (yikes!). (Still does not affect any running sessions).
Please keep these behaviors in mind as you plan your security strategy.
PS: Please be aware that Progress says it will be modifying security including adding run-time capabilities sometime in the near future.
OpenEdge ABL Memptr Pitfalls
Memptr is a very powerful datatype in the ABL/4GL. It allows the programmer to store any type of data including binary. However, as with all dynamic objects in Progress, one must be careful when using it.
Pitfall number one: Scope. Memptrs do not follow the rules of scope to which 4GL programmers have become accustomed.
What does this mean? Why do I care? Well, I’ll tell you. Since Memptrs do not follow the rules of scope, when a variable holding a Memptr HANDLE goes out of scope the associated memory is NOT released. You care because if this happens you now have a memory leak. Every time your program is executed it will leak memory equal to the amount allocated to your memptr.
Pitfall number two is the allocation process. Probably 99% of code you find will do this:
/* define a memptr variable */def var m as memptr no-undo./* Allocate the memory */set-size(m) = 1024./* now go ahead and start using it… */…
See anything wrong with the above? If not, don’t blame yourself. You are used to Progress doing this for you, but in this case, it does not. What am I referring to? You C programmers will know! The memory that has been allocated to m has not yet been “initialized”. This means the memory will contain random data: whatever happened to be in there before the allocation. If you are using put-string before your first get-string (without the numbytes parameter) then you have nothing to worry about since put-string automatically puts a NULL (0) as the next byte after the string and get-string will only read up to that NULL. But for other operations like put/get-byte or put/get-bytes or put/get-string with the numbytes parameter, grabbing random uninitialized data out of memory could bite you, so beware.
The final pitfall is also related to allocation. In your code, you may define a memptr at the beginning of a procedure and then use it in several places throughout the code. In each use you will want to allocate the appropriate amount of memory. So you may code something like this:
def var m as memptr no-undo.set-size(m) = 128./* do stuff with it here… */set-size(m) = 1024./* do other stuff with it here … */and so on…/* now we clean up and return */set-size(m) = 0.return.
This looks great right? We are allocating and cleaning up just as we should, right? Well, yes and no. The pitfall is that the second set-size where we allocate 1024 bytes doesn’t actually allocate anything. It essentially does nothing at all. AND it does not raise an error condition. So now we have a potential bug if the code attempts to put more than 128 bytes into that memptr.
This is solved by setting the memptr to 0 first.
Moral of the story, memptrs need special care as they do not particpate in conventional scoping and cannot be resized until they are cleared.
Hope this helps to save you some time in your coding efforts!
OpenEdge Record Lock Anti-pattern
Everyone knows that if you need an exclusive lock on a database record in the 4gl you do this:
find customer exclusive-lock where customer.id eq 1 no-error.
However, this will not wait forever. It waits for lkwtmo, which defaults to 30 minutes, then raises the STOP condition. Well, what if you really want to wait longer than 30 minutes (or whatever lkwtmo is set to)? We have occasionally seen developers do this (gasp!):
find customer where customer.id eq 1 exclusive-lock no-wait no-error. do while locked customer: find customer where customer.id eq 1 exclusive-lock no-wait no-error. end.
This will certainly work. However, if there is actually a lock on that row, this code will SLOW the database AND saturate the CPU of the client machine on which this code is running. That is an unbridled loop that will iterate thousands of times a second. Each iteration will cause the database to do work as well slowing all access (each lock attempt will show in promon as 1 Commit and as 3 DBRequests).
For example, on a 2.1ghz CPU that loop will iterate 40,000 times per second.
There are a number of good alternatives to the above design. One would be the following:
getLock: do on stop undo getLock, retry getLock:if retry then do:/* you may want to log something here or possibly put an upper limit on the wait time by counting the number of times it hits this code The below will cause an error to be returned after 60 minutes assuming the default lkwtmo */ cnt = cnt + 1. if cnt eq 2 then return error “Unable to get a lock on customer”.end. find customer exclusive-lock where customer.id eq 1 no-error. end.
As you can see, it is much, much better to let the Progress VM deal with getting the lock in this scenario.
OpenEdge Memory Management Anti-pattern
Hello all,
I’ve recently been reviewing some Progress 4GL and have found an all too common anti-pattern related to memory management.
When a variable is defined, the Progress runtime client (Virtual Machine) allocates memory at runtime for that variable. Once the variable is out of scope, the memory is released and everyone is happy. Progress programmers have grown comfortable with this design and obliviously define variables whenever they are needed knowing that they will be de-allocated automagically by the Progress VM.
Then came dynamic objects.
Progress programmers were overjoyed! They could now create temp-tables, buttons, queries all on the fly at runtime. No more convoluted if-then statements or .i’s or having to code a different “for each” for every combination of where clause.
However, as with any power bestowing feature, there is a dark side to this wonderful new world of dynamic 4gl: memory management. Most programmers never really stopped to consider the fact that if something is created dynamically at runtime, the VM has no way of knowing the scope. It cannot tell when to release the memory required for the dynamic object. REMEMBER: the scope of the variable you happen to assign the object to HAS NO BEARING on the scope of the OBJECT since it can be passed around. In other words, the scope of the variable holding the HANDLE to the object is NOT bound to the OBJECT itself. The scope of ALL OBJECTS are always at the SESSION. This applies to GUI widgets, dynamic queries, temp-tables, etc.
Java (and other VM’s) solve this through the use of a separate execution thread running concurrently called a Garbage Collector. Its job is to scan memory and find dynamic objects that are no longer “reachable” and release their memory. Unfortunately, the Progress VM has no such thread/concept.
To add insult to injury, not only does this leak memory but it also causes progressively worse performance: The more widgets in memory, the more time it takes to create another widget. Here are three examples (run on a 2.1ghz processor):
Button Handles
Minimum memory required/lost per Button Handle: 512 bytes
Query Handles
Minimum memory required/lost per Query Handle: 1024 bytes
Temp-Table Handles
Min. memory required/lost per Temp-Table Handle: 512 bytes
So, as you can see, from both a memory and CPU footprint standpoints, it is very important to be sure to clean up your objects.
This may seem like a large number of handles, but remember two important points:
1. This is at the session level. This means that if a.p calls b.p which creates objects then those will exist for the life of the session: THERE IS NO SCOPE other than SESSION FOR DYNAMIC OBJECTS and they are NEVER automatically reclaimed!
2. If the programs are running as part of a long-running session such as AppServer, EagleIQ server or Webspeed, then you have to consider the cumulative affect over days, weeks or months.
Also note that if it is a temp-table, it could potentially have a much larger memory footprint.
So, what must be done?
It is up to the Progress programmer to clean up each and every dynamic object created using the “delete object” command.
It may be appropriate to create a widget-pool in which to assign your objects so you can just delete the pool and all the objects within will be released as well. In fact, if you create a non-persistent widget-pool, it will be automatically deleted when it goes out of scope. Creating the object into a non-persistent pool will make it behave as if it were scoped at the level that the widget-pool is created: in effect, making it behave as if it were statically defined.
If you don’t use a non-persistent widget-pool, then It is also important to be sure the “clean up” code is executed even when there is an error. For example, the following will bleed memory if an error condition is raised within the blah blah:
procedure doQuery: def var qh as handle. create query qh. /* so some business logic here */ do while true: blah blah end. delete object qh. end.
However, if you create a non-persistent widget pool, then it is automatically deleted when it goes out of scope. So the following will not leak memory even if an error condition happens:
procedure doQuery:
def var qh as handle.
create widget-pool “wp”.
create query qh in widget-pool “wp”.
/* so some business logic here */
do while true:
blah blah
end.
delete object qh.
end.
The widget-pool may be defined at the .p level as well. In this case the pool is deleted when the .p is exited.
Oh, by the way, persistent procedures and memptr’s are two other constructs that have a session level scope. However, they cannot be part of a widget-pool and therefore must be handled individually.
OE Architect Editor Corruption Bug Fix in SP3
An editor corruption issue that causes OpenEdge Architect to crash or stop responding has been fixed in the just released OpenEdge Service Pack 3.
This issue was cropping up frequently enough during heads down coding sessions to cause use of OE Architect to be a hit or miss proposition for some of us.
While more prevelant on multi-core and HT processors single-core processors are not immune.
At Solvepoint we’re long-time Eclipse users so we’re eager to see OpenEdge Architect (based on Eclipse) to be stabilized and advanced.
Reference Kbase ID P124071 and Service Pack 3 Reference ID OE00150179.
Designing Systems That Work
“Design is not cleaning up the mess, or adding ornamentation at the end. It’s a process of thinking, organizing, trying, testing, reworking, creating anew, refining, honing, and more. Successful systems work because they are well conceived, and responsive to user needs, styles, wishes, and habits. They continue to work because they are well structured, and can be easily maintained and enhanced.” -Arthur Fink
Premier Worldwide Progress Event - Exchange, 2008 is at Disneyland Resort Paris, France
Details for Exchange, 2008 (June 8-11 at Disneyland Resort Paris, France) are posted at Progress Exchange, ‘08. Exchange is an annual event sponsored by Progress Software drawing over 1,000 users, and technical and business experts from around the world. Exchange events include several days of general and technical sessions, workshops and hands-on demonstrations and training sessions on the latest Progress technologies.
Need more info on OpenEdge and Progress? Join a Progress User Group.
Progress User Groups (PUGs) can be found all over the world. PUGs enable you to attend local technical presentations on current Progress topics, exchange ideas with other Progress users, and discuss the technology that is impacting today’s businesses. If you live in the Delaware Valley/Greater Philadelphia Area consider joining us for lunch at DVPUG. If you live elsewhere visit PSDN’s list of Progress User Groups to find one nearest you.
What are “Spectrum” and “RDL”?
“Spectrum” and “RDL” are product names that preceded “Progress 4GL”, “OpenEdge” and “ABL”.
“Spectrum” dates back to 1980.
-
Sponsored by Solvepoint
This blog is sponsored by Solvepoint, a leader in Progress performance tuning, database administration, software development, maintenance and integration. More Information