Memory Leaks in Siebel

This post has been contributed by  Timur Vafin. This post deals about What, Why and How of Memory Leaks in Siebel and he has also shared a set of SQL and Perl Script and a tool that could help you uncover possible causes of memory leaks due to not nullifying the objects after use.


I think that many Siebel Administrators know about problem with Siebel process memory leakage when Siebel OS process memory grows constantly and process can not fit to memory boundary (be it LDR_CNTRL environment variable or 2GB process memory limit on 32 bit platform or server resource boundary).

As a result of this problem Object Manager process crashes and generates core dump or FDR file. Siebel Server (siebsvc process) restarts this process again but the session of the users becomes invalid and all their unsaved data is lost. They are required to re-login in application. If this is repeated then UNIX OS start swapping process memory to swap device or file and Siebel performance become too slow and the only way to solve this is to restart Siebel Server.

What causes Memory Leaks?
Configuration: Recursive self joins and complex or wrong expression in calculated fields can result in extensive memory usage and subsequently crashing Siebel Object Manager process (Although this is a rare scenario).

Scripting: One main cause of Memory leaks in my experience is badly written scripts that omit Garbage Collection (GC), let me explain what I mean.

When we assign something to a variable in eScript Siebel reserves some memory to hold the reference and values of that variable. The amount of memory that is reserved for it depends on the type the variable. The only way to get this memory back is to nullify the object used in the script at the end of the script. Assigning a variable as null notifies Siebel GC to destroy the variable release the memory at the next GC run.

For example

var bc = this.GetBusComp(); //siebel reserves some memory for variable bc.
bc = null; //now siebel can release the memory assigned to this variable.

If I don’t nullify an object and the script is executed too frequently Siebel will keep using more and more memory till the point it runs out of it causing the crash. So, if you have lot badly written scripts without appropriate GC implemented you could run in Memory Leaks. The real embarrassing thing for me is that experienced people know about this problem but sometimes they trying to forget about it.

How to find cause of Memory Leaks?

FDR Files:

When Siebel Object Manager process crashes, at that time system will generate FDR file in Siebel Server bin directory. You can analyze these FDR files and find out the last steps before crash.

Log Files:

Log files contain very important information about the crash. Monitor OS process that consumes too much memory and then find the component name from Siebel Log using PID. You can search the log files for lines like

“Starting <Component Name> with PID <PID>”.

You can find the Log files at the following location

UNIX: [Siebel Root]/siebsrvr/enterprises/[Enterprise Name]/[Server Name]/log

Windows: [Siebel Root]\siebsrvr\log

Siebel SARM:

To find out what objects are used frequently by Object Manager you can enable 2nd level SARM logs for some time and analyze it. I will omit the details on this but you can find the most useful information by typing command line “sarmquery –tips” and in Siebel Documentation:

Siebel Performance Tuning Guide > Monitoring Siebel Application Performance with Siebel ARM

Siebel Performance Tuning Guide > Analyzing Siebel ARM Data

How to prevent Siebel Crash from Memory Leaks?

Component Parameter:

For multithreaded components Oracle has introduced “Memory-Based Multithread Component Recycling” feature to prevent Siebel Server from crashing due to memory leaks. This feature can be enabled by setting MemoryBasedRecycle parameter as true.

Once this feature is enabled (by setting parameter MemoryBasedRecycle=true) for a component, Siebel Server stops opening new sessions when Object Manager process reach memory limit defined by parameter MemoryLimit (in mega bytes, default 1500) for this process and starts another process to handle requests. It waits until all tasks are completed on problem process and shuts it down. Although this approach doesn’t fixes the root cause of the memory leak.

More information about this feature you can find in Siebel Documentation:Siebel System Administration Guide > Configuring Siebel Servers > Advanced Configuration Tasks > Configuring Memory-Based Server Component Recycling.

Garbage Collection: To overcome the problem of Memory leaks caused because of objects not nullified in script I have come up with set of SQL’s and Perl Script that can easily help you to:

  1. Analyzes eScript variables that has been opened but not closed (“null” are not assigned to variable) in the script
  2. Searches for objects that has been opened using the following functions such as GetAssocBusComp, GetPicklistBusComp, this.BusObject, ActiveBusObject, GetBusObject, GetService, NewPropertySet but not nullified
  3. Skipped eScript comments in code
  4. Takes under consideration multiple variables assignments like : var1 = var2 = var3 = <Something>

Furthermore, is you used Oracle Database for Siebel, you can use my tool that I call “Siebel Open Variable Analyzer”. I’ve combined Oracle Instant Client 10g with SQL*Plus, Strawberry perl 5.12, and my perl script.

There are steps to use it:

  1. Extract SiebelOVAnalyzer.rar to appropriate directory
  2. Edit connection information for Oracle Database Server (Siebel Database):
    1. tnsnames.ora file in root directory – change TNS
    2. run.bat file in root directory – change LOGIN environment variable: Siebel Database User that has sse_role \ tblo_role, password, TNS
  3. Run “run.bat”, it will produce “out.csv” file in root directory
  4. Open “out.csv” in Excel and analyze data for your purposes

So now you have a tool to produce list of variables that you should review in your code. You can download the SiebelOVAAnalyzer tool from the link below

http://uploadbox.com/files/82588e5ff8/

Password: siebelunleashed.com

Here is the link to download the SQL and Perl script in case if you don’t want to download the utility

SiebelOVAPerlScript

SQL to download eScript from Oracle DB (Siebel Repository)

I hope my article will help. Below are some good articles about Memory Leaks on Oracle My Support

  • How To Troubleshoot Memory Leaks on UNIX [ID 477522.1]
  • How To Troubleshoot Memory Leaks on Microsoft Windows [ID 477521.1]
  • How To Analyze the FDR Output in Siebel Versions 7.7.x, 7.8.x and 8. [ID 473939.1]

From Neel:

I have tried the utility and it works. Although I had to tweak a few setting to make it work. I will posting an article soon with details of the utility and my experience of using it. Please rate the article and provide your comments.

This Post has been viewed : 31,892 Times


14 Responses to Memory Leaks in Siebel

  1. Grt post . Equally useful for both experienced and newbie Siebel admins . Learn these concepts and you will go long way in becoming good siebel admin

    Neel, Timur Vafin – Thanks a lot for sharing it with the forum.

  2. Gentlemen,

    I exchange some e-mails with Timur and decided to create an open source project for this tool. It is available at Google Code in the URL http://code.google.com/p/siebel-code-analyzer/.

    Code is licensed by GNU GPL. It’s use only Perl code and should works in anywhere where Perl DBI works (my tests were applied using an ODBC connection).

    There is no download available, but the code can be reached from the Subversion repository and since is just two files, anyone should be able to download and setup the tool.

    Regards

  3. This is very important for me.
    Neel, I need to ask you something about the MemoryBasedRecycle

    Where can i contact you?

Leave a Reply

Contribute