New Comment System@Siebel Unleashed.. Do Try and give me feedback | Next Tip »Home

Middleware – first or last choice?

OkAvarageGoodVery GoodExcellent (5 votes, average: 4.60 out of 5)
Loading ... Loading ...

I was involved in giving a Siebel Demo to some clients few days back and had an interesting conversation regarding Middleware while presenting the demo.

Client had an assumption that having implementing integrations using middleware will result in reduced complexity and increase manageability and doing the same integration with web services will result in increased complexity and will be difficult to manage.

I was really surprised to hear this but I believe this is a general assumption with business users. It seems like they think middleware to be of some kind of component that requires just one time effort to setup and one time investment. Once it is setup then you don’t need any effort and integration in future will need minimal effort to achieve.

This assumption might be true in theory but it is rarely true in real projects. I believe with introduction of middleware we just transfer the effort and complexity on middleware rather than actually reducing it. Let me try to explain this with example:

Let’s assume that we need to send data from Siebel to 3 external applications. Each external application expects a different XML for same information.

Approach 1:
We create 3 web services each generating a different XML and send it to 3 different applications.

Approach 2:
We use middleware and send XML to middleware. Middleware will transform the XML and send to different applications as per the requirement.

In the first look it seems that middleware is indeed simplifying the integration but let’s consider the following points in detail

Failure Points:
With Approach 1 we have just two failure points, Source – Target and it is easy to find out error as Web Services are synchronous but with middleware in picture we have 4 failure points Source – Middleware and Middleware – Target.

Cost:
Web Services are industry standard and uses HTTP, SOAP and XML there is no additional cost except from the effort of developing it but middleware will have additional license fees and other associated costs

Effort Reduction:
On the contrary to general belief we are not actually reducing the effort to develop this integration we are just passing the effort to Middleware. So, now instead of putting in effort in Siebel somebody will be putting effort in Middleware to develop the integration.

Manageability:
Approach 2 definitely makes integration more manageable in Siebel but we are introducing complexity in Middleware, which means instead of Siebel, Middleware will be difficult to manage.

All the above points show that there is nothing that we are gaining by introduction of middleware.
Please note: I am not saying that using middleware will never help. I am just trying to emphasis the fact that usually that is not the case.

If we go back 5-7 years when SOA standard was not well developed and Siebel integration capabilities were still developing, Middleware was first and probably the only choice at that time.

But today with so many integration strategies at our disposal, (EBC, Symbolic URL, VBC, Web Services) I think Middleware should be our last choice for integration.

What do you think???

  • Share/Bookmark

Related Posts


Article by neel

Authors bio is coming up shortly. neel tagged this post with: , Read 387 articles by neel
  • Piyush Gupta
    There are two things common across business:
    1. Business Logic/Complexity.
    2. Interaction different systems.

    Ideal architecture would be to move complex business logic to application/systems like CRM,Mainframe, billing, jave or SAP etc.

    Let middle ware take care of interaction between different systems.

    Ideally middle ware should do transformation of data between systems and error handling part.
  • An interesting discussion.

    One thing you need to remember is that the EAI Object Manager, and the various transports... taken together are actually a small middleware in themselves.

    They perform most of the functions that a 3rd party middleware can (Transformation, Routing, even BPM) but in a very focussed and system centric fashion.

    So, if your entire organisation is focussed around your CRM system then that's going to be great. But if it's not then you start introducing business problems that the current Siebel EAI solution isn't designed to handle whereas a "full" middleware would be. You can try and design your way out of those technically but in all honesty the problems are best solved by a solution other than Siebel, at which point a middleware makes perfect sense.
  • siddu
    hi guys,
    I am able to do my requirement.
    i have just modified the colour code in the jcntrl.csss file for the following tag.

    JBACKGROUND-COLOR-READONLY: #DEDEDE;


    then i could see the different colour to read only fields.
  • siddu
    Hi masters ,
    I have one requirement,
    I have customer requirement, we need to change the background color of the Read Only fields from light gray(#CCCCC) to dark gray color(#99999). i tried modifying in the main.css in the below section


    .qpField
    { color:#000000; font-weight:normal;}


    /*------------------------------*/
    /* Rich Text Component Classes */
    /*------------------------------*/
    .rtcEmbedded {font-family:tahoma; font-size:8pt; border:1px solid #CCCCCCC; width:600px; height:138px; border-bottom : 4px solid #ccccff; }
    .rtcPopup {font-family:tahoma; font-size:8pt; border:1px solid #CCCCCC; width:600px; height:138px; border-bottom : 4px solid #ccccff; }
    .rtcReadOnly {font-family:tahoma; font-size:8pt; border:1px solid #999999; width:600px; height:138px; border-bottom : 4px solid #999999;}
    .rtcTextarea {font-family:tahoma; font-size:8pt; border:1px solid #CCCCCC; width:600px; height:138px; border-bottom : 4px solid #CCCCCC;}

    ----------

    you can see my changes where i replced #CCCCCC to #999999. after that i closed all the siebel realted applications, cleared the coockes from the brower. but it's not replecting in the UI. please suggest me the solution. if any of you can post this as the seperate Question, I tried i unable to post it thats why .

    thanks in Advance.

    I also mailed neel seperately to post it, may he is busy.
  • If you say SOA doesn't need a Middleware, aren't you throwing the concepts of ESB / OSB out of the window ??

    Middlewares have their own strengths, and that is why they are there and continue to evolve. BPM is an entire industry based on these concepts.

    If using Web Services, we mean we need a P2P Integration, lots of things would experience an erosion in value very fast.
  • Mayand Tiwari
    but.. just because we CAN achieve these using webservices doesn't mean that we should!! Webservices should be ideally used in scenarios where we are doing a query in a target system or small insert or update transaction followed by an acknowledgment (or no acknowledgment) which does not again make changes in the source system asynchronously.
    Eg: > query for the country based on IP address. (we already have open source webservices for this)
    > query to get a new TN from another system
    > exposing siebel functionality into some other application, which can query, upsert and delete siebel data.

    in all these cases the transaction has a typical request response structure. and once a particular local application service is exposed as a web service, any application can use these web services and once a proxy service etc are created and is working, these will be as good as using these services as local application services.

    a message oriented middleware should be ideally used in a long or complex transaction which doesn't have a typical request response structure. involving series of updates from an order management system about the status of the order after the order is submitted from siebel to this order management application. here, even if siebel is not sending the request again and again, the target order management system is required to send the status of order asynchronously so that it can also be updated in siebel. there are various other scenarios where it is desirable to use middleware.
  • Mayand Tiwari
    exactly!! as i pointed out the discussion is more of whether it should be message oriented or request response.

    in my second post i was just trying to say that yes... if your system is not very big and complex then you can also do this using web services (even though its not preferred). the scenario which i was talking about is obviously best done with a message oriented middleware, which can retrigger the target system again and again based on some predefined rules if a message has not been delivered. another thing are the fallouts, using webservices it is difficult to send the exact SOAP message again (without touching the source system) if it failed for the first time due to some problems in the target system and we really want to push it through by making some configuration changes in the target system. However if we are using a middleware and a queue, then we can just pickup that message that failed to deliver to the target system, and whenever the target system is ready, we can just resend it.
  • Maurice
    @Mayand Tiwari

    So how would you solve the issue if Siebel is not your only application that uses the functionality in the billing system? What if a second system that doesn't support Web Services? You would need to change the billing system to support this system and support two integration methods for the same functionality. Putting complex business logic in one place makes much more sense.

    Also what happens if your Siebel application is down or unreachable when the billing system sends its response Web Service call, it fails so you'd need to construct some retry mechanism in the billing system. A middleware solution can also add queueing.

    What happens if there is an issue with the messages themselves? Of course both systems should do validation and provide error handling. Using middleware you could centralize error handling and correction (kinda like an error hospital).

    Another bonus of using a middleware solution is that you can standardize your messages. Every time you use some piece of functionality you use the same message "template" and let the middleware convert it into the appropriate format on the other side (Eg. map from UTF-16 to UTF-8, from UTF-8 to EBCDIC, from XML to JSON, inline attachments into MIME, SOAP 1.1 to SOAP 1.2).
  • Mayand Tiwari
    Cont.. the above requirement can also be done using WebServices... for triggering the Billing systrem from Siebel, we can have use a Billing System's web service (outbound webservice for siebel) and have it invoked from siebel when the order is submitted from Siebel and when this request is processed in the billing system, the billing system can send the response (actually result would be a better term here) by invoking a siebel side webservice (i.e. Siebel Inbound webservice). But there will be 2 cycles of request response here.
    so even though the webservice call and the request response is a synchronous process, the overall system will still be asynchronous which means that user will not have to wait for the billing system to send the response or ack immediately and he can continue with his task execution.
  • Mayand Tiwari
    this question is more of "Message oriented middleware" vs "Web Services". If the transport of Webservices is HTTP, then it is synchronous. Which means that it has too follow HTTP rules:
    1) A Server cannot answer (response) without a question (request)
    2) When a question is asked, a connection is open and client waits until the response is delivered or the timeout period expires.

    Now, the question is how do you have asynchronity in such an environment (HTTP)? So, the way the modern still become asynchronous in HTTP environment is using AJAX, ActiveX or Flash in the client side which will enable them to override the typical page by page model, and thus let the user send asynchronous requests. To let the server send asynchronous response, we use COMET or reverse AJAX which can be ping at a regular interval of say 5 seconds, or let the connection open. Broadcast is also used as asynchronous messaging. This is how the websites like meebo, orkut, facebook achieve to have asynchronous model even in the HTTP environment.

    Now, in the environments where we have the option to have transports other than HTTP, we can have a middleware which will have a message queue. This middleware will send the messages in the queue as asynchronous requests or response if required. In large projects both webservices and middleware is used depending upon whether it has a realtime synchronous structure or asynchronous structure.

    Exmaple of using middleware: when you submit the order in siebel, a billing system is trigerred, now depending on the type of the response from this billing system the asset is created in siebel. the response from billing system can take one hour to one whole day, so it is asynchronous and thus these responses are submitted in the message queue in middleware.
  • Maurice
    While point-to-point integration is easy to implement I would say that integration using middleware is much more desirable. Integration using middleware (when implemented properly) allows you to define complex business processes (that may span multiple applications) once and use them across applications and provide a consistent interface.

    Lets take a scenario where you open a bank account and want to have a debit card with it. Most banks I know use separate systems for bank accounts and debit cards. In middleware you would define a process that opens a bank account in system A and (based on parameters specified) creates a request for a debit card in system B. With point-to-point integration you'd need to know the details of both systems and call both systems, possibly using different technologies.

    Now lets say that because of regulations the process needs to be extended with a credit check. With a middleware solution you'd add the credit check to the business process but with point-to-point integration you'd need to know the details of yet another system.

    Also take in consideration that application landscapes change, in the example above system B is replaced by a new version or worse still a different system that requires a different message format (but the overall information used is the same, we're still talking about bank cards). Change the business process in middleware and the applications that use your middleware service will never know that anything has changed.

    Also consider that organizations can (and in my experience will) use other applications besides Siebel (or multiple Siebel instances) that need to use the same functionality. Imagine the costs involved for designing, building, testing and maintaining the same functionality for each application (or Siebel instance).

    Lets give another example. An insurance company sells its products through insurance brokers and banks. Of course it provides applications to those brokers and banks to set up policies but what if a large bank wants to incorporate the process into its Siebel application, do you as an insurance company really want to open up all your applications to this bank and let them integrate their Siebel application using point-to-point (remember there will be more than one back-end system) or do you give them an interface to use. I'd say that you'd want to give them an interface. And where do you build that interface, I'd say in middleware.

    From my own personal experience (actually the two examples are based on a large project I worked on) middleware (implemented properly) is the way to go.
blog comments powered by Disqus

Polls

Do you like the new Comment System and new look to Site?

View Results

Loading ... Loading ...