OAM and EBS Breaks after Cloning

We are in the process of integrating E-Business Suite with Oracle Access Manager (OAM). Its setup in DEV and TEST but not in PROD yet.    There was a requirement to reclone DEV and as a result the integration with OAM broke.

We ran all the configuration steps on the EBS side, such as:

• $FND_TOP/bin/txkrun.pl -script=SetSSOReg -registerinstance=yes
• Verifying ASADMIN was active and the correct password.
• Profile options to configure authentation:
• Applications SSO Type
• Applications SSO Login types
• etc...
• Business Events to make sure account creation and modifications synchronizes correctly.
• We skipped all of the tasks to be performed on the OAM side since they were already done.

However, when restarting Access Gate we would see the following error in the logs:

weblogic.application.ModuleException:

        at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:302)

        at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:199)

        at weblogic.application.internal.flow.DeploymentCallbackFlow$1.next(DeploymentCallbackFlow.java:517)

        at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)

        at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:159)

        Truncated. see log file for complete stacktrace

Caused By: weblogic.common.resourcepool.ResourceSystemException:

Could not connect to 'oracle.apps.fnd.ext.jdbc.datasource.AppsDataSource'.

The returned message is: ORA-01017: invalid username/password; logon denied

We triple checked the datasource credentials defined in the Access Gate server but still nothing worked.  It turns one of the steps to deploy Access Gate was to copy the dbc file from $FND_SECURE over to the $MW_HOME/appsutil/accessgate/dev directory.   Since DEV was recloned the APPL_SERVER_ID value had changed.    We re-copied this file, bounced the Access Gate server and everything started working again.

Since we don't have EBS integrated with OAM in PROD yet i'm not sure if we will need to do any additional steps after a clone.   I'm thinking all the steps above will apply except for maybe business events as they will already be configured properly in PROD.   I'll update this post if I have to do anything else.

Web Cache 11g as a Reverse Proxy for E-Business Suite Virtual Tier

We are currently in the process of implementing a virtual tier setup for our E-Business Suite environment.    The reason is because we are planning on exposing some modules to the internet.   Since we are a small shop a separate physical tier isn't necessary.  

A complicating factor tho is that our DEV environment is in house while our TEST and PROD servers are located at a hosting provider.     TEST and PROD will be able to take advantage of an F5 Loadbalancer, while DEV doesn't have one available.

To setup the virtual tier i'm following the steps in note Case History: Implementing a Reverse Proxy Alone in a DMZ Configuration - R12 [ID 726953.1]

The note suggests using Web Cache 10g to simulate a reverse proxy:  Installing and Configuring Web Cache 10g and Oracle E-Business Suite 12 [ID 380486.1]

We don't have any 10g webcache environments, but we use 11g for our Fusion Middleware environment.  I couldn't find any notes on Metalink which described using WC 11g.  Technically tho, there should be no reason why it wouldn't work with 11g, so I decided to give it a try.

NOTE: I am not sure if 11g Web Cache is certified with R12, I don't believe it is.   So you should confirm with Oracle Support before using this in a production environment.   Since this is just a POC and in TEST and PROD I will be using an F5 Load Balancer I decided to give 11g WC a try.

For each under Section 5: Configuration details, I will show the 11g setup.

5.1.1 - Login to the Web Cache Admin page

• http://server:8091/webcacheadmin

5.1.2 - Listen Port

• Add a Listen Port.
• On the left hand side, under Ports click on Listen Ports.   Once the Listen Ports page has loaded click on the Create... button.
• IP Address Field -> ANY
• Port Number Field -> 7777 (or whichever port your EBS Virtual Tier is listening on.)
• Since this is a DEV environment I am not using https so I left Protocol as default.
• Click on Submit.

5.1.3 - Configure Web Cache Site definitions for E-Business Suite.

• On the left side under  Origin Servers, Sites, and Load Balancing click on Site Definitions.
• No changes from the 10g document.   Here is a screenshot of my setup:

• myserver  is your WebCache server.
• If your not using HTTPS, don't enter / for HTTPS Only Prefix, leave it blank or it will not work.

5.1.4 - Configure E-Business Suite as an Origin Server

• On the left hand side click on Origin Servers and Add... on the right hand side once the page loads.  The setup here is almost identical to the 10g document.   The only thing missing is the option to select the protocol.
  
  Here is a screenshot of my setup.  Remember, the Origin Server is the server where your EBS virtual tier is running.  In my case its configured to listen on port 7777.

• On the left hand side click on Site-to-Server Mapping 
• Click on the Add...  button once the page loads.   
• Below is a screenshot of my configuration.  Host Name (in this case server) is your Web Cache server.   Under Origin Servers select the one you created above for your EBS environment.  (Sorry for blanking out some of the server names, i'm paranoid. ;)   This is  a copy of the default settings for Web Cache 11g for our fusion middleware environment.

• The above configuration seems to work fine but its quite different that what your instructed to do in 10g.   In 10g your told to create the following mappings: 
• 1, , 7777, -, Unrestricted, , 80, No
  2, *, 7777, -, Exclude Fragments, , 80, No
  3, *, *, -, Exclude Fragments, , 80, No
• These mappings confused me a little until I looked at the 10g documentation: http://docs.oracle.com/cd/B14099_19/caching.1012/b14046/basics.htm#i1042797
• After seeing this I configured my Site-To-Server Mappings as following:

• The file site-to-site mapping above was the default one.   The next 3 are the ones I added for this EBS configuration.  The second one is the same as the screen shot above.   The next two are the same as that one but hostname is set to * for both, and for the last the port number is a *.   ESI Content Policy is set to Exclude fragments for both.

As you can see, the 11g setup is pretty much identical.  You can follow the rest of Installing and Configuring Web Cache 10g and Oracle E-Business Suite 12 [ID 380486.1] as is.   If your trying this configuration and running into issues with the configuration on the EBS side for the Virtual Tier let me know.  

R12 Cloning: adcrdb.sh missing from file system.

This weekend I had to build an R12 environment.   We use Oracle VM in our development environment so this made the who process much easier.    Before I would have to setup a new server, install the OS, patch it, put in the Oracle pre-req's then clone.   Now I just need to run pre-clone, shutdown the environment, copy it, rename/reconfigure the vm and run post clone.

When I ran adcfgclone.pl dbTier tho I hit the error described in the following note:

On Release 12 adcfgclone is failing with AC-00423, RC-50014 adcrdb.sh missing from file system [ID 549872.1]

The solution is to copy adcrdb.sh from the source system and place it under $ORACLE_HOME/appsutil/template

However, when I executed adcfgclone.pl again I hit the same error.   adcrdb.sh would get deleted from the template directory.

The solution was to run autoconfig, preclone and recopy the db files and software over.

Well, at least I didn't have to build a new server.. Saved me a little time anyways.

If your having cloning issues you may want to take a look at the other issues I encountered as well.  Just click on the EBS Cloning label on the right side.

R12 Cloning: Can't login as apps

This past week I was cloning a DEV environment from PROD.   I noticed that the adcfgclone.pl script appeared to be hanging.  Looking at the logs it was stuck on:


Attempting to connect to database using :

jdbc:oracle:thin:@myserver:1553:DEV,APPS
        DbSid     :DEV
        DbHost    :myserver
        DomainName:mydomain.com
        DbPort    :1553

So the first thing I tried was logging into the database using tns, ex:  sqlplus apps/apps@DEV.   This also hung.   However, other accounts such as dbsnmp, etc were fine.

Using Grid Control I noticed there was a lot of contention in the database:

   

Drilling down into concurrency it was very easy to see that the issue was related to library cache locks.

Looking at the sessions involved I found that they were related to our SOA server and other integrated applications.    Since I am in the process of cloning this database from PROD, the databases are identical, down to the account passwords.   So all of these connection attempts were using the incorrect password.   I reset the passwords to their appropriate DEV values a little later in the cloning process.

I did a quick search on Metalink and this note jumped out at me:

LIBRARY CACHE LOCKS DUE TO INVALID LOGIN ATTEMPTS [ID 1309738.1]

The recommendation is to set the following event:

EVENT="28401 TRACE NAME CONTEXT FOREVER, LEVEL 1"

I set this parameter, bounced the database, reran adcfgclone.pl and it ran with no issues.   

So, just an FYI.  If your cloning an EBS environment and have alot of integrated applications/processes that you may run into this issue until you either reset the account passwords or set this event.

Mac OSX: Disable Java 7

As most of you are probably aware if you are running Java 7 on your desktop there is a bug which will prevent forms from launching:

Steven Chan covers the certifications and the workarounds on his blog.

https://blogs.oracle.com/stevenChan/entry/java_jre_1_7_0


My Mac was updated to Java 7 a short period of time ago and of course I had issues opening forms.    I'm late to the game so i'm sure most people have already hit this issue.  I'm just documenting it here as a personal reference for when I need to back the fix out.

There are two solutions to this problem:

1.  Patch your EBS environment as per Steven's article above.
2.  Disable Java 7 on your Mac.


I choose to do number 2 as I needed to login ASAP and didn't have time to wait for patches to get through QA.

To disable Java 7 and re-enable Java 6 follow the steps on Apple's Support site:

http://support.apple.com/kb/HT5559?viewlocale=en_US


While I was trying* to track down when my system was updated to Java 7, I found the following site from Oracle which provides alot of information for Mac users of Java:

http://www.java.com/en/download/faq/java_mac.xml


* I'm not sure how my Mac was updated to 7.   I'm assuming it was when our tech support upgraded my Mac to Mountain Lion and updated it.