Openworld 2009 - Thursday Summary

I think its safe to say that getting up and having to be at Moscone West by 9am was pretty difficult after the big party last night.   I managed to drag myself out of bed and was only a few minutes late for the session.

Sessions I attended:

Oracle Advanced Compression in Oracle Database 11g 

This was a really good session. Daniel Morgan and Kevin Closson were the presenters. Kevin pretty entertaining and did a great job presenting the material.  The material for the presentation changed slightly since the abstract was posted due to the release of 11gR2 in September.  The new title was now Hybrid Columnar Compression.  You can download these presentation from Morgans Library.

The session started with why you may need to compress data, a brief history of it since its inception in and how it works underneath the hood.  There are many new features in 11.1 such as compressed tablespaces, segment compression and advanced options such as data guard network compression, data pump compression, improvements to rman compression, oltp table compression and securefile compression and deduplication.  11.2 brought some changes to segment compression allowing you to compress for OLTP, Query or Archive data.

The presentation continues to talk about Hybrid Columnar Compression, which is only available on the Exadata V2, and how it compresses the data.  It also has two new features, Archive and Warehouse compression.

Warehouse compression provides you with 10x average storage savings, 10x reduction in scan IO and is optimized for speed.

Archive compression provides you with 15x average storage savings but customers have reported upto 70x on some data.  This is primarily meant for cold or historical data and is optimized for space.
Slide 6 of Kevin’s presentation contains a table which shows how many CPU cores are required to fully saturate the IO channels as compared to a generic system.  What surprised me was that with Exadata is that 112 cpu cores could be used for data decompression and filtration!

Upgrading to Oracle Fusion Middleware 11g

We are in the process of upgrading from 10g to Fusion 11g so this session was perfect timing.  I haven’t reviewed the upgrade process yet so this session provided a good overview of the process.   They talked about how they have worked hard to reduce the manual steps required and tried to automate the process as much as possible.   The presentation covered the required upgrade version starting points and then they broke down the upgrade into 4 pieces:  WebLogic, Forms/Reports, Portal and Identity Management.

If your upgrading, definitely check out this presentation.

Oracle E-Business Suite Install and Cloning Best Practices

Like other apps dbas, i’ve performed my share of cloning so I wasn’t expecting to learn anything new from this presentation.  The Rapid Install and Cloning processes were discussed and it included some great slides on what happens during the cloning process when you pass certain parameters to

For example, when running on the database tier it covered the different actions taken behind the scenes the different parameters dbTechStack, dbconfig and database.  I kinda new what each one did but now I have a much better understanding. 

They also talked about new virtualization features that may be coming down the pipe.  Basically utilizing Oracle VM templates to quickly deploy new environments.  Looks interesting
After the session I talked to the presenter and asked him some questions about automating cloning.  Once I finish my scripts i’ll post about it.

New Oracle Recovery Manager (RMAN) 11g Features in Release 1 and Release 2

Michael Messina from Rolta TUSC is another entertaining presenter.  Throughout the session he asked questions and if you had the answer he was looking for (or not in some cases :) he gave you a tshirt… Thats a pretty good way to get audience interaction at Openworld.

Michael covered both the benefits and limitations of rman.  The limitations being backups not including password files, failed backups may leave files around and online redo log backups not supported.

11gR1 features discussed included data recovery advisor, rman proactive health check, better catalog management and duplicate database improvements.

11gR2 features discussed included automatic block repair, set newname clause enhancements, tablespace point in time recovery(TSPITR) and duplicate without connecting to the target database. 

Before I left I picked up two books at the Oracle bookstore, 11g New Features for Administrators Exam Guide and Oracle Automatic Storage Management: Under-the-Hood & Practical Deployment Guide.  Some light reading for the long plane ride home.


Openworld 2009 – Wednesday Summary

Today was off to another rocky start.  Unfortunately I received some bad news from home and spent part of the morning trying to make travel arrangements.   I was hoping to attend an unconference session by Paul Vallee and Alex Gorbachev from The Pythian Group about the pressures behind support a 24x7 environment.

While I was waiting for my next session I stopped by the exhibits and went to the fun zone to play some guitar hero and basketball to clear my mind.  I lost track of time and almost missed my next session so I hurried off to find it. 

Sessions I attended:

Meeting Name:Implementing an Advanced Architecture for Oracle E-Business Suite

Excellent session!  I would say the best one I attended at the conference. Elke Phelps, co-author of the Applications DBA Field Guide (which by the way I always carry), covered all of the HA features particular to the E-Business Suite and injected personal experiences, tips from working in a large environment which upto 65,000 concurrent users (correct me if i’m wrong).

I have implemented alot of these features but I received the most benefit from the section on external clients, which discussed methods to secure internet/external access to the E-Business Suite.     I haven’t done this year but I may have to in the next few months for a project I am currently working on.

The reason tho why this session was so good was the audience interaction at the end.  Once the floor was opened for questions quite a few members of the audience participated.  It even spilled out into the hallway after.    For me, this is what Openworld is about and I find not enough sessions have this level of discussion.  Most people scatter to leave as soon as the presentation is over, if not before.

As well, I finally introduced myself to Steven Chan, who’s blog I follow quite a bit.

Quick Tips for Database Performance Tuning

I really didn’t stay too long at this session… The presenters spent alot of time showing the SQL Monitoring and Tuning features of Enterprise Manager.   I was sitting pretty close to the back and the projection was pretty small.. I heard on twitter that people were lining up to see Larry’s keynote so I decided to head over there and try to get a good seat.

On the way over to the keynote area they had the hallway shutdown between Moscone South and North.  I am not sure why they do this but i’m guessing its because of the early access for press/bloggers/gold club members.

I showed the security guard my blogger pass and he let me through the door which i’m sure a few people behind me weren’t too happy about.  Its the first time I took advantage of the perks during the conference.

Larry Ellison Keynote

The Infosys keynote started first and was probably the driest one I have seen and I was afraid the guy next to me was going to fall out of his chair or get whiplash from dozing.  I can’t really say I listened to the full keynote but it seemed to be about IT driving innovation.

IMG_1921 What can you say about Larry?  He’s a very charismatic speaker and it wouldn’t be an OpenWorld for me without going to see his keynote.  He talked about Linux and how OEL and VM Server uptake has been better than they expected.

The audience laughed when he mentioned that IBM was contesting the performance stats of the Exadata 2 saying it doesn’t run 16x faster but only 6x faster. “They may be right.”, he continued with a chuckle.

He continued with a review of Exadata V1 which is the fastest machine for data warehousing, being 10-50x faster than other DW systems.  Customer stories included queries which took 24 hours were now taking less than 30 minutes, performance was 10x-72x better and a query taking 30 minutes was now only taking 1 minute.

Exadata V2 is the worlds fastest machine for OLTP with extreme performance for random IO and 3x faster than V1 for data warehousing performance.

A few surprise guests were here today.  Roger Daltrey and no other than the Terminator himself, Governor Arnold Schwarzenegger!  I have to say, i’ve been a big fan of his movies and I was really impressed with his speech.  I’m sure its up on youTube by now. 

IMG_1946 After Mr. Schwarzenegger left the stage Larry continued and once again challenged IBM to enter the 10 million dollar contest. The next topic was about their Unified Support System in which My Oracle Support will be integrated into Enterprise Manager.  Customers will be encouraged to upload their configurations so that information can be processed and scanned for potential problems.

The last topic was Fusion.  Larry re-iterated his commitment to the unlimited application support program and even tho EBS, Siebel, PSFT are all based on outdated technology, Oracle will continue to support them.  Until customers are ready to upgrade to the next generation of middleware which is based on open standards to ensure interoperability.

The keynote ran late, so I missed the last session of the day.  By the time I managed to get out of the keynote area is was well past the start time and it would have taken another 10 minutes to get to the room.   I headed back to the hotel to get ready for the appreciation event.

FireTrucks I had hoped to be on the first bus out so I could get a good seat at the concert but just as I was about to leave the hotel I heard a ton of fire trucks.   They were stopped in the intersection by my hotel and I could tell immediately that this wasn’t a false alarm as the air was hazy with smoke.     A couple of blocks in the area were shutoff or restricted by police so it caused alot of issues for the busses.  We finally managed to get a bus and headed out to treasure island.

Shortly after getting some food and beverages the concert started.   Aerosmith were simply amazing and it was a great concert.  Once it was finished we headed over to the carnival games and I tried to win my kids some stuffed animals.   At this point the amount of beer I had consumed had a detrimental  affect but I managed to win one toy.

IMG_1990However, having two children I couldn’t simply leave with just one stuffed animal.  I tried a few more games and came close to winning but in the end I was toyless.   I struck up a conversation with a fellow attendee, who was carrying quite a few stuffed toys.  He seemed like a good person to get some tips from.  A short time later he found me and gave me one of his stuffed animals, which thankfully ended my fruitless attempts at trying to win on my own.

By this time it was around 12:30 and having my first session at 9am the next day I took the bus back to the hotel.


Openworld 2009 – Tuesday Summary

Everyone awoke Tuesday to a pretty heavy rainstorm…  The news reported winds strong enough to tip over motorcycles and there were a few reports of trees falling on cars.  It seemed everyone at the conference was trying to get a cab, so after awhile I decided to try and hunt down an umbrella.  I managed to find one at a store close to the hotel and headed towards the Moscone Center.  

Openworld broadcasts the keynotes to various rooms around the conference center, so I decided to go to one of those instead of the keynote hall.  It was pretty empty so I took off my shoes and hoped my feet would dry out before the keynote was over and I had to put them back on.  I missed quite a bit of it, so I would rather wait until I had a chance to watch it via Oracle On Demand before I talk about it.

Sessions I attended during the day:

Making Oracle E-Business Suite Highly Available: What’s the Path? 

This presentation was ok but I would recommend Elke Phelpes session: Implementing an Advanced Architecture for Oracle E-Business Suite for a better overview of HA features for E-Business Suite.  
Some good points were made tho about defining high availability and the importance of an SLA (Service Level Agreement).

Hidden Gems of Oracle Data Pump   

I was hoping for more from this session.. When a title says ‘hidden gems’ i’m expecting to hear something about a feature that isn’t well known or used but the developers think has alot of potential.  I remember a session from many years ago about flashback technology.  At the time people were only using it for certain data related recovery scenarios.  I sat in on a presentation where they described using it for refreshing development environments.    I really like it when I find a presentation that uses a feature in unexpected ways.

Back to this one… It did give a good overview of the new features including Compression and Encryption.  Unfortunately you have to purchase the ASO and compression features in order to take advantage of it.    One item of note is that these features do not work over a network link.   The level of compression is comparable to the unix compress utility but not as good as gzip.

There are multiple ways to encrypt the data.  By specifying a password, transparently by using a wallet to provide the encryption key and a combination of both.

They spent alot of time talking about the Metadata Differ, which is a new feature of the Oracle Data Pump metadata API used to compare 2 objects, show their differences and generate ALTERS (if possible) to modify one object to be identical to the other.  Unfortunately this requires the OEM Change Management Pack license.  If your interested in how this feature works beneath the scenes then take a look at this presentation.

Measure, Interpret, and Analyze Oracle I/O Performance Data    

This session started with a video clip about the company the presenter worked for.  I don’t mind a shameless plug but it was a bit of overkill especially since 90% of the video had nothing to do with the company but technology highlights from the past 30 years.

The line up for this session was insane but luckily it was for people on standby.   There were a number of real life I/O performance examples, useful queries and some excel functions to help interpret the data.
In the ASM presentation I went to yesterday, which was given by the ASM team members from Oracle they mentioned that double stripping (which means that you have been provided LUNS which have been striped over multiple disks and then you use ASM to stripe data across the LUNs) increases performance.  This presenter refers to this as  Plaid Stripe and shows a configuration which could reduce IO performance and another where it is done correctly.  He also mentions that RAID 5 isn’t as bad as it used to be and that it can be used for everything but OLTP systems.  I’m not entirely sure I agree with this statement.  I have systems which have RAID 5 and anytime there is intensive IO performance suffers greatly.  Personally, if I had enough disk to use RAID 10, I would do it.

The top 11 New Features of Oracle Database 11g Release 2     

Even though Tom Kytes presentations are mainly focused towards developers I always try to make it to one.  He’s a great presenter and always has alot of great information.  You can download the presentation from askTom website. 

He discussed many (well, 11 or was that 10 ;) features of 11g but I have to say I enjoyed the execute on a directory, file waiters, flash cache and edition based redefinition the best.  For a much better description than I can do, download the presentation.

Blogger Meetup

After the day was finished I dropped by the blogger meetup.  I was hesitant because I didn’t really know anyone there but I surprised myself by staying almost towards the end.  I introduced myself to a couple of other bloggers and had some interesting discussions on technology, blogging and the conference.

I have to say, that after meeting some of these guys and talking to them I am going to try to blog harder about E-Business Suite. If I look at the content I have posted some of it, I think, is pretty good and seems to help a few people. (Based on the feedback i’ve received.)   Other pieces of content really aren’t that great and its solutions you can easily find on Metalink.

There are a few bloggers, we all know who they are, who post some really great content and I hope someday I can look at my blog and feel the same way.


Openworld 2009 – Monday Summary

Monday started with a keynote by Safra Catz and Charles Phillips.  The main point they drove home was that they have been focusing very hard on integrating the various products they have acquired and engineering them to work together.  The many acquisitions have been so that Oracle has all the parts to offer their customers a complete solution.   They are slavishly devoted to open standards, so that if you wanted, you could buy software from multiple vendors. But they hope that over time many of those pieces will be from Oracle.   Addressing the worries of what will happen to Sun, Mysql,etc, they will do what they have always done, make the products better.

I changed around my schedule for Monday at the last minute.   I had registered for the RAC hands on lab but I was afraid it would be similar to the Weblogic lab I attended yesterday.   A friend of mine was at the Active Data Guard lab and he mentioned that it was following steps in a workbook.   I can do these things at home so I thought my time would be better spent attending sessions.

I skipped my first session because it was at the Hilton and my second was back at Moscone.  With only 15 minutes between them it would have been tight to make it back and I really wanted to attend the second one.   The exhibition center opened at 10:30 so I spent 45 minutes or so in there exploring.

Real-Life Tuning and Monitoring of Your Oracle Weblogic Server and Oracle Database

This presentation by Doron Lev-Ari was pretty good.  It started with an review of how Oracle handles sql statements and transitioned to how Weblogic manages them.   He offered some tips on using prepared statements, caching within the weblogic server, performance and troubleshooting tips.

Based on the description of this session I was hoping for more tuning from a system administration standpoint but it was mainly focused at developers.

Introducing Oracle Identity Management 11g: Key New Features and Product Directions

I’ve only been using OID for the past 6 months, so i’m not very familiar with it yet.  We primarily use it for authentication so this session provided a good overview of the product in general.  It showed me that I have alot left to learn about this particular product.

Oracle E-Business Suite Technology Road Map and Vision: Release 12.1 and Beyond

There was alot of information in this presentation and if you are an EBS dba I would highly recommend reviewing the presentation.   The presenters covered the various management packs which provide you with centralized management, pro-active monitoring and automated cloning.  You can also scramble sensitive data during cloning with the Application Management Pack, which you cannot do manually.

End to end tracing so that you can trace a session from the midTier down to the database tier and back.  I haven’t looked into this yet but I can attest to the difficulty in doing this without tools, although over time I figured out how to do it.

Oracle Database 11g: Performance Innovations 

We have just started a project in which we will be using the 11g database, so I thought this session would provide a good overview of some of the performance enhancements and I wasn’t wrong.  Features discussed included SQL Results Cache, In-memory Parallel Execution, Database Smart Flash, OCI Consistent Client Cache, Times Ten In Memory Database Cache Grid, Active Dataguard Query Offload and storing all types of files within the database.

Slash Storage Costs with Oracle Automatic Storage Management 

I’ll admit that up until now I have been very hesitant to use ASM outside of a RAC environment.  The reason is that up until recently I have worked for big companies with dedicated storage teams and large SANS.  It made sense to rely on their expertise to manage storage and work with them to tune it for database performance.

Early this year I joined a small company, one which I wear multiple hats, including system administration.  Now I am responsible for everything from bare metal to the applications that reside on them.  After seeing this presentation I think its time for another look at this product.

The key features of ASM were discussed such as automatic I/O load balancing, automatic mirroring and automatic re-balancing when storage changes.  New features such as the Dynamic Volume manager which allows you to create ACFS and 3rd party file systems such as ext3 on top of volumes.   ACFS snapshots which are used to aid in recoverability.

Another nice feature is Intelligent Data Placement which puts frequently accessed data on the outside of a disk platter.  One thing to note is that ASM doesn’t know if it sees a physical disk or LUN and bases the fast/slow parts of the disk on sector numbers.  You can still see a performance benefit tho by grouping hot data together.

In the future they are planning on implementing ACFS Replication for disaster recovery, similar to SRDF.

IMG_1808After a busy day of sessions I skipped the OTN party and went to supper with some colleagues.  Our first stop was the Crab House on Pier 39, followed by a Ghirardelli Hot Fudge Brownie Sunday an a trolly ride back to our hotels.  

After a 17hr day I was pretty drained and feel asleep pretty quickly.


Openworld 2009 - Day 1 Summary

With a reported 37,000 attendees this year one assumption you could make would be that registration would be a long process.  I registered Saturday afternoon and there wasn’t even a line up at Moscone West.  As you ride the long escalators you get a birds eye view of the registration and over my multiple trips today the lineups were at a minimum.   A number of twitter updates have also mentioned how seamless registration was.

My first session of the day was OUG: Demystifying Oracle Real Application Clusters Workload Management with Alex Gorbachev of The Pythian Group.  Its pretty easy to determine he knows his stuff and has really taken the time to understand how everything works beneath the surface.   I briefly supported a RAC environment but most my my experience with it is experimenting in a VM environment.  This session provided an excellent overview of how workload is distributed in a RAC environment.

Another session I attended was Oracle Database 11g Best Practices for Using Partitioning in HA Environments and VLDBs by Ami Aharonovich.  If you missed this presentation, download the materials and demo source.   It was an excellent overview of the new features and effective demos which solidified the concepts.

At the OAUG Database SIG Meeting there was mention of a utility called Trace File Event Timeline which analyzes a trace file and creates a graphical event timeline. There are a number of blog posts which provide more information about the utility, why it was created and how the script itself works.

The next session I visited was the OAUG E-Business Suite Applications Technology SIG.   Steven Chan, Senior Director in the Oracle Applications Technology Integration group at Oracle, posted a summary of his presentation (and the slides) on his blog.  Using his words, “summarizes our latest techstack certifications, desupport notices, support policy updates, and other important topics from the last year in a single, cohesive presentation”.    Very useful to have all of this information consolidated into a single presentation.   If your an E-Business Suite admin, I highly recommend you review it.

My last session of the day was a hands-on lab, Oracle Weblogic Suite 11g for System Administrators.  I have to say I was disappointed by this session…  I followed the steps listed in the guide but being new to Weblogic I really didn’t have any idea of what or why I was doing them.

Finally the day ended with a keynote by Scott McNealy of Sun and Larry Ellision.  Scott started the evening with an entertaining IMG_1700 top 10 list developers out of control, which included the gas mask bra, windows and the Java ring.  On a more serious note he also talked about the top 10 innovations from Sun including NFS, Solaris, Java, ZFS and their Open Source contributions.
IMG_1720Each time a company is purchased, its competitors launch an aggressive marketing campaign trying to exploit the air of uncertainty around the merger.  In this case, what will happen to Sparc, Solaris, Java and MySql?     In short, Oracle promises to spend more money on these products than Sun did.

Before he left the stage there was one new product announcement, the Sun Storage F5100 Flash Array which as its name states is a 2 TB array of flash storage.  The benefits compared to disk were discussed such as performance and energy savings.   As Scott said at one point of the presentation, “I think Larry is going to enjoy playing with his new toy.”

Larry took the stage and the first item on his agenda was to reinforce their commitment to SUN Customers.  They promise to spend more money developing SPARC, Solaris and MySQL than Sun did. 

To improve the performance of Oracle products on Sun hardware and to have twice as many hardware specialists selling and servicing Sparc/Solaris systems.

Larry transitioned from his commitment, to how together Sun and Oracle will outperform anything on the market today, primarily focusing on IBM.   In addition to the performance gains of 25% more throughput and 16X better responsive time, he also talked about how much greener their product was.  IBM’s offering requires 6X more power 8X more floor space and on top of that isn’t even fault tolerant.   He finished with a challenge for anyone to prove that their database application doesn’t run twice as fast on Sun hardware as it does IBM, with the winner receiving 10M$.


Agent Configuration Assistant Failed

Today I hit the following error while trying to perform a standalone Grid Control Agent install:


There could be a number of reasons as to why the agent configuration assistant would fail while you are trying to install the Grid Control Agent.   The commands which failed are listed in the $AGENT_HOME/cfgtoollogs/configToolsFailedCommands file.

[oracle@myserver agent10g] cat cfgtoollogs/configToolFailedCommands
# Copyright © 1999, 2009, Oracle. All rights reserved.

If you search metalink with “oracle.sysman.emcp.agent.AgentPlugIn”  you will get a number of hits but the 2 most useful ones I have looked at are:

Troubleshooting the error oracle.sysman.emcp.agent.AgentPlugIn raised by the Agent Configuration Assistant (AgentCA)
Doc ID:

Troubleshooting the 'oracle.sysman.emcp.agent.AgentPlugIn has failed' error
Doc ID:

In my particular case, the agent failed because I entered an incorrect agent password during the install.   Unfortunately there is no way to re-enter the password within OUI at this point so login to your server and  follow the steps below:
  1. cd $AGENT_HOME/bin
  2. ./emctl stop agent
  3. ./emctl unsecure agent
  4. ./emctl secure agent 
    • At this point you will be prompted for the agent registration password.  If you input the incorrect password it will fail.   Run this again until you enter the correct password.

    1. ./emctl start agent
    Now you should be able to login to your Grid Control console and see this target.


    T-5 Days

    Until I depart for OpenWorld!   The last time I was there was 2006 and i’m looking forward to it again this year.   I guess it would be appropriate for me, given i’ll be posting from the conference, to state that I will be attending under the Blogger program.  I submitted an application over the summer and was pleasantly surprised to be accepted.   I don’t know what requirements are that your blog has to fulfill so I wasn’t sure how it would play out.

    This year I will be attending a couple of the Hands-on Labs.  On Monday I will be at the Oracle Real Application Clusters 11g Release 2, Part I/II labs and on Friday the Oracle Active Data Guard, Part I/II labs.   The rest of the sessions are a mix between E-Business Suite, Fusion Middleware and some database sessions.

    Of course on Sunday I will be attending the SIG meetings.  OAUG Database SIG, Fusion Middleware SIG and the E-Business Applications Technology SIG..  In the past I have found they have provided alot of useful information.

    I don’t normally attend many of the keynotes, especially if there are some interesting sessions at the same time.   I don’t spend a tremendous amount of time in the tradeshow area either but somehow I always manage to get a fair bit of swag.

    Outside of the conference I can’t wait to visit the Cheesecake Factory in Union Square.   Walking through China town and Fisherman’s Wharf is always a must.  I’ve been to Alcatraz before so I probably won’t go this time.  I’ll also try to check out a few of the places listed in Part 1 and Part 2  of things to do around the Moscone Center.  There is also a blogger meetup scheduled for Tuesday but i’m not 100% sure i’ll be able to attend yet.

    The rest of this week will be spent finishing some tasks, knowledge transfer to a colleague and finalizing my OpenWorld schedule.