Openworld 2011– Day 3 Summary

I can’t believe Openworld has come and gone.  The last two days were a blur!  With the appreciation event and traveling home I didn’t get a chance to post a summary.   And of course, once I made it back home I had a ton of family activities, work to catch up on, etc.   So I’m only getting around to going over my notes now.    Since the presentations are available online, I thought I would still talk about some of the sessions I’d recommend downloading.

On Wednesday I attended 3 presentations, I skipped one so I could like up like a fanboi for Larry’s keynote.   On Sunday night I managed to pick absolutely the worst seat in the hall.    (The camera people were directly in front of me, which blocked my view of the stage and the pillars on each side blocked the screen showing the speakers!)  So I wanted to make sure I sat in a good section.

The first presentation I attended was Oracle E-Business Suite Technology Vision, Roadmap, and Customer Successes by Lisa Parekh.  Unfortunately tho, I hadn’t intended on going there.   I was in a rush, they scanned my badge, it showed green and I went on in.     It was a good session tho and provided a lot of information on their priorities, such as ease of use, security and performance.   There is a lot of information in this presentation and its available for download here: https://oracleus.wingateweb.com/published/oracleus2011/sessions/17248/S17248_1609600.pdf

My next session was Best Practices for Maintaining and Troubleshooting Oracle Fusion Applications.  I quickly discovered tho that Fusion Applications != Fusion Middleware.   Luckily tho, Lisa (from the last presentation) mentioned an EBS Patching session.  So I left and went over to that one.

Oracle E-Business Suite Directions: Slashing Downtimes with Online Patching, presented by Kevin Hudson was very informative.  This was a very detailed session getting down to how patching works underneath.  In a nutshell patching will take place while the environment remains online and a quick bounce after for the cutover.

Behind the scenes there will now be two APPL_TOP’s.   One will be the called the Run Edition and the other the Patch edition.   As the name says, the patch edition will be used for patching and the run edition is currently used by the application.  After applying a patch you cutover to the patch edition, which now becomes the run edition.

One thing I liked was that after applying a patch you don’t have to do the cutover right away.  You can wait until scheduled downtime.   So, you could apply the patch during business hours (if it doesn’t affect performance much) and do a quick bounce off hours. 

Some points raised by questions from attendees :
- In the future it may be possible to roll back patches.
- At any point until you cutover you can abort the patch.
- Shared APPL_TOP still supported but now you’ll be sharing a dual filesystem.
- You can still merge patches.

If you’re an EBS DBA then you should read this presentation as it really digs down into how online patching works behind the scenes:
https://oracleus.wingateweb.com/published/oracleus2011/sessions/17255/17255_Cho139369.pdf

Next was Larry’s keynote.  It was probably the most entertaining one he had in quite a few years.  The last time I remember it being like this was many years ago bashing Microsoft.   This time Salesforce.com was the unlucky recipient.  My favorite quote:  “Sticker than a roach motel!” I left towards the end of the keynote because it was starting to overlap the last session of the day.




The last session I attended on Thursday was Oracle Enterprise Manager Deployment Best Practices.  Best practices such as having multiple OMSs so that you don’t have a single point of failure either due to an outage or change.  A common mistake is to distribute OMSs in geographically separate locations under the assumption that it will improve performance.  Its recommended to keep your OMSs and repository close since performance is better and agents can tolerate the latency.

If you are in the planning phase of a new Grid Control implementation plan on an incremental rollout, “big bang” deployments are harder to deliver.  Monitoring the EM environment is important such as the status of DBMS_SCHEDULER, housekeeping tasks, incoming loader backlog notifications, etc.

One feature talked about, which has been around for awhile,  was out of band notifications.  When setup, this feature allows an agent to monitor the OMS and send an email notification if it goes down.   The metalink note on how to set it up can be found here: 
Setting Up Out of Bound (OOB) Notification for 'OMS and Repository' Target in Grid Console [ID 429257.1]


The session also covered some new features of 12c.

Before heading out to the appreciation event concert I stopped by the hotel for a jacket and a quick video chat with the family.   I thought a jacket on top of the sweater I had on all day was enough. You think I would have known since this wasn’t my first trip to Treasure Island!   Anyways, thankfully they had patio heaters setup everywhere so after watching Sting perform I met up with some friends for some food and watch them make fools of themselves trying to win some stuff toys.

Openworld 2011 - Day 2

Wow, the conference is flying past fast.  Its already the morning of day 3.  The appreciation event is tonight, hopefully it won't rain!   Tomorrow is the last day, yikes!

I didn't sleep well Sunday night (remember my neighbours?), so I decided to sleep in Tuesday morning if I could.   So unfortunately I missed the keynote in the morning.  I will definitely catch the replay later.

My first session Extreme Database Administration: New Features for Expert DBAs talked about some of the new features coming up on Oracle Enterprise Manager 12c.   It covered some of the same material as a session I attended Monday but if you don't diversify your schedule your bound to get some overlap.

The session talked about Real-Time ADM which allows for real-time analysis of hung or slow database systems.   It connects to the environment via a lightweight connection that doesn't acquire any additional latches or locks and by passes the SQL layer.  It detects resource constraints/limits, hangs, memory issues, deadlocks, etc...

Comparative performance analysis is another feature which will allow you to run a full ADDM analysis over two AWR snapshots.    Used to help answer the question, performance was good yesterday but terrible today.

Ash Analytics is an advanced ASH report which looks similar to the top activity screen but you are not limited to selecting certain time frames but can expand the analysis by dragging the graph to any point in time.

Next up the presenter talked about change management features including difficulties in obtaining accurate test data.   Application Data Model (ADM) is a new feature to help address those issues.  It scans application schemas to model the relationships between tables and columns.  Enables data masking and subsetting operations to be application aware.  Data subsetting can be done by percentages of tables or by columns (regions, year, etc) at runtime.   One example showed subsetting 100GB of data to 20GB took 12 minutes using the export method. Secure Database Testing is another new feature which integrates real time application testing with data masking.

Database Upgrade Automation looks like an interesting feature as well.  Grid Control will detect that a new version is available and suggest the best upgrade path.  It will analyze the database for potential upgrade issues such as space, version, invalid objects, etc.   You can mass deploy binaries to all targets and create out of place copies, then switch the instances to new installations.    You can switch back easily if issues are encountered.

Exadata Management allows for comprehensive tools to manage all of your Exa servers.   I didn't take many notes here since I won't be fortunate enough to use this hardware anytime soon.   Lastly Database as a Cloud Service was discussed.     This seems like a really good feature for large DBA shops who provision and support alot of database environments.    Easy provisioning and deployment of prepackaged, preconfigured databases with metering and charge back capabilities.

I missed the next session due to a call from my manager, so explored the exhibition halls.   Maybe i'm getting older but this is the first year I have not sat in on any presentations.    I find you sit there for 5-15 minutes to listen about a product i'll never use to get a trinket that my kids will never play with. ;)   I end up walking around with a bunch of extra stuff weighing me down all day and then risk having my luggage overweight on the return flight.  Its happened many times before!   Don't worry tho, my kids will still get a gift, one they will hopefully actually use tho. ;)  

Afterwards I attended Steven Chan's Oracle E-Business Suite Technology Certification Primer and Roadmap presentation.   This one is a must for me each year.  As the abstract says, "Is your Oracle E-Business Suite technology stack up-to-date? Are you taking advantage of all the latest options and capabilities? This Oracle Development session summarizes the latest certifications and roadmap for the Oracle E-Business Suite technology stack...." This is a centralized place to make sure you on supported versions, the desupport timelines, certifications with other products and features.    If you didn't attend be sure to keep an eye on Steven Chan's blog as i'm pretty sure he will post it there at some point.

I did a last minute switch up and went to Oracle Recovery Manager (Oracle RMAN) and Oracle Data Guard: Seven Cool Tips from Oracle . If we take advantage of any features coming up it may be data guard. So I thought this would be a good presentation to take in.  The "Tips" were ok, 3 on RMAN, 3 on Dataguard and one from the Customer Testimonial.    They talked about using Guaranteed Restore Points with no IO impact and the tip was that you didn't need to explicitly enable flashback database.

"Super" quick restore by pointing the system to an image copy of a datafile in the FRA in the event of a failure.  I'm not sure how practical that one is.. I don't know anyone that uses image backups.  Most people take advantage of RMAN because of the way it skips un-used blocks, compression, etc.

One of the data guard tips for 11.2 is flushing redo.  Automatically shipping redo logs over to the standby. Previously this had to be done manually because there was a potential for dataloss even in maximum availability unless it was done manually.

The customer testimonial was pretty good.   Enterprise is a 16 billion dollar car rental company and I was surprised to learn that they don't use RAC.   Lisa Reinheimer  described their environment and how they are using dataguard.  I always like listening to how other companies architect their environments.  It gives me ideas and lets me know if i'm on the right track (if i'm already using the features).   If your interested in seeing their setup you should download the presentation.

Finally I attended Best Practices for Oracle E-Business Suite Performance Tuning . I can't even begin to tell you how much material was covered in this presentation.  This presentation was conducted by members of the EBS performance team.    You really need to download this presentation if you support an EBS environment.   There was too much content to even dream of capturing it all to post a summary.   Its basically a centralized presentation listing out tons of tuning tips, metalink notes discussing best practices and performance patches.   Enhancements to new FND_TRACE in 12.1.2 , Note: 1362660.1 for example.   Recommended performance patches for EBS can be found in note 2440401.   Great presentation.   I'll be referring back to this one quite a bit.

That was my day.   Afterwards I went to supper with some old colleagues to an Italian Restaurant in North Beach.    A nice walk down to the pier to burn off supper and a trolly ride back to the hotel.  

Openworld 2011 - Day 1

The day started off with a keynote featuring Joe Tucci from EMC, Mark Hurd and Thomas Kurian.  I won't say much about the keynote, i'm sure more than enough analysis has been already done.    All I will say tho, similar to last evening, the focus was too heavy on hardware and the "Exa's".     Oracle has sold a thousand of these servers and plan to sell 3000 over the next year.   While impressive, it still means that the majority of customers currently don't have Exa servers in house.   So, in my humble opinion, the keynotes should try to appeal to a wider audience.

First up I had a hand's on lab, Using Oracle Application Management Suite for Oracle E-Business Suite.  We just purchased this pack and plan on implementing it over the next few months.   The lab had a few issues, some my own fault, some with the environment.   It was only an hour long and given the amount of material in the guide i'm not sure if the goal was for us to complete it all.   Really to cover the amount of material they had listed you would need at least half a day.  One good thing tho is that they offered to email us the documentation (or copy it if you had a USB key) so we could continue/read up on it on our own time.

Next I attended Who Ate My DB Time? Advanced Techniques for Database Performance Analysis.  It was a good presentation.  It covered some new features of OEM R12 and how they work behind the scenes.  It finished with a customer testimonial from Boeing.

Automating Patching with Oracle Application Management Suite for Oracle E-Business Suite, covered features from both the application management pack and change management pack.   Monitoring, lifecycle management, configuration management, etc.    Being able to compare two instances from profile options to context values.  AMP for 12c is still under development, so if you use this pack you won't be upgrading to the new OEM version right away.   The customer testimonial by Prithwi Barman at the end was very interesting.   I know that before we can even use AMP we need to apply the latest ATG patches plus various others.   But after hearing him talk it looks like its going to take a fair bit more effort than I thought.   I didn't add it up, but it took them quite a few man months to implement it from DEV to PROD and test features such custom patches.  

Prithwi also talked about the process they follow migrating patches and displayed a chart summarizing the process flow.   Alot of time goes into building these flows so its always nice to see how other companies do this.  It helps give you a kickstart designing your own and/or justifing some of the choices you make.

For example, in the attached slide you can see that the developers package the custom patches, pass them along through an approval layer before reaching the DBA's.   This makes perfect sense to me but I know will make a difficult sell.   Being able to say X company is doing it this way for Y and Z reasons is going to help alot.

The next session I attended was Maximize Efficiency of Oracle E-Business Suite 11i and 12 Through Diagnostics.   Not something that really falls under my responsibility other than to make sure its setup.   So my main goal was just to get some more information on how its used by developers, some issues they may encounter etc.    Quite frequently when they open an SR oracle asks them to run various diagnostics reports and upload them.    

It was very beneficial, the presenter Karen Brownfield, talked about some security issues such as the CRM HTML administration role having super user access to diagnostics.   How tests in 11i timeout after 5 minutes, RDA's timeout in R12.   Features such as save sets and groups.  If you have a set of tests you plan to execute at the same time frequently you can put them in a save set.   Groups are similar but available to all users instead of just the one who created it.

Last but not least was Oracle E-Business Suite Directions: Deployment and System Administration.    Features in the next version of R12 (12.2) include a dual file system which allows online patching to occur.   Basically two appltops are installed, the application runs out of one of them (runtime appltop) while patches are applied to the other (patch appltop).   Once patching is complete a brief downtime occurs during the switch over.  Downtime is predictable, minutes not hours/days.

The benefits of switching the application tier over to Weblogic were also explained.   WLS has strong support for high availability and failover.   Consolidated administration, configuration and deployment.   A very indepth troubleshooting framework.   Comprehensive, low-overhead application and diagnostics.

A brief stop by the OTN party and then supper with an old co-worker.  We haven't seen each other in years even tho we live in the same city.   We were actually on the same flight and ran into each other after deplaning in San Francisco.

Openworld 2011 - Day 0

It didn't seem like long ago that I registered for #oow11 but the first day (or is it day 0?) has come and gone. I guess because of the timezone difference I was up about an hour before my alarm was set to go off, which is pretty strange for me. After a quick shower I was out the door by 7:30am and on my way to register. I expected long lines but I had my kit by 8am.

Day 0, like previous years, was a day for Oracle user groups to hold meetings. My day started with Thomas Canty's presentation titled Sysadmin 101. Thomas talked about RAID, the pros/cons of each level, NFS performance, direct NFS and various command line tips. This is a great presentation for someone new to supporting Oracle on Linux that would like to learn a few linux tips.

I take a fair bit of time leading up to the conference to go through Oracle's schedule builder and arrange my day. However, its still easy to miss good sessions. So I always keep my eye out for long lines outside a session and the TV screens showing the titles. As I was making my way to a session I happened to notice Mark Rittman's session on OBIEE 11g Architecture and Internals.

We are currently using OBIEE 10g and there has been talk of upgrading to 11g. So I went in and grabbed a seat. We currently use Oracle Fusion Middleware so alot of the concepts such as weblogic domains, system/non-system components, etc. So the session was a perfect fit to get me up to speed on OBIEE11g. I'm really glad I spotted that one as I walked by.

Next was the E-Business (EBS) Applications Technology SIG . I was a little late and walked in on an overivew of WebLogic. Afterwards Steven Chan gave an abbreviated overview of his Certification Primer and Roadmap session that he's presenting on Tuesday. I actually just realized it was abbreviated and not the full presentation while looking at his website (http://blogs.oracle.com/stevenChan/). So I just modified my schedule to include the full session which is at 1:15 on Tuesday.

I have used RMAN since 9i and its one of those features which I feel you always have to keep up on. Whats more important than being able to recover your database in the event of a failure? So I attended Kamran Agayev's session on RMAN 11g new features. One thing I liked about his presentation was the demo's and real examples/benchmarks. Alot of presentations either have slides of commands and their output or they try and do the demo live which almost always runs into some sort of issue. Kamran recorded his demo's and played them back. I found that for me personally it helped understand the topics better.

Kamran talked about new features such as compression types (ZLIB), Virtual Private Catalogs, changes to commands such as VALIDATE, etc. If you use RMAN and need to get up to speed on 11g new features then i'd check out this presentation.

Finally I attended the Sysadmin SIG. The session was dominated by a discussion on Oracle's Critical patch updates. Its nice to know that i'm not alone in finding that the documentation around this is very confusing. I'm not sure when it changed, maybe it was with release 12 but its definitely not as straight forward. I think part of the reason is that they are trying to reuse documentation. So you'll click on one note and it will reference another and you are left to fit all the pieces together. The Fusion Middleware charts list various products such as infrastructure home, middle tier home, OC4J, forms, jdeveloper, etc, The first time I tried to apply a CPU patch for R12 I had to open an SR as I had no idea which patches to apply. The state of the documentation has left me with an uneasy feeling that i'm not capturing all the patches I need to be applying.

Maybe my memory is playing tricks on me, but i'm pretty sure when I supported an 11i environment all the patches were self contained in one note and it was very easy to find the patches you needed to apply.

On top of the confusing documentation other issues such as determining if your environment is affected by the vulnerabilities, risk analysis and testing requirements also came up.

One of the benefits of coming to Openworld, in my opinion, is being able to provide feed back to the right people at Oracle and having an open discussion with them. Hopefully they will go back with our comments and improve the CPU process.

Lastly the day concluded with the opening keynote. I have to say that over the past few years I haven't looked forward to them as much as I used to. Ever since Oracle acquired Sun, Larry's keynote seems to be dominated by hardware. Not that its a bad thing because I think its a good direction for Oracle. But honestly, my organization is too small to take advantage or be able to afford any of the Exa's (Exadata, Exalogic, Exalytics). Oracle mentioned they sold 1000 in the past year and in line to sell 3000 over the next. Good numbers but that also means the vast majority of Oracle's customers aren't using it or plan to in the near future. So, to me, it doesn't make sense for it to dominate the entire keynote.

After the keynote I stopped by for a quick bite to eat and went to a few stores to take a few pictures of purses for my wife so she could do some virtual shopping.

Well, it seems that the headboard of my hotel neighbours room has stopped hitting off our shared wall, so its time to catch some sleep. I have to get up again in 3.5 hours! Please ignore any typo's or grammatical errors as my eye's aren't quite focusing. ;)

Openworld 2011

UPDATE: Opps, I just updated the title, no idea why I said 2012!

-=-=-=-

Just completed registration for my Blogger pass. I am really looking forward to the conference this year as the sessions look really good.

If your in San Francisco for the conference and would like to meet up drop me an email. My address is on the right side of this page.