Tags >> VMware
May 13
2013

Subtle Benefits of Virtualization

Posted by Solutions Architects in VMware , VM , vCOps , vCenter , Template Standardization , Policy Compliance , Performance Triage , Memory Savings , Hardware Upgrades , Benefits of Virtualization

By David Klee (@kleegeek)

Earlier this week, a blog post by guest blogger Christian Wickham was posted at VMware.com entitled: “Hidden benefits of virtualisation – reboot time and the impact on server availability and regular operations”. In this blog, Christian discussed a hidden benefit of virtualization – the positive impact to business on the shorter reboot times of servers. This post hit the proverbial nail on the head, but after taking the time to digest it, a few more subtle benefits of virtualization popped into my head. Some of these might be even more important to you than others, but all are greatly beneficial to businesses everywhere.

These benefits include:

  • Memory savings through Transparent Page Sharing (TPS)
  • Quick and simple hardware upgrades
  • Quicker infrastructure performance triage
  • Policy compliance through template standardization

Memory Savings

VMware vSphere has the ability to detect duplicate memory blocks on a host and deduplicate them – lending to reduced physical memory consumption on a host.

Shared_Storage

The amount of memory block deduplication potential grows if you run multiple VMs with the same operating system or programs on the same host. As you can see in the screen shot above, the amount of memory that can be effectively shared between virtual machines on a host can be quite substantial. This savings, in turn, can yield a greater VM consolidation ratio, which saves you more money because you have less hardware to license. Saving money is always good!

Simple Hardware Upgrades

Think about how much time over the last couple of decades you have spent doing server upgrades – not because of the software, but because of the hardware. The operating system and core software was probably just fine, but since the hardware was aging out, you had to put together a long-winded project. Downtime was planned. Late weekend nights were usually required to make it work with the least impact to the business.

Nowadays, thanks to virtualization, this is a complete nonissue. Hardware upgrades are a drag and drop event that can be performed whenever you wish, all with no interruption to service. Just add the new hardware to the virtual cluster and migrate your virtual machines over. Remove the old hardware from the cluster and you’re done!

Quicker Performance Triage

VMware vCenter builds performance statistic histories for you over time. vCenter Operations Management (including the free vCOps Foundation edition) builds a historical performance baseline automatically. If you suspect something in the environment is not healthy, a quick glance at the vCOps dashboard can help you see what is abnormal in the environment.

vCOps
(Image source: http://www.vmware.com/products/datacenter-virtualization/vcenter-operations-manager/overview.html)

Even just the historical views within vCenter can be enough to help you start down the right path of performance troubleshooting and remediation by showing you the performance metrics that matter.

Performacne_Metrics

Policy Compliance

Last, but certainly not least, comes the means to ensure server policy compliance, thanks to the standardization that comes from using virtual machine templates. Pre-load all of your best practices, policies, and organizational requirements into that VM template. At the deploy time, all of these normally time-consuming requirements become part of the deployed based VM. It is a gold standard, and the potential for human error or inconsistencies because of things like hardware differences or a different installer are now completely removed.

Hopefully these subtle benefits can help you as you work towards total virtualization of your datacenter. What less mentioned and more commonly overlooked benefits can you think of? Tweet your answers to @kleegeek.

Apr 26
2013

VMware Virtualization - RDM to VMDK Migration

Posted by Solutions Architects in VMware , vMotion , VMDK , SQL Server , RDM-V , RDM-P , RDM , Migration , Fusion-io , ESXi 5.0

By David Klee (@kleegeek)

A fun customer of ours asked me today if you could convert an RDM directly into a VMDK. They had a SQL Server instance residing on a Fusion-io SSD card through a pass-through Raw Device Map (RDM) on a VMware ESXi 5.0 host. They wanted to migrate the SQL Server to the SSD-based area on their SAN so that the VM could benefit from VMware HA between ESXi hosts in the vSphere cluster rather than pinned to a single compute node.

At first, I did not think this was directly possible. However, we found a VMware knowledge base article (1005241) that says you can do it. The methodology behind it makes perfect sense.

The method behind the migration depends on your current RDM configuration. Two kinds of RDMs exist – RDM-Physical (RDM-P) and RDM-Virtual (RDM-V). As my wizard-like colleague Jim Hannan (@HoBHannan) noted in his recent blog post ‘VMFS vs. RDM’, RDM-V specifies full virtualization interception of the SCSI commands to the storage device, and snapshots can be taken. With RDM-P, the SCSI commands are directly passed-through to the storage. VMware-level operations are very limited, but full SAN tooling exists.

Raw Device Map – Virtual Options

If your storage pointers are configured in the RDM-V manner, your options are better than RDM-P. You have to methods – cold migration and Storage vMotion (if the licensing level allows for it).

First, if you have the option for a period of downtime, the simplest method is to gracefully shut down the VM and perform a cold migration. You don’t even have to migrate between hosts. The migration process will create a new virtual disk and will clone the RDM into the virtual disk.

The original mapped LUN is left alone after the migration completes, and can be removed as necessary to reclaim space.

You can also use a Storage vMotion operation to perform the same migration task.

If you are moving SQL Server or Oracle data from RDM to VMDK, please use the advanced tab in the migration wizard to specify ‘Thick Provisioned Eager Zeroed’ as the virtual disk format. It ensures that the entire space required for the database objects is contiguous on the storage, yielding optimal storage performance.

Raw Device Map – Physical Options

If you have the RDM configured as RDM-P, you cannot migrate the storage using a Storage vMotion. Cold migrations are still fully supported.

The end result is that you can successfully (and simply, to boot!) migrate from RDMs to VMDKs using fully supported methods.

Apr 19
2013

Oracle Benchmarking and Tier 1 Readiness

Posted by Solutions Architects in VMware , Tier 1 , Swingbench , Storage , SQL Server , SLOB , OS Metrics , Oracle RAT , Oracle Orion , Oracle , NMON , Metrics , iperf , IOMeter , ddm , CPU , Benchmarking , Baselines , AWR Reports

By Jim Hannan (@HoBHannan)
I strongly believe one of the most beneficial exercises an IT staff can undergo is benchmarking. There are truly some substantial benefits to benchmarking:

  • Building of baselines
  • Establishing key metrics
  • Communicating the key metrics across the organization
  • Identifying scalability
  • Testing for Tier 1 readiness
  • Validating the infrastructure (network, storage, CPU, software, drivers, and configurations)

I think many organizations agree with this concept, but benchmarking can be resource intensive and inaccurate if done improperly. This tends to deter organizations from doing this kind of work.

Because I am a consultant, I have the benefit of working with many different organizations. I get to see what works and what does not. In fact, I spend quite of bit of time with customers doing benchmarking and Tier 1 readiness testing. In this blog I will discuss the various methods and present what works and what does not.

 

stopwatch

Why benchmark? Benchmarking provides a measurement of your hardware and scalability. Often during benchmarking, customers expose weaknesses or misconfiguration. Additionally, benchmarking establishes baselines for future benchmarking and performance troubleshooting. But most importantly, benchmarking tests the platform for Tier 1 readiness.

I think it is important to clarify a few things before continuing on. What does it mean to baseline? A baseline establishes a performance profile when things are running well. The baseline can record things like TPS (transactions per second), I/O peaks, CPU usage, maximum I/O throughput, and many other performance indicators. The baselines can come from tools like AWR reports, NMON statistics (OS metrics), storage tools, and user response time.

What does Tier 1 readiness mean? Tier 1 readiness is simply a key set of benchmarks designed to validate and record the performance of a platform (hardware and software) in its ability to service the workloads.

There are several different approaches to benchmarking. In the table below we categorized the different approaches and identified the level-of-effort for each category. A more detailed description of each test is below the table.

Test

Category Description

Industry Trends

Load Generation

Category 1 – Validates configurations and hardware. Pass/false results. (Example: Swingbench)

Common, typically short in duration (1 – 2 days).

Tier 1 Readiness

Category 2 – Benchmarking tools (Examples: IOMeter, dd, Oracle Orion, SLOB, iperf)

Common, short execution. (Less than 1 day)

Right-Sizing

Category 3 – Very accurate and low cost. Dependency that existing baselines and metrics exist. (Examples: AWR, perfmon, NMON)

Very accurate, predictable outcome.

Capture and Replay

Category 4 Validation – Recommended for VBCA and first time deployments on new infrastructure. (Example: Oracle RAT)

Highly recommended for first time deployments and VBCA.

Load Generation

Working from the top down, lets look at Load Generation techniques. Load Generation tools like Swingbench and even Spec 2006 (CPU/hardware analysis) have a preset of tests. They may not behave anything like your database or application. They have the benefit of being typically the easiest to setup. They fall short in accuracy of results.  That said, people who are very familiar with Swingbench can accurately measure performance by running tests across many different hardware configurations. The results have a point of reference – meaning that they can compare the previous results to the new results.

Tier 1 Readiness

I should tell you that some of my favorite tools live in this category. These tests tend to be very focused on particular elements like, for example, storage, network, or CPU. I like to tell customers these test expose the tipping points. What I mean by that is, these tests give you the absolute best case for throughput. Another reason that I am a really big proponent of these tests is because they are the most approachable. They do not take a lot of time to setup. They don't need an Oracle installation. In fact, our SQL Server team will use a lot of the same tools.

Right-Sizing

It could be argued that right-sizing does not fit into the benchmarking category, but I would disagree with that argument. It’s a process, and probably the most important of the four. After several edits of this paragraph, I realized something. I was originally going to say that right-sizing is a virtualization only step. That is simply not true. Right sizing is just as important for physical hardware as it is for virtual.

What data do you need for right-sizing? For Oracle, customers are most successful when they have collected OS metrics and AWR reports. The metrics should be collected every day. And metrics should exist as far back as a month ago – or even a year ago.

Why so much data? For your business critical applications, understanding their peaks is important. For example: what does month end look like? Does the application get very busy around Christmas or the beginning of the school year?

Capture and Replay

This is the most accurate of the testing options available. A capture and replay benchmark takes the application code and data and tests it on the new hardware. I have seen all kinds of various sets of tests. Some customers only replay critical or most prevalent SQL – others test all SQL over a defined period.

Summary

I mentioned in the beginning of the blog that I had the luxury of working with many different customers. I get to see what works and, in some cases, what does not work as well. For most customers a combination of the 4 different approaches in the right mix. Understanding when to use the appropriate tool is important. Additionally, the customers that are most successful with benchmarking have adopted their own methodology and processes. This allows the organization to be clear about what the key metrics are and how to run the test consistently each time. This is a deep topic with host of considerations. If you have questions or comments, I can be reached via Twitter @HoBHannan.

Apr 11
2013

VMware Certifications, Q2 2013

Posted by Solutions Architects in VMware Certified Professional , VMware Certifications , VMware , VCP5-DT , VCP5-DCV , VCP4-DCV , VCP-IaaS , VCP-Cloud , VCP , vCloud Director , VCDX5-Cloud , VCAP5-DTD , VCAP5-DTA , VCAP-CID , Datacenter Virtualization , Datacenter Design , Datacenter Administration

By David Klee (@kleegeek)

A few weeks ago I posted a short post on the current SQL Server certification paths from Microsoft. This week I’m following up with current VMware certification paths. The VMware certification paths are rapidly evolving, and VMware’s direction in the enterprise becomes immediately clear when looking through the list.

The original VMware certification started with the VMware Certified Professional back in the day for VMware ESX 1 – way back in 2003.

VMPro

Susan Gudenkauf  is VCP number one and tells her story at a great interview at TrainSignal.com.

Fast forward about ten years, and now four different certification tracks exist from VMware.

  • Cloud
  • Data Center Virtualization
  • End User Computing
  • Cloud Application Platform

While I feel all of these certification tracks are very valuable, I’m a bit less focused on the Cloud Application Platform certifications at this time, as they deal primarily with Spring software development and we do very little application development.  We try to stay ‘below the line’ when it comes to data.

For each of the three tracks that I am focusing on here, multiple certifications exist in a hierarchical progression.

Data Center Virtualization

First, the Data Center virtualization track is the granddaddy of the VMware certifications.

At the heart of the Data Center track is the VMware Certified Professional – Data Center Virtualization (VCP5-DCV – formerly VCP then VCP-DV). This certification is the starting point towards VMware virtualization mastery, and is the cornerstone of a solid VMware infrastructure understanding. This exam is quite challenging, and the candidate should be quite versed in VMware setup, configuration, and troubleshooting.

Pro5

As with other VMware base certifications, obtaining your VCP5-DCV requires that you attend a qualifying VMware-authorized class if you are not upgrading from the VCP4-DCV certification. A list of the qualifying courses can be found here.

Next come the VMware Certified Advanced Professional certifications for Data Center Administration and Data Center Design. Both certifications are great, and I personally feel that obtaining one or both of these certifications demonstrates that a person can properly manage any scale of a VMware infrastructure. I hold the VCAP5-DCD certification, and am looking to obtain the VCAP5-DCA certification by the end of the summer.

pro5dd pro5da

Finally comes the certification that demonstrates absolute mastery of the vSphere stack – VMware Certified Design Expert – Data Center Virtualization (VCDX5-DCV). This is such a high level certification that most people don’t strive to achieve because they simply do not need it. For those that do, they are usually technical architects or consultants who design large and/or complex cloud infrastructures and wish to demonstrate the technical capacity required of their profession. Only a few hundred individuals worldwide have completed the requirements for VCDX.

DE5

I am a glutton for punishment with my nonstop self-education regimen, and hope to achieve VCDX-DCV in the near future. Earning the VCDX-DCV designation is a personal goal that I have had since I learned about the VCP certification many years ago.

Cloud

Next, VMware’s huge push to the cloud is now backed by a number of certifications in the path to product mastery

VCPpath

First, comes the newly released VMware Certified Professional – Cloud (VCP-Cloud). It is a mix of the core vSphere information for traditional virtualization, mixed with a strong dose of subject matter surrounding the VMware vCloud suite of products. I took and passed the VCP-Cloud exam this past December, and it is a tough exam.

procloud

To be eligible to earn the VCP-Cloud designation, you must either already have obtained the VCP5-DCV certification and pass the VMware Certified Professional – Infrastructure as a Service (VCP-IaaS) exam, or take a ‘qualifying’ course from VMware and then pass the VCP-Cloud exam. Appropriate courses include:

These pre-requisite courses can be a barrier to VMware certifications because these classes are not cheap, but my colleagues and I have had very good experiences with these classes. Urge your employers to send you to these classes if you do anything at all with VMware virtualization management.

Additional information on the VCP-Cloud certification, along with learning blueprints and practice questions, can be found here.

Once you have gotten the VCP-Cloud certification under your belt, next in line are three VMware Certified Advanced Professional certifications – Cloud Infrastructure Design (VCAP-CID), Cloud Infrastructure Administration (coming soon), and Cloud Governance (also coming soon). Not much information exists on these certifications except for VCAP-CID because they are so new.

apcid

Finally, the pinnacle of VMware cloud certifications is the VMware Certified Design Expert – Cloud (VCDX5-Cloud). Very few individuals strive to achieve this level of certification, and even fewer make it.

As I grow with my vCloud experience, I am considering working towards the upper certifications in this track.

End User Computing

Last but not least, comes a certification track that VMware is actively building upon – End User Computing. It is centered on the VMware View platform for virtual desktops.

VCPenduser

First comes the VMware Certified Professional – Desktop (VCP5-DT).  Next comes the VMware Certified Advanced Professional – Desktop Administration (VCAP5-DTA – coming soon) and Desktop Design (VCAP5-DTD). Virtual desktops are exploding in adoption, and these certifications are a great way to prove that you are qualified to manage the largest and/or most complex View installations.

Conclusion

In the past, technical certifications were rather polarizing. Quite a few people thought they were irrelevant and unnecessary. HR managers and recruiters thought they were vital for recruiting the ‘right’ staff. Some certifications were (and still are) easy to pass, and do not have much value in the industry. The VMware certifications are far from it. Having passed a number of industry certifications myself, VMware creates some tough exams. The format of the tests is such that I consider them true tests of demonstrating the knowledge required to apply these technological platforms to business challenges. I hold the VMware certifications high, and when I see folks with a VMware certification, I can trust that they know their stuff.

If you work with VMware administration or architecture, I encourage you to explore these certifications, see which ones make sense for you, and go after them!

Mar 25
2013

Thoughts on Oracle’s Nimbula Purchase Announcement

Posted by Dave Welch in Xen , Wim Coekaerts , VMware , TechTarget , Tech Target , Red Hat , Real Application Clusters , RAC on VMware , RAC , Parent Partition , OVM , Oracle Virtual Machine , Oracle on VMware , Oracle E-Business Suite , Oracle Consulting Services , Oracle , OpenStack , Numbula , IBM , IaaS , HA , Dom0 , Beth Pariseau

By Dave Welch (@OraVBCA)

Tech Target’s Beth Pariseau asked for my opinion on Oracle’s recent Nimbula purchase announcement. She then posted an article titled “Oracle's Nimbula IaaS buy sparks open source speculation” on March 14th.

I always appreciate the opportunity to contribute to Beth’s pieces. On this occasion, I opted to give her my input in writing only. As I’ve thought about it, I think the OVM concerns that I shared with Beth are important enough that I’ve chosen to post my entire response to her. I’m posting it as-is, resisting the temptation to expound on how it relates to Oracle’s March 20 announcement of its hardware/cloud sales decline.

March 13, 2012

Beth,

It’s always great to hear from you. Quite frankly, I’m surprised I didn’t hear from you on the occasion of IBM’s announcement last week that they were committing to OpenStack. I’m inclined to believe that’s a much bigger deal than Oracle’s Nimbula announcement.

Oracle is floundering with virtualization. Oracle’s been on its Xen project for a full six years now (assuming it put at least a few months of prep into its OVM OOW 2007 announcement) and still hasn’t produced anything we consider remotely usable.

I’ve been verbal from the start about how VMware needed a technical and financial competitor on the Linux side. When Oracle announced OVM at OOW 2007, I was dearly hoping that Oracle would be that competitor. Over five years later, I’m still waiting for Oracle to step up.

Oracle announced OVM 3.0 at OOW 2011 with a lot of fanfare. Within two days, VMware blasted its partners a 15-point technical competitive analysis based solely on downloadable OVM 3.0 documentation. I thought VMware’s analysis totally missed the mark. I have maintained for years that OVM chose the wrong open source platform in Xen, and that would continue to be the case until Xen got rid of the parent partition, otherwise known as Dom0. The three Xen issues—any one of which was a bigger deal for me than anything on the OVM 3.0/vSphere partner compete sheet—were: 

    Tier 1 workload I/O & network scalability: difficult to accomplish when the same device driver code has to live and function in both the base native Linux install image as well as the parent partition virtual layer.
  • The parent partition is inherently anti-HA as there are lots of required Linux patches that require a full down of the environment yet have nothing to do with virtualization.
  • Security analysts agree that the ultimate way to reduce a security threat at the end of the day is to reduce the size of the attack surface. The installation of the parent partition mini-Linux image on top of the base native hardware image bloats rather than reduces the attack surface.

Oracle’s long-standing Senior VP of OVM/OEL Wim Coekaerts announced in his May 2011 blog post that the parent partition (Dom0) was finally going away as of the 2.6.39+ kernel. My hopes soared. At the time, I said that we were probably still a hardware depreciation lifecycle away from getting any respectable OVM GA adoption on the street. Meanwhile, Red Hat would have to muck with it and release. Then Oracle in turn would muck with it and release.

A year later in May 2012, Wim posted that OVM 3.1 with Dom0 was finally GA. Just a month ago, House of Brick followed Oracle Consulting Services into a shop that was trying to take a scaled workload live within days on Oracle E-Business Suite with RAC on OVM 3.1. OCS had done its own OVM configuration per its best practices for the customer. What we found was anything other than stable or even remotely usable. We retrofitted vSphere 5.1 per the customer’s plea, and they went live with the corrected stack days later. Takeaway: a year after Wim’s May 2011 Dom0 post, I’m regretfully predicting that we’re still at least a hardware depreciation life-cycle away from OVM becoming anything usable under Tier 1 workloads.

So what’s going on at Oracle? Wim Coekaerts won my respect a decade ago with his team’s engineering and support of Oracle Cluster File System. I wonder if this has anything to do with Wim and his team’s capabilities and rather wonder if his team is the victim of resource constraints. To me, it sure looks like another indicator that Oracle’s leadership’s primary objective with OVM may have been trade rag competition. Oracle’s leadership can’t be blind to the accelerating hemorrhage of accounts optimizing their Oracle red stacks on VMware, with the associated crippling reduction on Oracle processor-based licensing revenue.

Given the choices that Oracle appears to have made to-date and its current in-the-mire status with OVM, I think that for Oracle to bag Xen and go with OpenStack would have to be an improvement for two reasons: 

    Anything would be better than current state OVM.
  • Any other corporate entity overseeing design and development of the hypervisor and surrounding tooling would be an improvement.

I wouldn’t be at all surprised if Oracle went there.

Regards,
Dave

Mar 01
2013

Oracle Automated Deployments on VMware Part II

Posted by Solutions Architects in VMware , Virtualization , Oracle on VMware , Oracle , DBCA , Automation

by Jim Hannan (@HoBHannan)

DBCA2

In my previous post I discussed tools to automate deployments for virtualized Oracle workloads. I outlined some of the current industry methods and their tools. In this blog post I will take a deeper look at one of those tools, Oracle’s DBCA (Database Configuration Assistant). I cannot advocate for DBCA being the perfect solution for every company, but I think majority of IT teams will find it very well suited for their deployments. Before jumping into DBCA, we should first quickly revisit the table from the last blog post. The table below outlines the opportunities for automation. Each step has various methods available to it. In this blog we are targeting the final step, automated deployment of the Oracle database after the virtual machine and software has been deployed.

Table 1 - Oracle Automated Deployment and Tools

Steps

Tool(s)

Create a virtual machine

[1] Standard – Create VM through vSphere client of APIs (this step involves the process of allocating memory, CPU and storage)

Boot virtual machine

[1] Boot off media

[2] Network boot and receive boot image from network server

[3] Deploy virtual machine from template (template contains base OS with no additional software)

Automated OS install

[1] Linux kickstart

[2] Spacewalk

[3] VMware template with base image

Scripts for customization

[1] OS scripts to further customize the guest OS. Often includes network settings and deployment of organization scripts for managing Oracle.

Oracle software install

[1] Oracle Universal Install with a response file

[2] Zip of Oracle home with CPU patch already applied. Requires the creation of an Oracle inventory

Oracle Database

[1] Use Oracle DBCA command line

[2] Deploy Oracle database from zip files.

Most DBAs are familiar with DBCA and the GUI wizard. Probably less familiar is DBCA's ability to run silently from the command line. The command line options really lend themselves to automation. You can create a database template or a catalog of templates. In the following example I have created the template SAND-forblog.dbc. This is stored with the other default templates. After creating and selecting the configuration you can than override the setting by using DBCA command line options. An example of overriding a parameter is changing the character set or datafile location. From the command line you can type “dbca –help” to get a full list of options. I have listed some of the key options in the table below. Remember this command line options can override settings defined in the template, very flexible.
Creating the database:

[oracle@oel63-64-11g Desktop]$ dbca -silent -createDatabase -templateName SAND-forblog.dbc -gdbname FORBLOG -sid FORBLOG -responseFile NO_VALUE -sysPassword oracle -systemPassword oracle -storageType ASM -asmSysPassword oracle -diskGroupName DATA

Output of the database creation:

Copying database files

1% complete

3% complete

35% complete

Creating and starting Oracle instance

37% complete

42% complete

47% complete

52% complete

53% complete

56% complete

58% complete

Registering database with Oracle Restart

64% complete

Completing Database Creation

68% complete

71% complete

75% complete

85% complete

96% complete

100% complete

Look at the log file "/opt/oracle/cfgtoollogs/dbca/FORBLOG/FORBLOG.log" for further details.

You can delete a database with the command:

[oracle@oel63-64-11g Desktop]$ dbca -silent -deleteDatabase -sourceDB FORBLOG -sysDBAUserName sys -sysDBAPassword oracle

And output of the delete command:

Output

Connecting to database

4% complete

9% complete

14% complete

19% complete

23% complete

28% complete

47% complete

Updating network configuration files

48% complete

52% complete

Deleting instance and datafiles

76% complete

100% complete

Look at the log file "/opt/oracle/cfgtoollogs/dbca/DEV2.log" for further details.

I thought it would be worth mentioning that customers that have successfully implemented automation typically have matured their processes first by building standards. This reminds me of software development. First, build the flow chart (standard process), and then look for automation opportunities.

If you have any questions about DBCA automation or previous topics I can be reached on twitter @HoBHannan.

Feb 15
2013

A Fast-Track Conceptual Approach for Management and Peer Technical Disciplines of Oracle DBAs

Posted by Dave Welch in VMware , Oracle9i Database Concepts , Oracle VBCA , Oracle on vSphere , Oracle on VMware , Oracle memory , Oracle DBA team building , Oracle DBA manager , Oracle DBA management , Oracle Database Concepts Guide , Oracle Database 12c , Oracle Database 11g , Oracle Database 10g , Oracle Business-Critical Applications , Oracle 2-Day DBA , Oracle , Grow-from-within Oracle DBA

By Dave Welch (@OraVBCA)

I advocate that IT management at all levels and administrators of peer technical disciplines read the Oracle Database Concepts Guide’s Introduction chapter. Yes, that includes the C-level. That read will go a long way in preparing you to participate in database-related discussions at any level. Although this is beneficial if your organization deals with Oracle databases at all, it will prepare you to understand and accept a qualified architect’s suggestion that running Oracle business-critical workloads on the vSphere platform can provide remarkable benefits with minimal if any risks.

As I post this, it’s interesting for me to introspect on how little my mentoring activities have shifted since I authored my first version of this document. This is the centerpiece of guidance I developed and field-tested over seventeen elapsed years of Oracle DBA team building and mentoring.  If you intend to become a DBA, study my selected subset of the Introduction chapter as if you were preparing to take a test on it. In 1995, one of my protégés started with the ingredient goo that evolved into this guidance table. He went on inside a year to become a successful consultant working for Oracle Corporation.

Why am I recommending a document and release version that ostensibly became obsolete years ago when Database release 10g went into mainstream adoption? Certainly not as an excuse to save time by repurposing existing collateral. I just rewrote this approach table from scratch. I find the Database Concepts Guide’s Introduction chapter got cumbered as of release 10g, and the nice self-contained introductory chapter is no longer what it used to be. I’m not saying I could have done a better job authoring the newer Concepts Guides. You can return later to more recent versions’ concepts guides to spot look up newer features as needed. But I believe the benefit of a coherent, consolidated read exceeds the risk of wasting time in older release documentation. It can be difficult to find quality vendor-provided documentation. Oracle’s documentation, at least up through 9i, is a notable exception.

The 9i Concepts Guide’s Introduction chapter is 67 pages. In the approach guidance table that follows, I’m suggesting that you read only about two thirds of that chapter. Your first read needs to be slow and careful to understand the concepts. Then return to review the concepts sufficient to suit your purposes.

The following approach table follows all the way through for DBA candidates’ needs. Non-DBA candidates will know where short of that to stop. DBA candidates should follow this overview with a study of the 2-Day DBA guide first published with Database 10g. I have my own selective approach to that document as well, which is beyond the scope of this post.

Here is your syllabus:

Oracle9i Database Concepts
Release 2 (9.2)
March 2002
http://docs.oracle.com/cd/B10501_01/server.920/a96524.pdf

Heading

PDF Page

Doc Page

Initial Read

After Initial Read

Notes

Database Structure and Space Management Overview

46

1-2

Deep

 

 

Data Access Overview

54

1-10

 

 

 

SQL Overview

54

1-10

Deep

 

 

Objects Overview

56

1-12

 

If job goes there

 

PL/SQL Overview

57

1-13

Deep

 

 

Java Overview

58

1-14

 

Skim

 

XML Overview

59

1-15

 

Skim

 

Transaction Overview

61

1-17

Deep

 

 

Data Integrity Overview

63

1-19

Deep

 

 

SQL*Plus Overview

65

1-21

Deep

 

 

Memory Structures and Processes Overview

65

1-21

Memorize it

 

This is the most critical part of the chapter. DBA candidates: be prepared to reproduce the diagram’s key elements off the top of your head however you want to draw them and explain any aspect of it on demand.

 

Ignore everything here and in this document that has to do with Shared Server processes, the User processes to the left of the Shared Server processes, and the D000 box associated with those.

Application Architecture Overview

76

1-32

Deep

 

 

Distributed Databases

77

1-33

 

 

 

Replication Overview

79

1-35

 

Skim

 

Streams

80

1-36

Skim

 

Streams has pretty much been replaced by Oracle’s acquisition of Golden Gate which probably took place in 2009-2010. But do skim this to understand the high-level solution to the business problem of asynchronously synchronizing very large databases of different versions/platforms to allow cutover with just moments of downtime.

Advanced Queuing Overview

82

1-38

 

Skim

 

Heterogeneous Services Overview

83

1-39

 

Skim

 

Data Concurrency and Consistency Overview

84

1-40

 

 

 

Concurrency

84

1-40

Deep

 

 

Read Consistency

84

1-40

Deep

 

 

Locking Mechanisms

86

1-42

Skim

 

 

Quiesce Database

86

1-42

Skim

 

 

Database Security Overview

87

1-43

Skim

 

 

Database Administration Overview

93

1-49

 

 

 

Enterprise Manager Overview

93

1-49

 

Skim

This stuff has been totally replaced in 10g (April 2004) with Enterprise Manager Grid Control.

Database Backup and Recovery Overview

94

1-50

Memorize it

 

As I approached home growing into being a DBA in 1994, I decided that a DBA who couldn’t confidently restore/recover a production database during a 2 AM emergency, was otherwise of little use to his employer.

Data Warehousing Overview

97

1-53

Skim

 

 

Differences between Data Warehouse and OLTP

98

 

1-54

Deep

 

 

Data Warehouse Architecture

99

1-55

Skim

 

 

Materialized Views

102

1-58

Deep

 

Yes, it is two paragraphs. But they’re important.

OLAP overview

102

1-58

 

Skim

 

Change Data Capture Overview

103

1-59

Skim

 

 

High Availability Overview

104

1-60

Skim

 

 

Transparent Application Failover

105

1-61

Skim

 

 

On-line Reorganization Architecture

106

1-62

 

Skim

 

Data Guard Overview

107

1-63

Deep

 

 

Log Miner Overview

109

1-65

 

Skim

 

Real Application Clusters

109

1-65

Skim

 

 

Real Application Clusters Guard

109

1-65

 

 

Ignore this forever.

Content Management Overview

111

1-67

 

 

Ignore this forever.

Oracle Internet File System Overview

112

1-68

 

 

Ignore this forever.

 

 

Feb 07
2013

Right Sizing Your Virtual Machines

Posted by Solutions Architects in VMware , VM , vCPU , SQL Server , Right Sizing , Performance

by David Klee (@kleegeek)
You have allocated too many resources to your virtual machines, and now your business-critical server performance is suffering! How can this be? That does not make sense!

In this post, I will demonstrate how allocating too many vCPUs to a virtual machine with a low workload actually hinders performance instead of helps it.

Test Setup

To prove this cause and effect, we used a dedicated HP DL580 G7 server with four 10-core Intel Xeon E7-4850 CPUs at 2.0GHz per core and 512GB of RAM. An EMC DMX4 SAN was used for the storage underneath the virtual machines. VMware vSphere 5.0 Update 1 was used as the host hypervisor. One virtual machine was created, and was the only VM running on this host. The VM was configured with two vSockets and four vCPUs per socket, as well as 128GB of vRAM. SQL Server 2008R2 was installed on the VM and configured with all of our best practices.

Dell’s freely available DVDstore (a database benchmarking tool) was used to generate a synthetic workload against our SQL Server testbed.

A 50GB workload was generated and loaded into a new SQL Server database. A DVDstore workload test was performed for one hour. The vCPUs were then changed from 8 to 32 in a 4x8 configuration. The database was restored and the test rerun. The output from each test is in the form of ‘Orders Placed per Minute.’ For each test, the maximum degree of parallelism for the SQL Server instance, or MaxDOP, was adjusted from one to six (a requirement from the project).

Test Results

Threads

MaxDOP

8 vCPUs

32 vCPUs

2

1

19277

13589

2

2

19251

17858

2

3

18841

17453

2

4

15839

15640

2

5

15953

15779

2

6

16263

16055

8

1

76590

63910

8

2

76592

70705

8

3

75441

69335

8

4

57508

61412

8

5

55021

61579

8

6

56859

61151

16

1

152782

135484

16

2

151462

140577

16

3

147618

136376

16

4

86078

112365

16

5

81383

106862

16

6

84634

101230

32

1

298444

274629

32

2

291692

278024

32

3

280824

272659

32

4

108952

147444

32

5

102808

133270

32

6

106140

124293

64

1

487146

542351

64

2

429131

532679

64

3

368718

515461

64

4

113664

153877

64

5

117480

136862

64

6

0

127634

100

1

0

0

100

2

375301

539928

100

3

337446

480744

100

4

0

150850

100

5

0

132887

100

6

0

127498

The results are pretty clear. At a low volume of work, the SQL Server instance performs slower with more vCPUs assigned to the virtual machine. As the volume of work grows, the 32 vCPU VM eventually overtakes the 8 vCPU VM in performance.

Why?

The answer lies in the overhead of vCPU scheduling at the hypervisor layer. All vCPU activity is scheduled into a runnable queue, even if a vCPU is almost idle. You can see this measured indirectly via the vCPU Ready VMware performance counter. As the vCPU count goes up, the hypervisor schedules all activity in this queue. If some vCPUs are almost idle, they still have to get scheduled to run.

However, the priority of the request can decrease if VMware determines that a vCPU is idle, and the overhead of these tasks and queues has a cumulative effect. Now, if all vCPUs are busy, priority is given and these effects become negligible – for this VM. The effects of becoming deprioritized in the runnable CPU queue can potentially be felt by other VMs on the same host, however, so keep this in mind and constantly monitor CPU Ready times of all of your mission-critical virtual machines.

Therefore, baseline and benchmark your workload and determine the actual resource consumption of your workloads. Allocate your VM resources appropriately, and you might just see a noticeable jump in performance!

Note: This blog post was taken from an earlier engagement. For the full case study based on that engegement, please refer to this blog post: "SQL Server Performance on Itanium vs. x86 on VMware: A Case Study".

Feb 01
2013

Oracle Automated Deployments on VMware

Posted by Solutions Architects in VMware , Virtualization , vCPU , VBCA , Tier 1 , Oracle VM , Oracle , Automated VM

By Jim Hannan (@HoBHannan)

As virtualization adoption increases for Tier 1 workloads, many organizations Automated_VM_copyare looking for ways to automate deployments. Oracle VM has touted this as one of its premier features (Users Can Deploy Oracle Real Application Clusters Up to 10 Times Faster with Oracle VM Templates). In my opinion Oracle has done a really good job with this. As a virtualization software provider, Oracle recognized early on the importance of automating the deployment of Oracle workloads. Oracle primarily does this with Oracle VM templates, which I believe have some shortcomings (these shortcomings are the same for VMware templates). Templates for both Oracle and VMware have the same concept. A virtual machine is created and the copied to a template for future deployments. The problem with templates is that are they are out-of-date within the next release of a security patch or organizational standard change. Managing and updating the template itself becomes a burden with dimensioning returns the older the template gets.

Many organizations have recognized the shortcomings of templates and have moved to automated deployments. HoB has had the opportunity to assist and help customers with automation. Each organization we have worked with has had a slightly different approach to the solution, but fundamentally they all used a similar approach.

Common Approach:

  1. Create a virtual machine (vCPU, memory, network)
  2. Boot the virtual machine (typically using a boot image)
  3. Begin automated OS install (most common for Linux is kickstart)
  4. Use scripts to further customize the build
  5. Install Oracle software (process varies)
  6. Create the Oracle database (process varies)

The table below contains the different steps as outlined above with the various tool options. Take note that in the tool column, I have added numbers, e.g. [1] or [2]. This represents different tools for the particular step.

Oracle_Automated_Deployments_and_Tools

In my next blog, I will do a walk through of some of the tools mentioned above.

Jan 24
2013

VMware SMP Fault Tolerance Threatens a Disruption to the Clustered Relational Database Market

Posted by Dave Welch in vSphere , VMworld 2012 , VMware , SMP FT , SMP Fault Tolerance , Oracle RAC , Mendel Rosenblum , John Sloan , Jim Chow , High Availability , HA , David House , Clustered Relational Database

By Dave Welch (@OraVBCA)

The most impressive session I have taken in from the last two VMworlds was 2011 US session BCO2874 “vSphere High Availability 5.0 and SMP Fault Tolerance – Technical Overview and Roadmap”. This session was reprised at VMworld 2012 as session INF-BCO2655 “VMware vSphere Fault Tolerance for Multiprocessor Virtual Machines—Technical Preview and Best Practices”. Don’t miss these sessions’ live demo of a 4 vCPU workload failover.

I’ve been watching Fault Tolerance (FT) eagerly since Mendel Rosenblum did a live demo of FT alpha code at VMworld 2007. Alas, FT’s 2009 GA release has been little more than bait for future capability to House of Brick customers as none of those customers’ Tier-1 workloads fit inside FT’s current single vCPU limit.

I wonder if VMware’s executive leadership has any idea what it is sitting on with SMP Fault Tolerance (SMP FT). I’m wondering if SMP FT could turn out to be the most disruptive technology anyone has seen in years. SMP FT certainly threatens a massive disruption to the clustered relational database market. I make that prediction due to two key SMP FT features that Oracle RAC can’t touch: approximately single-second failover, and no client disconnect.

The VMworld 2012 US session included engineering’s confession that the alpha code still has very substantial performance latency (they use the word “overhead”, which I am replacing with “latency” for clarity). I got the impression that they’re quite worried about the latency, and wondered if they even considered their current performance numbers fatal to a GA release. Here’s a screen shot of their latency numbers (at video minute 35):

Performance_Numbers

The presenters then refer participants to the vSphere 4 FT Performance Whitepaper (“Section 3. Fault Tolerance Performance” beginning on p. 7) for further elaboration on their performance measurement methodology.

I’m betting there are plenty of extremely HA SLA-sensitive workload owners that will be more than happy to tolerate that much latency. For example, look no further than the stock exchanges where extremely distributed transactions must commit all transaction phases within a few seconds to avoid substantial revenue loss let alone regulatory penalties.

A corollary to Moore’s law is Intel executive David House’s prediction that chip performance would double every 18 months. I’ve read clarifications by other parties arguing that chip performance actually doubles every 20 to 24 months. Fine. Let’s call it 24 months. In his 2006 blog post, John Sloan extends that math to a 30-fold performance improvement each decade. (Related to that, my non-scientific mental survey tells me that less than 10% of House of Brick’s performance diagnostic engagements find CPU-constrained workloads.) In light of the fact that we are swimming in a world of CPU performance and it just keeps getting better, how obsessed do I imagine I’d be with SMP FT’s latency if I were the CTO of NYSE or NASDAQ? Not very. I could stand up my SMP FT-protected workload in a hardware refresh and still run circles around any x86 chip vendor's offering of just two hardware depreciation lifecycles ago.

Accordingly, I suggest the SMP FT product and engineering teams gate the product’s dot-zero release based on code stability only. Subsequent to that, I suggest the engineering/product teams prioritize their engineering efforts as follows:

  1. Stability
  2. vCPU horizontal scalability
  3. Latency (and a distant third at that)

During the 2012 VMworld SMP FT session Q&A, Jim Chow answered a question by saying that SMP FT engineers were still considering the subject of the question in their architectural discussions. That answer gave me the impression that SMP FT’s GA release was probably at least a year away at that time. Despite the wait, I believe enterprises would do well to evaluate SMP FT’s business continuity promise now and get project planning underway accordingly.

<< Start < Prev 1 2 3 4 Next > End >>