Dienstleister bittet die Datenbankadministration um die Beseitigung des folgenden (Zitat) Datenbankfehlers: ORA-00054: resource busy and acquire with NOWAIT specified

…der manchmal bei einem SELECT FOR UPDATE NOWAIT kommt.

Ist meine Datenbank kaputt?

Datenbank statt Gartenbank!

(vielen Dank an die zynischen Kollegen der Netzwerkadministration…)

Well, test server is test server: We (me as DBA, and a collegue as the responsible application developer) finally decided to step into the Oracle 11 world: The new system will be 11gR2

After some reading in I had no problems to understand the new separated architecture with its HA daemon (keyword: Grid Infrastructure) and install the whole software and create a database. And, very important, there was no, not one damn small, problem when importing 10gR2 datapump files (from the productive system). I heard some weird stories before, and I was a bit afraid about that. But nothing, all data was pumped in like a charm.

The first thing I changed, for me as DBA one of the most interesting features on those large databases, I really parallelized RMAN for backup. I hope I can reduce the backup time to a useful amount with that. This was one of the biggest discrepance in 10g: You can create very large datafiles, but you can’t backup them in a true parallel manner. I’ll check my logfiles today and hope…

Well, so far so good, I will see what 11g (especially R2 here) brings to me and, more important, the applications. It will take some time to really explore the pros.

For all of you still at 10g and loose a test system by accident (and have a developer that really wants it…): Try it

After configuring the ASM instance with the creation assistant and enabling the instance in /etc/oratab, I did a reboot. Finally I wrote a small shell function db_change() to change ORACLE_HOME, ORACLE_SID and PATH on the fly to operate on the different instances.

Nothing. I wasn’t able to connect with sqlplus. I verified the instance “+ASM” still was running, but I couldn’t connect. I checked my ORACLE_SID and ORACLE_HOME, but I couldn’t connect. I even called a friend of mine (thanks Martin!) to make sure I have no conceptual mistake. Nothing. Sqlplus always said “connected to an idle instance” – as if no ASM instance was running at all (or as if I had a typo in the SID).

Since the +ASM instance started fine during system startup, and started fine using the orarun-initscript from SuSE, something had to smell in my setup. After 30 minutes of checking all corners, I decided to compare my current process environment to the process environment of the started ASM PMON (thanks to Linux’ /proc/ filesystem and some shell vodoo, it’s not a big deal to get a good diff).

After eliminating all the unimportant differences, one was left:

ORACLE_HOME=/opt/oracle/product/10.2/asm_1

ORACLE_HOME=/opt/oracle/product/10.2/asm_1/

See it? Believe it or not, that was it! My shell function set an ORACLE_HOME with a trailing slash, and that drove sqlplus crazy! This may be documented somewhere, I didn’t look, but it’s definitely not something intuitive. Finally, that automatically means that things may fail if you put pathes with trailing slash in your /etc/oratab, too.

O.R.A.C.L.E!!!

Applies to the setup:

• SLES10SP2
• Oracle 10.2.0.4 w/ recommendec patchset 4
  

Have a nice day, and better don’t trust your common sense! Comments welcome, ’cause I’m sure this is a well known behaviour and I’m just too stupid

(and yes, I’ll rewrite my small shell function to actually parse /etc/oratab!)

super noob question , how do i query my oracle database online, do i need to setup bind9 and allow incoming connections so it can be accessed

I just wanted to share that

Have a good day!

• Backup/exports
• Shutdown DB, remove userspace files
• Cleanup files inside ASM
• DBCA to make new one (no scripts available)

You see, nothing unusual. So far so good, everything fine except the last step. The scripts I had from the first installation didn’t run through, so I just wanted to click something together. But DBCA spit out:

DBCA could not startup the ASM instance configured on this node. To proceed with database creation using ASM you need the aSM instance to be up and running. Do you want to recreate the ASM instance on this node?

Er.. what? ASM instance is up and running and some minutes before the old database ran fine with it! So what? Metalink, Google: Environment variables, Listener configuration etc etc etc. Nothing applied. The only thing I really knew 100% was that the ASM setup was rocksolid.

DBCA failed in the following cases in my experiments (I removed and re-added ASM instance):

• it detected offline ASM and failed trying to start it
• it detected online ASM and failed
• it detected no ASM, successfully created one, and failed to add disks etc..

Long story short: Every combination of DBCA and ASM actions failed.

After several hours we got the solution. I was about to give up and manually rewrote another DB installation script to use it for this DB environment. The last hours we turned around nearly every bit of that installation, we nearly did everything except reinstall the software. We even rebooted the server . My collegue was hopeless, too. But we both were fanatic to find the “why” – it was one of these things where workarounds may work, but don’t satisfy the admins.

I hacked a glogin.sql startup file several weeks ago, that displays a nice prompt with instance and username and some timing statistics when you use SQL*Plus. This is something fine and useful for me (and in general).

But we (infact my collegue) found out:

DBCA seems to communicate with the ASM instance using SQL*Plus. My changed output (infact only the prompt and the time statistics) made the SQL*Plus text parsers of DBCA going crazy. Without one word in the right direction in any log of DBCA. DBCA log always said it found the ASM disks and listed them (in the log). So, it was a simple text parsing issue! The developers of DBCA or the JAVA-classes that manage the communication seem to assume that nobody personaziles his SQL*Plus. Only bad software makes such assumptions

Now I know why ORACLE recommends to use different ORACLE_HOMEs. I didn’t try it, but I can imagine that the SQL*Plus of the ORACLE_HOME from the ASM instance is used when different.

The evening in numbers:

• applies to ORACLE software 10.2.0.4, most likely others, too
• happened on SLES10 SP2
• may be documented somewhere, but I didn’t find it on min. 200 pages I read on the Internet
• gave us min. 500 million new grey hairs
• took us 5 hours

I really begin to love ORACLE At least life isn’t boring…