Wednesday, April 02, 2008

11g and DBControl

After hearing and reading about so many of the wonderful new toys with 11g, I decided to give some of them a whirl; I tend to learn better when I can break it myself.

Unfortunately, I ended up breaking something else even before I could begin. When installing the software with the basic default options (starter database, DBConsole), emca blew chunks when trying to start dbconsole. This is not the kind of headache I wanted, but I am hoping to use this as a learning opportunity. Somehow. Hopefully I will be able to provide a resolution when this is over with.

For starters, the trace and emConfig.log are a bit hard to read. I did find some key occurences of "OC4J Configuration issue", so I turned to Metalink and google. Metalink was particularly unhelpful. The closest match I could find was Note 549079.1, which is good in and of itself, but 1) it says 10g, not 11g, 2) none of the 3 cases in step 5 applied (I tried all of them). I am still slogging through Google hits, but so far the only solutions I am finding are for 10g, and they do not seem to apply to 11g.

From emConfig.log
CONFIG: OC4J Configuration issue. /u01/app/oracle/product/11.1.0/oc4j/j2ee/OC4J_DBConsole_server1_TEMP not found.

Apr 2, 2008 2:12:05 PM oracle.sysman.emcp.util.PlatformInterface executeCommand
WARNING: Error executing /u01/app/oracle/product/11.1.0/bin/emctl start dbconsole
Apr 2, 2008 2:12:05 PM oracle.sysman.emcp.EMConfig perform
SEVERE: Error starting Database Control
Refer to the log file at /u01/app/oracle/cfgtoollogs/dbca/TEMP/emConfig.log for more details.
Apr 2, 2008 2:12:05 PM oracle.sysman.emcp.EMConfig perform
CONFIG: Stack Trace:
oracle.sysman.emcp.exception.EMConfigException: Error starting Database Control
at oracle.sysman.emcp.EMDBPostConfig.performConfiguration(
at oracle.sysman.emcp.EMDBPostConfig.invoke(
at oracle.sysman.emcp.EMDBPostConfig.invoke(
at oracle.sysman.emcp.EMConfig.perform(

Now, why would a required file not be found? Keep in mind this is coming straight out of the OUI, so I have this expectation that all required files would be automatically created.

I manually created the missing directories (recursive copy of existing directories) and the emca -repos create command was able to do a lot more. Now I get this really strange message:
CONFIG: Unable to determine local host vide /u01/app/oracle/product/11.1.0/server1_TEMP/sysman/config/ : No such file or directory

What is a "host vide"? Nothing on metalink or Google for that one.

UPDATE Fri Apr 4 13:56:56 CDT 2008
Resolved. See comments.


Dan Norris said...

For better or worse, it's events like yours that remind me why I'm still not terribly fond of exclusively relying on the tools to do the job. However, I do admit that DBConsole 11g is a lot nicer and more useful than any previous versions or flavors. So, I'll probably use it more when it's available, but I'm the type that always has to know what's under the hood--this post reminds me why I do that :). Good luck--I haven't seen these issues.

Charles Schultz said...

Amen to that. However, that inherently establishes a disconnect for those trying to learn the new features. All the books, manuals and Adobe slide shows show EM as the tool to "do" whatever is being talked about, and if you cannot utilize EM on your own, it is easy to associate the problem with the new feature, although they are not directly related at all.

It would also be nice if the code were instrumented such that the reason the program does not work becomes obvious when you read the log file. As it stands, the log file is scratching the surface by displaying symptoms. I love it when the logfile has a line that references itself as a source of more information.

Oliver said...

I think you have some troubles with name resolution of your host.
You need to have an entry for your host inside the /etc/host file and be sure to delete all ipv6 entries. When nslookup works fine for name and ip you can try again with the emca

Charles Schultz said...

That is an interesting take, since Oracle Support is suspecting something similar. What confuses me is that they want us to create a virtual fully qualified domain name (FQDN), but one that starts with a different letter. Is there something special about certain letters of a FQDN?

We are on hold as we wait for the sysadmin to make the necessary changes.

FYI - the current names are all fully resolvable; to wit, the host is actually one node of a 2-node RAC and has been running 10g RAC with no problems for about 6 months.

Charles Schultz said...

Turns out that my first guess was dead on, I just missed a crucial directory. I had to copy both the directory under $OH/oc4j/j2ee and $OH. Obviously, you have to create the emca repository first and let it bomb, but after doing that, you can use emctl to start the dbconsole.

A bug is being filed now.

jjap said...

The problem you described here is exactly what I am running into. I just do not understand the resolution. Can you explain what you mean by..."I had to copy both the directory under $OH/oc4j/j2ee and $OH. "?
Copy what to where?
Thanks so much!

Charles Schultz said...

Ironically, Oracle Support is still working this case; it is not yet fully resolved. However, we think we have found a bug, and way to compensate for it. And one of the workaround is this business of copying to a new directory.

Here are the directories I copied:

I copied these directories with short names:

So if your fully qualified hostname is, the shortname would be metalink.

Oracle Support thinks that our environment is somewhat to blame. We had/have 10g RAC installed in seperate Oracle Homes, but we are being told be DEV that 11g is actually picking these up somehow, causing havoc with the 11g Console. We are not exactly sure why, yet.

FYI - the bug we have identified so far is 6996516

jjap said...

Thanks for the info. I did not have [$OH]/oc4j/j2ee/OC4J_DBConsole_[fully_qualified_hostname]_[$SID], but I did have [$OH]/oc4j/j2ee/OC4J_DBConsole, so I copied that to [$OH]/oc4j/j2ee/OC4J_DBConsole_[short_hostname]_[$SID].
Now, emctl status dbconsole
gives me EM Configuration issue. [$OH]/[short_hostname]_[$SID] not found. But there isn't a directory [$OH]/[fully_qualified_hostname]_[$SID]to copy from. Any ideas?