webMethods Broker Failure - Publishing delayed while outbound store is draining

Problem

Publishing delayed while outbound store is draining. Service: wm.server.publish:publish

Solution

First to check if Broker is available and IS is connected to Broker.

If Broker is up and IS is connected to Broker, the situation normally happens when IS was shutdown during the process of publishing documents to Broker, and still had lots of uncommitted documents remaining in the local store or broker document queue before shutdown. It could happen:

1. After IS was brought up next time, the IS is trying to publish the remaining uncommitted documents in the local store to Broker which caused the traffic congestion;
2. The Broker considers the disconnected IS instance as a disconnected client. The disconnected client still carries out uncommitted documents in the broker document queue, which utilizes the Broker run time data. Once the total utilization of Broker run time data goes beyond of 10%, the Broker performance is going to be impacted.

If IS, Broker, MWS portal are up, the solution is to find out the disconnected clients and remove them to free up Broker utilization.

Worst scenario, the solution is to remove the IS DocumentStore and XAStore before starting up IS.

IS Start up Error - SERVER IS RUNNING IN SAFE MODE

Problem

If the Integration Server detects a problem with the master password or outbound
passwords at startup, it will place you in safe mode, which is a special mode from which
you can diagnose and correct problems.
When the Integration Server is in safe mode, it displays the Integration Server
Administrator, but the Integration Server is not connected to any external resources.
When you are placed into safe mode because of problems with the master password or
outbound passwords, you will see the following message in the upper left corner of the
Server Statistics screen of the Integration Server Administrator:

SERVER IS RUNNING IN SAFE MODE. Master password sanity check failed -- invalid
master password provided.

Solution

When there is a problem with these passwords, you can correct the problem by restoring
the passwords or resetting them. The method you choose depends on the problem with
the passwords. There are a number of reasons the Integration Server will automatically
go into safe mode.

• Passwords are Corrupted or Out of Sync
• You Entered the Wrong Master Password by Mistake
• Platform Locale Has Changed

By default, the master password file is <<IntegrationServer instance’s home directory>>/config/empw.dat and the outbound password file
is <<IntegrationServer instance’s home directory>>/config/txnPassStore.dat. The file <<IntegrationServer instance’s home directory>>/config/configPassman.cnf contains the location reference to master password file (master.password.field.fileName) and outbound password file (outbound.password.store.field.fileName).


If you do not have backups of these two files, by following the below steps, the problem can be resolved.

1. Shutdown IS
2. Remove files /config/txnPassStore.dat and /config/empw.dat
3. Reset the JDBC Connection Pools for webMethods IS by:

                 Enter Directory <<IntegrationServer instance’s home directory>>/config/jdbc/function
                 Run Below Commands:
find . -name "*.xml" | while read file
do
 sed -e  "s/connPoolAlias\">.*</connPoolAlias\"></g" "$file" > "$file.new"
 mv "$file.new" "$file"
done


 4. Disable package WmPRT by changing the content of file <<IntegrationServer instance’s home directory>>/packages/WmPRT/manifest.v3 from <enabled>yes</enabled> to <enabled>no</enabled>

5. Start IS

6. Enable package WmPRT from IS admin console.

7. Reconfig JDBC Pool Alias on IS admin console.

IS Start Up Error - FAILED TO INVOKE PROXY METHOD

Problem

After starting IS, get this Exception : -
com.wm.proxy.ProxyException: failed to invoke proxy method
null
at com.wm.proxy.Proxy.invokeStaticMethod(Proxy.java:168)
at com.wm.app.server.ServerProxy.ServerMain_main(ServerProxy.java:77)
at com.wm.app.server.ServerApp.start(ServerApp.java:105)
at com.wm.app.Application.launch(Application.java:468)
at com.wm.app.server.Main.main(Main.java:50)
Proxy invocation target stack trace:
java.lang.ExceptionInInitializerError: java.lang.IllegalArgumentException
at sun.net.www.ParseUtil.decode(ParseUtil.java(Compiled Code))
at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:531)
at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:501)
at sun.misc.URLClassPath$2.run(URLClassPath.java:297)
at java.security.AccessController.doPrivileged(Native Method)

Solution

+ Make sure Java Run time classpath is properly set up.
+ Make sure IS start script is properly written.
set it as follow:
set classpath=<<webMethods Installation Directory>>\jvm/linux160_64/jre/bin

then start your integration server as:
## .... run Integration Server
${JAVA_RUN} ${JAVA_D64} -DWM_HOME=${WM_HOME} -classpath ${CLASSPATH} ${IS_PROXY_MAIN} ${IS_DIR}/bin/ini.cnf $ ${PREPENDCLASSES_SWITCH} ${PREPENDCLASSES} ${APPENDCLASSES_SWITCH} ${APPENDCLASSES} ${ENV_CLASSPATH_SWITCH} ${SAVED_CP}: $*

content of ini.cnf can be:
#######################################################################
#     STARTUP PROPERTIES FOR WEBMETHODS INTEGRATION SERVER
######################################################################
application.name=webMethods Integration Server
application.productDirectory=/apps/webMethods8/IntegrationServer
application.version=8.0
application.classpath.additions=/apps/webMethods8/IntegrationServer/../WS-Stack/lib
application.updateVoidList=
application.classpath=/apps/webMethods8/IntegrationServer/lib/wm-isserver.jar\:/apps/webMethods8/common/lib/wm-isclient.jar\:/apps/webMethods8/common/lib/wm-wsclient.jar\:/apps/webMethods8/common/lib/wm-brokerclient.jar\:/apps/webMethods8/common/lib/wm-caf-client.jar\:/apps/webMethods8/common/lib/wm-jmsclient.jar\:/apps/webMethods8/common/lib/ext/jms.jar\:/apps/webMethods8/common/lib/ext/icu4j.jar\:/apps/webMethods8/common/lib/wm-scg-cluster.jar\:/apps/webMethods8/common/lib/wm-scg-core.jar\:/apps/webMethods8/common/lib/wm-scg-jdbcpool.jar\:/apps/webMethods8/common/lib/wm-scg-security.jar\:/apps/webMethods8/common/lib/wm-scg-audit.jar\:/apps/webMethods8/common/lib/wm-g11nutils.jar\:/apps/webMethods8/common/lib/glue.jar\:/apps/webMethods8/common/lib/wm-mapi.jar\:/apps/webMethods8/common/lib/wm-lwq.jar\:/apps/webMethods8/common/lib/ext/activation.jar\:/apps/webMethods8/common/lib/ext/wsdl4j.jar\:/apps/webMethods8/common/lib/ext/coherence.jar\:/apps/webMethods8/common/lib/ext/coherence-login.jar\:/apps/webMethods8/common/lib/ext/derby.jar\:/apps/webMethods8/common/lib/ext/enttoolkit.jar\:/apps/webMethods8/common/lib/ext/jaxen.jar\:/apps/webMethods8/common/lib/ext/jboss-common.jar\:/apps/webMethods8/common/lib/ext/jnpserver.jar\:/apps/webMethods8/common/lib/ext/log4j.jar\:/apps/webMethods8/common/lib/ext/mail.jar\:/apps/webMethods8/common/lib/ext/saxpath.jar\:/apps/webMethods8/common/lib/ext/xmlsec.jar\:/apps/webMethods8/common/lib/ext/commons-beanutils.jar\:/apps/webMethods8/common/lib/ext/commons-codec.jar\:/apps/webMethods8/common/lib/ext/commons-collections.jar\:/apps/webMethods8/common/lib/ext/commons-digester.jar\:/apps/webMethods8/common/lib/ext/commons-logging.jar\:/apps/webMethods8/common/lib/wss4j.jar\:/apps/webMethods8/common/lib/ext/commons-httpclient.jar\:/apps/webMethods8/common/lib/ext/commons-fileupload.jar\:/apps/webMethods8/common/lib/ext/sl53_cj40.jar\:/apps/webMethods8/common/lib/ext/tangosol.jar\:/apps/webMethods8/common/lib/ext/xalan.jar\:/apps/webMethods8/common/lib/ext/serializer.jar\:/apps/webMethods8/common/lib/ext/opensaml.jar\:/apps/webMethods8/common/lib/ext/saaj-api.jar\:/apps/webMethods8/common/lib/wm-directory.jar\:/apps/webMethods8/common/lib/wm-directory-components.jar\:/apps/webMethods8/common/lib/wm-mws-library.jar\:/apps/webMethods8/common/lib/wm-caf-server.jar\:/apps/webMethods8/common/lib/wm-caf-event-common.jar\:/apps/webMethods8/common/lib/wm-caf-schemas.jar\:/apps/webMethods8/common/lib/wm-caf-common.jar\:/apps/webMethods8/common/lib/wm-trang.jar\:/apps/webMethods8/common/lib/wm-caf-jsf.jar\:/apps/webMethods8/common/lib/wm-caf-jsf-mws.jar\:/apps/webMethods8/common/lib/wm-calendar-components.jar\:/apps/webMethods8/common/lib/saglic.jar\:/apps/webMethods8/common/lib/ext/axis.jar\:/apps/webMethods8/common/lib/ext/servlet-api.jar\:/apps/webMethods8/common/lib/ext/ldapfilt.jar\:/apps/webMethods8/common/lib/ext/ldapjdk.jar\:/apps/webMethods8/common/lib/ext/portlet-api.jar\:/apps/webMethods8/common/lib/ext/jaxb-api.jar\:/apps/webMethods8/common/lib/ext/jaxb-impl.jar\:/apps/webMethods8/common/lib/ext/jaxb-libs.jar\:/apps/webMethods8/common/lib/ext/jaxrpc-api.jar\:/apps/webMethods8/common/lib/ext/relaxngDataType.jar\:/apps/webMethods8/common/lib/ext/xsdlib.jar\:/apps/webMethods8/common/lib/ext/commons-discovery.jar\:/apps/webMethods8/common/lib/wm-calendar.jar\:/apps/webMethods8/common/lib/ext/ical4j.jar\:/apps/webMethods8/common/lib/ext/commons-lang.jar\:/apps/webMethods8/common/lib/ext/xercesImpl.jar\:/apps/webMethods8/common/lib/ext/jsp-api.jar\:/apps/webMethods8/common/lib/wm-caf-rules.jar\:/apps/webMethods8/common/lib/sin-common.jar\:/apps/webMethods8/common/lib/sin-misc.jar\:/apps/webMethods8/common/lib/ext/jdom.jar\:/apps/webMethods8/common/conf/\:/apps/webMethods8/common/lib/wm-metadata-core-centrasite.jar\:/apps/webMethods8/common/lib/wm-metadata-core.jar\:/apps/webMethods8/common/lib/cs-metadata-publisher.jar\:/apps/webMethods8/common/lib/wm-cdc-core.jar\:/apps/webMethods8/common/lib/wm-eform.jar\:/apps/webMethods8/common/lib/ext/jcr.jar\:/apps/webMethods8/common/lib/ext/jackrabbit-jcr-rmi.jar\:/apps/webMethods8/common/lib/sin-ssx.jar\:/apps/webMethods8/common/lib/sin-xmlserver.jar\:/apps/webMethods8/common/lib/CentraSiteCommons.jar\:/apps/webMethods8/common/lib/CentraSiteDynLoader.jar\:/apps/webMethods8/common/lib/CentraSiteJAXR-API.jar\:/apps/webMethods8/common/lib/CentraSiteLCM-api.jar\:/apps/webMethods8/common/lib/CentraSiteLCM-L10N.jar\:/apps/webMethods8/common/lib/CentraSiteLCM.jar\:/apps/webMethods8/common/lib/CentraSitePolicy-API.jar\:/apps/webMethods8/common/lib/CentraSiteResourceAccess-API.jar\:/apps/webMethods8/common/lib/CentraSiteUtils-L10N.jar\:/apps/webMethods8/common/lib/CentraSiteUtils.jar\:/apps/webMethods8/common/lib/CentraSiteVMS-10N.jar\:/apps/webMethods8/common/lib/CentraSiteVMS.jar\:/apps/webMethods8/common/lib/CentraSiteImportExport.jar\:/apps/webMethods8/common/lib/cstUtils.jar\:/apps/webMethods8/common/lib/INMConfigurationL10N.jar\:/apps/webMethods8/common/lib/INMConfiguration.jar\:/apps/webMethods8/common/lib/inmUtil.jar\:/apps/webMethods8/common/lib/inmUtilConf.jar\:/apps/webMethods8/common/lib/PolicyLogBindings.jar\:/apps/webMethods8/common/lib/TaminoAPI4J-l10n.jar\:/apps/webMethods8/common/lib/TaminoAPI4J.jar\:/apps/webMethods8/common/lib/uddiKeyConverter.jar\:/apps/webMethods8/common/lib/uddiv3ClientAPI.jar\:/apps/webMethods8/common/lib/CSIntegrator.jar\:/apps/webMethods8/common/lib/xqj-api.jar\:/apps/webMethods8/common/lib/xqj-ino-api.jar\:/apps/webMethods8/common/lib/wvcm.jar\:/apps/webMethods8/common/lib/ext/xmlbeans.jar\:/apps/webMethods8/common/lib/ext/jaxr-api.jar\:/apps/webMethods8/common/lib/ext/stax-api.jar\:/apps/webMethods8/common/lib/ext/wstx-asl.jar\:/apps/webMethods8/common/lib/ext/script-api.jar\: